There have been 82 NFL drafts and thus 82 pieces of evidence suggesting that teams do not quite know how to pick quarterbacks. If any other industry failed so often at identifying and developing the most crucial sector of its field, it’d probably devote more resources to figuring it out. If, say, the boating industry set up a system in which half the boats sunk for some reason, there’d be a meeting or two about it. And yet, NFL teams have mostly stuck to their original strategies.
The most important thing in building a football team—evaluating the quarterback—is especially important right now. This year’s draft could feature six quarterbacks selected in the first round, marking the first time that’s happened in 35 years. These are the types of drafts that ensure a boatload of new day-one starters and overhaul the entire landscape of the league.
At the very least, the NFL is certain to draft four quarterbacks in the first round. In the past decade, that happened only in 2011 and 2012. In one of those, the Panthers took Cam Newton and were on their way to annual competence as a franchise, but the teams that selected Jake Locker, Blaine Gabbert, and Brandon Weeden over those two drafts went in the opposite direction. Not all of the quarterbacks drafted early work out, and the ones that don’t can hurt a franchise.
What makes the quarterback-heavy drafts so fun is that they reveal large chunks of information about the sport. When the teams start picking quarterbacks, it’s classic football: teams overanalyzing something and still managing to not put enough thought into it. So many teams will overemphasize things that look impressive but mean nothing, while they overlook or ignore the traits that make quarterbacks great. It is, to use the scouting term, absolutely hilarious.
NFL teams have not gotten any better at picking quarterbacks over the years. Dak Prescott was the eighth quarterback drafted in 2016, behind Christian Hackenberg, Cody Kessler, and Connor Cook, and the Cowboys even admitted that Cook was their preferred prospect because of the offense he played in during college. Deshaun Watson, who looked as impressive as any rookie in recent memory in the seven games he played before his knee injury last year, was the third quarterback selected in the 2017 draft, and you could make a case that five teams who passed on him needed a quarterback. Hell, the Browns, who traded to Houston the pick the Texans used on Watson, technically passed on him twice. Then there’s the well-worn ground of pointing out that Tom Brady was readily available but teams said “nah” 198 times, while the same thing played out with Aaron Rodgers 23 times.
When it comes to collegiate passers, no one is certain about anything. Bill Walsh, who was one of the smartest football minds ever, once advocated for passing on Peyton Manning and Ryan Leaf in order to take Brian Griese in Round 2.
So here’s the question: Why? Why, in this era of seemingly unlimited data, an ever-growing sample size showing teams which traits matter, and expanding technology, are teams not getting better at assessing the most important position in their sport? The answer is that it takes something rarely understood or accepted in football. It takes nuance.
The first first-round quarterback was Sammy Baugh, a future Hall of Famer drafted by Washington in 1937. Two years later, Sid Luckman became the next first-round passer, and he also went to the Hall of Fame. Two picks after Luckman, TCU’s Davey O’Brien was selected with the fourth pick in the first round. That makes him the first first-round quarterback to not pan out—not because he was bad, but because he quit football after two years to join the FBI.
Ditching the sport for a career in law enforcement is not a concern that teams need to flag for modern prospects, but here’s a list of things that modern teams have, in fact, flagged this year: Sam Darnold’s “bad face,” the generation that Josh Rosen grew up in, Lamar Jackson’s agent, and Baker Mayfield reminding Ryan Leaf of himself. But let’s leave the pseudoscience of personality evaluation aside for now and focus on how teams miss on the actual on-field product.
Due to an increase in data at the college and pro levels, we now have a fairly good outline of which skills carry over from college to the NFL level. Although teams have not necessarily adapted, the race to put science into the alleged inexact science is on. Remember, most of this data is relatively new, so it’s too early to accuse NFL teams of ignoring it completely. But one good litmus test should arrive early on in Round 1 this year: Andrew Healy, whose database projecting college quarterbacks points directly in Mayfield’s direction, now works for the Cleveland Browns.
“NFL teams like players who are taller and faster, they think ‘Wonderlic’ means ‘smarter’ even though it’s not a test of football smarts, and they like guys who put up big numbers,” said David Berri, a sports economist and analytics expert who is a professor at Southern Utah and has studied the NFL draft. “That seems to all makes sense, but then you look at it and you say, ‘What of this predicts performance?’ And the answer is not really any of that.”
Matt Richner, an NFL draft consultant who has advised teams on quarterback evaluation, says that franchises struggle to marry the newfound data at their disposal with reliable scouting methods. He believes a hybrid of both is required to find the right quarterback, but that only “three or four” teams—the New England Patriots being one—have cracked the code. For instance, he’ll come up with a number that says a certain quarterback is exceptional at rollouts to his right. “[A team will] say, ‘That’s awesome!’ And I’ll say, ‘Well, you only call rollouts on 2 percent of your passes,’” Richner said. “They can’t really handle the information given to them.”
Like most people who study the college-to-pro transition, Richner puts an emphasis on completion percentage and players making throws that they’ll be asked to make at the NFL level. He finds it maddening when teams fall in love with a big-armed quarterback like Wyoming’s Josh Allen because he might be able to throw a 35-yard post route even though those account for only a minuscule percentage of throws over a season, while something like a 10-yard out route is far more common. He compares it to old-school baseball scouts who become enamored of players who can hit 500-foot homers, even if they do it far less often than a player whose homers go 350 feet.
Quantifying what kinds of throws a prospect will have to make in the NFL is a growing field. Pro Football Focus has developed a metric called “NFL throws,” which identifies the types of throws in which there’s a great variance between passers. One example PFF gives: an over-the-shoulder throw made in rhythm into tight coverage, where there’s plenty of room for disaster. “We identified the throws that separate Tom Brady from Brock Osweiler or DeShone Kizer or whoever you think the worst quarterback is,” said Pro Football Focus senior analyst Steve Palazzolo. Mayfield graded the best on these throws among draft prospects. Allen was sixth.
Palazzolo said his company is devoting more resources to explaining quarterback play. The company has three years of college data and is starting to get a firm grasp on projections. He said that the statistics that remain stable year-to-year when measured at the NFL level—performance from a clean pocket, avoiding negative throws, and hitting open throws—also can suggest which college prospects are more likely to succeed in the pros. These metrics favor Mayfield as well.
“It’s all been underresearched because there hasn’t been enough data,” Palazzolo said. He thinks NFL teams don’t do a good enough job of making eliminations based on evaluations. If, he says, a passer has a really bad completion percentage, he should be crossed off from high-pick contention. Then there’s the matter of teams confusing some consistent traits with things that coaches can change in a passer. Accuracy and decision-making, he said, very rarely improve at the pro level.
Richner believes that though most of the keys to projecting quarterbacks lie in the numbers, traditional indicators like the ability to perform in a collapsing pocket, keeping your eyes up during a throw, and comfort with audibles are all still key components of the evaluation process. Richner’s biggest key, however, is completion percentage, particularly on third down, when passing windows are tightest—and that is where he differs greatly from many NFL evaluators. He believes Mayfield’s numbers resemble Drew Brees’s and that he’s well worth a high pick, and he also has USC’s Darnold as a first-round pick. He thinks Allen, he of the 56 percent completion rate, is not a prospect, and he believes that UCLA’s Rosen is “awful” because of the dip in his completion percentage from second to third down. He said the 12-percentage-point decrease is on par with Gabbert, Jimmy Clausen, and Locker. None of whom was … good. On the other side of the spectrum: Seattle’s Russell Wilson is the best collegiate passer on third down since 2009.
Rosen, it should be noted, ranks second in PFF’s NFL throws grading system, and he excels in the intermediate part of the field—a place many NFL throws occur. Whichever indicators a team sides with, it finally has the tools to add some more certainty to the inexact science of drafting a quarterback. The question now is whether or not teams actually will.
