The Race to Make the NFL Draft an Exact Science
The NFL has more data on its players than ever before, but evaluating prospects still involves a lot of guesswork. Here’s how some teams are getting creative to find their perfect pick.The Race to Make the NFL Draft an Exact Science
The NFL has more data on its players than ever before, but evaluating prospects still involves a lot of guesswork. Here’s how some teams are getting creative to find their perfect pick.
Since last fall, I have reviewed material from the analytics databases of a handful of NFL teams relating to last year’s draft. The one commonality between all of them is they made me feel stupid regarding both football and math. The difference between them is everything else.
It is not a surprise that teams view players differently—traditional draft boards have wide variances even without the use of complex formulas. Each organization believed its in-house analysts had identified specific targets to build its draft strategies around. But what struck me is how very few of the teams’ targets overlapped with each other. One was obsessed with body composition. Another, using data collected from the traditional scouting process, thought it had found a formula to determine what constitutes a red flag.
There is nothing more interesting in this week’s draft than the race to make the NFL’s most inexact science an exact one. Here is what we know in 2019: There is more, and better, information about this year’s prospects than any in history, and that trend will continue for the foreseeable future. Whether teams are efficient in how they use that information is a different matter altogether.
On January 22, Mississippi State defensive end Montez Sweat, one of the top prospects in this year’s draft, topped out at 19.3 miles per hour at the Senior Bowl practice. He was not only the fastest defensive lineman that day, but he was faster than about half of all defensive backs, and no tight end came within one mile per hour of him. He was about two and a half miles per hour faster than Clemson receiver Hunter Renfrow. Sweat is 6-foot-6 and 260 pounds.
This information has been available for only two years, so there is no way of knowing how Sweat’s time would compare to, say, Khalil Mack’s time when he prepared for the draft five years ago. There is almost no tracking data available for college players, so NFL teams have to rely on a limited sample size collected during pre-draft workouts like the Senior Bowl and the combine. Sweat’s speed is part of a defining characteristic of football in 2019: It’s fascinating information and a cool nugget to repeat to your friends. In theory, it’s more revealing than the 40-yard dash, since Sweat will almost never run 40 yards on a play, and won’t do so without pads. It will be up to each team to decide whether this information is enough to make Sweat a better prospect than, say, Ed Oliver, or Rashan Gary, neither of whom were at the Senior Bowl.
When it comes to evaluating players already in the NFL, analytics are having a moment. Last year was the first time in-season player-tracking data was available, and it’s changed the way some teams view football. The NFL is still a coldly conservative league when it comes to many things—teams don’t go for it on fourth down enough and run the ball too much, for starters—but progress is being made. When I wrote about the league’s quiet analytics movement in December, I found it interesting that teams used tracking data to learn that some players were much faster in game situations than their speed drills at the combine suggested. Everyone, it seems, is on board with some form of analytics, even Raiders coach Jon Gruden, who’d taken a bemused stance on it last year. “I love analytics,” he said at the NFL owners’ meetings last month. “You people act like I don’t like analytics.”
But at the college level, it’s still an uphill battle to get that sort of hard data on prospects, which leads to a lot of guessing. According to John Pollard, the vice president of Zebra Sports, which provides player-tracking data for the NFL and the Senior Bowl, teams are evaluating tracking data for the best NFL players and seeing whether they can cross-reference it with stats from the combine that correlate to that kind of production. For example, if tracking data shows an edge rusher is quicker off the line of scrimmage than anyone else, teams will reverse-engineer his combine stats to find out how they could scout for that skill. Armed with new information—like the speed at which running backs explode through the offensive line—Pollard says teams are looking at film differently.
The Ringer’s 2019 NFL Draft Guide
In-depth scouting reports, mock drafts, and more analysis from Danny Kelly and our other NFL staff writers
Pollard says he’s received interest from college programs to use Zebra’s player-tracking technology in bowl games, though the NCAA has yet to sign off on it. “We would want to give that information to the NFL,” he says. Top teams use tracking data at the college level but release the information sparingly. For instance, we know that Alabama’s Kenyan Drake ran 23 mph on his kickoff return for a touchdown against Clemson in the 2016 national title game. But it borders on being little more than a fun fact (it’s also not that impressive compared to Sweat).
“There is no centralization for player tracking in college. A couple of conferences have it, a couple of teams have it for their own players,” says Michael Lopez, director of data and analytics for the NFL. “Teams want that data.”
What’s happening to the draft is a continuation of what has been happening in the league as a whole: Teams and analysts are trying to figure out what information matters. There’s just more of it than ever before.
“I’ve long held that football’s questions are more complicated and that our questions take longer and that’s a big reason teams are investing in this area,” Lopez says. “It’s not as simple as ‘who has the best on-base percentage’—though baseball is more complicated than that—but football starts at a much more complex starting point.”
“There is no centralization for player tracking in college. A couple of conferences have it, a couple of teams have it for their own players. Teams want that data.” — Michael Lopez, director of data and analytics for the NFL
“Analytics” is a moving target because it can generally mean anything involving a number. Teams like the Eagles, Vikings, and Jaguars have been praised for their analytics work, while other organizations … do not know what the term actually means. Six years ago, an NFL source told me that the Bills’ selection of quarterback EJ Manuel with the 16th overall pick in the 2013 draft was driven by analytics: Their research showed that the size of his hands would help in cold weather (it didn’t).
One of the most useful exercises of the draft is getting a yearly check-in on how teams view the sport. Ten years ago, it would have seemed miraculous that Baker Mayfield, an undersized, spread quarterback, would go first overall, since NFL front offices have spent the past few decades obsessing over quarterbacks’ height and college schemes (just ask second-round pick Drew Brees). It would have been inconceivable that Kyler Murray, a smaller quarterback than Mayfield, who played in the same college system, might go first overall the following year. The draft is a succinct way to figure out which NFL norms still exist and which have been toppled. It’s how we learn what teams think modern football is, so it’s fascinating to see how the data era of football impacts the draft, if at all.
Almost every data person I talk to believes the same things about the draft. The first is that the parts of the process that take place behind closed doors—interviews, prospects diagnosing plays on a whiteboard, character evaluations—are too difficult to quantify, and matter greatly. The second is that NFL scouts are pretty good at their jobs. “Scouts have a keen eye. They see things that are spot on. This is not ‘Oh, these guys not qualified and missing the picture and that’s why the draft is so hit-and-miss.’ It’s not it at all. They know what they see, and they tend to be right,” says Karl Pazdernik, cofounder of Deep Football, a company that applies machine learning to player evaluation. Pazdernik is a data scientist who became fascinated by the draft. Players age so fast in football, he says, that teams have to consistently hit on picks to replenish their talent, significantly more so than in other sports.
Pazdernik thinks teams can be smarter about identifying which traits are considered strengths and weaknesses. “The issue is that there are biases, that a quarterback has to be tall or a defensive lineman has to be strong,” he says. Along with his cofounder and brother-in-law Jacques Kvam, Pazdernik used machine learning to analyze NFL.com scouting reports. After identifying the key words that appeared most frequently, they ran statistical models to measure how reliable an indicator of a player’s success those characteristics were.
They found that players whose “strength” was listed as “short passing”—former Vikings first-round pick Christian Ponder, for instance—were not particularly successful. Kvam considered it to be a backhanded compliment in the scouting report. They also found it matters less to a wide receiver’s success if his size is listed as a weakness, citing DeAndre Hopkins as an example.
This sort of creative approach to player evaluation is getting more common in the NFL. There will never be a single statistic—VORP, WAR, or PER—to compare football players across positions, but little edges can be gained in the scouting process. Minnesota general manager Rick Spielman told me last season that the Vikings staff has built a database that compiles what the team’s scouts say about a particular player, and will consult it to see which scouts exhibit the most expertise for a particular position. Scouts are already good; how teams view scouting can get better.
“The point is to use data sources that haven’t typically been used to see if we can get extra information that is basically just sitting there, untapped resources,” Pazdernik says. Untapped resources are, of course, what this era of football is all about. The NFL draft is complicated. No sport has as many variables in the transition between college and pro. The wide variety of schemes makes statistical projection harder and ensures the eye test remains crucial to scouting. So any correlation is seen as interesting to NFL teams.
Pazdernik and Kvam say they are in discussions with “multiple” NFL teams about using Deep Football’s work, though as they point out, the moment they start evaluating a team’s proprietary scouting reports is the moment they stop talking publicly about their work. (They are also building out two non-draft algorithms: one that measures a quarterback’s pocket presence and another that will attempt to measure a player’s instincts using tracking data to judge how quickly they adjust to a play.)
The most interesting wrinkle in draft analysis is that it is a public endeavor as much as a private one. There are a lot of really smart people publishing work on what matters in the draft, many of whom don’t work for NFL teams. Football Outsiders’ Nathan Forster found, oddly, that passes defended was a key statistic for edge rushers. Great work is being done showing why teams miss on quarterbacks. Often, analysts publishing this work are offered jobs by NFL teams. The league had a “Big Data Bowl” competition at the combine to showcase academic work which Lopez says has already led to job offers from teams. Two names that popped up in my reporting are two recent college graduates: Jason Mulholland, who wrote a paper evaluating the tight end position while at the University of Pennsylvania, and now works for the Jets, and Namita Nandakumar, also from Penn, who landed with the Eagles after they were impressed by her work on hockey analytics.
Pro Football Focus started collecting advanced statistics from college games in 2014, but “we’re just scratching the surface,” says Eric Eager, a data scientist for PFF. The problem is finding a big enough sample set for the studies. “Football is so context-driven that once you drill down on what a guy is asked to do, your sample size shrinks considerably.” For example, Eager thinks the data is very good at projecting how good a player will be at pass protection because there are a lot of passing plays and only a few pass-protection schemes. Projecting how good a player will be at run blocking is more difficult because of the greater variety of schemes at the college and NFL levels.
“Football is so context-driven that once you drill down on what a guy is asked to do, your sample size shrinks considerably.” — Eric Eager, a data scientist for PFF
Quarterback evaluation is just as hard because there are so few elite passing prospects, and such a wide variety of college offenses. PFF focuses on what are called “NFL throws,” which are exactly as they sound. “A bubble screen is not an NFL throw because any NFL quarterback can make it and it’s not particularly valuable because it’s behind the line of scrimmage,” Eager says. “A deep post route into coverage separates Tom Brady and Patrick Mahomes II from Blake Bortles.” Even though Mayfield and Mahomes didn’t have the highest volume of NFL throws during their college careers, they excelled at them. (PFF, by these metrics, loves Kyler Murray.) PFF has found the plays that best predict a quarterback’s ability are when he’s throwing from a clean pocket. Eager says it’s easy to glamorize the Brett Favre touchdown pass as three linemen are hitting him, but it’s not a good indicator of repeatable success. This analysis helps when evaluating Big 12 quarterbacks like Murray, Mayfield, and Mahomes, who typically faced little pass rush.
Another thing PFF has found that relates to the draft is that highly graded coverage players are just as important, and perhaps more so, than highly graded pass rushers. “It’s something that, at first, super offended my sensibilities,” Eager says. “It’s a product of how we watch the game. The broadcast angle doesn’t show the coverage guys. Team success is correlated with how well coverage is. Pass rush and coverage are correlated, but the direction arrow points more towards coverage helping pass rush more than the other way around.” He points out that the smartest team in the league, the New England Patriots, has spent big on two cornerbacks this decade, Stephon Gilmore and Darrelle Revis, and not on pass rushers. This also helps explain why the Chiefs, another smart franchise, have put more emphasis on building the back end of their defense than their pass rush. Teams have more information than ever, but information hasn’t stopped them from making mistakes in the past. The beauty of the draft is that it tells you which teams are paying attention.