The New England Patriots are visiting the Pittsburgh Steelers on Thursday Night Football. This is an important game for Pittsburgh: the Steelers are 7-5, with a tenuous hold on their AFC wild-card slot. Starting quarterback Kenny Pickett is out with an ankle injury, so backup quarterback Mitchell Trubisky got the start, and they’re currently down to the Bailey Zappe–led Patriots, 7-3. It’s third-and-10 on the Pittsburgh 27-yard line for Trubisky. Here’s the play.
Trubisky drops back and is immediately under pressure—the Patriots sent a safety and a cornerback off his right side. He throws a panicked, fluttering ball to Pat Freiermuth up the seam, but Patriots safety Jabrill Peppers steps underneath the route and intercepts the pass on the run.
The Patriots offense converts the subsequent short field into a touchdown in just two plays. The score is 14-3, and the Steelers will never bridge the gap, losing 21-18.
Color commentator Kirk Herbstreit breaks down how the interception happened with slow-motion replay, as all broadcasts ask their color guys to do following key plays. “This is the book on [Trubisky],” he says. “Show him a presnap look, and then move at the snap of the football. … Watch the linebackers sink, and now they drop back into coverage, the safety was up close to the line of scrimmage, he drops back ... they bring pressure from his right, they drop from the left, and that’s been his Achilles’ heel: identifying that.”
The idea that Trubisky has a hard time identifying the Patriots blitzes and coverage rotations is a funny one. Not because it’s Trubisky or because it’s a particularly sneaky blitz. But because, presnap, an AI program identified the blitz. In real time. And broadcast it to every watcher of the game so that they could see what Trubisky couldn’t.
That AI is called Defensive Alerts. It’s a feature of Amazon’s Prime Vision broadcast—a weekly alternate stream available everywhere Thursday Night Football is. An alternate stream that may have the one thing so many prior alternate football broadcasts have never been able to find: a legitimate role in the world of NFL broadcasts.
Here’s a look at the same Trubisky interception, this time from the alternate Prime Vision broadcast. There are, as you can see right away, a lot of differences.
At first glance, this doesn’t even look like a broadcast—it looks like a breakdown, edited in post and released for the sake of analysis. On traditional broadcasts, the camera stays tight to the quarterback, keeping the line of scrimmage in view while receivers and defensive backs run off screen. On Prime Vision, a wide angle is used, and every player is visible at all times. On the traditional broadcast angle, when Trubisky releases the pass, viewers don’t know whether it’s a heroic throw made with pressure bearing down or a costly mistake—at least, until the receivers and defensive backs come into view. The Prime Vision view sacrifices that suspense, that uncertainty, for the sake of clarity.
As obvious of a difference as that might be, it may not be the most notable. As the offense gets set presnap, multiple Patriots players are highlighted with floating red circles. The “red orb” is the product of Defensive Alerts: a self-teaching artificial intelligence tool built by football experts working hand-in-hand with Amazon Machine Learning that identifies potential blitzers before the ball is snapped. As those defensive players shuffle (and then shuffle again), the AI program calculates and updates their likelihood of blitzing in real time, using the GPS data of each individual player and comparing it to thousands and thousands of defensive plays it has already watched. Once a player is deemed likely to blitz by Defensive Alerts, he’s highlighted with the red orb.
A presnap highlight isn’t a particularly new or radical thing to broadcasting. When Tony Romo took to the booth in the late 2010s, he sparked a craze of predictions for color commentators. It’s no longer a surprise now when broadcasters call out defenders following motion and predict man coverage or circle a particular matchup that they like at the bottom of the screen.
But Prime Vision does with computers what other broadcasts do with people. Defensive Alerts highlights potential blitzers. Prime Vision broadcasts also include a white orb that floats under a player of particular interest—especially one who lines up in a variety of spots, like Peppers, who was highlighted at the beginning of the play and ended up picking Trubisky off at the end of it.
A focus on predictive analysis was a huge part of the ideation of Prime Vision, according to Jared Stacy, director of global live sports production with Amazon. “So much of what we do on television today is an amazing job of breaking down a replay,” he said. “Well, what can we do leading into a play or as a play is unfolding that’s going to give the viewer some insight into that play? We know we can break down what just happened. Can we actually break it down in advance?”
The Prime Vision broadcast is predicated on questions like that one: Can we provide play analysis before the play happens? Here’s another: “Who’s a likely target on a play? When is the receiver actually open?” This question, asked by Stacy, was the genesis of Prime Targets, another feature of the Prime Vision broadcast, which places a green orb underneath pass catchers who are open and will likely convert a first down if targeted.
Over its two-year lifespan, Prime Vision has become a broadcast for asking big, bold questions, and then trying to answer them with new and revolutionary broadcast features. Stacy calls Prime Vision a “sandbox.” Sam Schwartzstein, producer of analytics and insights at Prime Video, has another name for it: the incubator. “There’s not a single week where we’re rolling out the same show, the same features,” he said. “We are trying new things every week: We will sunset things that don’t work, and we will boost things that do work.”
Schwartzstein makes the point that the Prime Vision broadcast isn’t all about machines and analytics. Defensive Alerts is almost entirely driven by AI, yes—but when Prime Vision chose to put a white orb under Peppers, it did that based on nothing but Schwartzstein’s preparation for the game. A football guy through and through, Schwartzstein played center at Stanford during the Andrew Luck years and, before joining Amazon, was the director of football operations, innovation, and strategy at the burgeoning XFL. When the Ravens played on Thursday Night Football, Schwartzstein put the white orb under Ravens fullback/tight end/extra offensive lineman Patrick Ricard because “I love Patrick Ricard.”
Schwartzstein’s role with Amazon extends beyond that of game researcher (though he does sit in with Al Michaels and Kirk Herbstreit during production meetings and help prepare the main broadcast for any and all analytic questions that might rear up). He also represents another new frontier explored by Prime Vision: He’s the first analytics expert to break into the broadcast.
Just as rules experts are consulted by most broadcasts to (theoretically) demystify challenging replays or clarify surprising calls, so too will Schwartzstein be summoned up on the Prime Vision broadcast to provide color to a coaching decision or other moment that provides red meat for analytics. Take the Seahawks’ decision to go for it on a fourth down late in a one-score game against the Cowboys in Week 13: “That was the right call to go for it,” Schwartzstein says on the ensuing play. “It’s because of the time left on the clock and that the Seahawks are missing that timeout. ... Unless they had another timeout, it could have been a probable opportunity to punt there, but by not having that timeout, they have one less tool in their toolshed to try to end this game.”
Schwartzstein’s analytics interludes don’t interject over the main broadcast—watchers of TNF on Prime continue to hear Al and Kirk—but on the Prime Vision broadcast, he can cut in over their audio to provide clarity on those fourth-down decisions. And those aren’t the only times that Schwartzstein is brought in. Schwartzstein and the Amazon Prime production team work directly with the NFL’s Next Gen Stats research team to uncover and develop a smorgasbord of noteworthy tidbits to sprinkle over the traditional Thursday night broadcast.
Consider another example from the Seahawks-Cowboys barnburner. Everyone from ardent fantasy managers to average fans knows that the Seahawks offense isn’t as good this year—so what’s been the issue? A Schwartzstein breakout at the top of the game gives some insight with a rarely used term on other, mainstream broadcasts: success rate.
Success rate is all over the Amazon Prime broadcasts—it’s even included in statistical graphics on the main broadcast. Why? “Although it’s not the perfect metric, it’s a metric that people can understand. It’s presented as a percentage,” Schwartzstein tells me. Expected points added, on the other hand, is presented as “fractional points,” as Schwartzstein calls them, which is much more abstract. “Our mantra is 90 percent purity, 100 percent understanding.”
Smarter football stats have been pushing their way onto broadcasts for years now. Keegan Abdoo, a Next Gen Stats analyst and Schwartzstein’s primary touchpoint from NGS, has been working on NGS packets for broadcasts for six seasons now. Each packet is filled with minute, proprietary data on both teams in the contest. You want to know how many yards after the catch over expectation Deebo Samuel has on in-breaking routes this season? How about the average get-off time of the Browns defensive line relative to the rest of the league? There’s an NGS packet for that.
Once the packet is made and delivered to a broadcast, how much of the information gets used is out of NGS’s hands—broadcasts must hit their contractual minimums but otherwise are free to use or pass on whatever they like. But the appetite for Next Gen Stats is “night and day” across broadcasts leaguewide, according to Abdoo, compared to what it was when he started. “We passed our total number of executions from last year in Week 9 this year.” That’s on all traditional broadcasts—it doesn’t include Prime Vision.
Prime Vision is its own beast entirely. Abdoo remembers when NGS first started sending stats over to Amazon Prime and Prime Vision. “It was like, 95 percent of our notes are making air? This is so different than anything we’ve ever done before.”
The relationship between Prime Vision and Next Gen Stats has become a two-way street. As Schwartzstein does his prep for the Thursday night game, he develops questions—remember, Prime Vision is a place to ask questions and answer them. While watching Steelers offensive film, he felt like wide receiver George Pickens was seeing a lot of cloud coverage. Cloud coverage is a handy way of getting vertical help on a dangerous downfield receiver like Pickens: It has a safety over top and a corner playing underneath.
So Abdoo built Schwartzstein a new stat in the NGS dashboard, catching all of the individual coverages that Pickens saw that would have a cloud safety on top of him, and there it was: The Steelers led the league in the amount of cloud coverage they’ve faced. That stat made the packet.
The same thing happens in the other direction: Abdoo and the NGS team can notice things and send it up the pipeline to Schwartzstein. A whopping 13 Next Gen Stats employees are in the office on Thursday night, supporting the Prime Vision broadcast live. When surprises appear, as they do during every NFL game, that amount of manpower comes in handy.
Take the Patriots’ offensive success against the Steelers last Thursday. While you and I were hitting up group chats with “Zappe > Mac” takes, the Next Gen Stats team figured out what was actually going on: With Bailey Zappe under center, the Patriots weren’t running their usual spread formations, but rather running much tighter formations. “We looked it up, and lo and behold: They’re at 50 percent (condensed formations). It’s their highest rate in a game this season by far. Sam builds a segment around it, talks about Zappe—they’re making the reads easier for him with condensed formations—we get that graphic up on the screen.”
This stuff? This is really for football nerds. Most dads in their La-Z-Boys don’t want to know how wide the Patriots’ offensive formations are—they just want to grumble that Jones should have been benched for Zappe ages ago. But that’s just because they’re seeing what they’re used to seeing. “There’s a reason why the main broadcast feed is tight: It tells you where to watch.” Schwartzstein says of traditional broadcasts. The all-22, as the wider shot is colloquially called, shows the moving safeties, all the routes developing in real time. It can be overwhelming without a guide.
“So we get the player name tags, we give the route trees, we give the Defensive Alerts,” Schwartzstein continues. “When we’re doing analytics and conversations, I want to give you that Leonardo DiCaprio moment—oh, look look look look look! A guided viewing experience.”
From George Pickens breakouts and fourth-down explanations to the Christmas-colored orbs floating under the players shown from a super-wide angle, the Prime Vision broadcast is weird. In a good way? In a bad way? Nobody’s really sure yet. It’s the sandbox to Stacy, the incubator to Schwartzstein. Abdoo calls it the “laboratory for broadcast enhancements.” Throw some new stuff at the wall and see what sticks.
Because Prime Vision is so weird, it can’t just hop into the NFL zeitgeist tomorrow. Like Schwartzstein said: The viewing experience must be a guided one. But traditional NFL broadcasts are pretty pleased with their current state of guided viewing. The camera angles are familiar, the voices on the call are familiar. Prime Vision is a radical change in a space that doesn’t really need radical change.
Perhaps that’s why, when I ask what “success” looks like for Prime Vision, Stacy goes back to the sandbox. “Success [for Prime Vision] is continuing to develop and refine features that can also migrate to the primary broadcasts,” he said. “The key is: We don’t ever stop thinking and questioning ourselves. We’re not always going to get it right. But we’re going to learn a lot, and I think those learnings are hopefully going to lead to really exciting features for fans.” Prime Vision isn’t really trying to achieve anything, like bringing in bigger numbers or reaching a younger audience, as many other alternate broadcasts aim to do. It’s just a place to try cool stuff.
Such a migration happened this past Thursday night, when Peppers intercepted Mitchell Trubisky. Defensive Alerts predicted the blitz on Prime Vision. Al Michaels called the play; Herbstreit broke it down. But when the main broadcast came back from commercial, there it was for the entire NFL audience to see: Prime Vision’s Defensive Alerts, yanked up onto the big stage. It was a moment for Amazon to say, “Hey, we tried something cool—and it worked!”
If an entire broadcast to try cool stuff sounds like an indulgence, well, perhaps it is. But Amazon is uniquely suited to have constructed this sandbox. In the live NFL space, Amazon is the only over-the-top provider: The company completely and totally bypasses cable and is available over the internet. Other networks have done alternative broadcasts: CBS partnered with Nickelodeon to produce broadcasts targeted at children, and just this year, Disney generated a Toy Story–themed alternate broadcast of the Jaguars-Falcons game in London. But those alternative broadcasts cost an additional channel if delivered on traditional television. Delivering them over streaming services requires staffing and technology, which can be a big lift for networks.
But Amazon invested in production teams from the day it acquired Thursday Night Football in 2021 and, with its entire operation in-house from Day 1, doesn’t have to endure these limitations. The company isn’t boxed in by Xfinity, DirecTV, or any other distributor of a block of channels. If Amazon wants to pop in a new alternate stream for TNF with, say, Dude Perfect—the famous YouTube trick shotters—it has the freedom to do so.
Being free to do something doesn’t necessarily mean there is wisdom in it. But in the case of Prime Vision, it seems like there is. Why wouldn’t traditional football broadcasts be interested in AI that could highlight blitzers before they come? That seems like the sort of thing NFL fans would enjoy. Or replays that automatically highlight open receivers missed by a quarterback on a key third down? When it’s all on the screen at once, it can feel like too great of a shock for a mainstream viewer—but piecemeal, the best of Prime Vision very well may become the standard of Sunday NFL viewing. As fun as the Nickelodeon playoff broadcast is, end zone slime has never sneaked its way onto the primary feed; Patrick Star hasn’t stepped in for Joe Buck. Prime Vision may become the first broadcast to truly bridge the gap.