Marcus Mariota and Misleading Metrics

In the aftermath of Monday night’s contest between the Houston Texans and Tennessee Titans, several people have pointed out the outstanding numbers Titans quarterback Marcus Mariota compiled in a losing effort:

Anyway you slice it, those numbers are crazy. While sports-analytics nerd (his words, not mine) Adam Steele pointed out that Adjusted Net Yards / Attempt, which includes Mariota’s six sacks taken, does a better job capturing his overall value, Mariota’s ANY/A comes out to 10.3, still an excellent total. For reference, New Orleans Saints QB and MVP frontrunner Drew Brees leads the NFL with a 9.7 ANY/A.

So if Mariota performed so well, why didn’t the Titans score more than 17 points?

Empty Yards and Completions

First, it’s important to note what Mariota’s 22 of 23 for 303 yards statline includes—and what it doesn’t. It includes Mariota going five of six for 64 yards in what amounts to garbage time—a drive that started with less than two minutes left, at his own nine yard line, with a 17-point deficit. It does not include the six sacks for 43 yards Mariota took. And these sacks were killer:

That’s right—all six sacks led immediately to punts or led to long-yardage situations that resulted in punts. Six sacks, five drives killed. While the negative yardage factors into ANY/A, and the extra plays into the denominator, it sells short the effect negative plays have on drives. An eight-yard loss and an eight-yard gain are not equal. The gain might convert a first down or set up a 2nd-and-2, but once the offense gets the first down, the O and D start back at square one: first and 10. An eight-yard loss is practically a drive killer, dramatically reducing the chance of converting a first down and virtually handling the ball to the other team. Rarely is this effect as dramatically shown as it was Monday night.

Mariota’s completion percentage was also raised by what Football Outsiders’ Scott Kacsmar calls “failed completions,” that is, completions that don’t help the team. For instance, Mariota was seven of seven passing on third down, but four of those completions didn’t net a first down. Chase Stuart identifies seven completions that produced a negative Expected Points Added. Add that to the six sacks and six passes in garbage time, and you only have 10 relevant pass plays that helped the Titans.

Also worth noting: Mariota added 28 yards on six carries, but these were also pretty hollow. He had a 14-yard scamper to convert a 2nd-and-13, but that was his only run that converted a first down.

Big Play Distortions

And there’s an issue with those 10, too—specifically, two of the plays were too good. Obviously the being good isn’t a problem, and the two big play touchdown passes, a 61-yarder to tight end Jonnu Smith and a 48-yarder to wideout Corey Davis, put Tennesee’s only two touchdowns on the board. The problem is the distorting effect these plays have on Mariota’s per play statistics. Two plays were responsible for more than a third of his yardage total, bumping up his yards per attempt from 9.2 to 13.1—an increase of almost four yards!

Statisticians have long known of the distorting effect outliers such as these have on averages, but they’re especially misleading when it comes to measuring football efficiency. That’s because plays are not a finite resource. Get a first down, and you can earn more plays. Many statistics will show one 40-yard completion as more efficient than eight five-yard gains, but the two teams are in the same state at the end: 40 yards downfield, facing a first-and-10. (Clock considerations make the two situations not exactly identical, but in many situations eating more time of possession in the second scenario is preferable). Had Mariota thrown five shorter passes instead of the one 61-yarder, still getting a touchdown, it’s worth the same on the scoreboard but would not have distorted his stat line so dramatically.

Mariota’s ANY/A was 74.5 on the two touchdown drives and 10.7 in garbage time, but a paltry 4.1 on the other seven drives despite a gaudy 15-for-15 completion mark:

Add it all up and Mariota’s outing is practically a perfect storm of the ways conventional statistics can be distorted—outlier big gains, failed completions, costly sacks, and garbage time production. The Titans passing game wasn’t completely toothless, but it struggled much of the night, a reality you’d never know from Mariota’s stat line.

What Wasn’t His Fault

The passing game was hardly the only problem for Tennesee Monday night. The defense gave up 34 points, and it also did the offense no favors, generating zero turnovers. That left the offense with an average starting field position of their own 25—a figure that would rank 31st over a full season.

The running game was superficially successful—23 carries for 105 yards—but it falls prey to the same big play distortions that the passing game does. Davis took an end around for 39 yards, and there were also Mariota’s contributions on the ground, discussed above. Running backs Derrick Henry and Dion Lewis combined for 15 carries for only 38 yards, so the down-to-down running game wasn’t helping any. Backup tight end Luke Stocker was also stuffed on a 4th-and-1 rush attempt near the goal line, a key play in a game that was 14-10 at the time.

Where We Go

Mariota’s stats are misleading, but that hardly means we should throw out all statistics. Savvy analysts need to understand what statistics tell us, and what they don’t. Superior metrics will punish quarterbacks for sacks and avoid excessively rewarding them for completions that don’t work towards moving the chains or garbage time yardage. It’s worth noting that in ESPN’s proprietary QBR, Mariota ranked only 24th among Week 11 QBs at 50.3. Perhaps other statistical innovations, such as moving from a per-play to a per-drive basis, have not been invented yet. There are two kinds of mistakes with statistics: ignoring them entirely, and using them without understanding limitations and context. Marcus Mariota’s line Monday is a perfect illustration of the latter.

One thought on “Marcus Mariota and Misleading Metrics

  1. Man haters will go to any depth to try and disrespect a certain player. He was excellent QB’s dont just complete 22of23 passes. If some short passes are all thats needed to throw at a 95.7% clip then why havent we seen more QB’s do it. Instead he set the 2nd highest Comp % of all time. He passed for almost 100 more yards than Watson despite Watson having one more attempt than Mariota, so how do short throws apply to him and not Watson? Answer they dont! He played great but they lost because he cant do it all on his own and his team didnt help. Mariota was getting smashed by the defense for 6 sacks and yet he was still able to put up those numbers. Did you not see the clutch long throws to Corey Davis? Including a long TD that was placed perfectly for Davis to avoid the defenders. Let’s see now… Watson had an offensive line that played way better, a rushing attack that was why they won the game and multiple talented receivers to throw to and yet he was still outplayed by Mariota. Mariota his having a great year. The only stat worth ignoring is passing TD’s its been a fluke of a season where everything that could go wrong around him has and yet hes on pace to set career highs including over 70% completion percentage for the season and tied with his best QB rating at 95.6%. One more thing about that Smith TD that was beccause of him skillfully finding the right guy. As Mathieu said it was not a busted coverage it was just a great play by the way that “garbage time” you speak of was when he threw his first and only incomplete pass, so acting like his high comp % was thanks to that garbage time is ridiculous. He began 19 for 19. Watson would have had to have 3 more completions at 32yds each just to match Mariotas 22 completions for 303 yards. Even if you take the Smith catch out he still averaged more yards per completion than watson. (which is ridiculous considering the combination of a great read and a surprise burst of speed). So yeah ummm the only thing that was misleading was the QBR which places to much emphasis on sacks considering most if not all of them were outside he control. It just goes to further boost his great day considering the mess of a pocket he did it from. I guarantee you this if Mariota was the one who got 260+ yards rushing the Titans would have won too

Leave a Reply

Your email address will not be published. Required fields are marked *