[dt_divider style=”thick” /]This millennium, the New England Patriots’ defense has consistently ranked better by points allowed than it has by yards allowed. For many of those seasons, the difference in its rankings has been notably large. Such disparities have led announcers, fans, and analysts alike to conclude that the Patriots employ a “bend but don’t break”-style defense (let’s call it BbDB for short). The logic goes: By playing a conservative defensive style and forcing an opposing offense to make relatively more plays to drive down the field, that offense has more chances to make a mistake like allow a sack, commit a turnover, make an errant throw or drop a pass, etc. This narrative has always seemed reasonable to me, but lately I’ve begun to wonder if it’s accurate. Since I suspect that asking coach Bill Belichick to reveal his defensive game-planning philosophy won’t get me anywhere, my best chance at shedding light on the subject is to look for clues in the data.
The data that I will be using are defensive drive statistics taken from Football Outsiders as compiled by Jim Armstrong. As I am particularly interested in comparing the Patriots of the Belichick / Brady-era to the rest of the league, I’m using every Patriots team-season from 2001-2016 as my test sample, and every other NFL team-season over the same period as my control sample. Note that the combined sets total 511 (not 512) team-seasons, as there were only 31 NFL teams in 2001 (the Houston Texans joined the NFL in 2002). Also, as I will later be using average defensive drive starting line-of-scrimmage (let’s call it DLOS), it should be noted that the kickoff touchback rule change for 2016 has probably affected the DLOS distribution this past season. However, this is OK since I will only be using DLOS as an independent variable, it is actually a benefit if the 2016 data expands the range of DLOS in the data. Finally, it could also be interesting to look at defensive results on a play-by-play basis, but I don’t have that data right now – perhaps a task for another day. OK, let’s jump right in.
I show in figure 1 each team-season average points-per-drive allowed, plotted against the respective average yards-per-drive allowed. The blue points represent the 495 non-New England team-seasons in the control sample, and red points the 16 New England team-seasons. The points are different sizes but don’t worry about that just yet. This figure illustrates nicely what I described above about the Patriots’ defensive rankings: They tend to give up more yards relative to points, as compared to the rest of the NFL:
Intuitively, it is easy to comprehend what I mean by a BbDB defense. However, before I start digging any further into the data, we need at least a boilerplate definition that I can statistically test. Given the motivation for this study and what we see in figure 1, I can start by saying that a BbDB defense allows fewer points-per-drive than is expected, based on league-average data, given its allowed yards-per-drive. I will expand upon this definition in a bit, but it’s a start.
The dotted line in figure 1 is the best-fit (in a least-squares sense) line to the control sample:
P = 0.079 x Y – 0.520
Where P is the season-average points allowed per defensive drive and Y is the season-average yards allowed per defensive drive. The resulting Pearson’s statistic is R = 0.830 +/- 0.014. The distribution of data-minus-fit residuals in the control is nice and Poissonian-looking (i.e., a bell curve), with a sample standard deviation of σ = 0.180. However, as I have already mentioned, the points representing the Patriots’ seasons are clearly not well represented by the control sample. Is this evidence for BbDB?
Obviously, there are several factors that affect points allowed per drive in addition to yards allowed per drive. For example, turnovers forced, red-zone efficiency, DLOS, etc., all should have some effect on points-per-drive. We now need to decide which, if any, of these factors we should exclude from our definition of BbDB, and then adjust our analysis accordingly.
The relative performance of defense in the red zone can be included as part of BbDB: A team will need to start taking more chances in the red zone if it is going to be more conservative outside of it. Regarding defensive turnovers, it is probably true that many are more properly attributed to a mistake by the opposing offense – or just plain luck – but again, forcing an offense to have more chances to make mistakes can be considered part of BbDB. Thus, I won’t try to compensate for either of these issues for now. However, DLOS would seem to be controlled primarily by the team in question’s offense and special teams. Furthermore, as a given defense plays with the (more-or-less) same offense and special teams each week, any effect from DLOS granted by the offense and special teams will be systematic, not random. As it turns out, the Patriots’ teams during the period in question have indeed been consistently excellent at giving its defense favorable average DLOS. So, let’s exclude DLOS from our definition of BbDB, and try to account for it in the data.
Actually, figure 1 already contains DLOS information: The points are sized inverse-proportionally to DLOS, in the sense that a smaller point represents a larger (i.e., less favorable for the defense) DLOS. If you are good at visualizing 3-D in a 2-D plot, you can imagine the smaller points are farther away (i.e., into the plane of the graph), and the larger points are closer. Viewing figure 1 in this way, the control sample appears to lie scattered about a plane in P–Y–DLOS space. In fact, fitting a linear plane is an excellent fit to the control-sample data. For those interested, the solution is:
P = 0.087 x Y + 0.049 x DLOS – 2.198
One way to visualize the effect of fitting a plane to the data is to “tilt” figure 1 so that we are looking at the data along the edge of the plane. Instead of showing a tilted 3-D figure, however, I’ll collapse the data onto the 2-D surface such that the third dimension (DLOS) is perpendicular to the page, and define a new horizontal axis, “predicted” (i.e., considering average yards allowed and DLOS) points allowed. This is shown in figure 2:
The figures appear very similar, but the scatter in the data about the fit is, as expected, decreased in figure 2. The dotted curve is, by construction, the line of equality, and the sample standard deviation of the residuals is now σ = 0.142, which is a 21% reduction from that of the 2-D fit. Furthermore, many more of the red, New England points now fall nicely along the plane of the control sample. Perhaps the putative BbDB-signal is in reality just an artifact of the Patriots’ consistently strong offense and special teams and the resulting favorable DLOS? Then again, there are still a number of red, Patriots points along the envelope of the control distribution. Let’s look at the same result a bit differently:
There is no new information in this third figure as I have simply subtracted the vertical axis from the horizontal, and plotted the resulting “expected – actual” points along the vertical axis. It is now easier to see that five¹ of the nine team-seasons with the most² BbDB defenses in the NFL during 2001-2016 belong to the Patriots, meaning 31% of the Patriots’ defensive seasons are in the 1.8% most BbDB of all NFL team-seasons during this period, which is remarkable. Have we now found evidence for BbDB? I think the answer is a tentative “yes”, though there are probably other factors affecting the data, which I haven’t included here, that should not properly be considered a part of BbDB. Either way, let’s look one step a bit deeper at what we do have.
I have already mentioned turnovers, which I decided to include as an inherent part of BbDB. In principle, turnovers should have a significant impact on the relative rates at which defenses allow yards and points: Most drives that end in a turnover have gained positive yards, but none of them result in defensive points against. Indeed, in each of the five New England seasons that stand out dramatically in figure 3, the Patriots were in the top seven of the NFL in defensive turnovers-per-drive that season. Thus, it could be quite interesting to see how big a part of BbDB could be attributed to turnovers. To quantify this, I simply repeated the earlier analysis, but now by fitting a hyperplane to the data in P–Y–DLOS-TO space. Here is a depiction of the result, again collapsing the hyperplane down to one dimension:
It seems that – not surprisingly – turnovers are a big part of a BbDB (or any, for that matter) defense. However, the distribution of New England seasons still skews somewhat to the lower-right. Notably, even after accounting for DLOS and turnovers in addition to yards allowed, the Patriots have not had any seasons where they allowed significantly more points than expected.
So, what should we conclude? It appears that there are a number of factors that combine to explain the consistent discrepancy between the Patriots’ rankings in points and yards allowed. Their favorable DLOS is one of these factors, and is likely due in large part to their consistently strong offenses and special teams. Importantly, DLOS is a controllable, not random, aspect of teams. Thus, while it is in some way fair to say that the quality of the Patriots’ defenses are not precisely as their points-allowed would imply, it is equally fair to say that the ability to prevent points of the Patriots’ teams is indeed much as said numbers state.
Going one step further, we found that, in seasons for which the Patriots were particularly good and / or lucky at producing turnovers, the points / yards discrepancies are particularly strong. Even after accounting for these two effects, however, there does seem to be some aspect of the Patriots’ defenses that finds a way to preferentially prevent points more so than yards, relative to that of other NFL teams. This preference could arise from, say, a focus on good tackling, or adeptness at limiting big plays. A more diverse data set could perhaps address these and other possibilities, but for now, let’s simply ascribe it to a BbDB philosophy.
¹ Note that these five outliers correspond to the 2001, 2010, 2004, 2006, and 2011 seasons, in order from largest outlier. Those defenses had different personnel and even different coordinators (Romeo Crennel, none, Crennel, Dean Pees, and none, respectively).
² I hope you noticed the crazy outlying point at approximately (1.5, -0.5). In case you are wondering, that was the 2001 Chicago Bears.
This something I’ve wondered about for a long time. Thanks for doing all that math.
I’ve returned to this article a number of times and just now shared it over on FO (the article on the AFC East). I’m fascinated by how NE’s defenses seems to always outperform their DVOA; this article is most rigorous on the subject that I’ve seen. My dream is to find a way to quantify the contribution coaching plays in team performance, although it may not be possible.