The Art of Disruption: Who Gets It and Why It Matters by Robert Binion
Guest Writer Robert Binion takes you through the Disruption Index
The college football offseason provides some much-needed time and space for those of us in the nerd corner of CFB fandom to crunch numbers. We might be interested in diving deeper into the season that was, honing models for the upcoming season, or looking over some longer term trends.
I am a life-long Georgia Tech fan and write about GT football over at From the Rumble Seat, so my offseason ideas often spring out of particular questions that have percolated from watching Tech play (poorly). For many of us wearing GT colors, it has been an undisputed axiom in recent years that the team was weak up front on both sides of the ball to an extremely damaging degree.
I set about trying to find numbers that would help test this and also to see what we could learn about the broader CFB landscape. Using the lovely tools at our disposal thanks to cbfastR and CollegeFootballData.com, I started looking at what I refer to as disruption metrics.
For a quick refresher:
Havoc rate is the percentage of plays on which a defense creates or an offense allows a tackle for loss, forced fumble, pass breakup, or interception.
Run stuff rate measures the percentage of running plays on which a defense creates or an offense allows a run of no gain or a loss.
The third disruption metric, pressure rate on called pass plays, is something that I chart each game for Georgia Tech but is not publicly available for all of CFB to my knowledge. Thankfully, a kind soul with access to the ESPN Stats and Information database was willing to share 2014-2021 pressure rates for P5 teams in aggregate.
Havoc rate, run stuff rate, and pressure rate all quantify team and player abilities that tend to jump off the screen at us on a Saturday afternoon. “We just can’t block them” means high havoc, run stuff, and pressure rates are happening. The painful experience of watching our arch rival from Athens field a devastatingly disruptive defense this year was case in point.
With our disruption metrics at hand, I wanted to try and understand the impact of these kinds of plays, as well as what goes into creating a disruptive defense and an offense that limits disruption. Historical EPA data, as well as recruiting ratings, jumped out as some significant metrics we might use in our exploration.
We went about standardizing each of the three metrics and then combining them into a Disruption Index. To evaluate overall team performance, we use EPA/play on offense and defense over the same time period. We use the 247 Composite’s average team recruiting rating as a proxy for the raw talent on a roster.
Disruption on Offense
Running a regression on our Disruption Index allowed against offensive EPA/play yielded a strong R-Squared value of 0.38. This means that allowing disruption (as characterized by the three factors in our Index) accounts for 38% of the overall variance in offensive performance in this robust 8 year period. At the top-left, you see many of the usual suspects who have dominated CFB over this time period. At the bottom-right, you have an interesting mix of the cellar-dwellers of the Power Five and teams with higher aspirations that have had significant and sometimes suboptimal coaching changes over this time period.
How does roster talent inform a team’s proclivity for allowing disruption on offense?
Here, we see definite statistical significance, but our R-Squared is 0.15, meaning about 15% of the variance in disruption allowed can be attributed to recruiting. Putting together recruiting ratings, Disruption, and EPA/play allows us to begin to draw some interesting inferences about various programs over this time period.
Three Teams that Stood Out:
1) Florida State immediately grabbed my attention, of course. They allowed the most disruption of any P5 football team over this time period, which includes a 2014 playoff appearance for the Seminoles. This is a program that has historically been able to recruit in the high level of the FBS, consistently bringing in classes ranked around the top 10 and sitting comfortably in the second tier that does not include Alabama, Ohio State, UGA, LSU, and Clemson. Over this time period, FSU experienced generationally bad Offensive Line play and fielded a number of quarterbacks who only exacerbated the situation with a lack of pocket awareness, often holding onto the ball for far too long. FSU shows us how disruption is caused by weak offensive lines, but it cannot be separated from quarterback play and offensive design.
2) On the other end of the spectrum, Clemson leads the way in not allowing any disruption. Clemson has not consistently recruited or put dominant offensive linemen in the NFL, but the Tigers have had two generational talents at QB and a system that minimized opportunities for disruption to happen. Here we see how quarterback play and coaching can vault what would be a very good team into a program capable of winning two national titles in this span.
3) Washington State shows us just how far schemes can tilt things. Over the course of most of our observed years here, WSU was running the Air Raid- throwing quickly and often. Despite below-average talent on the roster, the scheme enabled that offense to avoid disruption almost all of the time. And as you can see in the original chart, WSU came in just below the Disruption–EPA/play trend line. What do we see going on here? Using scheme to overcome talent deficiencies to produce efficient offense. I like it.
To take this analysis even further, player tracking data would likely be the necessary ingredient to parse out the contributions of the offensive line compared to the quarterback and other skill players. How long does the quarterback have to throw before pressure arrives? Does the quarterback move towards or away from potential pressure? Do running backs pick up their expected yards or needlessly run into defenders? Maybe one day we will have the opportunity to explore that treasure trove of data at the college level.
If you’d like to look at the metrics at a more granular level, here’s a team by team visual of the individual numbers that go into the index for all of the P5.
Disruption on Defense
Turning now to the defense, the impact of Disruption jumps out even more starkly.
Here, the R-Squared value is all the way up 0.63, about 70% higher than it was for offense. Disruption accounts for 63% of the variance in EPA/play allowed on defense. Above, you can see just how tightly teams line up along the trend-line. Once again, the top right contains many of the usual suspects who have dominated CFB recently. Clemson threatens to break the chart in the top right, checking in about half of a standard deviation better than any other defense in causing disruption over this period. After seeing Georgia Tech’s offense swallowed up by orange so many times recently, it is at least comforting to see that we were not alone.
How much of this is driven by talent?
Again, we see about twice as much impact on the defensive side of the ball, as the R-Squared for recruiting rating to Defensive Disruption is 0.27.
Looking together at recruiting, disruption, and EPA/play allowed, there are three teams or clusters that jump out to me on the defensive side of the ball.
Three Teams that Stood Out:
Washington, Iowa, and Northwestern form a fascinating grouping of teams that play excellent defense without getting much disruption. What might be going on here? All three of these programs have been at the very top of the list of defenses not allowing explosive plays. There is a strong emphasis on reliable, zone-heavy schemes on the backend that might minimize disruption opportunities but have led to very solid defensive play for the better part of a decade. There’s also been significant coaching continuity in all of these places; Mike Hankwitz was Northwestern’s DC for that entire period until 2021, Pete Kwiatkowski was DC for four years and on Washington’s defensive staff for all of those years but one, and Phil Parker has been Iowa’s DC for this entire period. These programs provide an interesting template for consistent defensive success without getting a lot of disruption.
Georgia Tech, Texas Tech, Arizona, and Oregon State form an underwhelming pod of teams that don’t get any disruption and greatly under-perform their talent level. These are teams who don’t have game-changing talents on defense but also employ schemes that do not maximize the personnel they do have on defense. There are blitzes that never get home, stunts that don’t confuse the offense, and coverages that consistency leak into big plays. Another interesting commonality here is that these four are generally low budget P5 teams who have employed low-budget defensive staffs. This hasn’t been an effective formula.
Finally, let’s talk about the Clemson defense. They are tops in the country in EPA/play allowed since 2014, and they’ve done so by over performing their recruiting more than any other team in the country. Clemson checks in a full two standard deviations better in the Disruption Index than their recruiting would predict. Brent Venables is taking a devastating defensive scheme to Norman, Oklahoma, and it will be fascinating to see how quickly he can jumpstart the disruption for the Sooners and to see if Wes Goodwin can keep things rolling in Death Valley.
Below, you can take a more in depth look at how every P5 defense has or has not created disruption against its opposing offenses.
Putting it all Together
Let’s put our Offensive and Defensive Disruption Indices together and highlight one team that jumps out at each end of the spectrum.
Wisconsin may shock some outsiders with just how effective they have been over this span. Outside of the no-surprise top 3, they have the best aggregated Index of any team in the country since 2014. Paul Chryst has righted the ship that Gary Andersen started to sink, and Jim Leonhard has crafted some of CFB’s most disruptive and effective defenses in the CFP era. He’s had 12 defensive players drafted in the last six years, far above what would be expected from a team who is below average in recruiting over that period.
For my Georgia Tech friends, our worst fears are confirmed: GT has the worst aggregate Disruption Index in all of the P5 over the past 8 years. Poor pass blocking throughout, quarterbacks who have tended to move into pressure, bad development of defensive players, and hilariously bad schemes on the backend come together to produce the Joker of the Disruption Index.
Winners and Losers
From the analysis we did above, Disruption on offense accounts for almost 40% of variance in EPA/play; disruption on defense accounts for almost 70% of variance in EPA/play allowed. Preventing disruption on offense and causing it on defense tells a big part of the story in college football. But only about 25% of a team’s performance in our Disruption Index seems to come from recruiting talent. The rest comes from development and deployment. As we wrap up, let’s highlight some coaches that stand out to us.
Winners:
Brent Venables: Clemson has recruited at the fifth-best level over the time period we have been looking at, but they have had far and away the most disruptive defense in the country. Coach Venables gets the absolute most out of his guys and knows exactly how to use them to maximize disruption and subsequent defensive performance.
Mike Leech: Washington State has generally been one of the toughest P5 programs to recruit to, and that trend has largely held over the past decade. But Mike Leech excels in those situations. His scheme fundamentally shifts expected outcomes; the Air Raid, in Leech’s application, simply does not allow disruption on offense, and the subsequent offensive performance is far better than what we would expect given WSU’s talent relative to the competition. So far, Leech seems to be taking things in the same direction at Mississippi State.
Paul Chryst and Jim Leonhard: Wisconsin is the best unsung program in the country. They recruit at a generally below-average level for the P5 and produce at a tier below the elite of CFB.
Losers:
Everyone involved in FSU coaching decisions from 2015-2020: It is stunning to see how bad FSU’s Disruption performance has been relative to its talent level, even if recruiting has slipped from the previous highs. The inability to develop offensive lineman, coach up a quarterback who could tolerate sub-optimal line play, or scheme away from the weakness is unforgivable for a program that should have no trouble living in the highest levels of CFB if everything is aligned correctly. 2021 did give some glimmers that Norvell is moving things in the right direction.
Geoff Collins and company: I did not set out to write a Georgia Tech centric piece here, but GT’s place at the absolute bottom of the P5 in our aggregate Disruption metrics can’t be overlooked. Of course, there are numerical impacts that overflow from the previous era of GT football given the time-span we are looking at, but GT’s numbers have plummeted over the past three seasons. Georgia Tech is just above the bottom 20 of the P5 in average recruiting talent but has underperformed, particularly on defense, as much or more than any team in the country. GT used to have an offensive scheme that did something similar to Leech’s in avoiding disruption, but that advantage is gone now too.
If you’re not Ohio State, Alabama, or Georgia, what can your team take from all this?
On offense, if you’re not in the top cluster of talented teams, you need an outside the box scheme to overcome personnel limitations, and you need to empower and trust a coach who has the capability of deploying in that direction.
On defense, you need to scheme your way to preventing explosives if you don’t have the kind of talent up front that can wreck another team’s offense. It may take a bit of time to build this into the program, so it might take giving a defensive coordinator a few years to deploy more limited talent in this way.
Disruption matters a lot in today’s CFB landscape, and you don’t have to be a blue blood to take advantage of this insight.
If you want to dive in to the data like I do, check out @CFB_Data and @cfbfastR on Twitter, where you can learn how to get started in the world of College Football data analysis!
If you want to see more amazing deep dives like this one, follow @robert_binion on twitter!
Thumbnail Photo: AP - Hakim Wright Sr.