Evaluating QB Metrics in CFB

Which stats should I use to talk about QB's?

Whether we want to admit it or not, most of football conversation is QB driven. If you want to test it out name a random NFL or college football team and see what is the first thing that comes to mind. If I ask how you think the Indianapolis Colts will do this season, I would bet Carson Wentz would be one of the first things on your mind. The Clemson Tigers? You’d probably think of Trevor Lawrence and then wonder how DJ Uiagalelei is going to follow a number 1 overall pick. QB’s have become the center of conversation in both the pro and collegiate ranks because of how important they are to a teams success. PFF created a CFB “Wins Above Average” Metric and based on positional value this is how important the QB position has become:

”Everyone understands the value of quality quarterback play, especially in the NFL. Comparing college to the NFL, though, quarterbacks are actually slightly more valuable compared to other positions, with their value over twice as much as the next closest position in college.” -Ben Brown

With this in mind, lets dive into some of the popular QB metrics, and see which ones we should be using when we evaluate QB play.

Stop using TD-INT Ratio

TD-INT Ratio is a very common stat used to evaluate a QB’s play, and if you’ve ever spent time around the analytics community, you’d see it gets dismissed rather quickly. The biggest reason why is it misses….. a LOT of context. Take the graphic on Sam Howells 2020 season. Howell finished the 2020 season with 30 TD’s and 7 INTs. That is 37 passes out of a total of 348 attempts, which equates to 10.6% of his passes. That doesn’t even include his rush attempts or his rushing TDs. What about all of those black dots? What happened on those plays were they good or bad plays? Limiting yourself to a tenth of a QB’s total plays can lead you to some bad inferences.

Ole Miss QB Matt Corral infamously had a 6 INT game against Arkansas, and a 5 INT game against LSU. One would assume from his 29-14 TD-INT Ratio that he had a pretty lackluster season. But, when you look at per play metrics like Expected Points Added (EPA) per play, he actually had a rather efficient season, finishing in the top half of QB’s in that metric.

Evaluating a QB just on their passing TD’s and INT’s also ignores the understanding of where TD’s and INT’s come from. Lets start with touchdowns:

Plotting out every passing TD in the CFP era (2014-2020), you can see that TD’s are primarily scored closer to the goal line (I know, next level analysis). If a team has the propensity to run the ball in the redzone, that QB is missing out on a big function of passing TD’s. If we only evaluated QB’s on their TD-INT ratio, these QB’s would be missing out on some potential TD’s. With what we know about the small sample size of this ratio, missing out passing touchdowns could alter your ratio immensely.

Lets turn to interceptions. They appear to be a bit more distributed than TD’s were, but you can see still that when you’re losing and have very little chance to turn it around, interception become more probable. Why should QB’s be punished heavily for trying to bring their team back from the brink of failure? There was actually an interesting study on QB’s (Aaron Rodgers) being too cautious when they’re down, and how they can affect a teams ability to comeback and win the game. I will link that piece here if you’re interested.

Ok…. so what do we use?

We have thrown TD-INT ratio out of the window. So what do we use? We have already established that we need to look at their entire body of work (per play metrics). Lets look at how some metrics translate year to year. Not to make this whole article ratio bashing, but one of these metrics sticks out like a sore thumb. It should also be noted that another popular stat to use, passing yards per game, is also at the bottom of the list. Even with its positioning on the list YPG has not nearly as weak a relationship year over year as TD-INT ratio. The champion of QB metrics belongs to ESPN’s QBR, which is essentially a per play efficiency metric with opponent adjustments and other variables mixed in. PFF’s CPOE is one of their newest creations by Tej Seth (@mfbanalytics on twitter), which you can read more about here. Basically the model looks at depth of pass and game context to determine the odds a pass is completed. Overall you can see most metrics fall under a moderate relationship year over year, which makes sense given the unpredictable nature of football.

Garbage Time

Last thing I want to touch on is garbage time. I can’t speak for QBR or CPOE, but in terms of EPA filtering out for garbage time appears to be a non issue for QB play on the college level. When you filter out plays by the win probability before the snap, the relationship grows weaker as you filter. This appears to be due to the same issue that plaques the ratio, sample sizes issues (albeit not nearly as extreme as TD-INT ratio). This also lines up with the idea that coaches pull their starters when games are out of hand. Another way to possibly filter out “garbage time” would be filtering out games against FCS opponents, which would be the next step in this analysis.

QB play will continue to dominate conversations as QB’s continue to put up numbers that were previously considered “video game numbers”. Using per play metrics like QBR, CPOE and EPA/Play will give you better ideas on the overall ability of the QB, as well as help you in predicting future performance. Its a new passing revolutionary era for football, so lets leave the old relic metrics in the past and start fresh with analytics!

If you want to dive in to the data like I do, check out @CFB_Data and @cfbfastR on Twitter, where you can learn how to get started in the world of College Football data analysis!

If you want to see more charts and one off analysis, follow my twitter page, @CFBNumbers