I saw someone's Top 10 QBs for 2025 list and wanted to compare it to xEPA/Play. Played around with a weighted average of the past 3 seasons (with most recent weighted more heavily) to construct a Top 10 (Jackson-Allen, Mahomes-Hurts, Purdy-Burrow-Tagovailoa, Prescott-Murray-Love, depending on how you weight the average and whether you take into account whole career instead of just the past 3 seasons).
Then I decided to expand it out, and found a PFF article ranking all 32 projected starting QBs posted this week. Ranking just based off Career xEPA/Play (without enforcing a total play cutoff), here were some of the biggest differences in rankings between PFF and xEPA/Play:
1. 17 positions away--Matthew Stafford (PFF rank: 7; Career xEPA/Play rank: 24). I get it a bit more now why you bring up Matthew Stafford as an example so often. He's still being ranked as a top 10 QB by PFF?? I also note that this could be an even bigger difference depending on where we slot the 4 QBs on PFF's list that do not have a qualifying xEPA/Play season. I've just chosen to leave them off, but if we put them in about where PFF has them in the rankings (not a wise move, but one that plays by their rules), McCarthy, Penix and perhaps even Ward would slot above Stafford. It's not even as if Stafford has seen a clear shift in how well he's been playing recently compared to his career norms. I could understand keeping him higher than 24, but top 10 seems ridiculous to me.
2. 15 positions away--Bo Nix (PFF rank: 19; Career xEPA/Play rank: 4) Sure, with only one (pretty good) season in his career, xEPA/Play is probably overrating him for now. I don't really put a lot of stock in that 4th place position, so discounting him somewhat is understandable. What's less understandable is the difference in treatment between Nix and Daniels under PFF's rankings. PFF puts Daniels at 6th, while Nix is down at 19th. There's a results-bias here (as Daniels had 0.202 EPA/play vs Nix's 0.090 EPA/play), but in terms of skill as measured by xEPA/Play, they are neck and neck (0.162 for Daniels vs 0.164 for Nix). Perhaps PFF grading is more results-oriented than I had thought--but they do acknowledge that after his first four starts, Nix was the 6th-best graded quarterback in the league. I guess I don't understand how you can acknowledge that level of sustained performance and still keep Nix this far down on your rankings.
3. 13 positions away--Russel Wilson (PFF rank: 22; Career xEPA/Play rank: 9). This one I think is more understandable, as Wilson is probably being propped up in career rank by all his good Seattle seasons and he's probably on a decline in his career. I think probably PFF is underrating him a bit, and career xEPA/Play is probably overrating him a bit in this ranking.
4. 13 positions away--Mason Rudolph (PFF rank: 31; Career xEPA/Play rank: 18). Based on their blurb, the PFF ranking seems more about the perception of Rudolph as a backup quarterback being ranked amongst starters than about his production or skill. I think that does a disservice to Rudolph, though, whose career xEPA/Play is actually higher than Trevor Lawrence's (perhaps partially due to only a couple of Rudolph's seasons qualifying--although their traditional rate statistics are remarkably similar, too, so it's not just that Rudolph has played poorly in seasons that he doesn't qualify for xEPA/Play). Listing Lawrence at 16 and Rudolph at 31 seems too strong a penalty for being a backup. When you factor in the number 1 overall pick (and generational one at that) vs 3rd round pick, though, it starts to make sense in the general perception of these players and speaks to the point you’ve made recently about draft position mattering.
5. 9 positions away—Baker Mayfield and CJ Stroud (PFF ranks: 13, 14; Career xEPA/Play ranks: 22, 23). Grouped together because they are next to each other on both lists
6. 9 positions away—Joe Burrow (PFF rank: 2; Career xEPA/Play rank: 11). Burrows’s ranking in terms of career xEPA/Play is lowered a bit by the inclusion of Nix and Daniels since we’ve removed the total plays threshold. That said, I do think he tends to be a bit overrated in NFL discourse, which generally puts him as a top 5 QB, where I think he belongs more in the next 5 Qbs.
7. 9 positions away—Joe Flacco (PFF rank: 29; Career xEPA/Play rank: 20). He’s buoyed a bit by his best seasons in Baltimore in terms of Career xEPA/Play, but the major discrepancy between the lists is the inclusion of a bunch of young passers (McCarthy, Penix, Ward, Stroud, Young, Williams) who either haven’t played before at the NFL level or whose skill doesn’t remarkably surpass what Flacco has demonstrated even just the past couple of seasons. He’s on the downhill of the aging curve while they are just coming into their own, but it wouldn’t surprise me if these rookies end up playing worse than Flacco this season.
David, you're a legend my friend. This analysis is fantastic. I've always had a bit of a gripe with PFF, and the reason for that is this EXACT thing, except in reverse. My gripe gestated from 2018 Mitchell Trubisky, a season where Mitch finished sixth in xEPA/Play, eighth in actual EPA/Play, but an ungodly 31st in PFF grade. At that moment I knew there had to be something going on over there. At that moment, I thought PFF grades had to be SO results-agnostic that they struggled to provide any value to me. However, with the analysis you've done here, perhaps I may have second thoughts.
I hate to be the one man anti-Matthew Stafford brigade, because I don't like just being so negative all the time, but the man has had a CPOE above zero and a sk%+ better than 100 in the same season just three times. His career high for xEPA/Play in a season is 0.118. It's just not all that, and I'm not sure what PFF or anybody else sees in this guy. I'm honestly not sure if I'm the hot take here or they are, because everybody had this right about Stafford until the point where he won a championship, at which point everybody went crazy overrating this guy.
If we do enforce a play minimum of some sort, I would hazard a guess that Stafford ends up about 16th amongst all active guys, which is roughly where he keeps finishing in the EPA/Play rankings, which means it's not even that Stafford's results cause people to overrate him. He's not been a top ten EPA/Play guy since 2021. He's not been a top five EPA/Play guy since 2019. That's a lot of years ago, so Matthew Stafford being all the way up in seventh cannot be credibly attributed to either results or skill. I'm not sure what it is about him. I've never been sure what it is about him, but there's something there that causes people to just go out of their minds. First overall draft status maybe?
It's interesting to me that Bo Nix being 11th in xEPA/Play in 2024 translates to him having the fourth highest career xEPA/play amongst active players. It's a testament to how many of them struggled as rookies I guess. This will even out as his career goes on I suspect, but if we use his 2024 rank of 11th instead of his actual xEPA/Play number, the discrepancy to PFF rank gets a lot smaller, although not small enough for my tastes. Is it that even the people over at PFF are like the general public, not valuing sack avoidance enough? In the absence of further evidence, this would be my guess, because a 19th place ranking fits Bo's 24th place CPOE ranking and eighth place rushing value ranking last year pretty well. It also must be acknowledged that Bo underperformed his xEPA/Play by more than Jayden overperformed his, meaning there should be some regression to the mean upwards for Bo as we move along in his career, and perhaps Denver gets him some better weapons.
The Russell Wilson thing is just recency bias, or the lack thereof. It's attributing his best seasons to the modern player, which of course is not correct. He finished 34th in xEPA/Play in 2022, 13th in 2022, and 18th last year, so if you throw a dart in this general direction, it does not offend me when it lands in 22nd place.
What does offend me is ranking Mason Rudolph 31st. What in the world is with that? In his 279 plays last year, he finished with an xEPA/Play of 0.124, and if we include his 2023 to make it a slightly more complete sample, his xEPA/Play in this time goes up to 0.175. That's better than Matthew Stafford has ever done in a qualifying season. There's that name again LOL. 0.124 is not better than every Trevor Lawrence season, but it's better than any recent one. I love that the Steelers are standing pat here. They've got the right guy. It comes down to my discussion about optics the other day. In terms of optics, having Mason Rudolph as the QB sucks, but in terms of real football, there's no way he can be 31st. I'd put Mason in roughly Justin Herbert tier. Until further notice, let's call him league average, with upside. That's very far from 31st.
You know what's interesting is also that Baker and CJ are right next to each other on the 2023 xEPA/Play leaderboard. 20th and 21st, respectively. However, in 2024, they diverged massively, with Baker going up to ninth place and CJ plummeting down to 37th. As such, Baker in 13th is not offensive to me, but CJ Stroud in 14th place. What the heck is that? That one placement to me would be enough to invalidate the whole list.
At best, that's a projection that CJ is going to get dramatically better this year. At worst, we've just caught PFF believing a career INT%+ of 115 on a negative career CPOE is sustainable. I've got news for them if they think that's the case. PFF of all bodies should know better than that. If you can put CJ Stroud 14th place in any ranking and try to claim said ranking has anything to do with real football on the field, you and I have a serious disagreement about the way this position is played.
CJ did finish 13th in EPA/Play in 2023, on a success rate ranking of 17th and an xEPA/Play ranking of 21st. Is this enough to keep repetitively throwing this guy parades like everybody on the internet seems wont to do? I'm not interested. If my alternative is this guy, give me Mason Rudolph every day of the week.
The Joe Burrow ranking seems to second my opinion I've put under Bo Nix. Sack avoidance is just not baked into the PFF analysis in a position of importance as high as I think it should be. Much like Bo Nix, if we were just grading arm stuff, Joe's position would make a lot of sense, although we do need to acknowledge that he did finish third place in xEPA/Play last year, and the second place guy just retired, so maybe second place Burrow isn't so wild after all, but to maintain this ranking with his inability to avoid sacks at an elite level, he basically has to maintain a CPOE of six or higher. I'm not saying he can't do this, but I am saying his CPOE in 2023 was just 1.2. In 2022 it was 2.7, so to bet on him to continue at six or higher seems like a rather long shot to me.
Thank you so much for this David. You've given me a lot to chew on here. I had limited respect for PFF's QB grades anyways, but in particular CJ Stroud at 14th just completely blew my head off my shoulders. Does PFF not market themselves as being immune to the kind of thing that would cause people to put CJ Stroud at 14th place on any ranking list? Apparently they absolutely are not. His draft position is clearly carrying the day in this ranking. I truly can't think of anything else that would cause them (or anybody) to like him so much.
I've been looking at this data on and off for the past week and every time I do I find something fascinating (to me anyway). Really excellent work.
Take Colin Kaepernick for example. He goes from having the 100th best season by xEPA/play at the beginning of his career to having the ~150th worst four years later. Every year is worse than the one before (except for a slight rebound from horrific to just bad in his last year). This is not a normal career trajectory. Most quarterbacks either get somewhat better or stay constant. (Unless they play long enough to fall off the aging cliff.) Looking at this it's obvious why no team thought he was worth the drama hiring him would bring.
These things do happen sometimes, but most of the time they're for injury reasons, and almost always to a lesser extent.
For instance, Roger Staubach is a light example of the same phenomenon, if we ignore 1977. He comes out the gate firing in 1971, but is never really the same again after the separated shoulder, except for one turn back the clock year in the aforementioned 1977. The most obvious comparison to Colin Kaepernick actually happens in his exact season, which is Robert Griffin III, who also puts up a stormer of an xEPA season in 2012, but monotonically gets worse for the entirety of the rest of his career.
Those two are basically two peas in a pod on that, but where RG3 had obvious knee issues that explain away some of his decline, there really is nothing like that for Colin Kaepernick. Is his sudden decline because he broke the ball of his foot in the second week of 2013? Given the nature of the decline that happens immediately afterwards, I would be surprised if they weren't related, but that just seems to me like an injury that a player can recover from.
As far as comparisons go, that's the end however. There is virtually nobody to compare to these two, and what makes each of these two unique is that they were never able to get it back. Not even for one year. Roger Staubach was never the same after 1971, but got it back for a year in 1977. Randall Cunningham never could come back from the torn ACL, except for 1998, when he did just that. Brian Griese was never the same after 2000, but got it back for a year in 2004. Chad Pennington was never the same after 2002, but got it back for one year in 2008, etc.. That is the common story. Most of the time there is one singular turn back the clock season. Sometimes two if you're lucky (i.e. Carson Palmer).
What makes Colin and Robert unique is that it never came back. They were never able to recapture it. These two pretty much stand alone as the only two in NFL history with the ability to have a season as good as each of them had in 2012, but not the ability to do it twice. There are plenty of people without the ability to do it three times (Roger Staubach, Chad Pennington, etc.), but pretty much everybody can at least do it twice, given the fact that they can do it once.
Terrific piece on a favorite topic of mine - skill versus circumstance. A quick question.
In the 1970s, Bill James proposed that a skill (at the time his Range Factor calculation) could be discerned by statistical attributes that remained relatively constant following a change in circumstances – trades. James saw this stability in Range Factor.
For example, Ozzie Smith had the same outstanding Range Factor in St. Louis as he did in San Diego. There were numerous other examples. James's point was that performance numbers that remain stable following trades are reflective of skill and not circumstance. Some variance is to be expected but is easily explainable. Things such as experience, injuries and aging will cause smaller expected variations.
The coup de grace came with his analysis of the rare swapping of identical position players – the third basemen for Texas and Cleveland. Buddy Bell, an excellent defensive player, was swapped for Toby Harrah, a defensive liability. Here was the perfect test. These two players swapped virtually every circumstance - pitching staffs, stadiums, short stops next to them, managers etc.
So, what happened? Nothing. Bell was again at the top of the Ranger Factor numbers while Harrah was near the bottom. Same as it ever was. Range Factor was a skill James persuasively argued because its statistical measurements traveled with the player to new environments and remained stable in the new environment. Today, Ranger Factor and other defensive metrics are taken as reflective of actual skill and not circumstance.
I might have missed it, but I am curious as to whether your chosen attributes of skill follow quarterbacks to new teams and remain relatively stable despite new circumstances? That would perhaps be an additional supportive argument here.
I wish I had a better answer, but it's yes and no Grant.
Things for NFL QBs that remain the very most stable when they change teams are things like time to throw, and depth of target. These are things that are mostly baked into the bread, and rarely change, but they also have exceedingly minimal correlation with results. Quite frankly, if you're a QB who can avoid sacks, and get the ball into your receivers' hands, it doesn't matter whether you throw quick (Tom Brady), throw slow (Patrick Mahomes), throw short (Joe Montana), or throw long (Joe Namath). You can succeed doing all these things, so in that way, they're not really skills. More like the football equivalent of batting stances. Since none systematically works better than any other, I don't care what you do. Just do what you do, and do it well.
The issue with this entire premise Grant is trying to find enough examples of prime QBs with mid-career team changes. For instance, Tom Brady changed teams because he was finished in New England. He was following the typical aging curve out of the league, which is why his 2019 was so bad, and then he went to TB, and all of a sudden wasn't aging anymore. I'm not a conspiracy theorist, but I need Tom's hookup for some HGH.
Using CPOE as the example, and excluding 2019, Tom's CPOE figures in 2016, 2017, 2018 in New England are 3.7, 3.7, one flat, respectively. Then we go to the Tampa years, where he's 3.7, 1.7, 0.7, so yes. I would say it's pretty constant. Peyton Manning in Denver is the same thing. His CPOE range in Indianapolis is mostly between four flat and 6.5. He goes to Denver and does 7.4, 6.4, 3.3, before the bottom falls out in 2015 for reasons that were not team change.
These are just examples, but it is mathematically proven as well that sack rate and CPOE are the two most consistent stats when QBs change teams. Results vary wildly, but these numbers do not vary as wildly. The problem is, this sport is still football. Players only control so much of their own statistics at the best of times. It's not like baseball, where players are largely by themselves out there.
In football, at least at this position, there are certain play styles, which some people are better at and some are worse at. My favourite example is John Elway. If I could trouble you to go over to my CPOE page, you can type his name in, and I bet you'd be able to tell when the Broncos got rid of Dan Reeves. The offence fundamentally changed, and the player became a fundamentally better player, no team change required, and the stats did not hold stable, because John was better at playing one way than the other.
I can name examples all day, but the purpose of what I'm saying is to drive home the point that these stats are not necessarily stable, but they are the most stable, because they're more stable than any potential alternatives. No stats are more stable through a team or offensive scheme change than CPOE and sk%+, but they are not hugely stable, because no statistic in football is hugely stable, because quite frankly, in my opinion, no level of skill is hugely stable.
I hope what I'm saying is making sense. The goal here is not to create a stable metric. It's to create a more stable metric, that remains quite unstable, because football is an unstable game. It uses the most stable metrics in existence, but the most stable metrics in existence are still quite unstable. Like I said in the article, the QB controls about 45 percent of his sack rate, and 33 percent of his CPOE. This is not a lot, but it's more individual control than the QB has over anything else.
Overall, the answer to your question is yes. The metrics I have built xEPA with are the very most stable ones available. I'm just tempering that yes with the crucial knowledge that they are still not very stable.
Taking it back to the "worst Super Bowl QB" discussion we had on your Rex Grossman post a few weeks ago:
Worst season by xEPA/Play for a QB who made the Super Bowl that year? Jim Plunkett in 1980 at -0.082 (joined that season in the Super Bowl by Ron Jaworski--the two lowest Career xEPA/Play QBs with over 4000 plays). However, he was not slated as the starter at the beginning of the season, and had 3 TDs in the Super Bowl, so he probably gets a pass.
The second worst season by xEPA/Play to end in a Super Bowl is 1987 Jay Schroeder--but he was replaced for the playoffs by Doug Williams, so he's not really a Super Bowl QB.
We turn next to 2007 Eli Manning as the new worst season by xEPA/Play to end in a Super Bowl at -0.055. He won the game, though, against the team led by the 18th best season ever in terms of xEPA/Play in 2007 Tom Brady. From narrative standpoint, the fact that he won the Super Bowl against the Patriots Dynasty in what otherwise would have been their perfect season is too powerful to have "worst Super Bowl QB" stick.
That brings us to 1977 Craig Morton for the Denver Broncos. In 1977, he won Comeback Player of the Year and tied for second in MVP voting--but he also put up -0.045 xEPA/Play. A sk%+ of 66 and completing passes basically at expectation makes for not a season demonstrating a lot of skill. Even his rushing numbers are nothing special. I'd have to do an investigation into why this season was so highly esteemed at the time.
What clinches it for me, though, is that Morton also has the fifth-worst xEPA/Play by a team's primary quarterback that ended in the Super Bowl (i.e. excluding the Plunkett and Schroeder seasons), for the 1970 Dallas Cowboys. He just barely gets into the negative range here at -0.000, but that means he is the only quarterback to go to multiple Super Bowls with negative xEPA/Play in their respective seasons. He also doesn't have any great seasons in terms of xEPA/Play in his career, lost both of the Super Bowls he played, and played pretty poorly in those games (4 interceptions in '77 resulting in him getting pulled from the game, 3 in '70). That combination of factors leads me to think the title of "Worst Super Bowl QB" is Craig Morton's to lose.
I really like your analysis here. I did not even think to check for this, but once again, I suppose you can always tell who did the analysis by looking at the results. I cannot believe that it turned out to be 1980 Jim Plunkett after all.
What an odd season that was. He is absolutely horrendous in the first two playoff games, and gets his team through the second round despite his immensely negative contribution in what is likely the worst combined QB performance in playoff history, Oakland vs Cleveland in 1980, but then becomes full blown playoff Kurt Warner for the final two games, which are the only two great playoff games of his entire life. I suppose he picked a great time to have them.
2007 Eli Manning will always get my goat. There remain members of the public who insist that he was a great (or even good) player, despite all evidence to the contrary, and I can't find a way for that not to get in my head, when there are legitimately elite Super Bowl seasons, such as the obvious 1976 Ken Stabler, the all-time xEPA/Play leader, that get nowhere near the attention they deserve.
I also have never been able to understand the fuss about 1977 Craig Morton. As far as results go, he was only sixth in ANY/A, and on much fewer total touches than everybody around him on that leaderboard. On my results-based 1977 tier list, he only comes in seventh, which is fine but should not have gotten him a second place MVP finish. The Denver passing offence wasn't even good. In terms of simple NY/A, they finished 15th in a 26 team league. Craig was extraordinarily good at avoiding turnovers in 1977, which is what makes his results look so good, but this is a 123 INT%+ season out of a player with a career rate of 99, so this is a clear outlier, which I don't feel the need to give him any real credit for.
That is what happened here. The story is clear as day, an absolutely ridiculous stretch of turnover luck, the likes of which would never happen to Craig Morton either beforehand or afterwards, but as for why this season was so overrated in its own time, I'm not actually sure. One would have to think that these are narrative MVP votes, which even the smart people in Craig's own time knew he did not deserve, but I'm only guessing.
Do take note though that Craig played that Super Bowl basically unable to move his left leg, and the sham that Super Bowl turned into is what caused the NFL to instate the ruthlessly strict injury reporting policy the league has today, so the betting public can at least be aware that something like this may happen, before it actually does. That doesn't change the fact that he was a poor QB on an okay offence in 1977, which does make him one of the worst QBs ever to make the Super Bowl, but I am willing to excuse his performance in the actual Super Bowl game just a little bit.
Even despite this, I do still concur with your choice that 1977 Craig Morton is the answer we've been looking for, if we operate with the restriction that QBs who won the game are exempt.
I was a little surprised how low Grossman ended up on the list (only two spots behind Morton in ‘77, with Ben Roethelisberger in ‘08 in between) after all the work you did to say he was underrated. Bad Rex was bad, and this leaderboard doesn’t exclude the all-time-terrible performances. But it does reinforce that they weren’t entirely just terrible results despite underlying good play.
I will also note that David Woodley (who you proffered as the competition when thinking in terms of overall career) doesn’t have enough touches in ‘82 to qualify, but with this raw completion and sack numbers being roughly in line with ‘80-‘81, I don’t think his xEPA/play in ‘82 would be in the negatives. I didn’t record the complete list, but from what I recall, he’d probably slot in a couple spots above ‘00 Dilfer. But Woodley wasn’t really in the discussion because of his performance in that specific season, anyway.
Rex on the whole in 2006 was bad. Nobody doubts that, but to remove three games and boost his CPOE all the way to barely below zero indicates that his performance was seriously biased downwards by those few bad games, even in this area.
The important thing to understand is that without the Bad Rex games, he generated around 0.16 EPA/Play, on a CPOE of -0.6 and a sk%+ of 114, if I recall correctly. However, the xEPA/Play for a season like this would be 0.096. This is a long way short of what he was actually doing, meaning he was seriously overperforming in the games that were not Bad Rex.
The point of that whole article was that I can see 2006 Rex Grossman as a 0.096 EPA/Play guy, if you ignore the bad stuff. This would still be one of the worst results-based QB seasons ever to make the Super Bowl, because it's just hard to get that far. We're not doubting that Rex is one of the worst. I was merely protesting the fact that he was the worst.
In real life, Bad Rex was tacked onto a seriously inflated statline, that was always going to come down anyways, but it's difficult to envision it coming crashing down as hard as it did. In an exercise like this, Bad Rex was tacked onto a much worse theoretical player than Rex Grossman was actually playing. However, it also hurts him less than it did in real life, as even with a -15 CPOE in these three games, you cannot forecast 13 turnovers in a three game sample, no matter how bad a player is.
For a stretch to be this bad, it took stupendously bad turnover luck, which a model like this accounts for. Therefore, with Bad Rex being so bad it was unable to be repeated, and with Good Rex also being so good that it was impossible for this particular player to repeat, it met in the middle at his xEPA/Play of -0.016 in 2006, which is shockingly close to his real result of -0.019.
In short, I think he was a fluke both ways. Was Good Rex as good as Rex was playing? No, but can anybody replicate 13 turnovers in three games, in the post-2004 easy passing era? Not a chance. No matter how bad they are. What I was trying to get out into the public was that the Rex Grossman that generated 0.096 xEPA/Play in the 13 regular 2006 games was a real person, and should not be overshadowed by just how bad Bad Rex was.
You're actually right about David Woodley. His real results were really bad, but this exercise has made clear that David was not quite as bad as met the eye. With a better offensive environment than the Don Shula Dolphins, his results probably would've been better. I would say his 1982 probably ends up basically in the same spot as his 1980 in xEPA/Play terms, around 0.02 somewhere. That's still below league average, but at least it's not negative. He's probably better than people like 2000 Trent Dilfer.
I have a lot of thoughts, so I'm going to try and present them in an organized manner
Future Applications:
I think it would be interesting to see what use xEPA would have in evaluating QB prospects. Because of college football's messed-up way of measuring sacks and rush yards, it wouldn't have use beyond the last 20 years.
Using it for individual games would be useful as well.
Play notes:
1. I did not expect Josh to have two of the top xEPA seasons of all time. Obviously, there is a bias toward high-touch modern players, but that is still impressive.
2. Kirk Cousins looking like not only a top 5 QB but arguably the best QB in back-to-back seasons is insane. This would be the hottest take of all time, as I'm convinced no one has ever thought this.
3. I'm surprised that Dak ended up being pretty clearly the worst of the great Cowboy QBs on a career basis.
General thoughts:
I think by showing how there are multiple ways to be accurate and avoid sacks that you have mitigated some hesitancy to the metric. As people, myself included, sometimes have a hesitancy to accept one number or one metric be the definitive metric. There is a sort of joy that comes from the variety and mystery of evaluating players, and the debates that follow from it. By showing there are multiple ways to be efficient and not making definitive claims about how to rank QBs all-time, you have kept some of that mystery alive.
The problem with using this statistic for college players is that there is no generally accepted model for college CPOE. The conferences are still separate enough that they all weakly have their own style of play, plus it likely requires some opponent adjusting, because a lot of college QBs can complete 85 percent of their passes against the FCS, or whatever very bad opponent these teams schedule for themselves. It has been done, but I don't believe it's publicly been done, so we'd have to wait on that.
I do think it could be useful, if we do ever get to point where we can use it. It's long been accepted that college CPOE is the best way to draft QBs, so I would not be shocked at all if xEPA could do it better, but the issue of college CPOE being mostly proprietary throws a monkey wrench in the middle of it all.
Using it for single games can be done though. I don't plan to do it in the imminent future, but it's absolutely a direction for future analysis.
It's true that total EPA is biased towards people with high touch counts, but it's not that biased. You still have to maintain an absurd level of skill over that massive amount of touches to get to the sharp end of the list. Dan Majkowski touched the ball 750 times in 1989, and I don't see him up there anywhere. We know how much easier it is to play QB when the offence is deleveraged from an individual, so any of the guys at the top that managed this level of skill over 700+ plays, they get my respect.
For the Kirk Cousins thing, you must also take into account the league environment he was in. You and I have coined the term 'weak league era' for a reason. QB play was extremely weak at this time. No disrespect to Ben Roethlisberger, but when the league's leader in xEPA/Play is Ben Roethlisberger (as it was in 2015), that's a weak league. 2015 is a down season for Peyton Manning, a down season for Tom Brady, a down season for Aaron Rodgers, a down season for Matt Ryan, a down season for Drew Brees, and a down season for Philip Rivers, all at once. Taking this all into account, is it really any surprise that the league's leader in Total xEPA is Kirk Cousins? Who else is left?
Basically, Ben Roethlisberger, Carson Palmer, and Russell Wilson are left, and I don't think it's particularly controversial to say that Kirk is better (not greater, but better) than all these guys. Russ may draw some argument, but I don't think Ben or Palmer should, so yeah. I would say that Kirk has an argument for being the league's best QB in 2015.
2016 is a fundamentally different story. Kirk leads the league in xEPA/Play himself, and it's not even close once you discard a few great Bryan Hoyer starts. I cannot immediately ascertain why his real results lagged so far behind. He had two top 20 receivers that year, and an offensive coach that's generally pretty well respected. I suspect turnover luck had something to do with it (five flat CPOE becoming an INT%+ of just 107 is unlucky), as he did finish second in success rate that year, behind only the Kyle Shanahan offence in Atlanta.
To put it bluntly, Kirk Cousins in Washington was a bona fide top tier QB in the NFL. People don't remember it that way, because he went to Minnesota and never got it back, but it was in there in the beginning. Kirk is sort of like a slightly worse version of Chad Pennington in my opinion. If everything had gone right, he could've been an all-time great, but after he left Washington, very little went right for Kirk.
I find the phrasing you used to describe Dak Prescott interesting. The worst of the Cowboy greats. It's not anything you've done wrong Marc. It's just a commentary on the way people think of the Cowboys in specific. Their QBs are generally so overanalysed and frequently scrutinized that they often end up underrated. Look at Tony Romo.
He played just seven full seasons, and in those seven full seasons, he was in the top five in xEPA/Play four times, and in the top two three times. If he could've just been on the field a little bit more in his career (particularly, if he could've got to the Kurt Warner nine season mark), Tony Romo is a Hall of Fame talent. No question about that. He's only as underrated as he is because he played for the Cowboys.
People hate the Cowboys, and so will never admit this, but the Cowboys are a team with an absolutely fantastic QB history. Probably one of the best in the league. I would put them fourth, behind only GB, SD, and SF. That's how good they are at finding QBs, so to say that Dak Prescott is the very worst of a great troop is not a particularly cutting insult, which is good, because I don't think you meant it as one. If I can ask you, who did you think would be worse than Dak Prescott?
It was never going to be Tony Romo, and Roger Staubach actually surprised me by coming in as low as 19th on the career list. I thought he would be better than that. This leaves the only option as Troy Aikman, but Troy from 1991-1996 was so consistently ruthless. Dak Prescott's yo-yo act really can't match that. He is slightly less all over the place in terms of skill than he is in actual results, but he still has oscillated wildly up and down over the course of his career, which of course means he will be fantastic again in 2025 LOL.
There are multiple ways to skin the cat of playing the QB position Marc. There always have been, and always will be. The most successful archetype throughout history has generally been combining the Peyton Manning style of sack avoidance with mushy middle throw selection, but more recently, the Patrick Mahomes style of throw selection has been picking up steam, so perhaps things will change as we move through history, but the great thing about it is that I've built this metric to be immune to that change.
I simply do not care how players do things. If they avoid the sacks, that's good enough for me. To me, that means they are good at sack avoidance, and there need not be anything more to it. Same thing with throwing accuracy. If on the throws they attempt, the QB makes more of the throws than the league average QB would have, they are good at making the throws. Perhaps for some guys this hinders them from even attempting some of the more difficult ones, but I don't care, because not attempting a throw you can't make is a skill. One that several QBs don't have. It should be accounted for.
The QB position is a mystery. Any time there are multiple ways to succeed at something, there will be mystery. I did not make any effort to maintain any mystery, but being true to the game entails not grading the method. If someday an Anthony Richardson combination of Lamar Jackson sack avoidance and Brett Favre throw selection comes along and is actually good, this metric will love him. If somebody comes along that's a pure Tom Brady sack avoidance, Tom Brady throw selection game manager, and is fantastic, this metric will love him too. This is the feature I'm the very most proud of, and it makes me happy you've brought it up.
I'm happy you've taken the time to engage my friend. Thank you very much for commenting.
To the point about teams with the best QB histories, by combined xEPA/Play from qualifying QBs, the list (rounded to the nearest thousandth, excluding seasons split across multiple teams) is
SF 0.120
DAL 0.109
NO 0.109 (would not have thought they would be this high, but I admit to being pretty ignorant of their pre-Payton/Brees history outside the infamous Ricky Williams trade)
MIA 0.106
MIN 0.103
GB 0.102
SD 0.097
IND/CLT 0.095 (can't fall too low when you have the majority of the career of the best ever)
CIN 0.092
KC 0.081
DEN 0.078
SEA 0.077
TEN/HOIL 0.074
BUF 0.072
BAL 0.072 (Lamar Jackson accounts for 60% of their franchise total xEPA in 22% of the total plays)
NE 0.068 (Tom Brady accounts for 76% of their total!)
ATL 0.067
PIT 0.059
HOU 0.053
WAS 0.051
TB 0.051
PHI 0.049
LAR/STL 0.048 (Largest outsize effect of a single QB that I've yet found: 35% of their total xEPA from just 6.6% of their total plays from Kurt Warner)
LV/OAK/RAI 0.042
JAX 0.038
CHI 0.035
ARI/CRD 0.034 (Second largest outsize effect of a single QB that I've yet found: 39% of their total xEPA from just 7.8% of their total plays from Kurt Warner)
CAR 0.031
CLE 0.030
NYG 0.029
NYJ 0.024 (The biggest media market, with multiple teams present in order to do so, is apparently unable to figure out the most important position in the biggest sport)
Yo my friend. Do you mind if I post this as a listicle? It's great stuff. Just for you, I'll throw some comments in here.
San Francisco number one (with John Brodie, Joe Montana, Steve Young, and Jeff Garcia, all top 25 of all time) is shocking absolutely nobody. I don't think it took a sophisticated mathematical analysis to show that the 49ers have the best QB history in the league, skill-wise. Note that this does not make them the best QB team in history results-wise. I've done that analysis already (although I haven't published it yet), and SF is not number one, largely because of their horrendous late 2000s run, but I suppose in an analysis like this, the negative effect of that is smaller than it is when looking at results.
Dallas is not at all shocking either, as I was talking about with Marc. Their skill outpaces their results (I gave away their position on that list) through history, but having them second place does not shock me whatsoever. Throughout their whole post-merger history, they're basically never without a QB capable of putting up a top ten xEPA/Play in the NFL. Even stopgap Steve Pelluer finished seventh in 1988. The only bad period in their whole history, from Roger Staubach straight on up, is the holding period between Troy Aikman and Tony Romo, which did not last long.
On the point of New Orleans, their skill also outpaces their results, and the main reason for this is their very good 1987-1995 run of QB play. Both Bobby Hebert and Jim Everett put up great xEPA numbers for them, with results that lagged behind just a bit, and then we get to the obvious, with Drew Brees generally very slightly underperforming his expected results, but doing this over and over and over again adds up eventually.
Minnesota is a real shocker here to me. I would not have expected to find them in the top five, primarily because there are a lot of Tommy Kramer seasons of being heavily overrated by the results that Minnesota has to get through. Fran Tarkenton and Daunte Culpepper are doing a lot of the heavy lifting here by the looks of it, with a bit of added contribution from some miscellaneous guys (i.e Randall Cunningham) along the way. This is an interesting way to have a great QB history. Two all-time greats in terms of skill, plus a lot of miscellaneous good stuff around the edges.
I would say I'm shocked by SD being so low, but there were a lot of bad years between 1989-2004, so I suppose upon further reflection I'm not as shocked as I thought I was. I also would've expected KC to be higher than tenth. They are slightly higher than that on the results list.
Tennessee/Houston at 13 is probably the biggest shock on this whole list to me. In terms of results, they have been dramatically worse than this throughout their history. I know 1980 Ken Stabler goes from one of the worst QBs in the league to one of the best when looking through this lens, but that alone cannot be enough to dictate a swing of this magnitude. Warren Moon's 1992 is also heavily underrated by its results, but that's really all that I can see. This leads me to believe that almost the entire Tennessee Titan history of being underrated by results concentrates into Steve McNair, who to his credit is one of only three players (along with Jay Cutler and Brett Favre, both of whom played for multiple teams for multiple seasons, instead of McNair dipping to the Ravens for one year) to be classed as 'significantly underrated' by my system, despite having as many as 5000 plays for things to even out. This would definitely explain why the Titans' results tend to be on the poorer end, despite a skill ranking that seems to be on the upper end of average.
The NE/IND divergence is actually interesting, because in terms of results, the Patriots are basically the Colts, but with a little more help for Tom. In terms of skill, it's the opposite. The Colts are basically the Patriots, except with just a little bit more help for Peyton. I looked into this, and it's because Tom Brady is not the only Patriot to be significantly overrated by their results. Steve Grogan and Tony Eason are both terrible for it as well. It's like a franchise tradition up there.
The Houston Texans at 19 threw me for a loop, but then I realised that they had legitimately elite QB play in 2007, 2008, 2009, and 2020, which once you get into the bad QB teams, four years out of 24 chances is all you need I suppose.
The Rams are the biggest negative outlier from results to skill. Basically, them and the Titans switch places. The Rams are fringe top ten in terms of historical results, and not even 20th in terms of skill, meanwhile the Oilers/Titans are exactly the opposite. I'm 100% sure this effect wholly concentrates in the 1970s, where the Rams could get top five QB play out of anybody, and were doing it regularly, switching QBs for absolutely no reason, and always finding another top five results guy to come in as a replacement. From Roman Gabriel to James Harris to Pat Haden to Vince Ferragamo, they were all part of this. Finally, they traded for Jim Everett to stop this carousel, but boy was it fun while it lasted. Not great on a QB skill ranking though.
I'm frankly shocked that Chicago is as high as 26. In terms of results, they're a stone cold 32, without even any close competition. I suppose this is more an exercise is other teams coming down from their results-based perches than Chicago actually moving up any.
The best example of this is Detroit, who did have the best results-based QB in the league in 1971 Greg Landry, but when you grade that season using xEPA, the result you get is a yikes. The same goes for 1995 Scott Mitchell, and without those two guys, the Lions are basically standing alone with one horrendous turnover luck Eric Hipple half season, and some good Jared Goff years. That's it. No wonder they're at the bottom.
One final note is that it's crazy to me that since the NFL merger, if you combine both New York teams together, there have been three elite QB seasons total. Norm Snead on the 1972 Giants, Vinny Testaverde on the 1998 Jets, Chad Pennington on the 2002 Jets, and curtains. That's all of them. There are several teams that have had three in a row, and yet each NY team cannot combine to do better than that over 55 years. There's been three in Cleveland for God's sake (1976, 1980 Brian Sipe, 1987 Bernie Kosar).
When you cannot even match Cleveland in terms of QB quality, you know IT IS that bad in New York.
Please do! I was just going to post the top few teams, but kept finding interesting tidbits—as someone who is a lot more well-versed in all things NFL than I am, it doesn’t surprise me that you have even more to contribute on it.
6 months ago, I viewed Staubach as head and shoulders the best Cowboys QB and Dak as the Cowboys 2B to Romo's 2a and Aikman's 2c. Your analysis hasn't changed my view of Dak, but rather a greater appreciation of Romo and Aikman. I thought their volume and Cowboy association made them a bit underrated, but now I think they are quite underrated.
Congrats, Robbie! xEPA is not only the culmination of your work but also the work of the NFL analytics community. It is the QB metric to rule all QB metrics. The career leaderboard provides a nice bit of confirmation bias, as I already had Peyton and Young as my two best QBs of all time
It is funny how from 2016-2020, there were a lot of debates about who was better between Aaron and Brady. The main pro-Aaron argument was that he was more talented than Brady, yet we see that they look identical from both a skill and support perspective.
Absolutely. They are right on top of each other, but in terms of skill, they are also right on top of Kirk Cousins and Bobby Hebert. We must understand that this does not make them the same, because maintaining the exact same skill level over more plays makes the player with more touches strictly better, but on a rate basis, the difference between Tom Brady and Kirk Cousins (especially Washington Kirk Cousins, where he led the league in total xEPA two years in a row) doesn't exist.
People love to call Kirk Cousins overrated, and have loved that for a decade now, and likely never will quit loving that, but as far as I'm concerned, he's actually one of the more underrated players out there. He missed the significantly underrated criteria by a hair, but his 0.134 career xEPA/Play translating into 0.096 actual EPA/Play does not exactly scream overrated to me. I actually think it's the opposite, as Kirk has spent his whole career surrounded by the overrated brigade. Matthew Stafford, Aaron Rodgers, Tom Brady, Patrick Mahomes, Andrew Luck, Justin Herbert, and the list can go on. All of these guys have results that outpace their skill level, and Kirk's do not.
I'm not sure if by 'the process' you mean just the xEPA process, or the entire Historical CPOE, sk%+, and then rushing augmentation progression. I'll see if I can't answer both.
The first benefit I've gotten is bestowing me with any opinions at all on the actual skill levels of players who played in the 1970s. It's hard to watch games from the 1970s, and play by play doesn't really exist in the 1970s beyond the Super Bowl game, so this is all I have. For instance, before I began this process, I really did not have any opinion of Fran Tarkenton. I'd never gotten the chance to watch him play, beyond the same 15 highlight plays that are shown everywhere all the time.
Now that I've done all this, I can see that Fran Tarkenton was an immensely accurate thrower of the football, who eventually grew into having very underrated sack avoidance skills, and that the importance of his scrambling was probably always a little bit overstated. All of this after having no opinion on him at all just a couple months ago. I'd say that's a pretty big change. This goes for every player in the entire decade.
If we're counting the entire progression process, I would say the biggest change of heart I've had on anybody is Ken Stabler. I used to think Stabler was a waste of a spot in the Hall of Fame that should have been used on Ken Anderson, as Stabler ranked in the top five in ANY/A just twice in his entire career. Now that it's all said and done, and Ken Stabler ranks 12th all time in xEPA/Play, it's safe to say that opinion has changed quite a bit. Ken Anderson still deserves a Hall of Fame spot, but I'm no longer in the camp of believing that it's Stabler's spot he should take. There are more deserving candidates to lose their spot, like 83rd ranked John Elway, about whom my opinion did not change one bit through this process.
For a downward shift in opinion, I'd say Dan Marino is a good candidate. I used to be a Marino over Montana guy, because strictly in terms of results, Dan is better. Way better in fact. Now it becomes a contest between the two, the result of which depends just how heavily we weight skill vs actual results when we're talking in an all-time sense. Dan is still 13th all time in xEPA/Play, but before this process, I had him as a certain top five guy.
Those are the heavy hitters, but there are also some players I've been quite down on that look marginally better after all this. For instance, xEPA/Play has Michael Vick as the 69th most skilled QB of all time. Justin Herbert is 61st. Andrew Luck is 79th. I would never have thought any of these guys to be anywhere near this high.
If you're looking for the absolute largest opinion change, the answer is without question Jay Schroeder. On my results-based QB tier lists, Jay has multiple top ten finishes, plus one top five, but in terms of xEPA/Play, he is 201st. That's the 17th worst QB of all time. Not Blake Bortles bad, but worse than Christian Ponder. This indicates to me that the Joe Gibbs Washington offence was doing an extreme amount of heavy lifting with Jay, and according to this methodology, I would be willing to bet that Jay Schroeder is the very most carried QB in the history of the league.
As far as modern guys, I believe I have to go with Steve McNair. Based on results, he falls barely short of Hall of Fame consideration, but in terms of skill, he ranks 28th all time in career xEPA/Play, and considering he did make the Kurt Warner nine full season threshold, this is a Hall of Famer. I would've just barely said no before on Steve, but I'll say yes now, especially considering all the doors he knocked down in the NFL.
I saw someone's Top 10 QBs for 2025 list and wanted to compare it to xEPA/Play. Played around with a weighted average of the past 3 seasons (with most recent weighted more heavily) to construct a Top 10 (Jackson-Allen, Mahomes-Hurts, Purdy-Burrow-Tagovailoa, Prescott-Murray-Love, depending on how you weight the average and whether you take into account whole career instead of just the past 3 seasons).
Then I decided to expand it out, and found a PFF article ranking all 32 projected starting QBs posted this week. Ranking just based off Career xEPA/Play (without enforcing a total play cutoff), here were some of the biggest differences in rankings between PFF and xEPA/Play:
1. 17 positions away--Matthew Stafford (PFF rank: 7; Career xEPA/Play rank: 24). I get it a bit more now why you bring up Matthew Stafford as an example so often. He's still being ranked as a top 10 QB by PFF?? I also note that this could be an even bigger difference depending on where we slot the 4 QBs on PFF's list that do not have a qualifying xEPA/Play season. I've just chosen to leave them off, but if we put them in about where PFF has them in the rankings (not a wise move, but one that plays by their rules), McCarthy, Penix and perhaps even Ward would slot above Stafford. It's not even as if Stafford has seen a clear shift in how well he's been playing recently compared to his career norms. I could understand keeping him higher than 24, but top 10 seems ridiculous to me.
2. 15 positions away--Bo Nix (PFF rank: 19; Career xEPA/Play rank: 4) Sure, with only one (pretty good) season in his career, xEPA/Play is probably overrating him for now. I don't really put a lot of stock in that 4th place position, so discounting him somewhat is understandable. What's less understandable is the difference in treatment between Nix and Daniels under PFF's rankings. PFF puts Daniels at 6th, while Nix is down at 19th. There's a results-bias here (as Daniels had 0.202 EPA/play vs Nix's 0.090 EPA/play), but in terms of skill as measured by xEPA/Play, they are neck and neck (0.162 for Daniels vs 0.164 for Nix). Perhaps PFF grading is more results-oriented than I had thought--but they do acknowledge that after his first four starts, Nix was the 6th-best graded quarterback in the league. I guess I don't understand how you can acknowledge that level of sustained performance and still keep Nix this far down on your rankings.
3. 13 positions away--Russel Wilson (PFF rank: 22; Career xEPA/Play rank: 9). This one I think is more understandable, as Wilson is probably being propped up in career rank by all his good Seattle seasons and he's probably on a decline in his career. I think probably PFF is underrating him a bit, and career xEPA/Play is probably overrating him a bit in this ranking.
4. 13 positions away--Mason Rudolph (PFF rank: 31; Career xEPA/Play rank: 18). Based on their blurb, the PFF ranking seems more about the perception of Rudolph as a backup quarterback being ranked amongst starters than about his production or skill. I think that does a disservice to Rudolph, though, whose career xEPA/Play is actually higher than Trevor Lawrence's (perhaps partially due to only a couple of Rudolph's seasons qualifying--although their traditional rate statistics are remarkably similar, too, so it's not just that Rudolph has played poorly in seasons that he doesn't qualify for xEPA/Play). Listing Lawrence at 16 and Rudolph at 31 seems too strong a penalty for being a backup. When you factor in the number 1 overall pick (and generational one at that) vs 3rd round pick, though, it starts to make sense in the general perception of these players and speaks to the point you’ve made recently about draft position mattering.
5. 9 positions away—Baker Mayfield and CJ Stroud (PFF ranks: 13, 14; Career xEPA/Play ranks: 22, 23). Grouped together because they are next to each other on both lists
6. 9 positions away—Joe Burrow (PFF rank: 2; Career xEPA/Play rank: 11). Burrows’s ranking in terms of career xEPA/Play is lowered a bit by the inclusion of Nix and Daniels since we’ve removed the total plays threshold. That said, I do think he tends to be a bit overrated in NFL discourse, which generally puts him as a top 5 QB, where I think he belongs more in the next 5 Qbs.
7. 9 positions away—Joe Flacco (PFF rank: 29; Career xEPA/Play rank: 20). He’s buoyed a bit by his best seasons in Baltimore in terms of Career xEPA/Play, but the major discrepancy between the lists is the inclusion of a bunch of young passers (McCarthy, Penix, Ward, Stroud, Young, Williams) who either haven’t played before at the NFL level or whose skill doesn’t remarkably surpass what Flacco has demonstrated even just the past couple of seasons. He’s on the downhill of the aging curve while they are just coming into their own, but it wouldn’t surprise me if these rookies end up playing worse than Flacco this season.
David, you're a legend my friend. This analysis is fantastic. I've always had a bit of a gripe with PFF, and the reason for that is this EXACT thing, except in reverse. My gripe gestated from 2018 Mitchell Trubisky, a season where Mitch finished sixth in xEPA/Play, eighth in actual EPA/Play, but an ungodly 31st in PFF grade. At that moment I knew there had to be something going on over there. At that moment, I thought PFF grades had to be SO results-agnostic that they struggled to provide any value to me. However, with the analysis you've done here, perhaps I may have second thoughts.
I hate to be the one man anti-Matthew Stafford brigade, because I don't like just being so negative all the time, but the man has had a CPOE above zero and a sk%+ better than 100 in the same season just three times. His career high for xEPA/Play in a season is 0.118. It's just not all that, and I'm not sure what PFF or anybody else sees in this guy. I'm honestly not sure if I'm the hot take here or they are, because everybody had this right about Stafford until the point where he won a championship, at which point everybody went crazy overrating this guy.
If we do enforce a play minimum of some sort, I would hazard a guess that Stafford ends up about 16th amongst all active guys, which is roughly where he keeps finishing in the EPA/Play rankings, which means it's not even that Stafford's results cause people to overrate him. He's not been a top ten EPA/Play guy since 2021. He's not been a top five EPA/Play guy since 2019. That's a lot of years ago, so Matthew Stafford being all the way up in seventh cannot be credibly attributed to either results or skill. I'm not sure what it is about him. I've never been sure what it is about him, but there's something there that causes people to just go out of their minds. First overall draft status maybe?
It's interesting to me that Bo Nix being 11th in xEPA/Play in 2024 translates to him having the fourth highest career xEPA/play amongst active players. It's a testament to how many of them struggled as rookies I guess. This will even out as his career goes on I suspect, but if we use his 2024 rank of 11th instead of his actual xEPA/Play number, the discrepancy to PFF rank gets a lot smaller, although not small enough for my tastes. Is it that even the people over at PFF are like the general public, not valuing sack avoidance enough? In the absence of further evidence, this would be my guess, because a 19th place ranking fits Bo's 24th place CPOE ranking and eighth place rushing value ranking last year pretty well. It also must be acknowledged that Bo underperformed his xEPA/Play by more than Jayden overperformed his, meaning there should be some regression to the mean upwards for Bo as we move along in his career, and perhaps Denver gets him some better weapons.
The Russell Wilson thing is just recency bias, or the lack thereof. It's attributing his best seasons to the modern player, which of course is not correct. He finished 34th in xEPA/Play in 2022, 13th in 2022, and 18th last year, so if you throw a dart in this general direction, it does not offend me when it lands in 22nd place.
What does offend me is ranking Mason Rudolph 31st. What in the world is with that? In his 279 plays last year, he finished with an xEPA/Play of 0.124, and if we include his 2023 to make it a slightly more complete sample, his xEPA/Play in this time goes up to 0.175. That's better than Matthew Stafford has ever done in a qualifying season. There's that name again LOL. 0.124 is not better than every Trevor Lawrence season, but it's better than any recent one. I love that the Steelers are standing pat here. They've got the right guy. It comes down to my discussion about optics the other day. In terms of optics, having Mason Rudolph as the QB sucks, but in terms of real football, there's no way he can be 31st. I'd put Mason in roughly Justin Herbert tier. Until further notice, let's call him league average, with upside. That's very far from 31st.
You know what's interesting is also that Baker and CJ are right next to each other on the 2023 xEPA/Play leaderboard. 20th and 21st, respectively. However, in 2024, they diverged massively, with Baker going up to ninth place and CJ plummeting down to 37th. As such, Baker in 13th is not offensive to me, but CJ Stroud in 14th place. What the heck is that? That one placement to me would be enough to invalidate the whole list.
At best, that's a projection that CJ is going to get dramatically better this year. At worst, we've just caught PFF believing a career INT%+ of 115 on a negative career CPOE is sustainable. I've got news for them if they think that's the case. PFF of all bodies should know better than that. If you can put CJ Stroud 14th place in any ranking and try to claim said ranking has anything to do with real football on the field, you and I have a serious disagreement about the way this position is played.
CJ did finish 13th in EPA/Play in 2023, on a success rate ranking of 17th and an xEPA/Play ranking of 21st. Is this enough to keep repetitively throwing this guy parades like everybody on the internet seems wont to do? I'm not interested. If my alternative is this guy, give me Mason Rudolph every day of the week.
The Joe Burrow ranking seems to second my opinion I've put under Bo Nix. Sack avoidance is just not baked into the PFF analysis in a position of importance as high as I think it should be. Much like Bo Nix, if we were just grading arm stuff, Joe's position would make a lot of sense, although we do need to acknowledge that he did finish third place in xEPA/Play last year, and the second place guy just retired, so maybe second place Burrow isn't so wild after all, but to maintain this ranking with his inability to avoid sacks at an elite level, he basically has to maintain a CPOE of six or higher. I'm not saying he can't do this, but I am saying his CPOE in 2023 was just 1.2. In 2022 it was 2.7, so to bet on him to continue at six or higher seems like a rather long shot to me.
Thank you so much for this David. You've given me a lot to chew on here. I had limited respect for PFF's QB grades anyways, but in particular CJ Stroud at 14th just completely blew my head off my shoulders. Does PFF not market themselves as being immune to the kind of thing that would cause people to put CJ Stroud at 14th place on any ranking list? Apparently they absolutely are not. His draft position is clearly carrying the day in this ranking. I truly can't think of anything else that would cause them (or anybody) to like him so much.
I've been looking at this data on and off for the past week and every time I do I find something fascinating (to me anyway). Really excellent work.
Take Colin Kaepernick for example. He goes from having the 100th best season by xEPA/play at the beginning of his career to having the ~150th worst four years later. Every year is worse than the one before (except for a slight rebound from horrific to just bad in his last year). This is not a normal career trajectory. Most quarterbacks either get somewhat better or stay constant. (Unless they play long enough to fall off the aging cliff.) Looking at this it's obvious why no team thought he was worth the drama hiring him would bring.
These things do happen sometimes, but most of the time they're for injury reasons, and almost always to a lesser extent.
For instance, Roger Staubach is a light example of the same phenomenon, if we ignore 1977. He comes out the gate firing in 1971, but is never really the same again after the separated shoulder, except for one turn back the clock year in the aforementioned 1977. The most obvious comparison to Colin Kaepernick actually happens in his exact season, which is Robert Griffin III, who also puts up a stormer of an xEPA season in 2012, but monotonically gets worse for the entirety of the rest of his career.
Those two are basically two peas in a pod on that, but where RG3 had obvious knee issues that explain away some of his decline, there really is nothing like that for Colin Kaepernick. Is his sudden decline because he broke the ball of his foot in the second week of 2013? Given the nature of the decline that happens immediately afterwards, I would be surprised if they weren't related, but that just seems to me like an injury that a player can recover from.
As far as comparisons go, that's the end however. There is virtually nobody to compare to these two, and what makes each of these two unique is that they were never able to get it back. Not even for one year. Roger Staubach was never the same after 1971, but got it back for a year in 1977. Randall Cunningham never could come back from the torn ACL, except for 1998, when he did just that. Brian Griese was never the same after 2000, but got it back for a year in 2004. Chad Pennington was never the same after 2002, but got it back for one year in 2008, etc.. That is the common story. Most of the time there is one singular turn back the clock season. Sometimes two if you're lucky (i.e. Carson Palmer).
What makes Colin and Robert unique is that it never came back. They were never able to recapture it. These two pretty much stand alone as the only two in NFL history with the ability to have a season as good as each of them had in 2012, but not the ability to do it twice. There are plenty of people without the ability to do it three times (Roger Staubach, Chad Pennington, etc.), but pretty much everybody can at least do it twice, given the fact that they can do it once.
Makes sense and thank you again.
Terrific piece on a favorite topic of mine - skill versus circumstance. A quick question.
In the 1970s, Bill James proposed that a skill (at the time his Range Factor calculation) could be discerned by statistical attributes that remained relatively constant following a change in circumstances – trades. James saw this stability in Range Factor.
For example, Ozzie Smith had the same outstanding Range Factor in St. Louis as he did in San Diego. There were numerous other examples. James's point was that performance numbers that remain stable following trades are reflective of skill and not circumstance. Some variance is to be expected but is easily explainable. Things such as experience, injuries and aging will cause smaller expected variations.
The coup de grace came with his analysis of the rare swapping of identical position players – the third basemen for Texas and Cleveland. Buddy Bell, an excellent defensive player, was swapped for Toby Harrah, a defensive liability. Here was the perfect test. These two players swapped virtually every circumstance - pitching staffs, stadiums, short stops next to them, managers etc.
So, what happened? Nothing. Bell was again at the top of the Ranger Factor numbers while Harrah was near the bottom. Same as it ever was. Range Factor was a skill James persuasively argued because its statistical measurements traveled with the player to new environments and remained stable in the new environment. Today, Ranger Factor and other defensive metrics are taken as reflective of actual skill and not circumstance.
I might have missed it, but I am curious as to whether your chosen attributes of skill follow quarterbacks to new teams and remain relatively stable despite new circumstances? That would perhaps be an additional supportive argument here.
Thanks as always for the great read.
I wish I had a better answer, but it's yes and no Grant.
Things for NFL QBs that remain the very most stable when they change teams are things like time to throw, and depth of target. These are things that are mostly baked into the bread, and rarely change, but they also have exceedingly minimal correlation with results. Quite frankly, if you're a QB who can avoid sacks, and get the ball into your receivers' hands, it doesn't matter whether you throw quick (Tom Brady), throw slow (Patrick Mahomes), throw short (Joe Montana), or throw long (Joe Namath). You can succeed doing all these things, so in that way, they're not really skills. More like the football equivalent of batting stances. Since none systematically works better than any other, I don't care what you do. Just do what you do, and do it well.
The issue with this entire premise Grant is trying to find enough examples of prime QBs with mid-career team changes. For instance, Tom Brady changed teams because he was finished in New England. He was following the typical aging curve out of the league, which is why his 2019 was so bad, and then he went to TB, and all of a sudden wasn't aging anymore. I'm not a conspiracy theorist, but I need Tom's hookup for some HGH.
Using CPOE as the example, and excluding 2019, Tom's CPOE figures in 2016, 2017, 2018 in New England are 3.7, 3.7, one flat, respectively. Then we go to the Tampa years, where he's 3.7, 1.7, 0.7, so yes. I would say it's pretty constant. Peyton Manning in Denver is the same thing. His CPOE range in Indianapolis is mostly between four flat and 6.5. He goes to Denver and does 7.4, 6.4, 3.3, before the bottom falls out in 2015 for reasons that were not team change.
These are just examples, but it is mathematically proven as well that sack rate and CPOE are the two most consistent stats when QBs change teams. Results vary wildly, but these numbers do not vary as wildly. The problem is, this sport is still football. Players only control so much of their own statistics at the best of times. It's not like baseball, where players are largely by themselves out there.
In football, at least at this position, there are certain play styles, which some people are better at and some are worse at. My favourite example is John Elway. If I could trouble you to go over to my CPOE page, you can type his name in, and I bet you'd be able to tell when the Broncos got rid of Dan Reeves. The offence fundamentally changed, and the player became a fundamentally better player, no team change required, and the stats did not hold stable, because John was better at playing one way than the other.
I can name examples all day, but the purpose of what I'm saying is to drive home the point that these stats are not necessarily stable, but they are the most stable, because they're more stable than any potential alternatives. No stats are more stable through a team or offensive scheme change than CPOE and sk%+, but they are not hugely stable, because no statistic in football is hugely stable, because quite frankly, in my opinion, no level of skill is hugely stable.
I hope what I'm saying is making sense. The goal here is not to create a stable metric. It's to create a more stable metric, that remains quite unstable, because football is an unstable game. It uses the most stable metrics in existence, but the most stable metrics in existence are still quite unstable. Like I said in the article, the QB controls about 45 percent of his sack rate, and 33 percent of his CPOE. This is not a lot, but it's more individual control than the QB has over anything else.
Overall, the answer to your question is yes. The metrics I have built xEPA with are the very most stable ones available. I'm just tempering that yes with the crucial knowledge that they are still not very stable.
Taking it back to the "worst Super Bowl QB" discussion we had on your Rex Grossman post a few weeks ago:
Worst season by xEPA/Play for a QB who made the Super Bowl that year? Jim Plunkett in 1980 at -0.082 (joined that season in the Super Bowl by Ron Jaworski--the two lowest Career xEPA/Play QBs with over 4000 plays). However, he was not slated as the starter at the beginning of the season, and had 3 TDs in the Super Bowl, so he probably gets a pass.
The second worst season by xEPA/Play to end in a Super Bowl is 1987 Jay Schroeder--but he was replaced for the playoffs by Doug Williams, so he's not really a Super Bowl QB.
We turn next to 2007 Eli Manning as the new worst season by xEPA/Play to end in a Super Bowl at -0.055. He won the game, though, against the team led by the 18th best season ever in terms of xEPA/Play in 2007 Tom Brady. From narrative standpoint, the fact that he won the Super Bowl against the Patriots Dynasty in what otherwise would have been their perfect season is too powerful to have "worst Super Bowl QB" stick.
That brings us to 1977 Craig Morton for the Denver Broncos. In 1977, he won Comeback Player of the Year and tied for second in MVP voting--but he also put up -0.045 xEPA/Play. A sk%+ of 66 and completing passes basically at expectation makes for not a season demonstrating a lot of skill. Even his rushing numbers are nothing special. I'd have to do an investigation into why this season was so highly esteemed at the time.
What clinches it for me, though, is that Morton also has the fifth-worst xEPA/Play by a team's primary quarterback that ended in the Super Bowl (i.e. excluding the Plunkett and Schroeder seasons), for the 1970 Dallas Cowboys. He just barely gets into the negative range here at -0.000, but that means he is the only quarterback to go to multiple Super Bowls with negative xEPA/Play in their respective seasons. He also doesn't have any great seasons in terms of xEPA/Play in his career, lost both of the Super Bowls he played, and played pretty poorly in those games (4 interceptions in '77 resulting in him getting pulled from the game, 3 in '70). That combination of factors leads me to think the title of "Worst Super Bowl QB" is Craig Morton's to lose.
I really like your analysis here. I did not even think to check for this, but once again, I suppose you can always tell who did the analysis by looking at the results. I cannot believe that it turned out to be 1980 Jim Plunkett after all.
What an odd season that was. He is absolutely horrendous in the first two playoff games, and gets his team through the second round despite his immensely negative contribution in what is likely the worst combined QB performance in playoff history, Oakland vs Cleveland in 1980, but then becomes full blown playoff Kurt Warner for the final two games, which are the only two great playoff games of his entire life. I suppose he picked a great time to have them.
2007 Eli Manning will always get my goat. There remain members of the public who insist that he was a great (or even good) player, despite all evidence to the contrary, and I can't find a way for that not to get in my head, when there are legitimately elite Super Bowl seasons, such as the obvious 1976 Ken Stabler, the all-time xEPA/Play leader, that get nowhere near the attention they deserve.
I also have never been able to understand the fuss about 1977 Craig Morton. As far as results go, he was only sixth in ANY/A, and on much fewer total touches than everybody around him on that leaderboard. On my results-based 1977 tier list, he only comes in seventh, which is fine but should not have gotten him a second place MVP finish. The Denver passing offence wasn't even good. In terms of simple NY/A, they finished 15th in a 26 team league. Craig was extraordinarily good at avoiding turnovers in 1977, which is what makes his results look so good, but this is a 123 INT%+ season out of a player with a career rate of 99, so this is a clear outlier, which I don't feel the need to give him any real credit for.
That is what happened here. The story is clear as day, an absolutely ridiculous stretch of turnover luck, the likes of which would never happen to Craig Morton either beforehand or afterwards, but as for why this season was so overrated in its own time, I'm not actually sure. One would have to think that these are narrative MVP votes, which even the smart people in Craig's own time knew he did not deserve, but I'm only guessing.
Do take note though that Craig played that Super Bowl basically unable to move his left leg, and the sham that Super Bowl turned into is what caused the NFL to instate the ruthlessly strict injury reporting policy the league has today, so the betting public can at least be aware that something like this may happen, before it actually does. That doesn't change the fact that he was a poor QB on an okay offence in 1977, which does make him one of the worst QBs ever to make the Super Bowl, but I am willing to excuse his performance in the actual Super Bowl game just a little bit.
Even despite this, I do still concur with your choice that 1977 Craig Morton is the answer we've been looking for, if we operate with the restriction that QBs who won the game are exempt.
I was a little surprised how low Grossman ended up on the list (only two spots behind Morton in ‘77, with Ben Roethelisberger in ‘08 in between) after all the work you did to say he was underrated. Bad Rex was bad, and this leaderboard doesn’t exclude the all-time-terrible performances. But it does reinforce that they weren’t entirely just terrible results despite underlying good play.
I will also note that David Woodley (who you proffered as the competition when thinking in terms of overall career) doesn’t have enough touches in ‘82 to qualify, but with this raw completion and sack numbers being roughly in line with ‘80-‘81, I don’t think his xEPA/play in ‘82 would be in the negatives. I didn’t record the complete list, but from what I recall, he’d probably slot in a couple spots above ‘00 Dilfer. But Woodley wasn’t really in the discussion because of his performance in that specific season, anyway.
Rex on the whole in 2006 was bad. Nobody doubts that, but to remove three games and boost his CPOE all the way to barely below zero indicates that his performance was seriously biased downwards by those few bad games, even in this area.
The important thing to understand is that without the Bad Rex games, he generated around 0.16 EPA/Play, on a CPOE of -0.6 and a sk%+ of 114, if I recall correctly. However, the xEPA/Play for a season like this would be 0.096. This is a long way short of what he was actually doing, meaning he was seriously overperforming in the games that were not Bad Rex.
The point of that whole article was that I can see 2006 Rex Grossman as a 0.096 EPA/Play guy, if you ignore the bad stuff. This would still be one of the worst results-based QB seasons ever to make the Super Bowl, because it's just hard to get that far. We're not doubting that Rex is one of the worst. I was merely protesting the fact that he was the worst.
In real life, Bad Rex was tacked onto a seriously inflated statline, that was always going to come down anyways, but it's difficult to envision it coming crashing down as hard as it did. In an exercise like this, Bad Rex was tacked onto a much worse theoretical player than Rex Grossman was actually playing. However, it also hurts him less than it did in real life, as even with a -15 CPOE in these three games, you cannot forecast 13 turnovers in a three game sample, no matter how bad a player is.
For a stretch to be this bad, it took stupendously bad turnover luck, which a model like this accounts for. Therefore, with Bad Rex being so bad it was unable to be repeated, and with Good Rex also being so good that it was impossible for this particular player to repeat, it met in the middle at his xEPA/Play of -0.016 in 2006, which is shockingly close to his real result of -0.019.
In short, I think he was a fluke both ways. Was Good Rex as good as Rex was playing? No, but can anybody replicate 13 turnovers in three games, in the post-2004 easy passing era? Not a chance. No matter how bad they are. What I was trying to get out into the public was that the Rex Grossman that generated 0.096 xEPA/Play in the 13 regular 2006 games was a real person, and should not be overshadowed by just how bad Bad Rex was.
You're actually right about David Woodley. His real results were really bad, but this exercise has made clear that David was not quite as bad as met the eye. With a better offensive environment than the Don Shula Dolphins, his results probably would've been better. I would say his 1982 probably ends up basically in the same spot as his 1980 in xEPA/Play terms, around 0.02 somewhere. That's still below league average, but at least it's not negative. He's probably better than people like 2000 Trent Dilfer.
Thanks! Much appreciated. I’ll take time later to study this.
Thank you very much my friend! I sincerely hope you enjoy what you find.
I have a lot of thoughts, so I'm going to try and present them in an organized manner
Future Applications:
I think it would be interesting to see what use xEPA would have in evaluating QB prospects. Because of college football's messed-up way of measuring sacks and rush yards, it wouldn't have use beyond the last 20 years.
Using it for individual games would be useful as well.
Play notes:
1. I did not expect Josh to have two of the top xEPA seasons of all time. Obviously, there is a bias toward high-touch modern players, but that is still impressive.
2. Kirk Cousins looking like not only a top 5 QB but arguably the best QB in back-to-back seasons is insane. This would be the hottest take of all time, as I'm convinced no one has ever thought this.
3. I'm surprised that Dak ended up being pretty clearly the worst of the great Cowboy QBs on a career basis.
General thoughts:
I think by showing how there are multiple ways to be accurate and avoid sacks that you have mitigated some hesitancy to the metric. As people, myself included, sometimes have a hesitancy to accept one number or one metric be the definitive metric. There is a sort of joy that comes from the variety and mystery of evaluating players, and the debates that follow from it. By showing there are multiple ways to be efficient and not making definitive claims about how to rank QBs all-time, you have kept some of that mystery alive.
The problem with using this statistic for college players is that there is no generally accepted model for college CPOE. The conferences are still separate enough that they all weakly have their own style of play, plus it likely requires some opponent adjusting, because a lot of college QBs can complete 85 percent of their passes against the FCS, or whatever very bad opponent these teams schedule for themselves. It has been done, but I don't believe it's publicly been done, so we'd have to wait on that.
I do think it could be useful, if we do ever get to point where we can use it. It's long been accepted that college CPOE is the best way to draft QBs, so I would not be shocked at all if xEPA could do it better, but the issue of college CPOE being mostly proprietary throws a monkey wrench in the middle of it all.
Using it for single games can be done though. I don't plan to do it in the imminent future, but it's absolutely a direction for future analysis.
It's true that total EPA is biased towards people with high touch counts, but it's not that biased. You still have to maintain an absurd level of skill over that massive amount of touches to get to the sharp end of the list. Dan Majkowski touched the ball 750 times in 1989, and I don't see him up there anywhere. We know how much easier it is to play QB when the offence is deleveraged from an individual, so any of the guys at the top that managed this level of skill over 700+ plays, they get my respect.
For the Kirk Cousins thing, you must also take into account the league environment he was in. You and I have coined the term 'weak league era' for a reason. QB play was extremely weak at this time. No disrespect to Ben Roethlisberger, but when the league's leader in xEPA/Play is Ben Roethlisberger (as it was in 2015), that's a weak league. 2015 is a down season for Peyton Manning, a down season for Tom Brady, a down season for Aaron Rodgers, a down season for Matt Ryan, a down season for Drew Brees, and a down season for Philip Rivers, all at once. Taking this all into account, is it really any surprise that the league's leader in Total xEPA is Kirk Cousins? Who else is left?
Basically, Ben Roethlisberger, Carson Palmer, and Russell Wilson are left, and I don't think it's particularly controversial to say that Kirk is better (not greater, but better) than all these guys. Russ may draw some argument, but I don't think Ben or Palmer should, so yeah. I would say that Kirk has an argument for being the league's best QB in 2015.
2016 is a fundamentally different story. Kirk leads the league in xEPA/Play himself, and it's not even close once you discard a few great Bryan Hoyer starts. I cannot immediately ascertain why his real results lagged so far behind. He had two top 20 receivers that year, and an offensive coach that's generally pretty well respected. I suspect turnover luck had something to do with it (five flat CPOE becoming an INT%+ of just 107 is unlucky), as he did finish second in success rate that year, behind only the Kyle Shanahan offence in Atlanta.
To put it bluntly, Kirk Cousins in Washington was a bona fide top tier QB in the NFL. People don't remember it that way, because he went to Minnesota and never got it back, but it was in there in the beginning. Kirk is sort of like a slightly worse version of Chad Pennington in my opinion. If everything had gone right, he could've been an all-time great, but after he left Washington, very little went right for Kirk.
I find the phrasing you used to describe Dak Prescott interesting. The worst of the Cowboy greats. It's not anything you've done wrong Marc. It's just a commentary on the way people think of the Cowboys in specific. Their QBs are generally so overanalysed and frequently scrutinized that they often end up underrated. Look at Tony Romo.
He played just seven full seasons, and in those seven full seasons, he was in the top five in xEPA/Play four times, and in the top two three times. If he could've just been on the field a little bit more in his career (particularly, if he could've got to the Kurt Warner nine season mark), Tony Romo is a Hall of Fame talent. No question about that. He's only as underrated as he is because he played for the Cowboys.
People hate the Cowboys, and so will never admit this, but the Cowboys are a team with an absolutely fantastic QB history. Probably one of the best in the league. I would put them fourth, behind only GB, SD, and SF. That's how good they are at finding QBs, so to say that Dak Prescott is the very worst of a great troop is not a particularly cutting insult, which is good, because I don't think you meant it as one. If I can ask you, who did you think would be worse than Dak Prescott?
It was never going to be Tony Romo, and Roger Staubach actually surprised me by coming in as low as 19th on the career list. I thought he would be better than that. This leaves the only option as Troy Aikman, but Troy from 1991-1996 was so consistently ruthless. Dak Prescott's yo-yo act really can't match that. He is slightly less all over the place in terms of skill than he is in actual results, but he still has oscillated wildly up and down over the course of his career, which of course means he will be fantastic again in 2025 LOL.
There are multiple ways to skin the cat of playing the QB position Marc. There always have been, and always will be. The most successful archetype throughout history has generally been combining the Peyton Manning style of sack avoidance with mushy middle throw selection, but more recently, the Patrick Mahomes style of throw selection has been picking up steam, so perhaps things will change as we move through history, but the great thing about it is that I've built this metric to be immune to that change.
I simply do not care how players do things. If they avoid the sacks, that's good enough for me. To me, that means they are good at sack avoidance, and there need not be anything more to it. Same thing with throwing accuracy. If on the throws they attempt, the QB makes more of the throws than the league average QB would have, they are good at making the throws. Perhaps for some guys this hinders them from even attempting some of the more difficult ones, but I don't care, because not attempting a throw you can't make is a skill. One that several QBs don't have. It should be accounted for.
The QB position is a mystery. Any time there are multiple ways to succeed at something, there will be mystery. I did not make any effort to maintain any mystery, but being true to the game entails not grading the method. If someday an Anthony Richardson combination of Lamar Jackson sack avoidance and Brett Favre throw selection comes along and is actually good, this metric will love him. If somebody comes along that's a pure Tom Brady sack avoidance, Tom Brady throw selection game manager, and is fantastic, this metric will love him too. This is the feature I'm the very most proud of, and it makes me happy you've brought it up.
I'm happy you've taken the time to engage my friend. Thank you very much for commenting.
To the point about teams with the best QB histories, by combined xEPA/Play from qualifying QBs, the list (rounded to the nearest thousandth, excluding seasons split across multiple teams) is
SF 0.120
DAL 0.109
NO 0.109 (would not have thought they would be this high, but I admit to being pretty ignorant of their pre-Payton/Brees history outside the infamous Ricky Williams trade)
MIA 0.106
MIN 0.103
GB 0.102
SD 0.097
IND/CLT 0.095 (can't fall too low when you have the majority of the career of the best ever)
CIN 0.092
KC 0.081
DEN 0.078
SEA 0.077
TEN/HOIL 0.074
BUF 0.072
BAL 0.072 (Lamar Jackson accounts for 60% of their franchise total xEPA in 22% of the total plays)
NE 0.068 (Tom Brady accounts for 76% of their total!)
ATL 0.067
PIT 0.059
HOU 0.053
WAS 0.051
TB 0.051
PHI 0.049
LAR/STL 0.048 (Largest outsize effect of a single QB that I've yet found: 35% of their total xEPA from just 6.6% of their total plays from Kurt Warner)
LV/OAK/RAI 0.042
JAX 0.038
CHI 0.035
ARI/CRD 0.034 (Second largest outsize effect of a single QB that I've yet found: 39% of their total xEPA from just 7.8% of their total plays from Kurt Warner)
CAR 0.031
CLE 0.030
NYG 0.029
NYJ 0.024 (The biggest media market, with multiple teams present in order to do so, is apparently unable to figure out the most important position in the biggest sport)
DET 0.020
Yo my friend. Do you mind if I post this as a listicle? It's great stuff. Just for you, I'll throw some comments in here.
San Francisco number one (with John Brodie, Joe Montana, Steve Young, and Jeff Garcia, all top 25 of all time) is shocking absolutely nobody. I don't think it took a sophisticated mathematical analysis to show that the 49ers have the best QB history in the league, skill-wise. Note that this does not make them the best QB team in history results-wise. I've done that analysis already (although I haven't published it yet), and SF is not number one, largely because of their horrendous late 2000s run, but I suppose in an analysis like this, the negative effect of that is smaller than it is when looking at results.
Dallas is not at all shocking either, as I was talking about with Marc. Their skill outpaces their results (I gave away their position on that list) through history, but having them second place does not shock me whatsoever. Throughout their whole post-merger history, they're basically never without a QB capable of putting up a top ten xEPA/Play in the NFL. Even stopgap Steve Pelluer finished seventh in 1988. The only bad period in their whole history, from Roger Staubach straight on up, is the holding period between Troy Aikman and Tony Romo, which did not last long.
On the point of New Orleans, their skill also outpaces their results, and the main reason for this is their very good 1987-1995 run of QB play. Both Bobby Hebert and Jim Everett put up great xEPA numbers for them, with results that lagged behind just a bit, and then we get to the obvious, with Drew Brees generally very slightly underperforming his expected results, but doing this over and over and over again adds up eventually.
Minnesota is a real shocker here to me. I would not have expected to find them in the top five, primarily because there are a lot of Tommy Kramer seasons of being heavily overrated by the results that Minnesota has to get through. Fran Tarkenton and Daunte Culpepper are doing a lot of the heavy lifting here by the looks of it, with a bit of added contribution from some miscellaneous guys (i.e Randall Cunningham) along the way. This is an interesting way to have a great QB history. Two all-time greats in terms of skill, plus a lot of miscellaneous good stuff around the edges.
I would say I'm shocked by SD being so low, but there were a lot of bad years between 1989-2004, so I suppose upon further reflection I'm not as shocked as I thought I was. I also would've expected KC to be higher than tenth. They are slightly higher than that on the results list.
Tennessee/Houston at 13 is probably the biggest shock on this whole list to me. In terms of results, they have been dramatically worse than this throughout their history. I know 1980 Ken Stabler goes from one of the worst QBs in the league to one of the best when looking through this lens, but that alone cannot be enough to dictate a swing of this magnitude. Warren Moon's 1992 is also heavily underrated by its results, but that's really all that I can see. This leads me to believe that almost the entire Tennessee Titan history of being underrated by results concentrates into Steve McNair, who to his credit is one of only three players (along with Jay Cutler and Brett Favre, both of whom played for multiple teams for multiple seasons, instead of McNair dipping to the Ravens for one year) to be classed as 'significantly underrated' by my system, despite having as many as 5000 plays for things to even out. This would definitely explain why the Titans' results tend to be on the poorer end, despite a skill ranking that seems to be on the upper end of average.
The NE/IND divergence is actually interesting, because in terms of results, the Patriots are basically the Colts, but with a little more help for Tom. In terms of skill, it's the opposite. The Colts are basically the Patriots, except with just a little bit more help for Peyton. I looked into this, and it's because Tom Brady is not the only Patriot to be significantly overrated by their results. Steve Grogan and Tony Eason are both terrible for it as well. It's like a franchise tradition up there.
The Houston Texans at 19 threw me for a loop, but then I realised that they had legitimately elite QB play in 2007, 2008, 2009, and 2020, which once you get into the bad QB teams, four years out of 24 chances is all you need I suppose.
The Rams are the biggest negative outlier from results to skill. Basically, them and the Titans switch places. The Rams are fringe top ten in terms of historical results, and not even 20th in terms of skill, meanwhile the Oilers/Titans are exactly the opposite. I'm 100% sure this effect wholly concentrates in the 1970s, where the Rams could get top five QB play out of anybody, and were doing it regularly, switching QBs for absolutely no reason, and always finding another top five results guy to come in as a replacement. From Roman Gabriel to James Harris to Pat Haden to Vince Ferragamo, they were all part of this. Finally, they traded for Jim Everett to stop this carousel, but boy was it fun while it lasted. Not great on a QB skill ranking though.
I'm frankly shocked that Chicago is as high as 26. In terms of results, they're a stone cold 32, without even any close competition. I suppose this is more an exercise is other teams coming down from their results-based perches than Chicago actually moving up any.
The best example of this is Detroit, who did have the best results-based QB in the league in 1971 Greg Landry, but when you grade that season using xEPA, the result you get is a yikes. The same goes for 1995 Scott Mitchell, and without those two guys, the Lions are basically standing alone with one horrendous turnover luck Eric Hipple half season, and some good Jared Goff years. That's it. No wonder they're at the bottom.
One final note is that it's crazy to me that since the NFL merger, if you combine both New York teams together, there have been three elite QB seasons total. Norm Snead on the 1972 Giants, Vinny Testaverde on the 1998 Jets, Chad Pennington on the 2002 Jets, and curtains. That's all of them. There are several teams that have had three in a row, and yet each NY team cannot combine to do better than that over 55 years. There's been three in Cleveland for God's sake (1976, 1980 Brian Sipe, 1987 Bernie Kosar).
When you cannot even match Cleveland in terms of QB quality, you know IT IS that bad in New York.
Please do! I was just going to post the top few teams, but kept finding interesting tidbits—as someone who is a lot more well-versed in all things NFL than I am, it doesn’t surprise me that you have even more to contribute on it.
6 months ago, I viewed Staubach as head and shoulders the best Cowboys QB and Dak as the Cowboys 2B to Romo's 2a and Aikman's 2c. Your analysis hasn't changed my view of Dak, but rather a greater appreciation of Romo and Aikman. I thought their volume and Cowboy association made them a bit underrated, but now I think they are quite underrated.
Congrats, Robbie! xEPA is not only the culmination of your work but also the work of the NFL analytics community. It is the QB metric to rule all QB metrics. The career leaderboard provides a nice bit of confirmation bias, as I already had Peyton and Young as my two best QBs of all time
It is funny how from 2016-2020, there were a lot of debates about who was better between Aaron and Brady. The main pro-Aaron argument was that he was more talented than Brady, yet we see that they look identical from both a skill and support perspective.
Absolutely. They are right on top of each other, but in terms of skill, they are also right on top of Kirk Cousins and Bobby Hebert. We must understand that this does not make them the same, because maintaining the exact same skill level over more plays makes the player with more touches strictly better, but on a rate basis, the difference between Tom Brady and Kirk Cousins (especially Washington Kirk Cousins, where he led the league in total xEPA two years in a row) doesn't exist.
People love to call Kirk Cousins overrated, and have loved that for a decade now, and likely never will quit loving that, but as far as I'm concerned, he's actually one of the more underrated players out there. He missed the significantly underrated criteria by a hair, but his 0.134 career xEPA/Play translating into 0.096 actual EPA/Play does not exactly scream overrated to me. I actually think it's the opposite, as Kirk has spent his whole career surrounded by the overrated brigade. Matthew Stafford, Aaron Rodgers, Tom Brady, Patrick Mahomes, Andrew Luck, Justin Herbert, and the list can go on. All of these guys have results that outpace their skill level, and Kirk's do not.
What player has your opinion changed on the most after going through this process? Is it Aaron, Kirk, or someone else?
I'm not sure if by 'the process' you mean just the xEPA process, or the entire Historical CPOE, sk%+, and then rushing augmentation progression. I'll see if I can't answer both.
The first benefit I've gotten is bestowing me with any opinions at all on the actual skill levels of players who played in the 1970s. It's hard to watch games from the 1970s, and play by play doesn't really exist in the 1970s beyond the Super Bowl game, so this is all I have. For instance, before I began this process, I really did not have any opinion of Fran Tarkenton. I'd never gotten the chance to watch him play, beyond the same 15 highlight plays that are shown everywhere all the time.
Now that I've done all this, I can see that Fran Tarkenton was an immensely accurate thrower of the football, who eventually grew into having very underrated sack avoidance skills, and that the importance of his scrambling was probably always a little bit overstated. All of this after having no opinion on him at all just a couple months ago. I'd say that's a pretty big change. This goes for every player in the entire decade.
If we're counting the entire progression process, I would say the biggest change of heart I've had on anybody is Ken Stabler. I used to think Stabler was a waste of a spot in the Hall of Fame that should have been used on Ken Anderson, as Stabler ranked in the top five in ANY/A just twice in his entire career. Now that it's all said and done, and Ken Stabler ranks 12th all time in xEPA/Play, it's safe to say that opinion has changed quite a bit. Ken Anderson still deserves a Hall of Fame spot, but I'm no longer in the camp of believing that it's Stabler's spot he should take. There are more deserving candidates to lose their spot, like 83rd ranked John Elway, about whom my opinion did not change one bit through this process.
For a downward shift in opinion, I'd say Dan Marino is a good candidate. I used to be a Marino over Montana guy, because strictly in terms of results, Dan is better. Way better in fact. Now it becomes a contest between the two, the result of which depends just how heavily we weight skill vs actual results when we're talking in an all-time sense. Dan is still 13th all time in xEPA/Play, but before this process, I had him as a certain top five guy.
Those are the heavy hitters, but there are also some players I've been quite down on that look marginally better after all this. For instance, xEPA/Play has Michael Vick as the 69th most skilled QB of all time. Justin Herbert is 61st. Andrew Luck is 79th. I would never have thought any of these guys to be anywhere near this high.
If you're looking for the absolute largest opinion change, the answer is without question Jay Schroeder. On my results-based QB tier lists, Jay has multiple top ten finishes, plus one top five, but in terms of xEPA/Play, he is 201st. That's the 17th worst QB of all time. Not Blake Bortles bad, but worse than Christian Ponder. This indicates to me that the Joe Gibbs Washington offence was doing an extreme amount of heavy lifting with Jay, and according to this methodology, I would be willing to bet that Jay Schroeder is the very most carried QB in the history of the league.
As far as modern guys, I believe I have to go with Steve McNair. Based on results, he falls barely short of Hall of Fame consideration, but in terms of skill, he ranks 28th all time in career xEPA/Play, and considering he did make the Kurt Warner nine full season threshold, this is a Hall of Famer. I would've just barely said no before on Steve, but I'll say yes now, especially considering all the doors he knocked down in the NFL.