Thanks for sharing not only the Jags story, but your story. I think the fact that this was a personal experience made it easier to empathize with you and other Jags fans. While not within the same context, I think many sports fans have felt similarly at one point or another, including myself.
On a lighter note, where do you place this Jags team amongst other teams of this era (was thinking 14-18 but there may be another timeframe that makes more sense)?
Thank you very much Marc! I voiced my concerns at the beginning that this wouldn't come out very well. Those concerns were very real. Normally I end up not very proud of my Jaguars work, but I quite like the way this turned out. If you don't mind my asking, what are your most heartbreaking moments, in football and in sports in general? I'd love to hear somebody else tell their story of a heartbreaking defeat.
Onto the second question, I think 2014-2018 is the right choice to define the era, because that was after the Broncos and Seahawks had aged out of being truly great teams (although they were still competitive ones), but before the Chiefs and Ravens come in in 2019 to jazz the league back up again. This leaves a very weak league for the whole 2014-2018 period. The Patriots were the lucky ones to not fall apart as the whole league aged out around them.
That being said, this 2017 Jaguar team is why I will forever insist that 2017 is the best playoff run either Tom Brady or the New England Patriots ever had, despite not winning a championship at the end of it. I don't know why the Brady stans don't talk about this more, because in the 'weak league' era, as I've always taken to calling it, after prime Manning but before prime Mahomes, I view the Jaguars as the second best team, and even this comes with a caveat, because the team I think is better than them is the 2015 Cardinals, and this only applies BEFORE the Carson Palmer injury.
It's either the Jags, with their best defence of all time, but fringe top ten offence, or the Arizona Cardinals with their EPAA+ of 130 (6th in 2015), and by far the best offence in the league before Carson broke his finger (which Larry Fitzgerald to this day says is the reason he never won a Super Bowl, and I agree with him). It's razor thin between these two options, but they are the two options, as from these two it's a long way down to the second tier, which consists of 2015 Panthers, 2014 Packers, 2018 Saints, who are all pretty similar in terms of quality to each other.
I hate to give a copout answer, and I hate having to talk about the 2015 Cardinals in general because the conversation always ends in copout answers, but if I can reserve the right to Carson Palmer having ten healthy fingers, I would pick the 2015 Cardinals, in a close fight. However, if Carson getting injured right at the end of the season was destiny, and I can't prevent it, I would pick this 2017 Jaguars team over their next closest competition (2015 Panthers) ten times out of ten.
I think your articles are important for keeping the memory of the 15 Cardinals and 17 Jaguars alive as these are the type of teams that are often lost to history.
It was a little unclear to me but would you put these teams over any Patriots team of this era?
Thank you my friend. These teams are like the team version of Trent Green. In my opinion, they're among the greatest of all time, but for whatever reason the NFL community treats them as if they were merely good. It feels like myself (and yourself) against the world sometimes trying to get my point across, but if I don't do it, who will?
When I said these two teams were one and two, I meant one and two. No Patriots team is even in the top five best teams of this era. Like I was saying in the other comment, this whole era was an exercise in the NFC failing to send its best team to the Super Bowl. I'll give you my top ten list of the weak league era:
2015 Cardinals
2017 Jaguars
2015 Panthers
2014 Packers
2018 Saints
2015 Seahawks
2016 Patriots
2018 Chiefs
2016 Falcons
2017 Eagles
You'll notice two things. First is that the 2015 NFC was loaded, so loaded that they all did double knockouts with each other, allowing the AFC to win the SB that year, and second is that all the competition was in the NFC in this era. Seven out of the top ten teams are over there, and the team that won the Super Bowl three times in these five years (Patriots) make this list just once. This is primarily because the NFC sent their best team to the Super Bowl just twice (2016, 2017) in this era.
I respect the 2016 version of the Patriots, who beat the best teams the league had to offer, but the 2014 version barely got out of there alive against the NFC's third best team. The 2017 version ought to have lost the AFC Championship, and did lose the Super Bowl, and the 2018 version got to play the NFC's third best team again.
You can only beat who is in front of you, and they do deserve their championships, but to somebody like me who is not a championship centrist, beating no top five teams on the way to a Super Bowl does not qualify you as a great team. I don't care about the championship. It's just a title. The real great teams are the ones that the Patriots missed, like the 2014 Packers, like the 2018 Saints, both of whom lost one possession playoff games, due primarily to randomness.
In this game, there's nothing you can do about that, which is why losing in the playoffs DOES NOT disqualify you from being a great team. It does in basketball, but it doesn't in this game. To me, one possession playoff results mean almost nothing, because you constantly run into things like this 2017 AFC Championship, where if the correct call had been made on the field, the Patriots would've lost. Since it wasn't, they won. That's just randomness. It has nothing to do with team quality. That to me is why championship centrism doesn't work in football like it does in other sports.
It's why I don't mind leaving three of the era's five champions off my best teams list. People who give lots of weight to championships (i.e. one possession playoff game results) will view that as blasphemy, but I don't.
I thought I had recently was that in the NFL, the regular season separates the good teams from the bad teams while the postseason separates the lucky from the unlucky.
There's an economics principle (I think you know I'm pursuing a PhD in economics, but if you don't, I am) that states that the closer two agents converge in terms of ability, the higher the chances that their market outcomes will be decided by something that isn't ability. The NFL and NHL and MLB playoffs run headlong into this principle almost every single year. Matching up two evenly matched teams is great in terms of entertainment value, but in terms of deciding who the better team is, it's quite possibly the worst imaginable way to do so.
As long as we accept that the champion is not the best team most of the time, this is okay, but when you make the claim that the best team is the champion, because they're the champion, I entirely disagree. More often than not, it's just the way the wind is blowing.
I'm not wholly willing to go as far as you went Marc, because occasionally there are multi-possession wins in playoff games, especially as we go further back in history. I hate to be THAT GUY, but this does mean to me that championships in years gone by tend to be worth more than championships now, because talent spreads used to be wider, and you could be more confident that the best team was the champion.
I don't think there is such a thing as an undeserving champion, because what makes you a deserving champion in the NFL is to win a three round, single elimination tournament. Every champion in history has done that, but not all deserving champions are great teams. Very far from.
#1 moment heartbreaking sports moments has got be the Warriors losing game 7 of the NBA finals in 2016. It is now more bittersweet as they ended winning 3 more championships and my home town team got its only title, but to this day I still haven't the 4th quarter of that game, even as some who likes Lebron James (though not at the time).
#2 would be the Seahawks losing the SB to the Patriots and in general any Tom Brady/Patriots playoff win from 2014-2020. I'm not sure if this game was the origin on me rooting against Brady, or if the reason I was heartbroken was because I was rooting against Brady. Thinking back on it, I'm not entirely sure why I was rooting for the Seahawks since they had demolished my favorite player's team the year before but I was pretty young back then so I don't think my rooting interest had much continuity.
#3 Is probably the Saints losing to the Rams in the 2018 NFC championships. I could also go with any Bills playoff lose in the past 3 seasons.
#4. If I had to add another one it be Michigan losing to Louisville in the National Championship back in 2013.
Those are all the ones I lived through, and then they are games I wasn't around for that as a fan suck but that would be a lot longer of list.
Are you not a fan of the Cavaliers? Why would that be a heartbreaking moment? It's heartbreaking to me because the championship centricity of the NBA causes that game to take the greatest team of all time from us, but why would it be heartbreaking to you?
Funnily enough, I was actually rooting for the Patriots in that 2014 Super Bowl, because that Seahawks team (and their fans) had become SO unlikeable at the time that even the Patriots seemed like a better alternative, so when New England snuck out of there with two late fourth quarter touchdowns to win a game they had no business winning, that actually made me very happy. The NFL community was so insistent that those Hawks were going to be a dynasty that I can't say that I wasn't happy when they fell well short of that.
If the 2018 Saints would've won the NFC Championship, they absolutely would've won the Super Bowl. A lot of the weak league era was an exercise in the NFC cannibalising itself, and the Patriots/Broncos taking on (and beating) a weak team in the Super Bowl. It would've been nice to see Drew Brees get another championship, after spending so much of a career spent on bad teams.
Despite living in Northeast Ohio my whole life, I've never been much of a Cavs fan. I became a huge Warriors fan during the 2015 season and rooted for them big time until they won their 4th championship of the Steph era. I was also rooting against Lebron at the time because I was a fan of his when he was with the Heat and cried when he came back to Cleveland (I know it is weird but I was like eight years old at the time so).
Hi Robbie, Remind me again why EPA per Play isn't set to 0? I tracked down your previous Jaguars article, which informed me the league average was half of their -0.192.
Regardless, my concern about the validity of this index is that thus center might be somewhat arbitrary but play an enormous role in the size of the ratio. I would need to understand what the center means in order to evaluate the validity of the ratio. Is it possible the distance from league average is a better gauge than the ratio?
The ratio is just a scaled distance from league average, but not in exact EPA/Play terms.
Robbie Marriage the stat inventor and Robbie Marriage the Substack writer are two wholly different people. I invented this stat many years ago, never intending anybody to see it, never intending to use it anywhere, and never looking to poke holes in its conception. At the same time, through all the years that have passed, I had entirely forgotten the exact specifics of the formula's specification. I lost the Excel spreadsheet containing what my formulas actually were, so I cannot even update the stat for the 2024 season.
Just for you though, I have managed to reverse-engineer my process, and it's not that much different than you said. I'll begin from the top. The formula is based upon the deviation between every defence in the NFL and the worst, with the added constant you were mentioning being the worst EPA/Play in the league again. This leaves the worst team in the NFL at defence with its own EPA/Play Allowed as the supposed value of its deviation, with every team in the league after that getting its actual deviation, plus the worst EPA/Play Allowed in the league. You're entirely correct that this addition on the end of every season is entirely arbitrary, is different for every season, and could've been done any way I selected, but it's not wholly conceptually meaningless. What structuring the additive constant like this was is meant to do is make the formula ruthlessly punish bad defences in good defensive environments, while going a little easier on bad defences in great offensive environments, which it does accomplish quite well.
This creates a measure of deviation between every team and the worst, scaled in EPA/Play. The '+' comes from comparing every team in the league's deviation from the worst possible team to the average deviation from the worst possible team. For instance, in 2017, the average deviation from the league's worst EPA/Play, centred around the league's worst EPA/Play like I discussed, is the Tennessee Titans, with their 0.16 flat. The Jacksonville Jaguars have 0.326 (0.259 points per play of deviation of their own, plus 0.067 more from the league worst defence of the Indianapolis Colts). Dividing these two things is where the 204 comes from.
The main criticism this formula has gotten in the past (which I suppose is the same criticism as I've gotten here, phrased in a different way) has actually been the insistence on measuring via deviation from the bottom. This is what causes it to like the 2017 Jacksonville Jaguars (a team that allowed -0.192 EPA/Play in a league where the average was -0.02) better than the 2019 New England Patriots (a team who allowed -0.217 in a league where average was 0). The reason for this is because it was so much easier to put deviations on the 2019 Miami Dolphins (0.125 EPA/Play, worst in the NFL) than it was to put them on the 2017 Indianapolis Colts (0.067 EPA/Play, worst in the NFL).
The system was fundamentally designed to reward deviations more in generally better defensive environments (where it's harder to create them) than to reward deviations in generally better offensive environments. Because most teams in 2019 had quite the deviation on the 2019 Miami Dolphins, the Patriots' staggering 0.337 EPA/Play of deviation looks a lot less impressive than the 2017 Jaguars' 0.259 points' worth.
Again, is this conceptually correct? I don't know. I suppose it's no different than the arbitrary constants in the WAR function or the Passer Rating function. I don't think it's a horrendous formula. It can tell you that the 2000 Baltimore Ravens (177) were slightly less impressive in their defensive environment than the 2013 Seahawks (179) were in theirs, despite being 0.07 EPA/Play points better in the raw data. I think where I erred is not necessarily the formula itself (which has some arbitrary bits, every formula does) but in the interpretation, and in explaining that interpretation, something not often thought about when designing formulas as young as I was then.
Looking back through how I designed these numbers, it should not be interpreted like it's ERA+. Jacksonville's 204 in 2017 does not mean the average EPA/Play that year was -0.09. What it means, in as literal of terms as I can put it, is that Jacksonville was twice as good as the league's average team (Tennessee) at putting distance between itself and the worst team in the NFL, in terms of defensive performance.
You're correct that you can change the constant to a million if you want to. Then the interpretation would become that the Jaguars were 1.00001 times better than the average team in the league at putting distance between themselves and a one million EPA/Play defence, because (obviously) every team would be all-time great at that.
Perhaps you're not entirely satisfied with the way I formulated this statistic, or what its actual meaning is, but I hope I've at least been able to explain what I've been talking about here. I use shorthand in saying that it should be interpreted like EPA+, because for the layman it probably just should be, to avoid all this stuff, but that is not its actual statistical interpretation. I've just explained what its actual statistical interpretation is. At least I hope I have.
My goodness, I hope I did not manage to be as accusatory as you suggest in just two or three sentences! I cannot imagine how I set off so many insecurities, but the positive is that it signals that you agonize to get your work right. I trust your analytical judgment, and think towards the end, with that comment you made about using the average Titans as the reference, I think I have the gist. Before, it sort of sounded like everything was just relative to the worst team, which would not be a reliable measure of the league and by extension of other teams, but that is obviously not what you did.
One thing that is interesting about football now is we have few big plays and few turnovers. That would seem to bring about few big shifts in EPA. It does seem like this is something that probably should be adjusted for in evaluating how great or terrible an offense or defense is. The answer to the best and worst EPA might be different if we were asking which was the performance that gave the highest probability of winning/losing, comparing one score amidst league data to another. It seems that high and low variance environments might differ in how conducive they are to dominance, but that is a kind of bias when you ask how good teams were in the abstract. In any event, in what I take to be our-low variance environment, we still certainly managed plenty of bad teams this year, by record at least.
You asked me a question that I had no answer for, about a statistic that I'd created long ago, that I had entirely forgotten the meaning of. Despite this, I had become way too full of my own shit on it, as one would say, never expecting anybody to question it. Once somebody did, only then did I realise I had no idea what this statistic meant, and totally panicked that I may have written complete gibberish in here, and in several articles in the past. That's the opposite of what I want to do here.
Insecurity is a good word to use for it. I never want to print incorrect things, so I spent all my mid-afternoon yesterday digging into what this stat actually is and what it meant, so I won't take that risk again. It took hours, but it was probably worth it in the end. If I can't even explain what this stat is to somebody else, I ought not to use it myself, but I've used it several times already, just resting on my laurels I suppose. That's where the panic came in.
I didn't mean to phrase anything like you were an accuser, or my enemy in any way. My apologies!
The high vs low variance thing is sort of like the example I was discussing in the 2019 Patriots vs 2017 Jaguars. The defensive performance measure in particular that I've created really rewards dominance in low variance environments (like 2017) over dominance in high variance environments (like 2019). 2024 was truly a low variance environment, at least in terms of defensive performance. Perhaps this closeness amongst the teams is why there was no dominance this season. Not on the defensive side at least.
As far as the big play discussion, the metric most correlated with winning is straight defensive EPA. This makes sense. It's measured directly in points, and points directly affect wins. The issue with this is that the plays which most affect EPA are mostly luck based. By this I mean the ability to generate turnovers is not correlated at all with performance on non-turnover defensive plays.
Therefore, including any type of provision that boosts teams extra, due to their ability to create big plays on defence, is inherently rewarding something that isn't skill. The best defences do not generate the most turnovers, although they end up with the best EPA/Play figures oftentimes. This is why it's so difficult to disentangle which defences are actually good sometimes. For instance, the 2024 Packers were fantastic at generating defensive splash plays. They were not particularly good at defence on plays that were not defensive splash plays.
Does this make them a very good defence? It depends on who you ask. EPA/Play thinks so. It ranks them fourth in the league this year, but in terms of defensive success rate (my preferred metric), they are on an island they don't belong, with their 44.7% success rate allowed (21st) translating into the fourth best EPA/Play in the league. To me, a statistical profile like this screams out to me that this is a defence reliant on turnovers and nothing else, which is a strategy doomed to fail once placed against better competition, who will not turn the ball over as much.
Many people choose to just ignore turnovers entirely, or artificially make every team's turnover rate equal when grading defensive performance. I think something like this is what you were getting at. An attempt to net out all the big swings to get at which team is best on a play-by-play basis. I can see the conceptual argument for this, but it requires an argument that turnovers are fundamentally the result of the offence, not the defence.
I haven't swam in those waters yet. Perhaps if I go back to inventing football statistics, I may give something like that a try.
Lots of interesting threads here. You are certainly being hard on yourself, but I assumed you made your extended forway into the index by choice and for your own good, not so much in specific response to what I said or out of the obligation to answer a question.
It's interesting that 2017 and 2019 were fundemantally different in defensive EPA standard deviation. If we don't have a great reason for that, if we can't trace it to rule changes or individual difference in summary stats ( i.e., like percentage of big plays, percentag of sacks,how often teams went on 4th downs), maybe we have to wonder what that was genuine or luck? And if luck, whether we should be leaning on it for a ranking?
Certainly also true that there could be a difference in a year's relative rank for EPA per Play standard deviation, looking play by play without distinction of teams, and a year's EPA per Play standard deviation using team averages. I'd be surprised if the two aren't related, but they are distinct.
Regarding the meaning of different stats and importance of offense/defense, I was intrigued that in one Football Outsiders table they showed scrimmage yard differential from season to season holding up better than anything else. It outperformed DVOA, although with a higher correlation of .01 or .02 or something.
I think this is where the utility of series success rate (or drive success rate) comes in, which is just how often teams convert a set of downs into a first down. It has a strong relation to points per drive, isn't heavily impacted by big plays or turnovers, and takes special teams out of the equation. It is also extremely easy to convert into points per drive (Credit to T Troy. Russell).
The Packers gave up a SSR of 69.57% which comes out to an Expected Points Per Drive of 1.97 (mean 2.07, highest 2.97). The 2017 Jags EPPD was 1.03 (mean 1.74, highest 2.18), and the 2019 Patriots EPPD was 1.11 (mean 2, highest 2.64).
I've never heard of this conversion process Marc. What is it?
Also, where can I find series success rate data for export? I thought I had datasets for everything saved on this computer, but I suppose I've never used that stat for anything before. I'd like to look at it if you could lead me to an easily exportable table.
The only thing I'd think about in terms of series success rate is it's almost certainly highly correlated with offensive environment, much more than basic EPA, and therefore would be of limited use comparing across seasons unless you era adjust. Such a process wouldn't be too hard I suppose, but when putting it into a Twitter graphic or something, I don't think series success rate would work too well.
I'm saying all this as somebody who hasn't even looked at the series yet, so perhaps I'm wrong, but I don't think I will be.
I unfortunately have not found a site that has the data for exporting. However, since the only components are first downs, drives, and touchdowns it is really easy to calculate. I'm going to be building my own full dataset soon so if you me to share my sheet I can (unless you build your own).
The conversion is done by calculating and adding touchdown probability and field goal probability. TD probability = % chance of series conversion ^ (yards from end zone / successful series length). Field goal probability is more complex, so I'll point you to the article which provides the numbers, the rationale behind the equations, and an adjustment for league environment (https://www.stampedeblue.com/2019/8/12/18256932)
I think because series success rate can be explained in one sentence, I think it could work well for mass communication, but I haven't tried it so I can't say for sure
Thanks for sharing not only the Jags story, but your story. I think the fact that this was a personal experience made it easier to empathize with you and other Jags fans. While not within the same context, I think many sports fans have felt similarly at one point or another, including myself.
On a lighter note, where do you place this Jags team amongst other teams of this era (was thinking 14-18 but there may be another timeframe that makes more sense)?
Thank you very much Marc! I voiced my concerns at the beginning that this wouldn't come out very well. Those concerns were very real. Normally I end up not very proud of my Jaguars work, but I quite like the way this turned out. If you don't mind my asking, what are your most heartbreaking moments, in football and in sports in general? I'd love to hear somebody else tell their story of a heartbreaking defeat.
Onto the second question, I think 2014-2018 is the right choice to define the era, because that was after the Broncos and Seahawks had aged out of being truly great teams (although they were still competitive ones), but before the Chiefs and Ravens come in in 2019 to jazz the league back up again. This leaves a very weak league for the whole 2014-2018 period. The Patriots were the lucky ones to not fall apart as the whole league aged out around them.
That being said, this 2017 Jaguar team is why I will forever insist that 2017 is the best playoff run either Tom Brady or the New England Patriots ever had, despite not winning a championship at the end of it. I don't know why the Brady stans don't talk about this more, because in the 'weak league' era, as I've always taken to calling it, after prime Manning but before prime Mahomes, I view the Jaguars as the second best team, and even this comes with a caveat, because the team I think is better than them is the 2015 Cardinals, and this only applies BEFORE the Carson Palmer injury.
It's either the Jags, with their best defence of all time, but fringe top ten offence, or the Arizona Cardinals with their EPAA+ of 130 (6th in 2015), and by far the best offence in the league before Carson broke his finger (which Larry Fitzgerald to this day says is the reason he never won a Super Bowl, and I agree with him). It's razor thin between these two options, but they are the two options, as from these two it's a long way down to the second tier, which consists of 2015 Panthers, 2014 Packers, 2018 Saints, who are all pretty similar in terms of quality to each other.
I hate to give a copout answer, and I hate having to talk about the 2015 Cardinals in general because the conversation always ends in copout answers, but if I can reserve the right to Carson Palmer having ten healthy fingers, I would pick the 2015 Cardinals, in a close fight. However, if Carson getting injured right at the end of the season was destiny, and I can't prevent it, I would pick this 2017 Jaguars team over their next closest competition (2015 Panthers) ten times out of ten.
I think your articles are important for keeping the memory of the 15 Cardinals and 17 Jaguars alive as these are the type of teams that are often lost to history.
It was a little unclear to me but would you put these teams over any Patriots team of this era?
Thank you my friend. These teams are like the team version of Trent Green. In my opinion, they're among the greatest of all time, but for whatever reason the NFL community treats them as if they were merely good. It feels like myself (and yourself) against the world sometimes trying to get my point across, but if I don't do it, who will?
When I said these two teams were one and two, I meant one and two. No Patriots team is even in the top five best teams of this era. Like I was saying in the other comment, this whole era was an exercise in the NFC failing to send its best team to the Super Bowl. I'll give you my top ten list of the weak league era:
2015 Cardinals
2017 Jaguars
2015 Panthers
2014 Packers
2018 Saints
2015 Seahawks
2016 Patriots
2018 Chiefs
2016 Falcons
2017 Eagles
You'll notice two things. First is that the 2015 NFC was loaded, so loaded that they all did double knockouts with each other, allowing the AFC to win the SB that year, and second is that all the competition was in the NFC in this era. Seven out of the top ten teams are over there, and the team that won the Super Bowl three times in these five years (Patriots) make this list just once. This is primarily because the NFC sent their best team to the Super Bowl just twice (2016, 2017) in this era.
I respect the 2016 version of the Patriots, who beat the best teams the league had to offer, but the 2014 version barely got out of there alive against the NFC's third best team. The 2017 version ought to have lost the AFC Championship, and did lose the Super Bowl, and the 2018 version got to play the NFC's third best team again.
You can only beat who is in front of you, and they do deserve their championships, but to somebody like me who is not a championship centrist, beating no top five teams on the way to a Super Bowl does not qualify you as a great team. I don't care about the championship. It's just a title. The real great teams are the ones that the Patriots missed, like the 2014 Packers, like the 2018 Saints, both of whom lost one possession playoff games, due primarily to randomness.
In this game, there's nothing you can do about that, which is why losing in the playoffs DOES NOT disqualify you from being a great team. It does in basketball, but it doesn't in this game. To me, one possession playoff results mean almost nothing, because you constantly run into things like this 2017 AFC Championship, where if the correct call had been made on the field, the Patriots would've lost. Since it wasn't, they won. That's just randomness. It has nothing to do with team quality. That to me is why championship centrism doesn't work in football like it does in other sports.
It's why I don't mind leaving three of the era's five champions off my best teams list. People who give lots of weight to championships (i.e. one possession playoff game results) will view that as blasphemy, but I don't.
I thought I had recently was that in the NFL, the regular season separates the good teams from the bad teams while the postseason separates the lucky from the unlucky.
There's an economics principle (I think you know I'm pursuing a PhD in economics, but if you don't, I am) that states that the closer two agents converge in terms of ability, the higher the chances that their market outcomes will be decided by something that isn't ability. The NFL and NHL and MLB playoffs run headlong into this principle almost every single year. Matching up two evenly matched teams is great in terms of entertainment value, but in terms of deciding who the better team is, it's quite possibly the worst imaginable way to do so.
As long as we accept that the champion is not the best team most of the time, this is okay, but when you make the claim that the best team is the champion, because they're the champion, I entirely disagree. More often than not, it's just the way the wind is blowing.
I'm not wholly willing to go as far as you went Marc, because occasionally there are multi-possession wins in playoff games, especially as we go further back in history. I hate to be THAT GUY, but this does mean to me that championships in years gone by tend to be worth more than championships now, because talent spreads used to be wider, and you could be more confident that the best team was the champion.
I don't think there is such a thing as an undeserving champion, because what makes you a deserving champion in the NFL is to win a three round, single elimination tournament. Every champion in history has done that, but not all deserving champions are great teams. Very far from.
#1 moment heartbreaking sports moments has got be the Warriors losing game 7 of the NBA finals in 2016. It is now more bittersweet as they ended winning 3 more championships and my home town team got its only title, but to this day I still haven't the 4th quarter of that game, even as some who likes Lebron James (though not at the time).
#2 would be the Seahawks losing the SB to the Patriots and in general any Tom Brady/Patriots playoff win from 2014-2020. I'm not sure if this game was the origin on me rooting against Brady, or if the reason I was heartbroken was because I was rooting against Brady. Thinking back on it, I'm not entirely sure why I was rooting for the Seahawks since they had demolished my favorite player's team the year before but I was pretty young back then so I don't think my rooting interest had much continuity.
#3 Is probably the Saints losing to the Rams in the 2018 NFC championships. I could also go with any Bills playoff lose in the past 3 seasons.
#4. If I had to add another one it be Michigan losing to Louisville in the National Championship back in 2013.
Those are all the ones I lived through, and then they are games I wasn't around for that as a fan suck but that would be a lot longer of list.
Are you not a fan of the Cavaliers? Why would that be a heartbreaking moment? It's heartbreaking to me because the championship centricity of the NBA causes that game to take the greatest team of all time from us, but why would it be heartbreaking to you?
Funnily enough, I was actually rooting for the Patriots in that 2014 Super Bowl, because that Seahawks team (and their fans) had become SO unlikeable at the time that even the Patriots seemed like a better alternative, so when New England snuck out of there with two late fourth quarter touchdowns to win a game they had no business winning, that actually made me very happy. The NFL community was so insistent that those Hawks were going to be a dynasty that I can't say that I wasn't happy when they fell well short of that.
If the 2018 Saints would've won the NFC Championship, they absolutely would've won the Super Bowl. A lot of the weak league era was an exercise in the NFC cannibalising itself, and the Patriots/Broncos taking on (and beating) a weak team in the Super Bowl. It would've been nice to see Drew Brees get another championship, after spending so much of a career spent on bad teams.
Despite living in Northeast Ohio my whole life, I've never been much of a Cavs fan. I became a huge Warriors fan during the 2015 season and rooted for them big time until they won their 4th championship of the Steph era. I was also rooting against Lebron at the time because I was a fan of his when he was with the Heat and cried when he came back to Cleveland (I know it is weird but I was like eight years old at the time so).
Hi Robbie, Remind me again why EPA per Play isn't set to 0? I tracked down your previous Jaguars article, which informed me the league average was half of their -0.192.
Regardless, my concern about the validity of this index is that thus center might be somewhat arbitrary but play an enormous role in the size of the ratio. I would need to understand what the center means in order to evaluate the validity of the ratio. Is it possible the distance from league average is a better gauge than the ratio?
The ratio is just a scaled distance from league average, but not in exact EPA/Play terms.
Robbie Marriage the stat inventor and Robbie Marriage the Substack writer are two wholly different people. I invented this stat many years ago, never intending anybody to see it, never intending to use it anywhere, and never looking to poke holes in its conception. At the same time, through all the years that have passed, I had entirely forgotten the exact specifics of the formula's specification. I lost the Excel spreadsheet containing what my formulas actually were, so I cannot even update the stat for the 2024 season.
Just for you though, I have managed to reverse-engineer my process, and it's not that much different than you said. I'll begin from the top. The formula is based upon the deviation between every defence in the NFL and the worst, with the added constant you were mentioning being the worst EPA/Play in the league again. This leaves the worst team in the NFL at defence with its own EPA/Play Allowed as the supposed value of its deviation, with every team in the league after that getting its actual deviation, plus the worst EPA/Play Allowed in the league. You're entirely correct that this addition on the end of every season is entirely arbitrary, is different for every season, and could've been done any way I selected, but it's not wholly conceptually meaningless. What structuring the additive constant like this was is meant to do is make the formula ruthlessly punish bad defences in good defensive environments, while going a little easier on bad defences in great offensive environments, which it does accomplish quite well.
This creates a measure of deviation between every team and the worst, scaled in EPA/Play. The '+' comes from comparing every team in the league's deviation from the worst possible team to the average deviation from the worst possible team. For instance, in 2017, the average deviation from the league's worst EPA/Play, centred around the league's worst EPA/Play like I discussed, is the Tennessee Titans, with their 0.16 flat. The Jacksonville Jaguars have 0.326 (0.259 points per play of deviation of their own, plus 0.067 more from the league worst defence of the Indianapolis Colts). Dividing these two things is where the 204 comes from.
The main criticism this formula has gotten in the past (which I suppose is the same criticism as I've gotten here, phrased in a different way) has actually been the insistence on measuring via deviation from the bottom. This is what causes it to like the 2017 Jacksonville Jaguars (a team that allowed -0.192 EPA/Play in a league where the average was -0.02) better than the 2019 New England Patriots (a team who allowed -0.217 in a league where average was 0). The reason for this is because it was so much easier to put deviations on the 2019 Miami Dolphins (0.125 EPA/Play, worst in the NFL) than it was to put them on the 2017 Indianapolis Colts (0.067 EPA/Play, worst in the NFL).
The system was fundamentally designed to reward deviations more in generally better defensive environments (where it's harder to create them) than to reward deviations in generally better offensive environments. Because most teams in 2019 had quite the deviation on the 2019 Miami Dolphins, the Patriots' staggering 0.337 EPA/Play of deviation looks a lot less impressive than the 2017 Jaguars' 0.259 points' worth.
Again, is this conceptually correct? I don't know. I suppose it's no different than the arbitrary constants in the WAR function or the Passer Rating function. I don't think it's a horrendous formula. It can tell you that the 2000 Baltimore Ravens (177) were slightly less impressive in their defensive environment than the 2013 Seahawks (179) were in theirs, despite being 0.07 EPA/Play points better in the raw data. I think where I erred is not necessarily the formula itself (which has some arbitrary bits, every formula does) but in the interpretation, and in explaining that interpretation, something not often thought about when designing formulas as young as I was then.
Looking back through how I designed these numbers, it should not be interpreted like it's ERA+. Jacksonville's 204 in 2017 does not mean the average EPA/Play that year was -0.09. What it means, in as literal of terms as I can put it, is that Jacksonville was twice as good as the league's average team (Tennessee) at putting distance between itself and the worst team in the NFL, in terms of defensive performance.
You're correct that you can change the constant to a million if you want to. Then the interpretation would become that the Jaguars were 1.00001 times better than the average team in the league at putting distance between themselves and a one million EPA/Play defence, because (obviously) every team would be all-time great at that.
Perhaps you're not entirely satisfied with the way I formulated this statistic, or what its actual meaning is, but I hope I've at least been able to explain what I've been talking about here. I use shorthand in saying that it should be interpreted like EPA+, because for the layman it probably just should be, to avoid all this stuff, but that is not its actual statistical interpretation. I've just explained what its actual statistical interpretation is. At least I hope I have.
My goodness, I hope I did not manage to be as accusatory as you suggest in just two or three sentences! I cannot imagine how I set off so many insecurities, but the positive is that it signals that you agonize to get your work right. I trust your analytical judgment, and think towards the end, with that comment you made about using the average Titans as the reference, I think I have the gist. Before, it sort of sounded like everything was just relative to the worst team, which would not be a reliable measure of the league and by extension of other teams, but that is obviously not what you did.
One thing that is interesting about football now is we have few big plays and few turnovers. That would seem to bring about few big shifts in EPA. It does seem like this is something that probably should be adjusted for in evaluating how great or terrible an offense or defense is. The answer to the best and worst EPA might be different if we were asking which was the performance that gave the highest probability of winning/losing, comparing one score amidst league data to another. It seems that high and low variance environments might differ in how conducive they are to dominance, but that is a kind of bias when you ask how good teams were in the abstract. In any event, in what I take to be our-low variance environment, we still certainly managed plenty of bad teams this year, by record at least.
I'm sorry David.
You asked me a question that I had no answer for, about a statistic that I'd created long ago, that I had entirely forgotten the meaning of. Despite this, I had become way too full of my own shit on it, as one would say, never expecting anybody to question it. Once somebody did, only then did I realise I had no idea what this statistic meant, and totally panicked that I may have written complete gibberish in here, and in several articles in the past. That's the opposite of what I want to do here.
Insecurity is a good word to use for it. I never want to print incorrect things, so I spent all my mid-afternoon yesterday digging into what this stat actually is and what it meant, so I won't take that risk again. It took hours, but it was probably worth it in the end. If I can't even explain what this stat is to somebody else, I ought not to use it myself, but I've used it several times already, just resting on my laurels I suppose. That's where the panic came in.
I didn't mean to phrase anything like you were an accuser, or my enemy in any way. My apologies!
The high vs low variance thing is sort of like the example I was discussing in the 2019 Patriots vs 2017 Jaguars. The defensive performance measure in particular that I've created really rewards dominance in low variance environments (like 2017) over dominance in high variance environments (like 2019). 2024 was truly a low variance environment, at least in terms of defensive performance. Perhaps this closeness amongst the teams is why there was no dominance this season. Not on the defensive side at least.
As far as the big play discussion, the metric most correlated with winning is straight defensive EPA. This makes sense. It's measured directly in points, and points directly affect wins. The issue with this is that the plays which most affect EPA are mostly luck based. By this I mean the ability to generate turnovers is not correlated at all with performance on non-turnover defensive plays.
Therefore, including any type of provision that boosts teams extra, due to their ability to create big plays on defence, is inherently rewarding something that isn't skill. The best defences do not generate the most turnovers, although they end up with the best EPA/Play figures oftentimes. This is why it's so difficult to disentangle which defences are actually good sometimes. For instance, the 2024 Packers were fantastic at generating defensive splash plays. They were not particularly good at defence on plays that were not defensive splash plays.
Does this make them a very good defence? It depends on who you ask. EPA/Play thinks so. It ranks them fourth in the league this year, but in terms of defensive success rate (my preferred metric), they are on an island they don't belong, with their 44.7% success rate allowed (21st) translating into the fourth best EPA/Play in the league. To me, a statistical profile like this screams out to me that this is a defence reliant on turnovers and nothing else, which is a strategy doomed to fail once placed against better competition, who will not turn the ball over as much.
Many people choose to just ignore turnovers entirely, or artificially make every team's turnover rate equal when grading defensive performance. I think something like this is what you were getting at. An attempt to net out all the big swings to get at which team is best on a play-by-play basis. I can see the conceptual argument for this, but it requires an argument that turnovers are fundamentally the result of the offence, not the defence.
I haven't swam in those waters yet. Perhaps if I go back to inventing football statistics, I may give something like that a try.
Lots of interesting threads here. You are certainly being hard on yourself, but I assumed you made your extended forway into the index by choice and for your own good, not so much in specific response to what I said or out of the obligation to answer a question.
It's interesting that 2017 and 2019 were fundemantally different in defensive EPA standard deviation. If we don't have a great reason for that, if we can't trace it to rule changes or individual difference in summary stats ( i.e., like percentage of big plays, percentag of sacks,how often teams went on 4th downs), maybe we have to wonder what that was genuine or luck? And if luck, whether we should be leaning on it for a ranking?
Certainly also true that there could be a difference in a year's relative rank for EPA per Play standard deviation, looking play by play without distinction of teams, and a year's EPA per Play standard deviation using team averages. I'd be surprised if the two aren't related, but they are distinct.
Regarding the meaning of different stats and importance of offense/defense, I was intrigued that in one Football Outsiders table they showed scrimmage yard differential from season to season holding up better than anything else. It outperformed DVOA, although with a higher correlation of .01 or .02 or something.
I think this is where the utility of series success rate (or drive success rate) comes in, which is just how often teams convert a set of downs into a first down. It has a strong relation to points per drive, isn't heavily impacted by big plays or turnovers, and takes special teams out of the equation. It is also extremely easy to convert into points per drive (Credit to T Troy. Russell).
The Packers gave up a SSR of 69.57% which comes out to an Expected Points Per Drive of 1.97 (mean 2.07, highest 2.97). The 2017 Jags EPPD was 1.03 (mean 1.74, highest 2.18), and the 2019 Patriots EPPD was 1.11 (mean 2, highest 2.64).
I've never heard of this conversion process Marc. What is it?
Also, where can I find series success rate data for export? I thought I had datasets for everything saved on this computer, but I suppose I've never used that stat for anything before. I'd like to look at it if you could lead me to an easily exportable table.
The only thing I'd think about in terms of series success rate is it's almost certainly highly correlated with offensive environment, much more than basic EPA, and therefore would be of limited use comparing across seasons unless you era adjust. Such a process wouldn't be too hard I suppose, but when putting it into a Twitter graphic or something, I don't think series success rate would work too well.
I'm saying all this as somebody who hasn't even looked at the series yet, so perhaps I'm wrong, but I don't think I will be.
I unfortunately have not found a site that has the data for exporting. However, since the only components are first downs, drives, and touchdowns it is really easy to calculate. I'm going to be building my own full dataset soon so if you me to share my sheet I can (unless you build your own).
The conversion is done by calculating and adding touchdown probability and field goal probability. TD probability = % chance of series conversion ^ (yards from end zone / successful series length). Field goal probability is more complex, so I'll point you to the article which provides the numbers, the rationale behind the equations, and an adjustment for league environment (https://www.stampedeblue.com/2019/8/12/18256932)
I think because series success rate can be explained in one sentence, I think it could work well for mass communication, but I haven't tried it so I can't say for sure