Introducing TEP: Turnover Equivalent Plays

TEP is a stat that accounts for all mistakes a player makes in order to give a better measure of mistake proneness than regular turnovers.

(If you need a primer on EPA, click here to read my beginner's guide. Enjoy!)

In the last post I talked extensively about how turnover statistics are overrated, and how if you're using turnovers as a measure of how good or bad a QB or team is, you're likely doing it wrong. I promised at the end of that article I would explain a better way to judge QBs if turnovers weren't going to be acceptable. Here I am.

My statistical pet project is titled Turnover Equivalent Plays, or TEP for short. Allow me to explain it to you.

First of all, the theoretical basis of this statistic borrows heavily from that of another statistic that's widely used in the football world, called Adjusted Net Yards per Attempt (abbreviated from here on as ANY/A). If you've read anything I've ever written, you'll have heard of this statistic, and it's generally accepted as one of the best stats out there to grade QBs with. It's formula goes like this:

You may note that (despite its name) this stat is not measured in terms of yards, nor is it measured per pass attempt. Nevertheless, it's very good for a stat designed to be able to cover all eras of NFL history. If you wish to analyse players from the 1960s or 1970s, I wouldn't recommend any stat above this one.

It's not the 1970s anymore.

I respect what ANY/A is going for, but with the benefit of hindsight, we can see its one crucial flaw. It (like many other things in the football world) dramatically overvalues turnovers. May I ask you why it takes two and a quarter touchdowns to make up for just one interception? The worst any team can do is score a touchdown off a turnover, so shouldn't you be square after one touchdown of your own?

I act like I don't know the answer to this purely for the purpose of the bit. I do know the answer. You see, this stat was designed to be a hybrid team-QB stat. These are the weights that maximised correlation with team points scored back in the day when the stat was designed.

Fortunately, we now have a stat that correlates even better with team points scored. It's called Expected Points Added (EPA), and I'm going to use it to create a better measure of QB performance.

When people think of mistake prone QBs, they think of galling interceptions. A turnover is indeed the worst kind of negative play a QB can make, but it's not the only one. There are many many more mistakes a player can make which determine their value. This is evidenced by the following list. These are the NFL's best ten players at avoiding turnovers on a per play basis in 2023.

1. Kenny Pickett: 4 turnovers in 392 touches

2. Tyrod Taylor: 3 turnovers in 244 touches

3. Justin Herbert: 8 turnovers in 588 touches

4. Kyler Murray: 5 turnovers in 341 touches

5. CJ Stroud: 9 turnovers in 594 touches

6. Dak Prescott: 11 turnovers in 719 touches

7. Tommy DeVito: 4 turnovers in 255 touches

8. Joe Burrow: 7 turnovers in 426 touches

9. Derek Carr: 11 turnovers in 612 touches

10. Matthew Stafford: 11 turnovers in 594 touches

There are some good names on this list, but it's about the worst top ten QBs list I've ever seen. Dak Prescott is a good name to have on any top ten QBs of 2023 left, but Kenny Pickett is a yikes, and Tommy DeVito? It's beyond words.

Clearly, simply avoiding interceptions is not a good barometer of QB play. It's not even a good barometer of how mistake prone or not a QB is. I'll give you an example: who is more mistake prone, Zach Wilson or Trevor Lawrence?

Obviously, the correct answer is Trevor Lawrence?

Wait, what?

I don't think it's a provocative statement to say that Trevor Lawrence is most definitely not more mistake prone than Zach Wilson, but strictly in terms of avoiding turnovers, Zach is better. In fact, in terms of turnovers per play, there's a few galling names that Zach Wilson is rubbing shoulders with, up to and including Josh Allen.

Yup. The Josh Allen. The same one who's been in the top five of my QB tier list each of the last four seasons. If we're talking strictly about avoiding turnovers, he's no better than Zach Wilson. I can give more examples like this, but what more evidence do you need that turnovers are just not that important of an indicator?

I'm going to rectify this problem by fixing the key flaw the ANY/A stat has. By not weighting sack yards at all, and weighting each interception by a factor of 45, taking into account that sacks lose 6.5 yards on average, this means that the ANY/A formula believes that turning the ball over is approximately seven times worse than getting sacked.

I'll ask you a question my dear reader: How bad do you think sacks are, relative to turnovers?

Go ahead and think of an answer. I'll wait.

The answer is 2.4. On average, a turnover costs a team -4.5 EPA. The average sack costs a team -1.856, meaning that the average sack is just 2.4 times worse than the average turnover. If a QB gets sacked three times, he's already cost his team more value than a turnover would have.

When people see turnovers and freak out, the reason is simple. Turnovers end drives, and when the other team has the ball you don't have the ball, and thusly are very unlikely to score. What is missed in that line of thinking is that sacks are drive killers too.

At any given time, a sack reduces the probability of getting a first down by 30 percent. Allow this to happen often and you're really beginning to hamstring your team. Remember the Russell Wilson Seahawks? I'm sure many people were thinking to themselves throughout that entire era why they didn't score points like the elite offences despite (surface level) elite QB play. The answer is that Russ constantly would get sacked on eight percent of his drop backs, which would force passing situations and cause punts.

I'm going to use two players as examples, for no reason at all. Their names are Patrick Mahomes, and Josh Allen. In terms of turnover avoidance, neither are anything special. In 2023, Patrick ranked 20th and Josh 31st. You would think with these numbers that their offences would underperform in a similar way, but they don't. Why?

These two are both elite at pocket presence, and are constantly getting the ball out of their hands, avoiding the 30% hit you take to your chances of a first down (and therefore scoring) whenever a sack happens. Not getting sacked is a skill, and it's the reason these two have been elite for so long.

Therefore, I present you a new stat. One that weights turnovers and sacks properly against each other. I also choose to put mine over all the plays, not just pass drop backs, because rushing attempts are chances to turn the ball over too:

Here is the leaderboard for avoiding mistakes on a per play basis once sacks are properly accounted for:

1. Kenny Pickett: 4 turnovers and 23 sacks in 392 plays

2. Justin Herbert: 8 turnovers and 29 sacks in 558 plays

3. Kyler Murray: 5 turnovers and 18 sacks in 341 plays

4. Dak Prescott: 11 turnovers and 39 sacks in 719 plays

5. Derek Carr: 11 turnovers and 31 sacks in 612 plays

6. Jordan Love: 14 turnovers and 30 sacks in 676 plays

7. Matthew Stafford: 11 turnovers and 30 sacks in 594 plays

8. Patrick Mahomes: 17 turnovers and 27 sacks in 716 plays

9. Joe Burrow: 7 turnovers and 24 sacks in 426 plays

10. Tyrod Taylor: 3 turnovers and 17 sacks in 244 plays

You see a lot of names in common here with the turnover avoidance list, because avoiding turnovers is a big part of avoiding mistakes, but the players who are elite at avoiding turnovers but quite bad at avoiding sacks (CJ Stroud and Tommy DeVito) are gone, replaced with players who are worse at avoiding turnovers, but better at avoiding mistakes overall (Patrick Mahomes and Jordan Love).

This is a much better list of the most mistake prone QBs in the game, but it's still not fool proof. Kenny Pickett being at the top is still not a great look for any QB stat. If anything it just gives more credence to the Josh Allen Theorem I talked about before. Kenny didn't just avoid mistakes in a showy, surface level way like CJ Stroud. He's actually up there, and he was still terrible at the position.

There must be some way to account for this. In fact, there is. This got me thinking of the folks over at Puntalytics and their concept of PUPTO (PUnts Plus TurnOvers).

It's extremely simple. Punts are turnovers too. They're just as much of a drive killer as any conventional turnover. They don't hurt your chances of scoring as badly (already the chances quite low if you've gotten to fourth down), but they still ought to be avoided at all costs, just like sacks and turnovers.

When you think of punts as turnovers, that gives me one more type of mistake I must account for. Plays that directly cause punts: third down failures. Moving from third down to fourth on average costs a team -1.3 EPA. This is not as bad as a sack or turnover, but it's bad enough that it should be accounted for in an analysis like this. It only takes roughly 3.4 of these to get to the same negative value of a turnover.

The final Turnover Equivalent Plays (TEP) formula includes properly weighted turnovers, sacks, and third down incompletions (subtract third down interceptions to prevent double counting) to give a more correct measure of just how mistake prone every QB in the NFL is. Divide it by total touches to generate TEP/Play:

These are the top ten QBs in the NFL in 2023 in order of TEP/Play:

1. Kyler Murray: 18.59 TEP in 341 plays

2. Dak Prescott: 39.41 TEP in 719 plays

3. Joe Burrow: 25.12 TEP in 426 plays

4. Patrick Mahomes: 42.82 TEP in 716 plays

5. Jalen Hurts: 45.71 TEP in 752 plays

6. Lamar Jackson: 41.47 TEP in 671 plays

7. Justin Herbert: 34.94 TEP in 558 plays

8. Jared Goff: 41.88 TEP in 668 plays

9. Josh Allen: 46.88 TEP in 742 plays

10. Matthew Stafford: 37.76 TEP in 594 plays

This is a much better top ten QBs list. You can argue about the order, but most of the right names are here.

This is the big list I've been wanting to get to, because this is the point I've been wanting to prove. Turnover Equivalent Plays really should be called the Jalen Hurts statistic, because once sack avoidance and third down performance are accounted for, Jalen parachutes in from his 28th place ranking in turnovers per touch to be one of the NFL's five least mistake prone QBs for the second year running.

Josh Allen also makes an appearance. Did you figure at the top of this article that it would be finishing with me telling you Josh Allen is the ninth best QB in the league at not making mistakes, despite 22 total turnovers? What they don't tell you when they're complaining about how many turnovers he commits is that Josh Allen is the NFL's very best at avoiding drive killing mistakes that aren't turnovers, which is an immensely valuable, and criminally underrated part of Josh's game.

Remember when I said that Zach Wilson and Josh Allen were roughly the same at pure turnover avoidance? When other ways of making mistakes are accounted for, here are the ten most mistake prone QBs in the NFL in 2023:

33. Bryce Young: 58.59 TEP in 660 plays

34. Desmond Ridder: 43.24 TEP in 482 plays

35. Trevor Siemian: 16.47 TEP in 182 plays

36. Ryan Tannehill: 26.65 TEP in 293 plays

37. Will Levis: 28.94 TEP in 313 plays

38. Daniel Jones: 22.29 TEP in 240 plays

39. Jimmy Garoppolo: 19.47 TEP in 206 plays

40. Tommy DeVito: 24.24 TEP in 255 plays

41. Zach Wilson: 45 TEP in 464 plays

42. Bailey Zappe: 28.82 TEP in 258 plays

Oh how the mighty can fall. Recall Tommy DeVito ranked seventh in turnover prevention. He fared so badly at avoiding all other mistakes that he ranks as the 3rd most mistake prone QB in the league once all is said and done. In terms of turnover prevention, Zach Wilson is rubbing shoulders with Josh Allen, but this list proves he's not close in any other facet of the game.

I've got love for Daniel Jones, but I must admit this is a pretty good worst ten QBs in the NFL list for last season.

Now that I'm mostly done pitching my new stat to you, I must admit there are some pitfalls to it. First and most obvious is that it includes no provisions for anything positive. At all. Nobody gets any credit for doing anything well, only not doing things badly. You can be a great NFL QB without being all that great at avoiding mistakes, evidenced by the fact that QBs number one (Brock Purdy), four (Tua Tagovailoa), eight (Jordan Love) and ten (Baker Mayfield) in my 2023 QB tier list make no appearance in this article at all until now.

Just because none of those four are particularly good at avoiding mistakes doesn't mean they aren't good at football. Quite the opposite. These players must overcome their quite frequent mistakes with big positive plays in order to be as good as they are. This is more difficult than just playing like the great QBs who don't make as many frequent mistakes, making what they do, in a way, even more impressive.

TEP is a stat that measures how mistake prone particular QBs are. It doesn't do anything else. Use it as such and I think it's a great statistic and will help you stay ahead of the curve in your football discussions.

Thank you very much for reading my article! I hope you found it interesting and perhaps use this stat going forward. Is there anything you found surprising? Let me know.