Some questions for you and a stray thought or opinion here and there, if you have the time.
Could you please explain your selection of defining the bad quarterbacks in terms of CPOE and not EPA? Not that the lists would diverge that much, but I'm curious. Is EPA looked upon as the bottom line, but CPA more predictive?
So, I get confused. What variables does the probability of completion for a pass incorporate? Distance downfield, plus lateral distance? Whether the quarterback is on the run or not? Does it take into account the distance between the receiver and the defender at the time the ball is thrown?
One advantage of EPA, in terms of measuring a quarterback's worth, it would seem to me, is it rewards how much a quarterback sees, and where he chooses to throw the ball, not just how well he does when he does throw the ball. EPA implicitly penalizes for omissions (or, I suppose, compares to an average rate of omission).
If CPOE does take in more than just how far downfield the pass is, I can see that it might yield some interesting results, in that a pass could have limited potential reward, but show up as just as skilled a play as one that went well downfield. Not because the throw downfield was one that the quarterback should have completed (although that could be true, too, if it were a blown coverage), but because the short throw vertically still covered a lot of distance, or was to a closely covered guy. This is sort of the equivalent of when the announcer says, "That was the best two-yard run I ever saw," which is sometimes a reasonable statement. There can be skill in short plays, and things we can glean from them. So, maybe that's why CPOE is more predictive than EPA, if it is.
I hadn't focused on just how many of our top quarterbacks received limiting playing time their rookie year. I do find that hard to combat, as an argument that sitting on the bench isn't hurting young quarterbacks, and may even be helping them.
I like Justin Fields' talent, but my first reaction is I don't agree with your logic in saying the Bears should have stayed with him. Why is everything about 2024? Would they have won the Super Bowl with Fields? Don't Luck, Goff, Stafford, and Burrow indicate that there is longitudinal progress with quarterbacks, and Williams could well turn out to be a better player than Fields, even though his first two weeks have been rocky? And I know about Bryce Young and Zach Wilson and Trey Lance, but I think it just stands to reason that the higher you're picked in the first round, the more likely you are to succeed. It might not be true, but it probably is, and if that hasn't turned out over a period of years, it probably only means that we haven't had sufficient sample size to see the opposite, although the effect is probably weak.
Hey David. How are you doing tonight? I hope you're well. Okay. Several questions. I'll attempt to answer in order.
1) EPA is strictly a measure of results, measured in terms of points (I have written an article on what EPA is, but I think I've gleamed from your comment that you know what it is). Results on a player level (even at a position as central as the QB) are very highly dependent on the environment surrounding a player in this game. It's not like baseball where you can have great players on bad teams. Football isn't really like that. A great player on a bad team will not put up bad results, but he likely will merely put up okay results. Therefore, results are not the best way to forecast individual players going forward (this is particularly true when changing teams, although that's not relevant in this case), which is what I was trying to do.
The best way to do that at this position is Completion Percentage Over Expected (CPOE), which is the superior measure of throw accuracy, and as close to a measurement of actual talent at this position you're going to get. What factors it takes into account vary with the expected completion percentage model you choose to use. I elect to use NFLFastR's version, which takes into account only air distance, and direction to determine the expected completion percentage.
I dislike the NGS version because it takes into account distance between receiver and defender when calculating expected completion percentage, which tends to reward arm talent guys who force balls into coverage, and penalize QBs (like Brock Purdy and Patrick Mahomes) who throw what the model considers 'easy' passes because they're constantly able to find and target receivers who are actually open (very far from defenders), and I don't like that. Player tracking does not yet exist at the level to determine whether a pass was thrown on the run or not or anything like that.
The main reason I use CPOE in this discussion is that it is the one of two stats that tend to persist (with the other being sack rate, which is what 'sacks are a QB stat' means). The season-over-season correlation between players' CPOE values is 0.44 if I remember correctly. That of actual results is less than half that, if my memory serves. Therefore, the definition of a bad QB is not necessarily one with bad results, but one with a very low CPOE.
This is sort of like baseball, in that the best measure of quality looking backwards is EPA/Play, but the best measure of quality looking forwards is CPOE, sort of like the difference between ERA and FIP. Much like ERA and FIP one being bad almost always means the other is bad, but not always. Think of a high ERA, low FIP pitcher. You would know the examples better than I, but the NFL version of this is Sam Darnold, who'd been a low EPA/Play, high CPOE guy, and once finally freed from the NYJ that talent is finally shining through. It in fact was quite easily predictable.
That was long winded, but I hope explains why a low CPOE is indicative of a bad player, much moreso than the actual results.
2) I have no understanding of why NFL teams insist on continuing to play these rookie QBs. They're almost always awful, and as I explained in the article and you've mentioned, just two of the NFL's top ten QBs last year started out of the gate as rookies, one of whom being the miracle fourth round Dak Prescott turning out to be the best rookie QB of all time, and the other being Josh Allen, who was horrendous in his rookie season.
To me, choosing to start a rookie QB is tantamount to tanking, which is fine, but gets a little less fine when it's being done by five teams across the league, and even less fine than that when it's being done by a team like Chicago, who are pretty good except for the anchor they've insisted on wearing around their own necks. That's when it begins to become a problem of a lot of games being non-competitive in a league where there only 16 games in a week.
3) I just went over that starting a rookie QB week one appears to have no effect at all on his future development, as most of the top guys in the league spent week one on the bench. Anecdotally, I think forcing a rookie to go through a really bad first season is actually a hinderance, although I have generated no data at all to support that hypothesis. Looking at the recent examples, it certainly didn't help Bryce Young, and I would argue it helped CJ Stroud only in that there is no QB controversy now, when if the Texans had a competent (non-rookie) QB guiding that offence to be the Super Bowl contender it should've been last year, there certainly would be.
Which brings us to the Bears. Now that we all understand that starting a rookie QB neither helps nor hinders QB development, why would you ever sell low on Justin Fields in order to make your team worse this year? Not everything needs to be about 2024, and my apologies if I made it seem that way, but this move sends a clear message to players and fans that nothing is about 2024, which is not correct either, especially when you're Chicago, a team long settled into a culture of losing.
We've seen it already. If the Chicago Bears had Justin Fields right now they'd be 2-0 without question, and likely would've made the playoffs this year, hopefully boosting the trade value of Fields beyond a sixth round pick, but definitely getting this organisation back used to winning or at least competing. Instead they've sold low on their former first round pick for the privilege of being terribly lucky to be 1-1, and a very long shot to make the playoffs. If there were any benefit to Caleb for doing this I would understand, but the data seems to show that there just isn't any.
Even if Caleb is going to grow to be a better player than Justin, the Alex Smith-Patrick Mahomes case shows that it's still better to keep the rookie on the bench so the team can compete right now. If the Chiefs had tried to play Patrick Mahomes he would've been poor (because all rookies are, nothing at all against Patrick), and the 2017 Chiefs team would've missed the playoffs, and what would that have gotten anybody? The season was all about Patrick's development anyway, but it didn't have to happen on the field in the games. The same goes with Brett Favre-Aaron Rodgers or Drew Bledsoe-Tom Brady or Doug Flutie-Drew Brees.
So we can say that we don't believe starting week one helps rookie QBs develop, but it requires actually believing it to understand my stance that trading Justin Fields was a mistake. If you truly believe (like I do) that there's no benefit whatsoever to starting year one, then the Bears just made themselves worse at the QB position for no reason.
BTW, even for the all time greats this seems to hold. Looking at my top ten QBs since 1981 list, three (Peyton Manning, Dan Marino, Josh Allen) straight up started as rookies, five (Patrick Mahomes, Joe Montana, Aaron Rodgers, Drew Brees, Tom Brady) did not see the field at all. Dan Fouts started but played so badly he got benched halfway through the season, and Steve Young gets thrown out for USFL reasons. Therefore, even for players with all time great potential, I'm not buying that they need to play right now.
That got really long winded, but I hope it answered your questions. Please let me know if I left anything unanswered. Thank you for your comment David!
Some questions for you and a stray thought or opinion here and there, if you have the time.
Could you please explain your selection of defining the bad quarterbacks in terms of CPOE and not EPA? Not that the lists would diverge that much, but I'm curious. Is EPA looked upon as the bottom line, but CPA more predictive?
So, I get confused. What variables does the probability of completion for a pass incorporate? Distance downfield, plus lateral distance? Whether the quarterback is on the run or not? Does it take into account the distance between the receiver and the defender at the time the ball is thrown?
One advantage of EPA, in terms of measuring a quarterback's worth, it would seem to me, is it rewards how much a quarterback sees, and where he chooses to throw the ball, not just how well he does when he does throw the ball. EPA implicitly penalizes for omissions (or, I suppose, compares to an average rate of omission).
If CPOE does take in more than just how far downfield the pass is, I can see that it might yield some interesting results, in that a pass could have limited potential reward, but show up as just as skilled a play as one that went well downfield. Not because the throw downfield was one that the quarterback should have completed (although that could be true, too, if it were a blown coverage), but because the short throw vertically still covered a lot of distance, or was to a closely covered guy. This is sort of the equivalent of when the announcer says, "That was the best two-yard run I ever saw," which is sometimes a reasonable statement. There can be skill in short plays, and things we can glean from them. So, maybe that's why CPOE is more predictive than EPA, if it is.
I hadn't focused on just how many of our top quarterbacks received limiting playing time their rookie year. I do find that hard to combat, as an argument that sitting on the bench isn't hurting young quarterbacks, and may even be helping them.
I like Justin Fields' talent, but my first reaction is I don't agree with your logic in saying the Bears should have stayed with him. Why is everything about 2024? Would they have won the Super Bowl with Fields? Don't Luck, Goff, Stafford, and Burrow indicate that there is longitudinal progress with quarterbacks, and Williams could well turn out to be a better player than Fields, even though his first two weeks have been rocky? And I know about Bryce Young and Zach Wilson and Trey Lance, but I think it just stands to reason that the higher you're picked in the first round, the more likely you are to succeed. It might not be true, but it probably is, and if that hasn't turned out over a period of years, it probably only means that we haven't had sufficient sample size to see the opposite, although the effect is probably weak.
Hey David. How are you doing tonight? I hope you're well. Okay. Several questions. I'll attempt to answer in order.
1) EPA is strictly a measure of results, measured in terms of points (I have written an article on what EPA is, but I think I've gleamed from your comment that you know what it is). Results on a player level (even at a position as central as the QB) are very highly dependent on the environment surrounding a player in this game. It's not like baseball where you can have great players on bad teams. Football isn't really like that. A great player on a bad team will not put up bad results, but he likely will merely put up okay results. Therefore, results are not the best way to forecast individual players going forward (this is particularly true when changing teams, although that's not relevant in this case), which is what I was trying to do.
The best way to do that at this position is Completion Percentage Over Expected (CPOE), which is the superior measure of throw accuracy, and as close to a measurement of actual talent at this position you're going to get. What factors it takes into account vary with the expected completion percentage model you choose to use. I elect to use NFLFastR's version, which takes into account only air distance, and direction to determine the expected completion percentage.
I dislike the NGS version because it takes into account distance between receiver and defender when calculating expected completion percentage, which tends to reward arm talent guys who force balls into coverage, and penalize QBs (like Brock Purdy and Patrick Mahomes) who throw what the model considers 'easy' passes because they're constantly able to find and target receivers who are actually open (very far from defenders), and I don't like that. Player tracking does not yet exist at the level to determine whether a pass was thrown on the run or not or anything like that.
The main reason I use CPOE in this discussion is that it is the one of two stats that tend to persist (with the other being sack rate, which is what 'sacks are a QB stat' means). The season-over-season correlation between players' CPOE values is 0.44 if I remember correctly. That of actual results is less than half that, if my memory serves. Therefore, the definition of a bad QB is not necessarily one with bad results, but one with a very low CPOE.
This is sort of like baseball, in that the best measure of quality looking backwards is EPA/Play, but the best measure of quality looking forwards is CPOE, sort of like the difference between ERA and FIP. Much like ERA and FIP one being bad almost always means the other is bad, but not always. Think of a high ERA, low FIP pitcher. You would know the examples better than I, but the NFL version of this is Sam Darnold, who'd been a low EPA/Play, high CPOE guy, and once finally freed from the NYJ that talent is finally shining through. It in fact was quite easily predictable.
That was long winded, but I hope explains why a low CPOE is indicative of a bad player, much moreso than the actual results.
2) I have no understanding of why NFL teams insist on continuing to play these rookie QBs. They're almost always awful, and as I explained in the article and you've mentioned, just two of the NFL's top ten QBs last year started out of the gate as rookies, one of whom being the miracle fourth round Dak Prescott turning out to be the best rookie QB of all time, and the other being Josh Allen, who was horrendous in his rookie season.
To me, choosing to start a rookie QB is tantamount to tanking, which is fine, but gets a little less fine when it's being done by five teams across the league, and even less fine than that when it's being done by a team like Chicago, who are pretty good except for the anchor they've insisted on wearing around their own necks. That's when it begins to become a problem of a lot of games being non-competitive in a league where there only 16 games in a week.
3) I just went over that starting a rookie QB week one appears to have no effect at all on his future development, as most of the top guys in the league spent week one on the bench. Anecdotally, I think forcing a rookie to go through a really bad first season is actually a hinderance, although I have generated no data at all to support that hypothesis. Looking at the recent examples, it certainly didn't help Bryce Young, and I would argue it helped CJ Stroud only in that there is no QB controversy now, when if the Texans had a competent (non-rookie) QB guiding that offence to be the Super Bowl contender it should've been last year, there certainly would be.
Which brings us to the Bears. Now that we all understand that starting a rookie QB neither helps nor hinders QB development, why would you ever sell low on Justin Fields in order to make your team worse this year? Not everything needs to be about 2024, and my apologies if I made it seem that way, but this move sends a clear message to players and fans that nothing is about 2024, which is not correct either, especially when you're Chicago, a team long settled into a culture of losing.
We've seen it already. If the Chicago Bears had Justin Fields right now they'd be 2-0 without question, and likely would've made the playoffs this year, hopefully boosting the trade value of Fields beyond a sixth round pick, but definitely getting this organisation back used to winning or at least competing. Instead they've sold low on their former first round pick for the privilege of being terribly lucky to be 1-1, and a very long shot to make the playoffs. If there were any benefit to Caleb for doing this I would understand, but the data seems to show that there just isn't any.
Even if Caleb is going to grow to be a better player than Justin, the Alex Smith-Patrick Mahomes case shows that it's still better to keep the rookie on the bench so the team can compete right now. If the Chiefs had tried to play Patrick Mahomes he would've been poor (because all rookies are, nothing at all against Patrick), and the 2017 Chiefs team would've missed the playoffs, and what would that have gotten anybody? The season was all about Patrick's development anyway, but it didn't have to happen on the field in the games. The same goes with Brett Favre-Aaron Rodgers or Drew Bledsoe-Tom Brady or Doug Flutie-Drew Brees.
So we can say that we don't believe starting week one helps rookie QBs develop, but it requires actually believing it to understand my stance that trading Justin Fields was a mistake. If you truly believe (like I do) that there's no benefit whatsoever to starting year one, then the Bears just made themselves worse at the QB position for no reason.
BTW, even for the all time greats this seems to hold. Looking at my top ten QBs since 1981 list, three (Peyton Manning, Dan Marino, Josh Allen) straight up started as rookies, five (Patrick Mahomes, Joe Montana, Aaron Rodgers, Drew Brees, Tom Brady) did not see the field at all. Dan Fouts started but played so badly he got benched halfway through the season, and Steve Young gets thrown out for USFL reasons. Therefore, even for players with all time great potential, I'm not buying that they need to play right now.
That got really long winded, but I hope it answered your questions. Please let me know if I left anything unanswered. Thank you for your comment David!
Hey Robbie. How do I get the nflfastR package? I don't understand.
What are you asking me?
Like the link to the site and how to use it/download.
Great, thanks. You are pretty cool. Anytime you want a nice compliment, I'll send you one, otherwise just keep it up.