Analytics, schmanalytics! (1 Viewer)

[ComfortablyNumb]

I'm really getting tired of seeing coaches make stupid decisions which are supported by "analytics" but usually seem to backfire and often cost them the game. Harbaugh last week, Staley every week, and McDaniel tonight (going for two). I feel like I should market myself to the NFL as a common-sense expert sideline consultant...

 

[The Dude]

I don't think tonight was about analytics. I think his defense couldn't stop Mahomes and he was looking to fire the defense up by getting a lead.

 

[ignatiusjreilly]

I don't think tonight was about analytics. I think his defense couldn't stop Mahomes and he was looking to fire the defense up by getting a lead.

Late-game decisions on 2PCs like tonight’s are almost entirely qualitative. Mostly comes down to trusting your offense vs trusting your defense. Chiefs had been killing the Raiders in the red zone all night and saw a chance to put the game away. Raiders saw a chance to take a lead over a team with a shaky backup kicker. Numbers and WP% had nothing to do with it

Meanwhile, “analytics” has become the new “gluten”

 

[Neil Beaufort Zod]

Totally agree it wasn't a cold numbers evaluation. No idea what Reid was thinking when he went for two, though.

 

[Deamon]

I'm more annoyed at people complaining about analytics than those who use it.

I'm for it. It's not always going to work, but it obviously works more than it doesn't.... or else the numbers wouldn't suggest it.

So much hindsight, so many armchair coaches, so much blaming the decision solely on the result.

Agreed though, tonight wasn't about analytics at all. I do really like the "go for two when you're down 14 and score a TD" thing though. That one makes a lot of sense to me even though it backfired this weekend.

 

[Chaka]

You never complain when it pays off, like Devante's 58 yard TD on 4th & 1 tonight. That seems to get forgotten in the outrage machine.

 

[titusbramble]

Going for 2 here is a completely standard play

 

[candian fantasy guy]

Try being a Blue Jays fan, freaking analytics!!

 

[zamboni]

So much hindsight, so many armchair coaches, so much blaming the decision solely on the result.

Yup - the technical term is outcome bias. Easy to say after the fact when decisions are made at the time based on probability.

All that said, I do think coaches sometimes use analytics as a fallback to justify their actions when it goes sideways - not every situation is different and sometimes the situation may not be appropriately represented by historical data.

 

[Dr. Octopus]

Somebody is a big fan of punting it seems.

 

[IHEARTFF]

Totally agree it wasn't a cold numbers evaluation. No idea what Reid was thinking when he went for two, though.

He was thinking we can make it a 2 score game and our backup kicker is no Butker

 

[rockaction]

Somebody is a big fan of punting it seems

Oh God, I love it. The majestic arc of the football. The waving of the hands before play stops dead. Even its practitioners call any botched ball a muff. All that stuff.

 

[Capella]

You have no idea what analytics is if you think that’s it

 

[Capella]

You never complain when it pays off, like Devante's 58 yard TD on 4th & 1 tonight. That seems to get forgotten in the outrage machine.

It’s the worst football related dialogue I’ve seen in a long time.

 

[rockaction]

It’s the worst football related dialogue I’ve seen in a long time.

The dialogue isn't helped by color commentators like Troy Aikman, who really have no idea what the term encompasses yet still toss it around like it means something at the wrong times.

Everything to them is analytics.

 

[ignatiusjreilly]

Totally agree it wasn't a cold numbers evaluation. No idea what Reid was thinking when he went for two, though.

“I wonder if they’ll cover Kelce this time.”
“Sooner we wrap this game up, sooner I can have a cheeseburger.”

 

[lod001]

I'm really getting tired of seeing coaches make stupid decisions which are supported by "analytics" but usually seem to backfire and often cost them the game. Harbaugh last week, Staley every week, and McDaniel tonight (going for two). I feel like I should market myself to the NFL as a common-sense expert sideline consultant...

Analytics has been a miserable failure so far this year costing numerous games. Lions lost because of it. Lions blew a gain because they used analytics not once but twice in the same game and analytics failed them both times.
Week 4 , the Falcons were the idiots of the week by going for it on 4th & 3 from the 4 instead of kicking the game tying FG.
This weeks loser is Cincy.
The Chargers should have joined them but the Browns inept play calling at the end caused them to have to try a 54 yarder instead of an easier one. They had 1:10 to get 15-20 yards and make it less than a 50 yarder but they called a deep out like idiots. Wasted down. Chargers DESERVED to lose because of it.
Now add in the Raiders this week.

 

[zamboni]

I'm really getting tired of seeing coaches make stupid decisions which are supported by "analytics" but usually seem to backfire and often cost them the game. Harbaugh last week, Staley every week, and McDaniel tonight (going for two). I feel like I should market myself to the NFL as a common-sense expert sideline consultant...

Analytics has been a miserable failure so far this year costing numerosu games. Lions lots becasue of it. chragres DESERVED to lose becasue of it.

That's because analytics doesn't guarantee results. It just provides probabilistic outcomes, which are typically much less than 100%, based on past data. If you have a 70% success rate, obviously doesn't mean it won't hit on the 30% failure rate.

 

[lod001]

You never complain when it pays off, like Devante's 58 yard TD on 4th & 1 tonight. That seems to get forgotten in the outrage machine.

They lost the game. Had they kicked the XP and won, then analytics would have worked but since they went back to the analytics well and failed, they lost BECASUE of analytics.

 

[Cowboysfan8]

Dave Roberts is the current king of losing games because of this. Just wait

 

[lod001]

At it's current rate, this year, its well below 50%.

 

[jtd13]

Brick, are you just pointing at things you don't like and calling them analytics?

(Saw it in a tweet this morning but can't find it now to give credit)

 

[steeler6]

Totally agree it wasn't a cold numbers evaluation. No idea what Reid was thinking when he went for two, though.

I think his replacement kicker missed 2 shortish FGs. One of those was negated by a stupid penalty. Plus if they convert the 2 point conversion, it puts them up by 2 scores.

 

[ChiefD]

I'm really getting tired of seeing coaches make stupid decisions which are supported by "analytics" but usually seem to backfire and often cost them the game. Harbaugh last week, Staley every week, and McDaniel tonight (going for two). I feel like I should market myself to the NFL as a common-sense expert sideline consultant...

Analytics has been a miserable failure so far this year costing numerosu games. Lions lots becasue of it. chragres DESERVED to lose becasue of it.

That's because analytics doesn't guarantee results. It just provides probabilistic outcomes, which are typically much less than 100%, based on past data. If you have a 70% success rate, obviously doesn't mean it won't hit on the 30% failure rate.

Does analytics and outcome probabilities factor in what I like to call "The Pyschology of Points"?

Because I feel like it does not. Probabilities are one thing since they can use data to determine success or failure rates based on past occurences. But scoring points when they are available can certainly affect momentum and the pyschology of the players.

Going for two and failing and now being in a losing position when you had a chance to even the score has to affect the mindset of both teams. And it seems like a lot of coaches completely ignore the momentum factor and just play the math.

Maybe I'm wrong but I think it's a huge factor.

 

[zamboni]

I'm really getting tired of seeing coaches make stupid decisions which are supported by "analytics" but usually seem to backfire and often cost them the game. Harbaugh last week, Staley every week, and McDaniel tonight (going for two). I feel like I should market myself to the NFL as a common-sense expert sideline consultant...

Analytics has been a miserable failure so far this year costing numerosu games. Lions lots becasue of it. chragres DESERVED to lose becasue of it.

That's because analytics doesn't guarantee results. It just provides probabilistic outcomes, which are typically much less than 100%, based on past data. If you have a 70% success rate, obviously doesn't mean it won't hit on the 30% failure rate.

Does analytics and outcome probabilities factor in what I like to call "The Pyschology of Points"?

Because I feel like it does not. Probabilities are one thing since they can use data to determine success or failure rates based on past occurences. But scoring points when they are available can certainly affect momentum and the pyschology of the players.

Going for two and failing and now being in a losing position when you had a chance to even the score has to affect the mindset of both teams. And it seems like a lot of coaches completely ignore the momentum factor and just play the math.

Maybe I'm wrong but I think it's a huge factor.

Excellent point. I think that is the crux of the argument between the continuum of pure analytics proponents and pure gut feel proponents. It resonates particularly in baseball, where analytics are typically more prominent.

The truth is that probably both have value because there is definitely a big human element in sports that analytics doesn't always factor in.

 

[Ministry of Pain]

I'm really getting tired of seeing coaches make stupid decisions which are supported by "analytics" but usually seem to backfire and often cost them the game. Harbaugh last week, Staley every week, and McDaniel tonight (going for two). I feel like I should market myself to the NFL as a common-sense expert sideline consultant...

If we are talking about that Kick last night


Down 30-29
4:27 4th Q, you just scored a TD and you go for a 2-pt conversion?

-KC is going to take the field with over 4 minutes left and you have 3 potential ways to start that drive
30-29, you are behind
30-30, you are in a tie ball game
31-30, you might be ahead but you still are going to have to stop Mahomes

Is the risk worth the reward? If there were 27 seconds left I might understand, you can end the game.

 

[cdclopper]

The thing where the coach goes for 2 to go up by 1, explain how analytics supports this? You're basically removing the possibility you get the ball back with a punt. To me, it only makes sense if the other team has no time left.

 

[GroveDiesel]

The Buffalo Bills are one of, if not THE, top team when it comes to following the analytics. Seems to be working out for the then pretty well.

I do think that folks on both sides of the analytics thing have done a pretty awful job of understanding exactly what analytics is and isn’t.

On the anti-analytics side, it seems that folks really don’t understand that it ultimately comes down to probabilities. There are no guarantees and often we’re talking about fairly small differences in probabilities, both of which may be low probabilities to begin with. But when one way or the other doesn’t pan out, it just reinforces our already existing biases and we’re more likely to focus on the failures than the successes.

On the pro-analytics side, they’ve done an absolutely awful job making it clear that the inputs matter. Often when we see these odds and recommendations through media/4th down bots/whatever, those are based on league-wide generic data. None of that data accounts for specific team or personnel circumstances.

When ESPN or Amazon or whoever pops up a stat saying that analytics says that they should go for it, their formula isn’t account for the fact that it’s Bailey Zappe and not Tom Brady under center. It’s not taking into account that the team may be down to their 4th offensive guard who has played nothing but OT his entire life but is in because of injuries.

Teams with good analytics departments not only have all their own customized inputs specific to their team, but they adjust them on the fly as well. A team’s analytics calculation on whether to go for it or not should be unique to their team and shouldn’t look exactly like the generic ones used by media/analysts. Now, some teams will also be better at this than others, so one team may have good analytics and another team have bad analytics. So whether it’s a good decision or bad decision based on that data is something we’ll never really know.

 

[Banger]

Totally agree it wasn't a cold numbers evaluation. No idea what Reid was thinking when he went for two, though.

I understand Reid going for 2 a WHOLE lot more than the Raiders going for 2. KC going for 2 goes up by 2 scores if they succeed and still up by 7 if it fails whereas the Raiders will automatically lose is you miss on your 2 pt conversion.

 

[titusbramble]

The thing where the coach goes for 2 to go up by 1, explain how analytics supports this? You're basically removing the possibility you get the ball back with a punt. To me, it only makes sense if the other team has no time left.

Having a lead is better than not having a lead. Not sure why you talk about getting the ball back with a punt - you just get the ball back slightly less in every case where you force a fourth down and they then don't convert (with better field position)

 

[zamboni]

The Buffalo Bills are one of, if not THE, top team when it comes to following the analytics. Seems to be working out for the then pretty well.

I do think that folks on both sides of the analytics thing have done a pretty awful job of understanding exactly what analytics is and isn’t.

On the anti-analytics side, it seems that folks really don’t understand that it ultimately comes down to probabilities. There are no guarantees and often we’re talking about fairly small differences in probabilities, both of which may be low probabilities to begin with. But when one way or the other doesn’t pan out, it just reinforces our already existing biases and we’re more likely to focus on the failures than the successes.

On the pro-analytics side, they’ve done an absolutely awful job making it clear that the inputs matter. Often when we see these odds and recommendations through media/4th down bots/whatever, those are based on league-wide generic data. None of that data accounts for specific team or personnel circumstances.

When ESPN or Amazon or whoever pops up a stat saying that analytics says that they should go for it, their formula isn’t account for the fact that it’s Bailey Zappe and not Tom Brady under center. It’s not taking into account that the team may be down to their 4th offensive guard who has played nothing but OT his entire life but is in because of injuries.

Teams with good analytics departments not only have all their own customized inputs specific to their team, but they adjust them on the fly as well. A team’s analytics calculation on whether to go for it or not should be unique to their team and shouldn’t look exactly like the generic ones used by media/analysts. Now, some teams will also be better at this than others, so one team may have good analytics and another team have bad analytics. So whether it’s a good decision or bad decision based on that data is something we’ll never really know.

You said it much more eloquently than I did - bravo.

 

[jobarules]

I don't think tonight was about analytics. I think his defense couldn't stop Mahomes and he was looking to fire the defense up by getting a lead.

And ironically they stopped them on the very next drive

 

[Capella]

It’s the worst football related dialogue I’ve seen in a long time.

The dialogue isn't helped by color commentators like Troy Aikman, who really have no idea what the term encompasses yet still toss it around like it means something at the wrong times.

Everything to them is analytics.

Totally agree. It’s horrible and very stupid.

 

[Capella]

As somebody else said you have to understand probabilities in order to understand what teams are doing here. Most Americans don’t, we’re just taught to look at things in black and white terms - did it work or not in this one very specific instance. And of course we just remember when it did not work, not the times when it did.

All of these teams have highly sophisticated teams analyzing this stuff - not just the “next gen stats” that they display on tv while some bozo like Herbstreit explains why he would have punted and played “the field position game” instead.

The most important part of football is having the ball. Trying to keep the ball is good. It will take a while for a lot of people to get out of the 1992 philosophy of football that they grew up watching but we’ll get there.

 

[The Dude]

wow did this go bad for the OP

 

[habsfan]

Is anyone else disappointed this thread isn't about a new form of analytics that factors in offensive / defensive schemes?

 

[humpback]

The Buffalo Bills are one of, if not THE, top team when it comes to following the analytics. Seems to be working out for the then pretty well.

I do think that folks on both sides of the analytics thing have done a pretty awful job of understanding exactly what analytics is and isn’t.

On the anti-analytics side, it seems that folks really don’t understand that it ultimately comes down to probabilities. There are no guarantees and often we’re talking about fairly small differences in probabilities, both of which may be low probabilities to begin with. But when one way or the other doesn’t pan out, it just reinforces our already existing biases and we’re more likely to focus on the failures than the successes.

On the pro-analytics side, they’ve done an absolutely awful job making it clear that the inputs matter. Often when we see these odds and recommendations through media/4th down bots/whatever, those are based on league-wide generic data. None of that data accounts for specific team or personnel circumstances.

When ESPN or Amazon or whoever pops up a stat saying that analytics says that they should go for it, their formula isn’t account for the fact that it’s Bailey Zappe and not Tom Brady under center. It’s not taking into account that the team may be down to their 4th offensive guard who has played nothing but OT his entire life but is in because of injuries.

Teams with good analytics departments not only have all their own customized inputs specific to their team, but they adjust them on the fly as well. A team’s analytics calculation on whether to go for it or not should be unique to their team and shouldn’t look exactly like the generic ones used by media/analysts. Now, some teams will also be better at this than others, so one team may have good analytics and another team have bad analytics. So whether it’s a good decision or bad decision based on that data is something we’ll never really know.

This has come up in other threads- I agree with this, but I'd take it a step further. In reality there is no way to definitively say what the exact probabilities are. It makes a lot more sense to use smaller sub-sets of data, but it's still subjective. Every single play in the NFL is unique, so even narrowing it down to extremely similar plays under extremely similar conditions in the past isn't perfect. Running this next play with these exact set of circumstances has never happened before (and will never happen again).

So again, I think you can make a much stronger case for a tighter range of probabilities by having a strong analytics department who customizes the inputs, but no one can give an exact probability. There is always going to be some subjectivity in the decision of which inputs to use, so while we can make our case for whether we agree or disagree with the decisions, we really can't (or at least shouldn't) declare that it was "right" or "wrong" (on the reasonably close ones, anyway). The outcomes are completely irrelevant.

 

[fred_1_15301]

I don't think tonight was about analytics. I think his defense couldn't stop Mahomes and he was looking to fire the defense up by getting a lead.

And ironically they stopped them on the very next drive

Yes they did but what if KC had an extra down to work with ? There is a pretty good chance IMO that KC scores a FG if they have 4 downs to work with on that drive. So does the upside of being up 1 (and giving Mahomes 4 minutes plus 4 downs) outweigh the downside of potentially never even getting the ball back? I personally don’t think so.

 

[ignatiusjreilly]

I don't think tonight was about analytics. I think his defense couldn't stop Mahomes and he was looking to fire the defense up by getting a lead.

And ironically they stopped them on the very next drive

Yeah, it was a totally rational decision for Reid to think, "I'd rather put my money on Mahomes converting a 2PC than on my defense stopping Jacobs on one." And yet the LV D did stop Mahomes, and the KC D did stop Jacobs.

Good process, bad results

 

[rockaction]

There is always going to be some subjectivity in the decision of which inputs to use, so while we can make our case for whether we agree or disagree with the decisions, we really can't (or at least shouldn't) declare that it was "right" or "wrong" (on the reasonably close ones, anyway).

I've been saying this for years about the over-reliance, in some instances, on even baseball situations and statistics. You can never duplicate an exact situation, and any stat proffered as the be-all end-all should have its inputs looked at. My position in football is that there are situations and longer-term probabilities to consider and that the coach ought to be apprised of both. I think the Chargers do this actually really well, in contrast to what most pundits think.

Anyway, your post was a good one, IMO.

 

[GroveDiesel]

The Buffalo Bills are one of, if not THE, top team when it comes to following the analytics. Seems to be working out for the then pretty well.

I do think that folks on both sides of the analytics thing have done a pretty awful job of understanding exactly what analytics is and isn’t.

On the anti-analytics side, it seems that folks really don’t understand that it ultimately comes down to probabilities. There are no guarantees and often we’re talking about fairly small differences in probabilities, both of which may be low probabilities to begin with. But when one way or the other doesn’t pan out, it just reinforces our already existing biases and we’re more likely to focus on the failures than the successes.

On the pro-analytics side, they’ve done an absolutely awful job making it clear that the inputs matter. Often when we see these odds and recommendations through media/4th down bots/whatever, those are based on league-wide generic data. None of that data accounts for specific team or personnel circumstances.

When ESPN or Amazon or whoever pops up a stat saying that analytics says that they should go for it, their formula isn’t account for the fact that it’s Bailey Zappe and not Tom Brady under center. It’s not taking into account that the team may be down to their 4th offensive guard who has played nothing but OT his entire life but is in because of injuries.

Teams with good analytics departments not only have all their own customized inputs specific to their team, but they adjust them on the fly as well. A team’s analytics calculation on whether to go for it or not should be unique to their team and shouldn’t look exactly like the generic ones used by media/analysts. Now, some teams will also be better at this than others, so one team may have good analytics and another team have bad analytics. So whether it’s a good decision or bad decision based on that data is something we’ll never really know.

This has come up in other threads- I agree with this, but I'd take it a step further. In reality there is no way to definitively say what the exact probabilities are. It makes a lot more sense to use smaller sub-sets of data, but it's still subjective. Every single play in the NFL is unique, so even narrowing it down to extremely similar plays under extremely similar conditions in the past isn't perfect. Running this next play with these exact set of circumstances has never happened before (and will never happen again).

So again, I think you can make a much stronger case for a tighter range of probabilities by having a strong analytics department who customizes the inputs, but no one can give an exact probability. There is always going to be some subjectivity in the decision of which inputs to use, so while we can make our case for whether we agree or disagree with the decisions, we really can't (or at least shouldn't) declare that it was "right" or "wrong" (on the reasonably close ones, anyway). The outcomes are completely irrelevant.

Completely agree. The goal is to get as close as possible to the true probabilities. You’ll never get the actual, but the closer you get, the better decisions you can make.

 

[TVT 0 N S T A]

I don't think tonight was about analytics. I think his defense couldn't stop Mahomes and he was looking to fire the defense up by getting a lead.

Late-game decisions on 2PCs like tonight’s are almost entirely qualitative. Mostly comes down to trusting your offense vs trusting your defense. Chiefs had been killing the Raiders in the red zone all night and saw a chance to put the game away. Raiders saw a chance to take a lead over a team with a shaky backup kicker. Numbers and WP% had nothing to do with it

Meanwhile, “analytics” has become the new “gluten”

If you don't trust your defense to get the stop then what makes you think a 1 point lead is going to be the difference maker? Situationally, that type of play call has a time and place. 4 minutes and 30 seconds left in the game and saying you don't trust your defense to get the stop isn't a very good bailout strategy. You're in no better position with a 1 point lead than you are with a tie anyway.

 

[ignatiusjreilly]

I don't think tonight was about analytics. I think his defense couldn't stop Mahomes and he was looking to fire the defense up by getting a lead.

Late-game decisions on 2PCs like tonight’s are almost entirely qualitative. Mostly comes down to trusting your offense vs trusting your defense. Chiefs had been killing the Raiders in the red zone all night and saw a chance to put the game away. Raiders saw a chance to take a lead over a team with a shaky backup kicker. Numbers and WP% had nothing to do with it

Meanwhile, “analytics” has become the new “gluten”

If you don't trust your defense to get the stop then what makes you think a 1 point lead is going to be the difference maker? Situationally, that type of play call has a time and place. 4 minutes and 30 seconds left in the game and saying you don't trust your defense to get the stop isn't a very good bailout strategy. You're in no better position with a 1 point lead than you are with a tie anyway.

It's all relative. It's not about not trusting your D, it's about trusting your offense more.

 

[ignatiusjreilly]

Somebody is a big fan of punting it seems

Oh God, I love it. The majestic arc of the football. The waving of the hands before play stops dead. Even its practitioners call any botched ball a muff. All that stuff.

That to me is the weirdest part of the anti-analytics crowd in football. I get the people who object to analytics in baseball and basketball on aesthetic grounds. Defensive shifts in baseball and corner threes on fast breaks in basketball make those games less enjoyable to watch. But in football it's the exact opposite!

It's a fallacy that analytics just means, "Always go for it on 4th down", but the net effect of teams relying on numbers more is that you get more 4th down attempts instead of punts/FGs, more all-or-nothing 2PCs instead of overtime, etc. And that's a bad thing? Seems to me those are some of the most exciting plays in football!

 

[rockaction]

corner threes on fast breaks in basketball make those games less enjoyable to watch

I agree with your whole post, but I actually enjoy the kick out and three. Tooting my own horn here -- I had advocated for that in basketball since the early-mid nineties when I was in college and played basketball regularly. Especially where two pointers were one point and threes counted as two points for our scorekeeping purposes. That made the disparity that much bigger, and we often used the pull-up three or kick-out three to our advantage. Helped us hang with teams we never should have hung with at times.

 

[Zow]

I'm really getting tired of seeing coaches make stupid decisions which are supported by "analytics" but usually seem to backfire and often cost them the game. Harbaugh last week, Staley every week, and McDaniel tonight (going for two). I feel like I should market myself to the NFL as a common-sense expert sideline consultant...

Analytics has been a miserable failure so far this year costing numerosu games. Lions lots becasue of it. chragres DESERVED to lose becasue of it.

That's because analytics doesn't guarantee results. It just provides probabilistic outcomes, which are typically much less than 100%, based on past data. If you have a 70% success rate, obviously doesn't mean it won't hit on the 30% failure rate.

Right.

Plus, analytics is only a part of the analysis. It may make sense to go for a 4th and 8 on the opponent's 38 yard line when you don't have Tucker as your kicker. It may not, however, be wise to then run a fullback dive.

Analytics is an effective tool to provide coaches the correct math for difficult decision-making. That said, the coaches still have to call a favorable play, and the players still have to execute. Those two factors are entirely outside the control of the math and math shouldn't be blamed.

 

[Zow]

corner threes on fast breaks in basketball make those games less enjoyable to watch

I agree with your whole post, but I actually enjoy the kick out and three. Tooting my own horn here -- I had advocated for that in basketball since the early-mid nineties when I was in college and played basketball regularly. Especially where two pointers were one point and threes counted as two points for our scorekeeping purposes. That made the disparity that much bigger, and we often used the pull-up three or kick-out three to our advantage. Helped us hang with teams we never should have hung with at times.

Same!

I/my team hung in a lot more games than we should have because we knew to shoot threes.

 

[Deamon]

I wish announcers could say if they liked the decision or not right before the play. They do the whole "play it safe" where they may softly question the decision, and if it doesn't work then they really dig their heels in and start saying how bad of a decision it was. And if it works, they talk about how gutsy it was.

This year has been so annoying listening to commentators and espn hosts and people on message boards all saying how bad a decision was. But like others have said above, where are all those people when someone makes a questionable decision based on analytics and it works out? There's no thread being made for all of those, and there has been a lot.

 

[ignatiusjreilly]

The Buffalo Bills are one of, if not THE, top team when it comes to following the analytics. Seems to be working out for the then pretty well.

I do think that folks on both sides of the analytics thing have done a pretty awful job of understanding exactly what analytics is and isn’t.

On the anti-analytics side, it seems that folks really don’t understand that it ultimately comes down to probabilities. There are no guarantees and often we’re talking about fairly small differences in probabilities, both of which may be low probabilities to begin with. But when one way or the other doesn’t pan out, it just reinforces our already existing biases and we’re more likely to focus on the failures than the successes.

On the pro-analytics side, they’ve done an absolutely awful job making it clear that the inputs matter. Often when we see these odds and recommendations through media/4th down bots/whatever, those are based on league-wide generic data. None of that data accounts for specific team or personnel circumstances.

When ESPN or Amazon or whoever pops up a stat saying that analytics says that they should go for it, their formula isn’t account for the fact that it’s Bailey Zappe and not Tom Brady under center. It’s not taking into account that the team may be down to their 4th offensive guard who has played nothing but OT his entire life but is in because of injuries.

Teams with good analytics departments not only have all their own customized inputs specific to their team, but they adjust them on the fly as well. A team’s analytics calculation on whether to go for it or not should be unique to their team and shouldn’t look exactly like the generic ones used by media/analysts. Now, some teams will also be better at this than others, so one team may have good analytics and another team have bad analytics. So whether it’s a good decision or bad decision based on that data is something we’ll never really know.

This has come up in other threads- I agree with this, but I'd take it a step further. In reality there is no way to definitively say what the exact probabilities are. It makes a lot more sense to use smaller sub-sets of data, but it's still subjective. Every single play in the NFL is unique, so even narrowing it down to extremely similar plays under extremely similar conditions in the past isn't perfect. Running this next play with these exact set of circumstances has never happened before (and will never happen again).

So again, I think you can make a much stronger case for a tighter range of probabilities by having a strong analytics department who customizes the inputs, but no one can give an exact probability. There is always going to be some subjectivity in the decision of which inputs to use, so while we can make our case for whether we agree or disagree with the decisions, we really can't (or at least shouldn't) declare that it was "right" or "wrong" (on the reasonably close ones, anyway). The outcomes are completely irrelevant.

Completely agree. The goal is to get as close as possible to the true probabilities. You’ll never get the actual, but the closer you get, the better decisions you can make.

First of all, I think it's a bit of a straw man to say that coaches blindly follow analytics. I'm not aware of any coach ever saying, "I wasn't sure that was the right call, but Nerdlinger up in the booth told me it increased our WP by 0.4% so we had to do it." I think most of them recognize it's a tool to aid decision making.

Second, one of my pet peeves when it comes to probabilities is that people tend to cherry pick the contrary data without considering a) factors that support the analytic recommendation and b) the magnitude of the confidence interval. Consider the controversial Chargers decision from this weekend, but put aside for a moment whether or not you agree with it and simply view it as a decision-making process. The data (as represented by the 4th Down Bot, which is only one potential method of modeling the situation) said that going for it increased WP by 11.7%. Many people objected that this data doesn't account for the fact that the Browns had Jacoby Brissett, which made it less likely they would put together a scoring drive if LA punted. That's definitely true, and should be taken into consideration by the decision maker. But it's also true that LA had Herbert and Ekeler and had been gashing the Browns all day, which perhaps made them more likely than an average team to convert the fourth down and put the game away. In addition, 11.7% WPA is a huge number! It's not just a matter of "The numbers say to go for it, but this one factor cuts against that recommendation so you shouldn't." It matters whether you're talking about a 51/49 recommendation or a 75/25 one.

Which isn't to say you should always default to the 75% option. To take one example, in the famous GB/AZ "Double Hail Mary" playoff game, Packers pulled to within one on the final play of regulation. I would imagine all the numbers would suggest going for two in that situation. GB was a road underdog who had needed two miracle plays to get themselves back in the game. Plus, more qualitatively, if ever there were a time to rely on momentum, that was the moment: GB was riding high and Cards were totally deflated after blowing a chance to close the game out.

But ...

According to interviews with McCarthy after the game, they did discuss going for two, but they had lost a bunch of receiving options by that point (remember, Rodgers' Hail Marys had both gone to Jeff Janis) and didn't feel confident in any of their 2PC plays with the personnel they had available. Was that the right call? I have no idea! I don't know what the plays would have been, or who they would have gone to. But I certainly think that's a rational factor to consider, one that's potentially significant enough to overrule whatever the numbers may be recommending.

By contrast, in the KC-Cleveland divisional game a couple years ago, Reid was lauded for the gutsy call of going for a game-clinching conversion even though Chad Henne was in at QB. But it sounds like a big factor in convincing him to go for it was that they had a play to Tyreek that they felt very confident in. Again, those are some of the things that none of us could possibly know about in the moment.

 

[Dr. Octopus]

I wish announcers could say if they liked the decision or not right before the play. They do the whole "play it safe" where they may softly question the decision, and if it doesn't work then they really dig their heels in and start saying how bad of a decision it was. And if it works, they talk about how gutsy it was.

This year has been so annoying listening to commentators and espn hosts and people on message boards all saying how bad a decision was. But like others have said above, where are all those people when someone makes a questionable decision based on analytics and it works out? There's no thread being made for all of those, and there has been a lot.

Daboll got a lot of praise for going for 2 and the win instead of kicking the XP for the tie - but yes generally people love to complain so you hear more about it when it doesn't work.