Concept Coop
Footballguy
Are you sure it predicts misses? It could - and I’m open to that - but I don’t think you’ve shown that it has any predictive value. You’d have to test it against other variables, at the very least.
Fantasy Football - Footballguys Forums
Welcome to Our Forums. Once you've registered and logged in, you're primed to talk football, among other topics, with the sharpest and most experienced fantasy players on the internet.
Spaceshuttle (1 Viewer)
- Thread starter Interseptopus
- Start date
Concept Coop
Footballguy
He’s not. He’s performing the exact same univariate analysis that you are. He simply changed the variable. In doing so, he found an even stronger correlation than you did. It’s a valid question.
Ilov80s
Footballguy
I don’t think it predicts misses. It doesn’t predict anything. It’s simply looking for commonalities. What do the top performers seem to have in common? Can we find other WRs who also have those things in common? Can we try to avoid WRs who don’t look like the top guys?
Last edited by a moderator:
Concept Coop
Footballguy
He was refuting your argument. You picked a variable and found a correlation. He picked a different variable and found an equally strong correlation. Why one would have predictive value and not the other is a valid question - and you haven’t answered that.
Concept Coop
Footballguy
I agree with some of this. I do think dom% and breakout age are general indicators of ability. My question is whether they keep being quality indicators once we control for draft spot. I know the best WRs tend to breakout earlier. But what is the track record for WR drafted early despite a later breakout age? How do 2nd round WRs with a late breakout compare to 3rd round WRs with an early one, etc., etc.?
I'll take a stab at trying to encapsulate the concerns.
Dr. Dan has pointed out the correlation about BA/DR and top 12 finishes. This is a good starting point to show that there are more players that meet the two criteria that have top finishes as compared to those that don't meet the criteria and have top finishes.
For arguments sake:
10/12 meet the BA/DR criteria
2/12 do not meet BA/DR criteria
At this point I would agree, it seems to make more sense to avoid potential misses (non BA/DR players).
Now to Clopp's point:
of the 2 exceptions, let's say they may make up 50% of the WRs that do not meet either BA/DR criteria (4 total WRs)
of the 10 players that meet BA/DR, these players make up 10% of the WRs that meet BA/DR crietria (100 total WRs)
I know I'm exaggerating the numbers here, but it is simply to point out where the method may fail as a miss predictor. If I decided to exclude players that don't meet BA/DR thresholds as above, I'm not actually improving my chances of avoiding a miss. While a small percentage of the top 12 finishers fail the metrics, they actually represent half of their total population. I would actually be better off taking one of these WRs as they have a combined higher chance of not being a miss.
Back to Dr. Dan's point:
While we don't know the exact population percentages, we certainly have good indications that there are fewer players (low percentage of all WRs) that meet both BA/DR criteria as compared to those that don't. We also see a higher percentage of these players show up in the top 12, meaning we see a higher population density of players meeting BA/DR AND landing in the top 12. So while we don't have the specific population percentages and breakdowns like we do for type of players that finished in the top 12, I do still think this method is useful for predicting misses.
Dr. Dan has pointed out the correlation about BA/DR and top 12 finishes. This is a good starting point to show that there are more players that meet the two criteria that have top finishes as compared to those that don't meet the criteria and have top finishes.
For arguments sake:
10/12 meet the BA/DR criteria
2/12 do not meet BA/DR criteria
At this point I would agree, it seems to make more sense to avoid potential misses (non BA/DR players).
Now to Clopp's point:
of the 2 exceptions, let's say they may make up 50% of the WRs that do not meet either BA/DR criteria (4 total WRs)
of the 10 players that meet BA/DR, these players make up 10% of the WRs that meet BA/DR crietria (100 total WRs)
I know I'm exaggerating the numbers here, but it is simply to point out where the method may fail as a miss predictor. If I decided to exclude players that don't meet BA/DR thresholds as above, I'm not actually improving my chances of avoiding a miss. While a small percentage of the top 12 finishers fail the metrics, they actually represent half of their total population. I would actually be better off taking one of these WRs as they have a combined higher chance of not being a miss.
Back to Dr. Dan's point:
While we don't know the exact population percentages, we certainly have good indications that there are fewer players (low percentage of all WRs) that meet both BA/DR criteria as compared to those that don't. We also see a higher percentage of these players show up in the top 12, meaning we see a higher population density of players meeting BA/DR AND landing in the top 12. So while we don't have the specific population percentages and breakdowns like we do for type of players that finished in the top 12, I do still think this method is useful for predicting misses.
Last edited by a moderator:
Biabreakable
Footballguy
I think it is a good idea to look at draft position in addition to the 2 other metrics if the goal is to look for patterns or trends.
2 data points isnt really enough.
Draft position being the strongest predictive measure of player success would be a useful to what this is trying to do.
I am somewhat iffy about the relevance of the two other metrics and the chosen margins.
2 data points isnt really enough.
Draft position being the strongest predictive measure of player success would be a useful to what this is trying to do.
I am somewhat iffy about the relevance of the two other metrics and the chosen margins.
cloppbeast
Footballguy
One other idea, another aspect, that is we're not as interested in avoiding misses as much as getting fantasy points. Using thresholds turns data points into binaries, which equates a Tyreek Hill to a Rishard Matthews. They both turn up as hit as a wr1 in this study, but really there's a big difference between the two.
Often a team selects a guy who hasn't produced in college because he has some tantalizing elite physical traits, a sort of swing for the fences pick. He'll either be awesome or terrible. Said player has a high chance to be Brian Quick or AJ Jenkins, but also shot at being Michael Thomas. Using a risk avoidance strategy could net you Michael Crabtree, which is nothing to sneeze at, I guess. But has Michael Crabtree won anybody a championship? Is it better trying to risk getting Dwayne Bowe in effort to snag Andre Johnson, or just settle for Jarvis Landry?
I'm using anecdotes here to illustrate a hypothesis. I think total fantasy points is a better measuring stick than a single wr1 season. Does drafting a high ba/low dom% of similar draft pedigree net you more or less fantasy points in the long run?
Often a team selects a guy who hasn't produced in college because he has some tantalizing elite physical traits, a sort of swing for the fences pick. He'll either be awesome or terrible. Said player has a high chance to be Brian Quick or AJ Jenkins, but also shot at being Michael Thomas. Using a risk avoidance strategy could net you Michael Crabtree, which is nothing to sneeze at, I guess. But has Michael Crabtree won anybody a championship? Is it better trying to risk getting Dwayne Bowe in effort to snag Andre Johnson, or just settle for Jarvis Landry?
I'm using anecdotes here to illustrate a hypothesis. I think total fantasy points is a better measuring stick than a single wr1 season. Does drafting a high ba/low dom% of similar draft pedigree net you more or less fantasy points in the long run?
Last edited by a moderator:
What makes someone considered to be a "false positive"? Or are you saying that any WR that did not meet the DR and BA criteria but finished in the top XX is a false positive because you are trying to use this to rule out players?
Concept Coop
Footballguy
I’m struggling to follow. If there were 2 false positives in the first round alone, how is Tyreek Hill the only one?
It might be easier to read if you break it up by those who met the criteria and those who didn’t, then again by those with a top 24 finish l and those who without. If you could give me those 4 numbers by round, I’d be in your debt.
It might be easier to read if you break it up by those who met the criteria and those who didn’t, then again by those with a top 24 finish l and those who without. If you could give me those 4 numbers by round, I’d be in your debt.
Last edited by a moderator:
Concept Coop
Footballguy
If my math is right, you think there is a ~72% chance of 0 top 24 season between the four guys you list? You’d take that bet?
Concept Coop
Footballguy
The odds of all 4 of them missing.
(Edit: And my was right the first time. Fixed it.)
0.92*0.92*0.92*0.92 = 0.72
So you think there is a 72%?chance that none of them ever finish top 24?
Last edited by a moderator:
Thanks Dr. Dan. My curiosity is piqued. Understanding the high percentage of a miss if a player misses one of the two metrics, which of the two metrics has a higher miss rate?
In other words, if I am taking a chance on a player in later rounds of fantasy, should I skip the player who doesn't meet the BA metric or skip the player who doesn't meet the DR metric?
It's definitely splitting hairs, I'm just curious how the numbers shake out.
Thanks!
In other words, if I am taking a chance on a player in later rounds of fantasy, should I skip the player who doesn't meet the BA metric or skip the player who doesn't meet the DR metric?
It's definitely splitting hairs, I'm just curious how the numbers shake out.
Thanks!
cloppbeast
Footballguy
Hopefully I understand your data correctly.
wrs drafted: 219
meeting criteria: 118 (54%)
NOT meeting criteria: 101 (46%)
So yeah, it looks like you could have something here considering 79% wr1s (to 2018 tho) meet the criteria compared to 54% of the total population. If you only had these 2 metrics, absolutely go with the low BA/high Dom.
Introducing the effect of these metrics on the draft does complicate things. Breakdown of receivers meeting the criteria be round:
1st: 78%
2nd: 76%
3rd: 71%
4th: 38%
5th: 54%
6th: 34%
7th: 41%
The NFL definitely accounts for BA/Dom%, perhaps inadvertently idk. But I think we have enough information here to say there is some inefficiency. Quantifying it requires different tools and a little more information. For instance, @Dr. Dan your post on the first page of this thread showing 79% of wr1s meet the criteria goes to 2018, while your data above only to 2016. Matching these up would allow for analyzing hit rates of by round drafted as well.
wrs drafted: 219
meeting criteria: 118 (54%)
NOT meeting criteria: 101 (46%)
So yeah, it looks like you could have something here considering 79% wr1s (to 2018 tho) meet the criteria compared to 54% of the total population. If you only had these 2 metrics, absolutely go with the low BA/high Dom.
Introducing the effect of these metrics on the draft does complicate things. Breakdown of receivers meeting the criteria be round:
1st: 78%
2nd: 76%
3rd: 71%
4th: 38%
5th: 54%
6th: 34%
7th: 41%
The NFL definitely accounts for BA/Dom%, perhaps inadvertently idk. But I think we have enough information here to say there is some inefficiency. Quantifying it requires different tools and a little more information. For instance, @Dr. Dan your post on the first page of this thread showing 79% of wr1s meet the criteria goes to 2018, while your data above only to 2016. Matching these up would allow for analyzing hit rates of by round drafted as well.
Ilov80s
Footballguy
Interesting idea. My only anecdotal rebuttal is that you have created a false premise. It isn't Tyreek or Rishard. Players who are just as dominant as Tyreek have met both thresholds and been difference makers: Julio, Hopkins, Davanta Adams, Antonio Brown, etc. One can play it "safe" and land an alpha.
cloppbeast
Footballguy
I wrote that piece before Doc got back with the distribution of all the drafted guys who meet the criteria. With that in mind, I have to say you better your chances of landing a stud in the later rounds targeting guys who produced in college at a young age. Still not sure about the high drafted ones.
Last edited by a moderator:
Ilov80s
Footballguy
Were there a disproportionate amount of young WRs in 2014? That might account for that phenomenon. In normal years some of those young guys opted to stay another year and would have part of 2015? Just speculating.
Concept Coop
Footballguy
If there is any predictive value, there’s no doubt they’re incorporating it in their models. How much stock the decision makers put in analytics is a question, but all 32 orgs have an analytics team running models that we hobbyists can’t compete with.
My biggest issue with most hobbyist metrics is that the creators don’t test against or control for draft position. It’s like creating a gambling model that doesn’t aim to beat the spread. Knowing that home teams win 55% of their games does a gambler zero good, as the betting lines already account for it. Your model has to beat the spread to have any value. If the league is aware of past trends and still highly values a player on the wrong side of those trends, we shouldn’t be ignoring that.
And that’s even assuming the hobbyist model has predictive value. You can’t apply a model to past happenings and call it predictive. Predictive models have to be measured by actual predictions.
cloppbeast
Footballguy
I was just thinking about this in regards to the Ravens first round pick. Marquise Brown going so high shocked me. The analytics went against it, no combine information, and the liz-franc injury - despite all this Hollywood was the first wr picked. I can't convince myself the NFL was just too stupid to consider all this and now find myself wondering what I was missing about him.Concept Coop said:
You could apply the same reasoning to one Josh Jacobs.
Ilov80s
Footballguy
barackdhouse
Footballguy
Good stuff, guys. I'm not clear on how much stock the GM of a given team puts into his analytics team and/or even their own board. I wonder if they sometimes freeze up OTC and go with their gut. I would assume that isn't the case *most* of the time, but I think the idea of a GM going after *his guy* is not unheard of. To the extent that it matters when looking at this, IDK but I don't think the gap between the analytic skills of hobbyists and NFL are all that disparate. Where I think the NFL has an edge is player scouting of specific football traits, like he has good hands, route running, football IQ, etc. And that edge the NFL has is enough to render draft position the likely most important factor in all this. But I am wondering if the x factor of GMs making weird (or seemingly) decisions is relevant. IDK.
Last edited by a moderator:
Concept Coop
Footballguy
I’m sure there are plenty of data scientists with fantasy football hobbies. In that sense, you could be right. But we’re doing incredible things with machine leaning/AI. The gap between cutting-edge analytics and guys like us using Excel and manually building models can’t be overstated.
barackdhouse
Footballguy
Concept Coop
Footballguy
Absolutely. The industry is incredibly agile and, by NFL standards, affordable. Most teams are probably using Amazon. That said, I’m certainly curious how much weight they put in it; I honestly have no idea.
Ilov80s
Footballguy
kittenmittens
Footballguy
Both of these stats have a huge opportunity component, and major overlap between he two. Breakout age actually uses a certain Dominator to determine it.
The idea is, I think, to use stats that show who the coaches of the NCAA team think is the best player. The coaches have the most intimate knowledge and largest sample to know who the best player is, and have large financial incentives (winning) to get that player the ball. That should be rather predictive unless coaching staffs make inefficient decisions.
I wonder how much the numbers change to be more predictive if you add some further analysis to some of the data? Obviously that's hard to do.... But if you can explain why a player missed a criteria it might explain a false positive or two and could be carried forward to achieve more predictive data.
What I think of is mostly players who are blocked by elite players or play in elite programs where dominating the offensive touches, especially at an early age, is a lot more difficult?
For Tyreek Hill, he may not even be a false positive because he went Juco and then had off the field issues his first NCAA season, so no way these metrics can really capture that accurately.
I'm not sure how you would do this uniformly... Maybe just look at the players you are eliminating and see if there is a good and obvious reason they couldn't get a high dominator?
The idea is, I think, to use stats that show who the coaches of the NCAA team think is the best player. The coaches have the most intimate knowledge and largest sample to know who the best player is, and have large financial incentives (winning) to get that player the ball. That should be rather predictive unless coaching staffs make inefficient decisions.
I wonder how much the numbers change to be more predictive if you add some further analysis to some of the data? Obviously that's hard to do.... But if you can explain why a player missed a criteria it might explain a false positive or two and could be carried forward to achieve more predictive data.
What I think of is mostly players who are blocked by elite players or play in elite programs where dominating the offensive touches, especially at an early age, is a lot more difficult?
For Tyreek Hill, he may not even be a false positive because he went Juco and then had off the field issues his first NCAA season, so no way these metrics can really capture that accurately.
I'm not sure how you would do this uniformly... Maybe just look at the players you are eliminating and see if there is a good and obvious reason they couldn't get a high dominator?
kittenmittens
Footballguy
I agree with this.
They don't have to be, there is pretty low financial incentive to win in the NFL. A huge percentage of the money is split evenly across teams.
It's not baseball where each team gets its own TV contract in addition to the national deal.
That said, I would bet that the smart teams are running circles around the rest of the league in this area. Winning for mostly glory is still incentive.
Biabreakable
Footballguy
While I do think NFL front offices have a analytics team that looks at data like this (and a lot more) to further inform their decisions, no two NFL team analytics groups are the same, some are more robust than others, and each group is looking at different factors based on the data they have.
The Vikings are the team I know better than others. I have heard Rick Spielman talk about this. He says they have data that goes back to 2005 in their modeling. He makes mention of similarity scores where they can find college prospects who are near clones of players who have come before by their metrics. The Vikings have been able to find good pass rushers to play edge in the 4th round of the draft consistently for almost a decade now. Maybe other teams should look at what they have been doing to accomplish that. I know Justis Mosqueda has done some good metric based research on what he calls force players, who meet certain criteria he has found that leads to good pass rushers. In his analysis the Vikings seem to have found something similar to what he has. Their team is full of players who fit his criteria.
As far as breakout age? I don't know if it is relevant or not. The Vikings selected Irv Smith this year, one of the youngest players in the draft. Daniel Hunter was a very young player when they selected him as well. A reporter asked Spielman if age is a factor that influences his decisions. Speilman said not at all. But I don't think he understood the question. Speilman basically said they won't rule out a player for being too young, which is the opposite of what was being asked. I got the impression from Spielman from his answer that age is not a factor they consider at all. Maybe I am wrong about that and Spielman lies, maybe does not want anyone to know the nuts and bolts of their process, but I got the impression from his answer that age is not a variable they consider to be important. At all.
The dominator rating and where that gets cut off is pretty arbitrary. I don't know how much weight if any that NFL teams place on this. If they do, I am guessing they do it differently than what has been done here. I think dominator rating misses context and a lot of factors that lead to that numbers results. I am not sure how useful it is.
Whatever the NFL teams have found to be useful for informing their decisions has been factored into the draft position. All of it. No two teams are the same however.
Based on the Ravens very poor track record with drafting the WR position for decades now, I am skeptical of their decision making process in regards to these players. Maybe Brown is great. The draftnniks and tape watchers largely seem to agree he is a very good WR. Teams still make most of their decisions based on subjective tape watching over anything else. The analytics is just something they will consider as well when building their draft boards. Much like in business. You can inform management all you want but they make decisions based on cronyism and other things anyways.
wgoldsph
Footballguy
Hardman seems to be the logical choice to lead wash at wr to me considering McLaurin is a clear out wr/special teams ace. Also, he had back to back 1000 yard years... How did that not reach the dominator threshold for a possession wr?
cloppbeast
Footballguy
I am reminded of the running back the 49ers selected a few years back, Joe something-or-other, I forget his name. The team supposedly chose him because Kyle Shanahan threw a fit about it, slamming the table. Im quite sure he wasn't getting emotional over the analyitics, only had a hunch.
To @Biabreakable 's point, I'm sure some teams consider statistical factors more and better than others. You see it here in SP. Where some stat heads come to threads like this; others draft Butler ahead of Isabella. Imo, it's not accurate to assume NFL front offices dont resort to their own judgement over historical trends just as often. NFL decision makers are football people, not statisticians.
Look how long it took baseball to take that nerdy moneyball guy seriously. Only Billy Bean really listened to him until the results made it abundantly clear. Well, for many reasons, football analytics will never be as obvious. Sample size, competition, schemes, and other factors make it more difficult to find a trend. Even if you find one, the results won't slap you in the face. Increasing your hit rate from 30% to 40% is pretty significant, but not really very noticeable to the naked eye.
Last edited by a moderator:
cloppbeast
Footballguy
Ilov80s
Footballguy
Looking at your missing data:
Julian Edelman did not meet the criteria given he played QB.
Braylon Edwards absolutely met the threshhold. He had 1000 and 10 at 19. He had 48% of the passing yards and 60% of the passing TDs. He was off the charts on DR and BA.
barackdhouse
Footballguy
Yeah Joe Williams. This is basically what I'm getting at. If the hardcore analytic hobbyists here are going to drill down so far in the weeds, then I'm not sure you can ignore how much noise there is in draft position. Every GM is different every decision is different. To me this shouldn't be thought of as a model that can or can't be rigorously defended in an analytics sense so much as an easy method for a gambler to pull down the low hanging fruit (likely misses). And of course it can be tested over the next few years.
cloppbeast
Footballguy
Credit Doc for the hypothesis, because I'm confident there's a trend here. Hardman and Campbell's draft position this year indicates at least some teams still dont care about these metrics; therefore the trend which has existed the last 10 years should continue.
I agree with you though, this analysis falls short of quantifying the inefficiency, just stopping at its demonstration.
Dinsy Ejotuz
Footballguy
I think this is absolutely an issue late in the draft. You've got a mix of special teamers and WR5s mixed in with high-risk/longshot starter hopefuls.
barackdhouse
Footballguy
I mean is there some super computer out there telling GMs they should take a kicker in the 2nd round? Or all the punters that seemingly get taken. I don't disagree that having one of the best at either of those positions can be an edge for a ballclub, but I find it hard to believe the analytics departments recommended those guys.
I'll admit I don't know enough about AI and some of the big data/machine learning possibilities that are out there, but it seems to me there is no way to get around the process of trying to separate the noise from the signal, and to parse out the differences in human scouting. And as far as how close NFL draft position can be modelled as as function of a GM/analytic department's methodologies, I think at the end of the day even the best of analytical tools are subject to the whims of the humans making judgement calls on whether something is more signal than noise. Can a super machine learn what makes a good route runner or a bad route runner? Aren't there different human schools of thought on such questions and as such a super computer has limitations too? I'm not saying don't employ the use of them, I just have some skepticism about it. If it says take a kicker in the 2nd round, I might have to fire someone.
Ilov80s
Footballguy
Not sure it can given how weird TE usage is in college. Kelce had 5th percentile breakout at 22.9, Kittle had 23rd percentile breakout age at 21.9, Ertz was 27th breakout age at 21.8. They all had great dominator ratings though. O.J. Howard also had a low breakout age and a less than 50% dominator. It seems like the top 4 TEs in most drafts this year weren't qualifiers for this.
CalBear
Footballguy
No, you don't. Because there are more players within the thresholds. Their success rate isn't greater.
There is no success predictor which has been shown to improve on NFL draft order. Which makes sense. NFL player evaluators have an actual job of reviewing NFL prospects. We have a remote control and an Excel spreadsheet.
cloppbeast
Footballguy
I have addressed this as a concern already, actually a few posters have. But, we found out only about 54% of wide receivers drafted meet Docs criteria, compared to 79% of wr1s.
bicycle_seat_sniffer
Smells like chicken
bicycle_seat_sniffer
Smells like chicken
The Alex Smith season? I would say Mahomes qualifies as elite......tyreek is a unicorn....
Ilov80s
Footballguy
I don't think anyone here is advocating choosing a 6th round pick who meets the criteria over a 1st round pick who didn't. But what happens if you are choosing between 2 guys with similar draft capital? Wouldn't it be helpful to have some other means to separate the players with good draft capital?
cloppbeast
Footballguy
You demonstrate one of the issues with using thresholds. The best you can get is showing there could be a trend, without establishing the actual difference between Deebo Samuel and Andy Isabella.
CalBear
Footballguy
It would be helpful. That doesn't mean it's possible. And there is a whole stack of statistical fallacies involved in looking at correlations in previous outcomes to predict future behaviors.
Ilov80s
Footballguy
Who is to say it’s not possible to gain even a 5-10% edge? You are right that correlation does not automatically equal causation. It doesn’t automatically mean there isn’t a meaningful relationship though either.
cloppbeast
Footballguy
Sounds like you call into question the entire field of predictive analytics.CalBear said:
Last edited by a moderator:
CalBear
Footballguy
Not at all. Predictive analytics spends a ton of time looking at which factors actually are predictors, and which are conflated. For example, ballpark factors are good predictors in baseball. OPS vs. lefty/righty is a good predictor. "Clutch hitting" is not a good predictor, so baseball quants don't use it.
Any quant would be very suspicious of a metric like "break out age" or BMI which is not meaningfully about performance. Especially when you're looking at a big spreadsheet with dozens of stats which are guaranteed to have multiple spurious correlations.
cloppbeast
Footballguy
A hs freshman wrestler winning a state championship - a 14 y.o. beating the best 18 y.o.'s the state has to offer, its no coincidence these are the guys who go on to win NCAA championships. The same goes in college football. A 19 y.o. putting up stats for his college team is more impressive than a 22 y.o.
CalBear
Footballguy
Sure. But he NFL draft is already taking that into account. You only gain anything if you're exposing areas where the NFL draft is inefficient.
Player 1 whose breakout age was 21 goes at 1.15. Player 2 whose breakout age was 19 goes at 2.15. Do you take Player 2 over Player 1 because of this metric?
Similar threads
