Confused by Waldman's Crank Scores Projections (1 Viewer)

[Bartguy]

I'm a big fan of Matt Waldman's thinking and was following his Crank consistency articles, but I'm frankly scratching my head about how he implemented them for his rankings.

I thought the idea he outlined was that a player who scored slightly lower, but scored more consistently week to week could/should be more valuable. Wouldn't that argue for calculating which players have proven to be more consistent over their careers and target them?

Instead, it looks like he correlated crank scores to the position ranks (ie. QB1 tends to have the highest elite scores, QBs 2-5 tend to have most good scores, etc.) and then plugged in the players he ranked at those spots. But what if the player you have ranked highly has a history of inconsistent scoring, but his position rank typically WAS very consistent? Case in point, Chad Ocho Cinco is highly ranked, but if memory serves, has almost always been a feast or famine type scorer.

Why wouldn't we look at the players individual CRANK scores and identify those who are very consistent but getting a lot less love than higher ranked players who are a lot more inconsistent? I thought that was the idea of crank scores in the first place -- to find value that your competition is missing.

Anybody else confused by this article? Or does it make perfect sense to you? If so, help me get my head around it. Thanks.

 

[GregR_2]

I agree that I don't understand the desire to use AVT in this sense.

I brought up back in the previous CRANK thread that I thought anyone wanting to make a system trying to predict the future or guide future decisions based on consistency needs to first demonstrate that with their method, future consistency is predictable enough based on past consistency to merit using.

I mentioned a couple of things Doug Drinen had done it which are now a few years old, which had suggested a player being consistent in the past wasn't being a good predictor of being consistent in the future.

Doug joined in after my comment and said:

...

While I don't really blame Doug v2000 for running and publishing this study, Doug v2008 wouldn't find it very convincing.

It's just a really tricky issue. I have studied it a lot, and I do still believe that week-to-week consistency is so difficult to predict that you're probably better off not trying to predict it. But I still haven't figured out a clear way to demonstrate that.

Matt later replied to the topic with the following:

See post #5 Greg R...I use it as something to talk about what's happened historically and then share my attempts to incorporate it into a predictive system. Again, I've had success, but much like BigSteelThrill alludes to, much of that probably comes from my experience evaluating players. I continue to experiment with it and I'm sure if I come up with something that has a plug and play quality to the process, I'll make it known

(post #5)

I think it's an interesting way to look at historical performance, but as a stand-alone model for predicting future performance it's not there yet. I use it for my personal draft and share my experiments with readers, but I need to continue looking at ways to create/identify a more systematic method for projecting consistency. I've had success with it, but I have to admit I haven't turned the projections part into a clear cut systematic process as done with AVT or VBD.

What worries me is that this was being used as a method to look at past performance as Matt says. But now it's being turned into that stand alone model to incorporate in your decision making about the future. But the step that shows whether it is useful to predict the future hasn't been done still, at least as far as I can tell from the article.

And that goes for whether it is using AVT or using player CRANK scores. I won't rule out that you can get something useful from using AVT, but that one needs to be proven even more than using the player's CRANK would, due to what we already know to be the shortcomings of AVT.

If it is a useful predictor at all, I would think it would be worse using AVT. It seems like the same failings of AVT for projections would apply to AVT for CRANK. AVT neglects our uncertainty that the guy we projected at RB2 actually ends up RB2. If CRANK does turn out to be a good predictor (which needs to be shown), I imagine the RB2 CRANK would do a better job predicting the CRANK for the guy who actually ends up RB2 than it would predicting the CRANK for the guy we rank at RB2 before the season. If it is a good predictor at all, one would think the player's CRANK would be the better predictor for the player. Though again, that should be proven either way before it gets turned into a system. My point in all of this is simply that while it can be turned into a system... before that is done some work should be done to see whether we have reason to think doing so will be more useful than using a random number generator to spit out consistency scores that we use.

 

[jackdubl]

What worries me is that this was being used as a method to look at past performance as Matt says. But now it's being turned into that stand alone model to incorporate in your decision making about the future. But the step that shows whether it is useful to predict the future hasn't been done still, at least as far as I can tell from the article.

And that goes for whether it is using AVT or using player CRANK scores. I won't rule out that you can get something useful from using AVT, but that one needs to be proven even more than using the player's CRANK would, due to what we already know to be the shortcomings of AVT.

If it is a useful predictor at all, I would think it would be worse using AVT. It seems like the same failings of AVT for projections would apply to AVT for CRANK. AVT neglects our uncertainty that the guy we projected at RB2 actually ends up RB2. If CRANK does turn out to be a good predictor (which needs to be shown), I imagine the RB2 CRANK would do a better job predicting the CRANK for the guy who actually ends up RB2 than it would predicting the CRANK for the guy we rank at RB2 before the season. If it is a good predictor at all, one would think the player's CRANK would be the better predictor for the player. Though again, that should be proven either way before it gets turned into a system. My point in all of this is simply that while it can be turned into a system... before that is done some work should be done to see whether we have reason to think doing so will be more useful than using a random number generator to spit out consistency scores that we use.

Wo! You're blowin' my mind, bro .