2006 FBG Predictions, Statistical Analysis of Accuracy (1 Viewer)

[GoBears84]

This took a little longer than expected...

This is the final summary of my attempt to analyze the predictions from 2006 Projections Dominator as compared to the year’s final results. The individual analysis by position can be found at the following links:

PK,

QB's,

RB's,

WR’s

TE’s and

Team D.

My analysis involved calculating the Points Per Game (PG) for each player based on the number of games claimed to have been played in the 2006.

From there I took Projections from Dodds, Henry, Smith, Tremblay and Wood for QB, RB, WR and TE and divided them by 16 and subtracted the predictions from the actuals to get a residual (a measure of how far off the prediction was). Then I squared the residuals.

I then sorted all the data by PPG scored. It was immediately apparent that QB’s had many more statistical outliers than any other position. There were 5 QB’s in the top 24 who had significantly better performances than predicted: Cutler, Young, Campbell, Feeley, and Leinart. None of these guys were starters at the beginning of the season and weren’t expected to start as soon as they did. They significantly out performed all of the expert predictions. They were not included in the analysis.

To balance the analysis I looked at the top 24 for QB’s, RB’s, WR’s and TE’s as individual averages and as one big pool. Since only Dodds and Smith predicted Team D’s and only Dodds, Herman, Smith predicted PK’s, those were kept out of the big analysis.

The means of the Points per Game residual from the predictions were (minus data from the outliers) for the top 24 of each position were.

Top 24…………..…..QB…………..RB…………...WR…………...TE……….…..Total

Dodds PG…………..3.16…………..4.87…………..1.87…………..1.41…………..11.30

Henry PG…………..3.43…………..4.78…………..1.85…………..1.62…………..11.67

Smith PG…………..2.71…………..4.85…………..1.66…………..1.76…………..10.97

Tremblay PG……....3.18…………..5.10…………..2.19…………..1.43…………..11.89

Wodd PG…………..3.21…………..4.71…………..1.81…………..1.54…………..11.27

The variances were:

Top 24……….……..QB…………..RB……….…..WR……….…..TE……….…..Total

Dodds PG…………..3.98…………..5.47…………..2.40…………..1.77…………..13.62

Henry PG…………..4.53…………..5.07…………..2.15…………..2.15…………..13.89

Smith PG…………..3.25…………..5.51…………..1.92…………..1.94…………..12.62

Tremblay PG…..…..3.86…………..5.62…………..2.32…………..1.77…………..13.57

Wodd PG…………..4.35…………..5.37…………..2.32…………..1.77…………..13.80

Looking across the prediction space, Smith is always on the lower end of the prediction mean and variance. All the other experts were pretty similar.

Pooling all the results into the top 100 (actually 96), the means of the residuals were:

Top 96………….......All

Dodds PG…………..3.32

Henry PG…………..3.39

Smith PG…………..3.19

Tremblay PG……....3.41

Wodd PG…………..3.41

And the variances were:

Top 96………….......All

Dodds PG…………..4.52

Henry PG…………..4.48

Smith PG…………..4.38

Tremblay PG….……4.54

Wodd PG…………..4.82

Again, Smith is slightly lower.

Our previous analysis of Team D’s, showed that the means and variance of the Points per Game residual from the predictions were:

Top 24 D’s..……Means……..Variance

Dodds …………..1.97…………..2.75

Smith …………..2.01…………..2.15

And for kickers, the means of the Points per Game residual from the predictions were (minus data for Gould who was an outlier):

Top 24 PK…..……Mean……...Variance

Dodds PG…………..0.89…………..0.94

Herman PG…………1.35…………..1.33

Smith PG…………..1.16…………..1.28

What does it all mean? My analysis have suggested that overall, the predictions of the experts are statistically similar and that the differences that do exist are minor and probably due to chance.

However, there are some differences and I think this is what make the PD so powerful. Instead of relying on just the projections of Dodds in DD, the weight can be spread across the experts. Therefore if someone misses badly on a prediction, the impact is not as significant.

I also think that there is a "group think" here. While the experts all claim to be independent thinkers, the fact is that all of the projections are fairly similar, with some outliers. This may be because they all start out from the same baseline (2005 EOY) or because they all read the same articles and boards and there are some built in biases to follow the leader. Best example: Willie Parker. Last year was his first full year as a starter, yet all the experts under projected him by 72 + 5 points. That's too close to be random variation.

Time will tell if one expert is consistently better than another at certain positions. But right now we only have one year of data and as I've maintained, past performance is no guarantee of future projections.

Joel

 

[GregR_2]

I also think that there is a "group think" here. While the experts all claim to be independent thinkers, the fact is that all of the projections are fairly similar, with some outliers. This may be because they all start out from the same baseline (2005 EOY) or because they all read the same articles and boards and there are some built in biases to follow the leader. Best example: Willie Parker. Last year was his first full year as a starter, yet all the experts under projected him by 72 + 5 points. That's too close to be random variation.

Would it be a lot more work for you to take the 2005 EOY PPG and put them through the same treatment?I'd be curious from it to see if the 2005 EOY show the same group think, which might suggest that's a big source of it, while if it doesn't then it could be similar articles as you said, or it could just be individually coming to the same conclusions about changed situations.

 

[GoBears84]

I also think that there is a "group think" here. While the experts all claim to be independent thinkers, the fact is that all of the projections are fairly similar, with some outliers. This may be because they all start out from the same baseline (2005 EOY) or because they all read the same articles and boards and there are some built in biases to follow the leader. Best example: Willie Parker. Last year was his first full year as a starter, yet all the experts under projected him by 72 + 5 points. That's too close to be random variation.

Would it be a lot more work for you to take the 2005 EOY PPG and put them through the same treatment?I'd be curious from it to see if the 2005 EOY show the same group think, which might suggest that's a big source of it, while if it doesn't then it could be similar articles as you said, or it could just be individually coming to the same conclusions about changed situations.

I don't believe the data exists - I pulled this from last years PD. And yes, it is a lot of work...about 2-3 hours..per position.

 

[nygiants56]

This took a little longer than expected...

This is the final summary of my attempt to analyze the predictions from 2006 Projections Dominator as compared to the year’s final results. The individual analysis by position can be found at the following links:

PK,

QB's,

RB's,

WR’s

TE’s and

Team D.

My analysis involved calculating the Points Per Game (PG) for each player based on the number of games claimed to have been played in the 2006.

From there I took Projections from Dodds, Henry, Smith, Tremblay and Wood for QB, RB, WR and TE and divided them by 16 and subtracted the predictions from the actuals to get a residual (a measure of how far off the prediction was). Then I squared the residuals.

I then sorted all the data by PPG scored. It was immediately apparent that QB’s had many more statistical outliers than any other position. There were 5 QB’s in the top 24 who had significantly better performances than predicted: Cutler, Young, Campbell, Feeley, and Leinart. None of these guys were starters at the beginning of the season and weren’t expected to start as soon as they did. They significantly out performed all of the expert predictions. They were not included in the analysis.

To balance the analysis I looked at the top 24 for QB’s, RB’s, WR’s and TE’s as individual averages and as one big pool. Since only Dodds and Smith predicted Team D’s and only Dodds, Herman, Smith predicted PK’s, those were kept out of the big analysis.

The means of the Points per Game residual from the predictions were (minus data from the outliers) for the top 24 of each position were.

Top 24…………..…..QB…………..RB…………...WR…………...TE……….…..Total

Dodds PG…………..3.16…………..4.87…………..1.87…………..1.41…………..11.30

Henry PG…………..3.43…………..4.78…………..1.85…………..1.62…………..11.67

Smith PG…………..2.71…………..4.85…………..1.66…………..1.76…………..10.97

Tremblay PG……....3.18…………..5.10…………..2.19…………..1.43…………..11.89

Wodd PG…………..3.21…………..4.71…………..1.81…………..1.54…………..11.27

The variances were:

Top 24……….……..QB…………..RB……….…..WR……….…..TE……….…..Total

Dodds PG…………..3.98…………..5.47…………..2.40…………..1.77…………..13.62

Henry PG…………..4.53…………..5.07…………..2.15…………..2.15…………..13.89

Smith PG…………..3.25…………..5.51…………..1.92…………..1.94…………..12.62

Tremblay PG…..…..3.86…………..5.62…………..2.32…………..1.77…………..13.57

Wodd PG…………..4.35…………..5.37…………..2.32…………..1.77…………..13.80

Looking across the prediction space, Smith is always on the lower end of the prediction mean and variance. All the other experts were pretty similar.

Pooling all the results into the top 100 (actually 96), the means of the residuals were:

Top 96………….......All

Dodds PG…………..3.32

Henry PG…………..3.39

Smith PG…………..3.19

Tremblay PG……....3.41

Wodd PG…………..3.41

And the variances were:

Top 96………….......All

Dodds PG…………..4.52

Henry PG…………..4.48

Smith PG…………..4.38

Tremblay PG….……4.54

Wodd PG…………..4.82

Again, Smith is slightly lower.

Our previous analysis of Team D’s, showed that the means and variance of the Points per Game residual from the predictions were:

Top 24 D’s..……Means……..Variance

Dodds …………..1.97…………..2.75

Smith …………..2.01…………..2.15

And for kickers, the means of the Points per Game residual from the predictions were (minus data for Gould who was an outlier):

Top 24 PK…..……Mean……...Variance

Dodds PG…………..0.89…………..0.94

Herman PG…………1.35…………..1.33

Smith PG…………..1.16…………..1.28

What does it all mean? My analysis have suggested that overall, the predictions of the experts are statistically similar and that the differences that do exist are minor and probably due to chance.

However, there are some differences and I think this is what make the PD so powerful. Instead of relying on just the projections of Dodds in DD, the weight can be spread across the experts. Therefore if someone misses badly on a prediction, the impact is not as significant.

I also think that there is a "group think" here. While the experts all claim to be independent thinkers, the fact is that all of the projections are fairly similar, with some outliers. This may be because they all start out from the same baseline (2005 EOY) or because they all read the same articles and boards and there are some built in biases to follow the leader. Best example: Willie Parker. Last year was his first full year as a starter, yet all the experts under projected him by 72 + 5 points. That's too close to be random variation.

Time will tell if one expert is consistently better than another at certain positions. But right now we only have one year of data and as I've maintained, past performance is no guarantee of future projections.

Joel

you lost me at hello on this one!great work. do you have the Cribb Notes version , and more importantly, can this be translated to English ?

I know it took a while to do this, I'm thankful you did it, but basically all you're saying is that most `experts ` think the same.

so that's why LT,Sjax,LJ,Gore are ALWAYS listed as the top 4 RB's in every single fantasy mag and fantasy website right now. and look, I didn't need math to figure this out!

I'm not sure you needed to go thru all the work to post this, we all know that most pundits think the same,as do most fantasy GM's. I don't rely solely on Dodds' information, or anyone else's for that matter, to make my draft picks in fantasy football. I'll take a general feeling for what the pundits think, and tweak it to my benefit. Some guys are higher on certain players than others. I weigh all of this in my decision-making process.

sometimes you guys go overboard with mathematical projections and theories..Dodds is notorious for this..I wonder, in the end, whether or not all of his hard work with number-crunching, actually pans out.. I mean, is he a better prognosticator than he was ,say, 5 years ago? If you take out the math and science part, is he really that much better? not knocking the guy for his work ( although a lot of it is overkill), but I wonder if people get too carried away with the numbers? Do they lose sight of the fact that this is fantasy football..its not supposed to be quantum physics..its not supposed to be hard to decide what player to grab as a #4 RB..

The best thing I've seen,is Fantasy Index's Experts polls , where they poll the top 20 experts on the top 20 players at each position...its just a good idea to get the consesus opinions of the top experts in the field.

I'm not splitting atoms, I'm not finding the square root of 8, I don't need to know what a prime number is. I look at the consensus opinions and make my own judgements from there.

people who get all worked up with numbers, I find, don't draft well..they are too consumed with numbers, to notice that Peyton Manning is sitting there,waiting to be drafted.

if you're a professional poker player, THEN you need to have a mathematical mind..

NOT in the world of fantasy football.

 

[Ksquared]

GoBears84,



I have followed these threads and appreciate all the work you did. This may not be as valuable this year, but if we can do this for the next couple of years some tendencies may show up. From this study, I agree with spreading the PD % across the experts and a few user input values thrown in for players I feel strongly about. Last year I had my worst year and I really only used a couple of the experts values in the PD. I think this shows that strategy may not be the best way to go.

NYGIANTS,



(Rant) We get it. You want no math or statistics in fantasy football. How many threads are you going to keep lecturing/preaching this position. Some of us do not have "math phobia" and enjoy looking at a game so full of statistics/numbers by using statistics. Give props to the person that did all this work, ask some questions, or just don't post. We all understand, you laugh at all of us that use math. (Rant over)

 

[Bob Henry]

The best thing I've seen,is Fantasy Index's Experts polls , where they poll the top 20 experts on the top 20 players at each position...its just a good idea to get the consesus opinions of the top experts in the field.

Maybe this is more along the lines of what you'd like to see.. The following is an email from Ian Allan from Fantasy Index following the calculations of last year's Experts Poll winner.

Just a heads up. It finally occurred to me last night how to make sense ofthe overall, over-the-years standings in the Fantasy Football Index expertspoll.With so many guys in the poll, and the number of entries changing over theyears (it used to be 15, then 20, and now it's 25), it's very difficult towin. And how do we weigh a guy finishing in the top 5 six years in a rowversus a guy winning it once put finishing outside the top 15 twice? Stufflike that.From now on, I'll be keeping track of a new stat: OVERALL WINNINGPERCENTAGE.Each year, I'll tabulate wins and losses, based on how many other guys arein the poll. Scott Pianowski, for example, won the poll last year. So hegets 24 wins and 0 losses for that one. Dan Bakley finished 2nd, so he gets23 wins and 1 loss. And so on.Anyway, below is the current all-time top 10 leaderboard, including allexperts who've appeared in at least 60 games worth of competition over thelast 17 years. W L Pct. 91 14 .867 Bob Henry 126 22 .851 Ian/Jon Millman 100 29 .775 David Dorey 93 36 .721 Scott Pianowski 88 36 .710 Joe Bryant 60 31 .659 William Del Pilar 57 38 .600 Doug Lott 72 56 .563 Greg Ambrosius 91 76 .545 Michael Nazarek 78 70 .527 Carl Weatherford

 

[Jene Bramel]

FWIW, as I know he is going to be his usual modest self, in the five years Bob Henry has competed in this poll he has never finished out of the top seven. He has won it twice outright and I believe narrowly missed a third title. He finished fourth last year. Nobody else can come close to that level of consistency, IIRC, except the FBG collective. Between David, Joe, Woodrow and later Bob, I believe FBG has placed a staff member in the top five for the last four seasons.

 

[Jason Wood]

FWIW, as I know he is going to be his usual modest self, in the five years Bob Henry has competed in this poll he has never finished out of the top seven. He has won it twice outright and I believe narrowly missed a third title. He finished fourth last year. Nobody else can come close to that level of consistency, IIRC, except the FBG collective. Between David, Joe, Woodrow and later Bob, I believe FBG has placed a staff member in the top five for the last four seasons.

Bob is the king of the FF Index poll, no question.

 

[LHUCKS]

FWIW, as I know he is going to be his usual modest self, in the five years Bob Henry has competed in this poll he has never finished out of the top seven. He has won it twice outright and I believe narrowly missed a third title. He finished fourth last year. Nobody else can come close to that level of consistency, IIRC, except the FBG collective. Between David, Joe, Woodrow and later Bob, I believe FBG has placed a staff member in the top five for the last four seasons.

Bob is the king of the FF Index poll, no question.

If I had to choose just one staffer's rankings that I use to compare to my own it would be Bob's. I always keep an eye on where are major variances lie.

 

[LHUCKS]

I also think that there is a "group think" here. While the experts all claim to be independent thinkers, the fact is that all of the projections are fairly similar, with some outliers. This may be because they all start out from the same baseline (2005 EOY) or because they all read the same articles and boards and there are some built in biases to follow the leader. Best example: Willie Parker. Last year was his first full year as a starter, yet all the experts under projected him by 72 + 5 points. That's too close to be random variation.

This is partially correct...my ranking methodology is significantly different than most of the staff from what I've gathered over several years and thus my variances are greater on average than the staff's. I'm not saying my methodology is more right than the staffers because I've never done an exhaustive comparison, but I will say that the FBG consensus rankings are by far and away the best rankings you will find from any major publication...they just are.

 

[GoBears84]

This took a little longer than expected...

What does it all mean? My analysis have suggested that overall, the predictions of the experts are statistically similar and that the differences that do exist are minor and probably due to chance.

However, there are some differences and I think this is what make the PD so powerful. Instead of relying on just the projections of Dodds in DD, the weight can be spread across the experts. Therefore if someone misses badly on a prediction, the impact is not as significant.

you lost me at hello on this one!great work. do you have the Cribb Notes version , and more importantly, can this be translated to English ?

I know it took a while to do this, I'm thankful you did it, but basically all you're saying is that most `experts ` think the same.

so that's why LT,Sjax,LJ,Gore are ALWAYS listed as the top 4 RB's in every single fantasy mag and fantasy website right now. and look, I didn't need math to figure this out!

I'm not sure you needed to go thru all the work to post this, we all know that most pundits think the same,as do most fantasy GM's. I don't rely solely on Dodds' information, or anyone else's for that matter, to make my draft picks in fantasy football. I'll take a general feeling for what the pundits think, and tweak it to my benefit. Some guys are higher on certain players than others. I weigh all of this in my decision-making process.

sometimes you guys go overboard with mathematical projections and theories..Dodds is notorious for this..I wonder, in the end, whether or not all of his hard work with number-crunching, actually pans out.. I mean, is he a better prognosticator than he was ,say, 5 years ago? If you take out the math and science part, is he really that much better? not knocking the guy for his work ( although a lot of it is overkill), but I wonder if people get too carried away with the numbers? Do they lose sight of the fact that this is fantasy football..its not supposed to be quantum physics..its not supposed to be hard to decide what player to grab as a #4 RB..

The best thing I've seen,is Fantasy Index's Experts polls , where they poll the top 20 experts on the top 20 players at each position...its just a good idea to get the consesus opinions of the top experts in the field.

I'm not splitting atoms, I'm not finding the square root of 8, I don't need to know what a prime number is. I look at the consensus opinions and make my own judgements from there.

people who get all worked up with numbers, I find, don't draft well..they are too consumed with numbers, to notice that Peyton Manning is sitting there,waiting to be drafted.

if you're a professional poker player, THEN you need to have a mathematical mind..

NOT in the world of fantasy football.

There are many different approaches to doing well in FF, and one established method is using VBD. It's a numbers based approach, so you may not be familiar with it Since it's inception, the Draft Dominator which is built on the principles of VBD has been based on the projections of Dodds. Last year the FBG's introduced the Projections Dominator so that you could use any expert, or a combination of experts.Over in the application forum, where I like to hang out, the big question has been: "What weights should we use?" So for those who care, I took it as a little project to see if I could find any differences in the 2006 Actuals vs. projections. I think the people who use PD might find it interesting.and possibly useful..if you don't, my feelings aren't hurt.

FWIW, in my two big leagues, last year I won one, missed winning the second by two points and by steering two buddies to the DD, watched one go from worst to first and one make the playoffs for the first time in 4 years...

Joel

 

[GregR_2]

...I know it took a while to do this, I'm thankful you did it, but basically all you're saying is that most `experts ` think the same....people who get all worked up with numbers, I find, don't draft well..they are too consumed with numbers, to notice that Peyton Manning is sitting there,waiting to be drafted....

Actually, he's displaying the difference between himself and those who don't like stats. People like him test hypothesis against reality and determine if what he (or we) THOUGHT is actually true.I find many people who react negatively to the use of statistics in FF tend to follow conventional wisdom and rules of thumb that very well may be wrong, but they'll never know because they won't bother to test it.One of the reasons that FBGs is such a great site is because people come in here knowing how to draft by the stud RB theory, and they leave understanding why the stud RB theory works, and even more important, how to tell when it isn't best. People come here often believing falsehoods like guys who were consistent last year should be expected to be consistent next year, or that being injured previously is an indication the player is likely to miss a significant amount more time in the future than those who were healthy.Luckily we have people who as well as having gut feelings about players, realize there is a wealth of information to be gained by actually testing reality and not just assuming that our perception of it is correct.

 

[SSOG]

Hey GoBears, this is awesome, and I really appreciate the time you put into it. My one request is could you possibly create a "FBGs Composite Rankings" and compare them using the same methodology, as if there were an extra ranker whose predictions were the average of all the other FBGs? It'd be interesting to see if the whole really was greater than the sum of the parts.

 

[rdstorm]

...I know it took a while to do this, I'm thankful you did it, but basically all you're saying is that most `experts ` think the same....people who get all worked up with numbers, I find, don't draft well..they are too consumed with numbers, to notice that Peyton Manning is sitting there,waiting to be drafted....

Actually, he's displaying the difference between himself and those who don't like stats. People like him test hypothesis against reality and determine if what he (or we) THOUGHT is actually true.I find many people who react negatively to the use of statistics in FF tend to follow conventional wisdom and rules of thumb that very well may be wrong, but they'll never know because they won't bother to test it.One of the reasons that FBGs is such a great site is because people come in here knowing how to draft by the stud RB theory, and they leave understanding why the stud RB theory works, and even more important, how to tell when it isn't best. People come here often believing falsehoods like guys who were consistent last year should be expected to be consistent next year, or that being injured previously is an indication the player is likely to miss a significant amount more time in the future than those who were healthy.Luckily we have people who as well as having gut feelings about players, realize there is a wealth of information to be gained by actually testing reality and not just assuming that our perception of it is correct.

I appreciate the work GoBears, even if I do have to look up what some of it means. I have been thinking of doing something similar across the various FF sites for a couple of years but never found the time.I do agree that it could be even more effective if translated into language that statistics novices or even laymen could understand.

 

[GoBears84]

Hey GoBears, this is awesome, and I really appreciate the time you put into it. My one request is could you possibly create a "FBGs Composite Rankings" and compare them using the same methodology, as if there were an extra ranker whose predictions were the average of all the other FBGs? It'd be interesting to see if the whole really was greater than the sum of the parts.

This actually wouldn't be too hard to do, if somebody has the rankings from last year. I have the data actually arranged as RB01- RBXX, QB01 - QBxx, etc. All one would need to do is put the expert predictions in order and in the same format and calculate the residuals.My question is, if you are considering the impact on VBD, how accurate is it? I realize most of the experts here give rankings, not predictions, but for VBD, aren't predictions more important? Even in tier based drafting, you have to understand where each tier is divided.If somebody has last years rankings I can take a quick look and see if I can make a quick analysis.

 

[GoBears84]

I appreciate the work GoBears, even if I do have to look up what some of it means. I have been thinking of doing something similar across the various FF sites for a couple of years but never found the time.I do agree that it could be even more effective if translated into language that statistics novices or even laymen could understand.

I tried to break down into raw numbers and, in the end, the implication of the results. Any suggestions you might have for simplifying my explanation of the analysis would be appreciated.

 

[Musesboy]

If somebody has last years rankings I can take a quick look and see if I can make a quick analysis.

I have the 2006 rankings saved in email format copied directly from the site. They are sortable by position and by staff member. If you PM me your email address, I'll send them now.

 

[BGP]

FWIW, as I know he is going to be his usual modest self, in the five years Bob Henry has competed in this poll he has never finished out of the top seven. He has won it twice outright and I believe narrowly missed a third title. He finished fourth last year. Nobody else can come close to that level of consistency, IIRC, except the FBG collective. Between David, Joe, Woodrow and later Bob, I believe FBG has placed a staff member in the top five for the last four seasons.

If I'm reading this correctly, the Index has 15, then 20, then 25 experts? To be top 5 among 15 people doesn't tell me much. The top 33%? That's about like the top 11 out of 32 NFL teams. That's like equating the 06 Giants with the 06 Colts. Even out of 25, the top 5 are still the top 20%. Its time to brag if maybe you make the top 5% or something of similar, of course that leaves you with at best the top 2 in a 25-man field.Its more interesting to note that Bob Henry ranks 1st in the all-time polling than to "brag" that FBGs finishes in the top 20%.

 

[GoBears84]

This is the final summary of my attempt to analyze the predictions from 2006 Projections Dominator as compared to the year’s final results. The individual analysis by position can be found at the following links:

PK,

QB's,

RB's,

WR’s

TE’s and

Team D.

Pooling all the results into the top 100 (actually 96), the means of the residuals were:

Top 96………….......All

Dodds PG…………..3.32

Henry PG…………..3.39

Smith PG…………..3.19

Tremblay PG……....3.41

Wodd PG…………..3.41

Average..........3.34

And the variances were:

Top 96………….......All

Dodds PG…………..4.52

Henry PG…………..4.48

Smith PG…………..4.38

Tremblay PG….……4.54

Wodd PG…………..4.82

Average............4.43

What does it all mean? My analysis have suggested that overall, the predictions of the experts are statistically similar and that the differences that do exist are minor and probably due to chance.

However, there are some differences and I think this is what make the PD so powerful. Instead of relying on just the projections of Dodds in DD, the weight can be spread across the experts. Therefore if someone misses badly on a prediction, the impact is not as significant.

Joel

I received a request to look at the Average of all the experts compared to each of them individually. This was very easy to do since I already have the data formatted. The data is added above with a bold emphasis. Essentially it appears to prove my original point - by using the weighted averages of the experts in the PD, your overall variance will be less than if you focused on just one expert. This is even more apparent when you look at a graph of the data by expert and see that the Average has no extreme outliers, but all but one of the experts does.

Joel

 

[Jene Bramel]

FWIW, as I know he is going to be his usual modest self, in the five years Bob Henry has competed in this poll he has never finished out of the top seven. He has won it twice outright and I believe narrowly missed a third title. He finished fourth last year. Nobody else can come close to that level of consistency, IIRC, except the FBG collective. Between David, Joe, Woodrow and later Bob, I believe FBG has placed a staff member in the top five for the last four seasons.

If I'm reading this correctly, the Index has 15, then 20, then 25 experts? To be top 5 among 15 people doesn't tell me much. The top 33%? That's about like the top 11 out of 32 NFL teams. That's like equating the 06 Giants with the 06 Colts. Even out of 25, the top 5 are still the top 20%. Its time to brag if maybe you make the top 5% or something of similar, of course that leaves you with at best the top 2 in a 25-man field.Its more interesting to note that Bob Henry ranks 1st in the all-time polling than to "brag" that FBGs finishes in the top 20%.

I was "bragging" about the consistency of the finish. I don't disagree that one "top 20%" finish isn't particularly impressive in a vacuum. Consistent top five finishes within the same group of "experts", none of whom have produced similar results, is noteworthy. Using your example, that would make Henry the Patriots of the Fantasy Index Experts Poll.

 

[GoBears84]

FWIW, as I know he is going to be his usual modest self, in the five years Bob Henry has competed in this poll he has never finished out of the top seven. He has won it twice outright and I believe narrowly missed a third title. He finished fourth last year. Nobody else can come close to that level of consistency, IIRC, except the FBG collective. Between David, Joe, Woodrow and later Bob, I believe FBG has placed a staff member in the top five for the last four seasons.

If I'm reading this correctly, the Index has 15, then 20, then 25 experts? To be top 5 among 15 people doesn't tell me much. The top 33%? That's about like the top 11 out of 32 NFL teams. That's like equating the 06 Giants with the 06 Colts. Even out of 25, the top 5 are still the top 20%. Its time to brag if maybe you make the top 5% or something of similar, of course that leaves you with at best the top 2 in a 25-man field.Its more interesting to note that Bob Henry ranks 1st in the all-time polling than to "brag" that FBGs finishes in the top 20%.

I was "bragging" about the consistency of the finish. I don't disagree that one "top 20%" finish isn't particularly impressive in a vacuum. Consistent top five finishes within the same group of "experts", none of whom have produced similar results, is noteworthy. Using your example, that would make Henry the Patriots of the Fantasy Index Experts Poll.

Although you've the implication of your post proves my point.

Between David, Joe, Woodrow and later Bob, I believe FBG has placed a staff member in the top five for the last four seasons.....Consistent top five finishes within the same group of "experts", none of whom have produced similar results, is noteworthy.

The perfromance of each expert varies from year to year. Sometime one is better than another. Consistent top five finishes within the same group of experts suggests that pooling their projections will give you better results than trying to guess which expert will give you the best projections in a given year.Joel

 

[patsfangr]

Hey Joel! You steered me here from the Applications Questions Forum when I asked the "How do you weigh the experts" question. Great stuff! I want to add my thanks to those of others for all your work. Although, having spent far too many hours on "meaningless" hobby stuff myself, I know that it was a "labor of love". (Addictions come in all forms, right? For some of us, hobbies like FF are our cocaine. We just can't get enough of this stuff! But at least it doesn't make our noses disintegrate!)

Seriously, I understand the point of "NY Giants". I feel the same way about the people who spend countless hours figuring out the Salary Cap situation for teams, because the "business side" of football both pisses me off (I HATE Free Agency!), and bores the hell out of me. If I want financial news, I'll read the Wall Street Journal instead of sports rags! ;-)

What "NY Giants" is missing is that this IS a hobby. We are all free to put into it as much or as little time as we choose, and nobody is hurt by it. So if some of us like to play with the stats, let it go. No need to berate a guy because he likes to travel by train, and you prefer to get there faster on a plane.

Let's face it, FF always comes down to a lot of luck. Injuries are totally unpredictable in the NFL. An injury to any player on a team can significantly affect the numbers of your player, even if he's NOT injured. How good will the stats of Wayne and Harrison be if Manning goes down? At least the use of stats created by people with some inside knowledge of the game - people who do this for a living, and spend much of their time studying the reports and data from NFL teams - give us a better chance of making good decisions than our "gut feel" from watching lots of games on TV, and reading a few mags in our leisure time.

Thanks again for the work, Joel. Keep it up. The more, the better, as far as I'm concerned. I now consider you the "5th expert" at FBG site!

 

[patsfangr]

Joel, I ran into a significant problem. As I was beginning work with the PD, I noticed that not all experts rated all players. For example, Smith has no ratings entered for Jamarcus Russell. (Lots more. That's just one example.) This seriously screws up the weighted average!

Now what?

 

[GoBears84]

George,

The best place to answer this question is back in the applications area. Check out this thread on the Projections dominator and bring up the question again there. The best you can do is set Russels projections yourself.

Joel