Why do rankings use averages for consensus? (1 Viewer)

[freshly_shorn]

I'm fresh out of a graduate statistics class and I learned something that applies to all those rankings we see out there. Did you know that it is incorrect to use an average for rankings? You should be using a median. This is because rankings are ordinal, meaning while the order of numbers has significance, the difference between numbers do not (example- being the 1st ranked player doesn't mean the 10th place player is 10 times worse). It is not meaningful to average or mathematically manipulate ordinal (ranking) data in any way.

Now, you *can* average projected fantasy points or other stats- that is ratio data, meaning you can use whatever math you'd like to interpret the number (averages, ratios, differences for VBD, and the like).

Medians do generate a different- and accurate- consensus. You can prove it using Excel.

Forgive the geek factor- but this game is all about stats, and we should be interpretting it the correct way.

 

[phthalatemagic]

I'm fresh out of a graduate statistics class and I learned something that applies to all those rankings we see out there. Did you know that it is incorrect to use an average for rankings? You should be using a median. This is because rankings are ordinal, meaning while the order of numbers has significance, the difference between numbers do not (example- being the 1st ranked player doesn't mean the 10th place player is 10 times worse). It is not meaningful to average or mathematically manipulate ordinal (ranking) data in any way.Now, you *can* average projected fantasy points or other stats- that is ratio data, meaning you can use whatever math you'd like to interpret the number (averages, ratios, differences for VBD, and the like).Medians do generate a different- and accurate- consensus. You can prove it using Excel.Forgive the geek factor- but this game is all about stats, and we should be interpretting it the correct way.

you aint the only one my friend!

 

[Clayton Gray]

Consider the following possible rankings:

Player A - 1, 1, 1, 3, 3, 3, 3

Player B - 2, 2, 2, 2, 4, 4, 4

Are you saying it is more accurate to list Player B ahead of Player A?

 

[Clayton Gray]

Consider the following possible rankings:

Player A - 1, 1, 1, 3, 3, 3

Player B - 2, 2, 2, 2, 4, 4

Are you saying it is more accurate to say these players are evenly ranked?

 

[David Yudkin]

Consider the following possible rankingslayer A - 1, 1, 1, 3, 3, 3Player B - 2, 2, 2, 2, 4, 4Are you saying it is more accurate to say these players are evenly ranked?

Are you saying that this post should come first?

 

[David Yudkin]

Consider the following possible rankingslayer A - 1, 1, 1, 3, 3, 3Player B - 2, 2, 2, 2, 4, 4Are you saying it is more accurate to say these players are evenly ranked?

Or are you saying that this post should come first?

 

[videoguy505]

Consider the following possible rankingslayer A - 1, 1, 1, 3, 3, 3, 3Player B - 2, 2, 2, 2, 4, 4, 4Are you saying it is more accurate to list Player B ahead of Player A?

4 out of 7 think A is #3.4 out of 7 think B is #2.

 

[Chase Stuart]

Consider the following possible rankingslayer A - 1, 1, 1, 3, 3, 3, 3Player B - 2, 2, 2, 2, 4, 4, 4Are you saying it is more accurate to list Player B ahead of Player A?

I don't care what anyone else says, Clayton Gray knows math.

 

[Chase Stuart]

Consider the following possible rankingslayer A - 1, 1, 1, 3, 3, 3, 3Player B - 2, 2, 2, 2, 4, 4, 4Are you saying it is more accurate to list Player B ahead of Player A?

4 out of 7 think A is #3.4 out of 7 think B is #2.

6/7 think A is better than B, while 1/7 think B is better than A (and only by one spot, too).

 

[videoguy505]

Consider the following possible rankingslayer A - 1, 1, 1, 3, 3, 3, 3Player B - 2, 2, 2, 2, 4, 4, 4Are you saying it is more accurate to list Player B ahead of Player A?

4 out of 7 think A is #3.4 out of 7 think B is #2.

6/7 think A is better than B, while 1/7 think B is better than A (and only by one spot, too).

... if we assume they are listed in order of who's doing the picking, and haven't been re-ordered by increasing order of number to figure out the median.If these are expert rankings by 7 staffers, and they went:

Staffer	1 2 3 4 5 6 7A Rank	 1 1 1 3 3 3 3 B Rank	 4 4 4 2 2 2 2

Then 4 of 7 staffers think B is better than A.Also, "by one spot, too"... remember that being 1 spot better isn't an ordinal amount of improvement. #1 isn't necessarily the same amount better than #10 as #91 is from #100.

 

[fatness]

Rock, paper, scissors is the only real method.

 

[massraider]

Rock, paper, scissors is the only real method.

Obviously a man that doesn't own a Magic 8-Ball.I pity you.

 

[Couch Potato]

Poor mode just gets no love around here.

 

[phthalatemagic]

Euclid was a #####

 

[Clayton Gray]

Consider the following possible rankingslayer A - 1, 1, 1, 3, 3, 3, 3Player B - 2, 2, 2, 2, 4, 4, 4Are you saying it is more accurate to list Player B ahead of Player A?

4 out of 7 think A is #3.4 out of 7 think B is #2.

6/7 think A is better than B, while 1/7 think B is better than A (and only by one spot, too).

What about thislayer A - 1, 1, 1, 5, 5, 5Player B - 2, 2, 2, 4, 6, 6Player C - 3, 3, 3, 3, 7, 7Is that a three-way tie?

 

[phthalatemagic]

Clayton - are you 100% satisfied with mean finding a consensus? I think it's clumsy, but I don't have a better solution. I like using a median, but I'm not looking for a consensus when I look at the rankings. I'm looking at the typical ranking for each guy.

 

[Maurile Tremblay]

If these are expert rankings by 7 staffers, and they went:

Staffer	1 2 3 4 5 6 7A Rank	 1 1 1 3 3 3 3 B Rank	 4 4 4 2 2 2 2

Then 4 of 7 staffers think B is better than A.

That's a hand-picked example trying to get a majority to favor B. There are many more combinations of A=1113333/B=2222444 that will have the majority favor A.You're right that rankings aren't proportional such that the difference between 1 and 2 will necessarily be the same as the distance between 2 and 3. But the distance between 2 and 3 will generally be smaller than the distance between 1 and 4. (For any particular person, this is necessarily true unless 1, 2, 3, and 4 are all tied with each other. But even comparing rankings from different people, the distance between one person's 2 and 3 will be smaller, on average, than the distance between another person's 1 and 4.)I think any combination of A=1113333/B=2222444 should put A ahead of B.

 

[Clayton Gray]

Medians do generate a different- and accurate- consensus. You can prove it using Excel.

Also interested in this proof.

 

[Clayton Gray]

Clayton - are you 100% satisfied with mean finding a consensus? I think it's clumsy, but I don't have a better solution. I like using a median, but I'm not looking for a consensus when I look at the rankings. I'm looking at the typical ranking for each guy.

I honestly haven't put too much thought into it.

 

[Maurile Tremblay]

Clayton - are you 100% satisfied with mean finding a consensus? I think it's clumsy, but I don't have a better solution. I like using a median, but I'm not looking for a consensus when I look at the rankings. I'm looking at the typical ranking for each guy.

I honestly haven't put too much thought into it.

I would take the mean of the mean, median, and mode. Or would I take the median of the mean, mode, and median?I would probably do it both ways and then average the results.So take the mean of the mean of the mean, median, and mode and the median of the mean, mode, and median. Yep, that's the ticket.

 

[BGP]

FBG should develop a system awarding points for 1st place votes, 2nd place votes, etc., and develop its own BCS.

 

[phthalatemagic]

Clayton - are you 100% satisfied with mean finding a consensus? I think it's clumsy, but I don't have a better solution. I like using a median, but I'm not looking for a consensus when I look at the rankings. I'm looking at the typical ranking for each guy.

I honestly haven't put too much thought into it.

The rankings list is one of your products. Who does think about it?

 

[Maurile Tremblay]

So take the mean of the mean of the mean, median, and mode and the median of the mean, mode, and median. Yep, that's the ticket.

By the way, even though I've goofed off a bit in this thread, I do agree with freshly_shorn's criticisms of using the mean.I'm just not sure that using the median would be any less problematical. It wouldn't be that hard to figure out by playing around in Excel for a few minutes, which I'll do later. But if freshly_shorn wants to post his proof that would be helpful as well.

 

[freshly_shorn]

Consider the following possible rankingslayer A - 1, 1, 1, 3, 3, 3Player B - 2, 2, 2, 2, 4, 4Are you saying it is more accurate to say these players are evenly ranked?

Nope. I'm saying using averages for ordinal data is incorrect. A median is the only valid way to represent middle ground between lists of rankings. We all agree that we're trying to find a consensus between rankings, correct? Well, the only statistically valid way to do that is with a median.The better way to represent if a player is better or worse than another is- as the footballguys do- is to give you projections of actual stats. Then you're talkin' turkey!

 

[Chase Stuart]

So take the mean of the mean of the mean, median, and mode and the median of the mean, mode, and median. Yep, that's the ticket.

By the way, even though I've goofed off a bit in this thread, I do agree with freshly_shorn's criticisms of using the mean.I'm just not sure that using the median would be any less problematical. It wouldn't be that hard to figure out by playing around in Excel for a few minutes, which I'll do later. But if freshly_shorn wants to post his proof that would be helpful as well.

I'm not sure any method is perfect. Using the mean has some technical drawbacks, in that it does equate a different of X ranks to be a difference of X value, across the rankings spectrum. But I don't think that's such an outlandish proxy as to distort the results in any meaningful way.

 

[JayMan]

Clayton - are you 100% satisfied with mean finding a consensus? I think it's clumsy, but I don't have a better solution. I like using a median, but I'm not looking for a consensus when I look at the rankings. I'm looking at the typical ranking for each guy.

I honestly haven't put too much thought into it.

I would take the mean of the mean, median, and mode. Or would I take the median of the mean, mode, and median?I would probably do it both ways and then average the results.So take the mean of the mean of the mean, median, and mode and the median of the mean, mode, and median. Yep, that's the ticket.

What if the mode of the mean, median and mode is not unique?... do you also include the mode of the mean, median and mode in the mean of the mean of the mean, median, and mode and the median of the mean, mode, and median?

 

[freshly_shorn]

You also have to extend this argument to ADP. Again, we're talking rankings. ADP is not a statistically valid measurement. Nor is standard deviation.

If you really want to get a handle on what's happening between rankings, you probably want to use a median and frequencies. How many times was someone picked 1st, 2nd, 3rd. etc.

I will post an example of 'proof' later tonight, but all I did was copy the WR rankings from football guys, dump them in Excel, put in the median, and resort on the median. You'll definitely see some players shuffled around- not necessarily the highly ranked ones, where there is a lot of consensus- but the mid to bottom players, where rankings differ quite a bit.

 

[Buckna]

FWIW, I think it would be cool to see median's across a large mock draft dataset. I think that would be a useful tool in addition to averages.

 

[freshly_shorn]

FWIW, I think it would be cool to see median's across a large mock draft dataset. I think that would be a useful tool in addition to averages.

Except that averages are 'meaningless'! Think about this- the BCS system is based on a statistically invalid operation of averaging polls! What the hell do these college people know!

 

[Clayton Gray]

Clayton - are you 100% satisfied with mean finding a consensus? I think it's clumsy, but I don't have a better solution. I like using a median, but I'm not looking for a consensus when I look at the rankings. I'm looking at the typical ranking for each guy.

I honestly haven't put too much thought into it.

The rankings list is one of your products. Who does think about it?

I put a lot of thought into my rankings. However, until this thread, I put zero thought into whether the mean was the most valid method of finding consensus. Please forgive me. Now, after seeing major flaws in using the median, I still prefer the mean.

 

[Clayton Gray]

I will post an example of 'proof' later tonight

 

[Maurile Tremblay]

I've played around a bit in Excel.

I consider a set of consensus rankings to be "correct" if they accurately reflect the mean underlying projections.

By playing around with a bunch of projections, and using those projections to generate rankings, and then looking at the mean of the rankings and the median of the rankings in order to compare them to the rankings based on the underlying projections, I have come to the following conclusions:

1. The mean of the rankings is not always an accurate reflection of the mean of the underlying projections.

2. The median of the rankings is not always an accurate reflection of the mean of the underlying projections.

3. Often when the median ranking is off, the mean ranking is similarly off (i.e., the mean rankings and median rankings agree with each other even when they don't agree with the rankings based on mean projections).

4. When the median ranking and mean ranking disagree with each other, there's no obvious pattern I've seen as to which is more likely to be closer to the ranking based on mean projections.

So in conclusion, using the mean doesn't seem to have a huge advantage over using the median or vice versa, at least in my initial playing around.

But if freshly_shorn has a proof that using the median is more likely to work better, I'd be happy to take a look at it.

 

[CalBear]

One problem is that neither mean nor median reflects "consensus." It would be better just to report it as mean (or median, or, better, both) instead of using an inappropriate word. If 10 people rate someone as #1, and I rate him as #10, that indicates that there is no consensus at all; arguing over whether the consensus is #1 or #1.9 really misses the point.

For the record, Wall Street makes the same mistake, too, taking an average rating of a number of analysts and referring to it as a consensus rating.

 

[videoguy505]

I've played around a bit in Excel.I consider a set of consensus rankings to be "correct" if they accurately reflect the mean underlying projections.By playing around with a bunch of projections, and using those projections to generate rankings, and then looking at the mean of the rankings and the median of the rankings in order to compare them to the rankings based on the underlying projections, I have come to the following conclusions:1. The mean of the rankings is not always an accurate reflection of the mean of the underlying projections.2. The median of the rankings is not always an accurate reflection of the mean of the underlying projections.3. Often when the median ranking is off, the mean ranking is similarly off (i.e., the mean rankings and median rankings agree with each other even when they don't agree with the rankings based on mean projections).4. When the median ranking and mean ranking disagree with each other, there's no obvious pattern I've seen as to which is more likely to be closer to the ranking based on mean projections.So in conclusion, using the mean doesn't seem to have a huge advantage over using the median or vice versa, at least in my initial playing around.But if freshly_shorn has a proof that using the median is more likely to work better, I'd be happy to take a look at it.

I see that point as well. What would be more useful in judging ranking would be to see the projections that each of the staffers put behind them. If the spread between the #8 and #16 guys, or the #50 to the #65 guys, are very wide or very narrow, that is very important.

 

[freshly_shorn]

I've played around a bit in Excel.

I consider a set of consensus rankings to be "correct" if they accurately reflect the mean underlying projections.

By playing around with a bunch of projections, and using those projections to generate rankings, and then looking at the mean of the rankings and the median of the rankings in order to compare them to the rankings based on the underlying projections, I have come to the following conclusions:

1. The mean of the rankings is not always an accurate reflection of the mean of the underlying projections.

2. The median of the rankings is not always an accurate reflection of the mean of the underlying projections.

3. Often when the median ranking is off, the mean ranking is similarly off (i.e., the mean rankings and median rankings agree with each other even when they don't agree with the rankings based on mean projections).

4. When the median ranking and mean ranking disagree with each other, there's no obvious pattern I've seen as to which is more likely to be closer to the ranking based on mean projections.

So in conclusion, using the mean doesn't seem to have a huge advantage over using the median or vice versa, at least in my initial playing around.

But if freshly_shorn has a proof that using the median is more likely to work better, I'd be happy to take a look at it.

First of all, Secondly, didn't mean to touch off a firestorm, and I'm not necessarily endorsing median as the best way to get a consensus.

Finally, all I wanted to accomplish is to point out that DERIVING THE MEAN/AVERAGE OF ORDINAL (RANKING) DATA IS STATISTICALLY MEANINGLESS. You could rank players using 1-100, 1000-1100, or 1,234,567- 1,234,667: the point is, the numbers in and of themselves are meaningless; therefore any calculation using those numbers is meaningless- technically speaking.

Here is a link that may help explain better than I can:

http://www.graphpad.com/faq/viewfaq.cfm?faq=1089

Personally, I think a combination of median and frequencies is likely the 'best' way to derive some sort of consensus- or lack thereof.

Even better would be to just create the list from projections- that is truly the best way. But not everyone actually does projections, so this is what we have.

 

[freshly_shorn]

One problem is that neither mean nor media reflects "consensus." It would be better just to report it as mean (or median, or, better, both) instead of using an inappropriate word. If 10 people rate someone as #1, and I rate him as #10, that indicates that there is no consensus at all; arguing over whether the consensus is #1 or #1.9 really misses the point. For the record, Wall Street makes the same mistake, too, taking an average rating of a number of analysts and referring to it as a consensus rating.

Yep- we're dealing with a phantom quantity here. What we are trying to do is show who is *most likely* the consensus choice to be ranked at such and such position.

 

[Chase Stuart]

I've played around a bit in Excel.

I consider a set of consensus rankings to be "correct" if they accurately reflect the mean underlying projections.

By playing around with a bunch of projections, and using those projections to generate rankings, and then looking at the mean of the rankings and the median of the rankings in order to compare them to the rankings based on the underlying projections, I have come to the following conclusions:

1. The mean of the rankings is not always an accurate reflection of the mean of the underlying projections.

2. The median of the rankings is not always an accurate reflection of the mean of the underlying projections.

3. Often when the median ranking is off, the mean ranking is similarly off (i.e., the mean rankings and median rankings agree with each other even when they don't agree with the rankings based on mean projections).

4. When the median ranking and mean ranking disagree with each other, there's no obvious pattern I've seen as to which is more likely to be closer to the ranking based on mean projections.

So in conclusion, using the mean doesn't seem to have a huge advantage over using the median or vice versa, at least in my initial playing around.

But if freshly_shorn has a proof that using the median is more likely to work better, I'd be happy to take a look at it.

First of all, Secondly, didn't mean to touch off a firestorm, and I'm not necessarily endorsing median as the best way to get a consensus.

Finally, all I wanted to accomplish is to point out that DERIVING THE MEAN/AVERAGE OF ORDINAL (RANKING) DATA IS STATISTICALLY MEANINGLESS. You could rank players using 1-100, 1000-1100, or 1,234,567- 1,234,667: the point is, the numbers in and of themselves are meaningless; therefore any calculation using those numbers is meaningless- technically speaking.

Here is a link that may help explain better than I can:

http://www.graphpad.com/faq/viewfaq.cfm?faq=1089

Personally, I think a combination of median and frequencies is likely the 'best' way to derive some sort of consensus- or lack thereof.

Even better would be to just create the list from projections- that is truly the best way. But not everyone actually does projections, so this is what we have.

Touching off a firestorm isn't a bad thing. I think we're all in agreement that averaging ordinal data may be flawed. The key, there, is may be.Let's assume I've got projections of WRsA-D of 400 FPs, 399 FPs, 398 FPs and 100 FPs. Maurile's got projections for WRsA-D of 400 FPs, 300 FPs, 100 FPs and 102 FPs.

Maurile and I, on average, expect WR C to score 249 FPs and WR D to score 101 FPs. That's a pretty significant difference; "we" think WR C is much better than WR D. But our rankings would put them at 3, 4 and at 4, 3, respectively. Averaging the ranks improperly makes them appear equal. That's why you shouldn't average ordinal numbers. It eliminates a difference of 147 FPs with the click of a mouse.

On the other hand, I could have WRsA-D projected at 400 FPs, 300 FPs, 200 FPs and 100 FPs, and Clayton could have WRsA-D projected at 300 FPs, 400 FPs, 200 FPs and 100 FPs. Clayton and I project WRA and B evenly: both at 350 FPs. If we were to average our rankings...WR A would be 1.5 and WR B would be 1.5. In other words, the average rankings would perfectly mirror how we feel.

The question, then, is how often will the FBG projections/feelings behind the ranking reflect the disparty between my rankings and Maurile's rankings, and how often will they mirror the Clayton/Chase rankings. If it's the latter, we don't have to worry much about this averaging problem. If it's the former, we do. I suspect practically, that it's much more like the latter, although I'm unsure exactly how close it is.

(Note: this has nothing to do with using the median instead of the mean, but more generally, whether averaging the rankings is permissible.)

 

[guderian]

Consider the following possible rankingslayer A - 1, 1, 1, 3, 3, 3, 3Player B - 2, 2, 2, 2, 4, 4, 4Are you saying it is more accurate to list Player B ahead of Player A?

Player A - 1, 1, 1, 1, 1, 1, 100Player B - 2, 2, 2, 2, 2, 2, 2Are you saying it is more accurate to list player B ahead of player A?

 

[Chase Stuart]

Consider the following possible rankingslayer A - 1, 1, 1, 3, 3, 3, 3Player B - 2, 2, 2, 2, 4, 4, 4Are you saying it is more accurate to list Player B ahead of Player A?

Player A - 1, 1, 1, 1, 1, 1, 100Player B - 2, 2, 2, 2, 2, 2, 2Are you saying it is more accurate to list player B ahead of player A?

It depends how you define accurate. If you have reason to believe that all rankers are equally likely to be correct, then yes, it is more accurate to list player B ahead of player A. That may not always be the case, though. For example, maybe Jeff Pasquino heard that Antonio Gates was out for the season, so he ranked him 100. If Pasquino's information was wrong, then we shouldn't have player B ahead of player A. If Pasquino's information is correct, then it was correct to have player B to be placed ahead of player A (and all the other rankings are wrong). If you think that a string of similar values increases the likelihood of being accurate, then it is inaccurate to list player B ahead of player A. If six people view Player A at 1, and only one at 100, we might think the 7th person is nuts, and we should ignore those rankings. On the other hand, FBG gets accused all the time of "group think", so perhaps this isn't such a hot idea.Really depends on your definition of accurate.ETA: This is really a different problem than the one we've been addressing in this thread, so this may be a poor sidetrack. While mode or mean/median after deleting outliers is a useful measure of central tendency, the question hear is how to deal with ordinal rankings that need to be combined. Using projections gets around that problem; using projections does not get around the outlier problem you've described here. If we saw Player A projected with 300 FPs by 6 people and 0 FPs by a 7th, and Player B projected with 290 FPs by 7 people, the ordinal problem would disappear but the outlier problem would not.

 

[guderian]

Consider the following possible rankingslayer A - 1, 1, 1, 3, 3, 3, 3Player B - 2, 2, 2, 2, 4, 4, 4Are you saying it is more accurate to list Player B ahead of Player A?

Player A - 1, 1, 1, 1, 1, 1, 100Player B - 2, 2, 2, 2, 2, 2, 2Are you saying it is more accurate to list player B ahead of player A?

It depends how you define accurate. If you have reason to believe that all rankers are equally likely to be correct, then yes, it is more accurate to list player B ahead of player A. That may not always be the case, though. For example, maybe Jeff Pasquino heard that Antonio Gates was out for the season, so he ranked him 100. If Pasquino's information was wrong, then we shouldn't have player B ahead of player A. If Pasquino's information is correct, then it was correct to have player B to be placed ahead of player A (and all the other rankings are wrong). If you think that a string of similar values increases the likelihood of being accurate, then it is inaccurate to list player B ahead of player A. If six people view Player A at 1, and only one at 100, we might think the 7th person is nuts, and we should ignore those rankings. On the other hand, FBG gets accused all the time of "group think", so perhaps this isn't such a hot idea.Really depends on your definition of accurate.

Ask Clayton, he posed the original question. There are problems with both methods, for analyzing stocks we've always used both methods and even through in an "adjusted mean" that omits the high and low. I was merely trying to highlight the fact that a single example doesn't lay the issue to rest.

 

[freshly_shorn]

Again, whether or not you *think* averaging rankings is OK, every trained statistician on Earth will tell you that is a meaningless exercise. But Chase Stuart used FP- and as I mentioned before, that is the best way to do this.

Aside from that, here is my demonstration, taken from today's Top 30 Expert Rankings (I don't know how to make in line up pretty, sorry):

Mean = Average (not appropriate but included for comparison)

Median = The value of the number in the exact middle of all the values

Mode = The most frequently occuring value (#NA means nothing occurred more than the others)

Sorted by Mean

Mean Median Mode

1 RB LaDainian Tomlinson, 1 1 1

2 RB Steven Jackson, STL 2.4 2 2

3 RB Larry Johnson, KC 2.8 3 3

4 RB Frank Gore, SF 5.5 5 4

5 RB Shaun Alexander, SEA 7 8 9

6 RB Brian Westbrook, PHI 7.6 7 5

7 RB Willie Parker, PIT 8.5 9 10

8 RB Joseph Addai, IND 10.4 8 6

9 RB Rudi Johnson, CIN 10.5 8 8

10 QB Peyton Manning, IND 12 13 14

11 RB Reggie Bush, NO 12.4 13 13

12 RB Clinton Portis, WAS 13.4 13 7

13 WR Chad Johnson, CIN 13.8 14 9

14 RB Laurence Maroney, NE 14.8 14 14

15 RB Travis Henry, DEN 15.3 13 #N/A

16 WR Steve Smith, CAR 17.8 16 15

17 RB Maurice Jones-Drew, 17.9 17 17

18 WR Larry Fitzgerald, 19.8 19 14

19 RB Willis McGahee, BAL 20.9 16 10

20 WR Torry Holt, STL 21.7 21 15

21 RB Ronnie Brown, MIA 22.4 22 22

22 TE Antonio Gates, SD 22.9 23 18

23 WR Reggie Wayne, IND 24.5 25 25

24 RB Edgerrin James, ARI 25.1 24 24

25 WR Terrell Owens, DAL 25.6 25 31

26 QB Carson Palmer, CIN 27.6 27 26

27 WR Marvin Harrison, IND 28.1 31 33

28 WR Roy Williams, DET 30.8 33 37

29 RB Cedric Benson, CHI 32.6 30 #N/A

30 QB Tom Brady, NE 33.9 37 19

Sorted by Median

` Mean Median Mode

1 RB LaDainian Tomlinson, 1 1 1

2 RB Steven Jackson, STL 2.4 2 2

3 RB Larry Johnson, KC 2.8 3 3

4 RB Frank Gore, SF 5.5 5 4

6 RB Brian Westbrook, PHI 7.6 7 5

5 RB Shaun Alexander, SEA 7 8 9

8 RB Joseph Addai, IND 10.4 8 6

9 RB Rudi Johnson, CIN 10.5 8 8

7 RB Willie Parker, PIT 8.5 9 10

10 QB Peyton Manning, IND 12 13 14

11 RB Reggie Bush, NO 12.4 13 13

12 RB Clinton Portis, WAS 13.4 13 7

15 RB Travis Henry, DEN 15.3 13 #N/A

13 WR Chad Johnson, CIN 13.8 14 9

14 RB Laurence Maroney, NE 14.8 14 14

16 WR Steve Smith, CAR 17.8 16 15

19 RB Willis McGahee, BAL 20.9 16 10

17 RB Maurice Jones-Drew, 17.9 17 17

18 WR Larry Fitzgerald, 19.8 19 14

20 WR Torry Holt, STL 21.7 21 15

21 RB Ronnie Brown, MIA 22.4 22 22

22 TE Antonio Gates, SD 22.9 23 18

24 RB Edgerrin James, ARI 25.1 24 24

23 WR Reggie Wayne, IND 24.5 25 25

25 WR Terrell Owens, DAL 25.6 25 31

26 QB Carson Palmer, CIN 27.6 27 26

29 RB Cedric Benson, CHI 32.6 30 #N/A

27 WR Marvin Harrison, IND 28.1 31 33

28 WR Roy Williams, DET 30.8 33 37

30 QB Tom Brady, NE 33.9 37 19

So, you'll note that these are close- but different. And statistically legal.

I was thinking of writing an article on this.. wonder if there's any interest??

 

[Clayton Gray]

Consider the following possible rankingslayer A - 1, 1, 1, 3, 3, 3, 3Player B - 2, 2, 2, 2, 4, 4, 4Are you saying it is more accurate to list Player B ahead of Player A?

Player A - 1, 1, 1, 1, 1, 1, 100Player B - 2, 2, 2, 2, 2, 2, 2Are you saying it is more accurate to list player B ahead of player A?

Could be. If there is a legitimate reason for Player A to be ranked at 100, then it could better to consider Player B as the better player.

 

[Clayton Gray]

Consider the following possible rankingslayer A - 1, 1, 1, 3, 3, 3, 3Player B - 2, 2, 2, 2, 4, 4, 4Are you saying it is more accurate to list Player B ahead of Player A?

Player A - 1, 1, 1, 1, 1, 1, 100Player B - 2, 2, 2, 2, 2, 2, 2Are you saying it is more accurate to list player B ahead of player A?

It depends how you define accurate. If you have reason to believe that all rankers are equally likely to be correct, then yes, it is more accurate to list player B ahead of player A. That may not always be the case, though. For example, maybe Jeff Pasquino heard that Antonio Gates was out for the season, so he ranked him 100. If Pasquino's information was wrong, then we shouldn't have player B ahead of player A. If Pasquino's information is correct, then it was correct to have player B to be placed ahead of player A (and all the other rankings are wrong). If you think that a string of similar values increases the likelihood of being accurate, then it is inaccurate to list player B ahead of player A. If six people view Player A at 1, and only one at 100, we might think the 7th person is nuts, and we should ignore those rankings. On the other hand, FBG gets accused all the time of "group think", so perhaps this isn't such a hot idea.Really depends on your definition of accurate.

Ask Clayton, he posed the original question. There are problems with both methods, for analyzing stocks we've always used both methods and even through in an "adjusted mean" that omits the high and low. I was merely trying to highlight the fact that a single example doesn't lay the issue to rest.

Agreed.When there are more than 10 FBGs that submit ranking, we throw out the high and low as well. I like that more than the straight mean and more than the median.

 

[glock]

Clayton - are you 100% satisfied with mean finding a consensus? I think it's clumsy, but I don't have a better solution. I like using a median, but I'm not looking for a consensus when I look at the rankings. I'm looking at the typical ranking for each guy.

I honestly haven't put too much thought into it.

I would take the mean of the mean, median, and mode. Or would I take the median of the mean, mode, and median?I would probably do it both ways and then average the results.

So take the mean of the mean of the mean, median, and mode and the median of the mean, mode, and median. Yep, that's the ticket.

<johncleese>What if the mode of the mean, median and mode is not unique?... do you also include the mode of the mean, median and mode in the mean of the mean of the mean, median, and mode and the median of the mean, mode, and median?</johncleese>

 

[Clayton Gray]

So, you'll note that these are close- but different. And statistically legal.

You still haven't explained what makes one list better than another.

 

[freshly_shorn]

So, you'll note that these are close- but different. And statistically legal.

You still haven't explained what makes one list better than another.

Dude, I've said it many times. At the risk of beating a dead horse and putting it as plainly as possible:IT IS WRONG TO AVERAGE RANKINGS.You can derive the median, mode, count the frequency at which rankings occur, but, statistically, it is INCORRECT to average them.Just food for thought. When I said the median is a better way to do it, I should have said the median is the *proper* way to do it, if you're not using projections.

 

[Clayton Gray]

IT IS WRONG TO AVERAGE RANKINGS.

OK....why is this?The link you provided earlier stated it was incorrect to average zip codes. I'll readily agree with that.

 

[Chase Stuart]

So, you'll note that these are close- but different. And statistically legal.

You still haven't explained what makes one list better than another.

Dude, I've said it many times. At the risk of beating a dead horse and putting it as plainly as possible:IT IS WRONG TO AVERAGE RANKINGS.You can derive the median, mode, count the frequency at which rankings occur, but, statistically, it is INCORRECT to average them.Just food for thought. When I said the median is a better way to do it, I should have said the median is the *proper* way to do it, if you're not using projections.

I don't think it's automatically wrong. I noticed you didn't respond to my earlier post, so maybe you missed it.

Let's assume I've got projections of WRsA-D of 400 FPs, 399 FPs, 398 FPs and 100 FPs. Maurile's got projections for WRsA-D of 400 FPs, 300 FPs, 100 FPs and 102 FPs.Maurile and I, on average, expect WR C to score 249 FPs and WR D to score 101 FPs. That's a pretty significant difference; "we" think WR C is much better than WR D. But our rankings would put them at 3, 4 and at 4, 3, respectively. Averaging the ranks improperly makes them appear equal. That's why you shouldn't average ordinal numbers. It eliminates a difference of 147 FPs with the click of a mouse.On the other hand, I could have WRsA-D projected at 400 FPs, 300 FPs, 200 FPs and 100 FPs, and Clayton could have WRsA-D projected at 300 FPs, 400 FPs, 200 FPs and 100 FPs. Clayton and I project WRA and B evenly: both at 350 FPs. If we were to average our rankings...WR A would be 1.5 and WR B would be 1.5. In other words, the average rankings would perfectly mirror how we feel.The question, then, is how often will the FBG projections/feelings behind the ranking reflect the disparty between my rankings and Maurile's rankings, and how often will they mirror the Clayton/Chase rankings. If it's the latter, we don't have to worry much about this averaging problem. If it's the former, we do. I suspect practically, that it's much more like the latter, although I'm unsure exactly how close it is.

 

[Chase Stuart]

IT IS WRONG TO AVERAGE RANKINGS.

OK....why is this?The link you provided earlier stated it was incorrect to average zip codes. I'll readily agree with that.

He's correct about this, as a matter of statistics. It's less clear whether he's correct with respect to FBG rankings.

 

[Clayton Gray]

IT IS WRONG TO AVERAGE RANKINGS.

OK....why is this?The link you provided earlier stated it was incorrect to average zip codes. I'll readily agree with that.

He's correct about this, as a matter of statistics. It's less clear whether he's correct with respect to FBG rankings.

OK....why is this?