Has anyone used the KUBIAK projections? (1 Viewer)

[SSOG]

Schatz actually just posted something relevant to this point. He has an article up ranking fantasy strength of schedule. Here's a good quote from it:

Let it be noted that these projections were done fairly quickly, and are not perfect. No projections are perfect, of course. On that note, while I’m talking about projections, I wanted to point out something I wrote on the new fantasy football open discussion board. Normally, I don’t plan on responding to stuff in there — I would prefer to be e-mailed directly — but I thought this was an important point.

The issue was the high projection for Donovan McNabb. If this projection is wrong, does it mean the tool is flawed? Here was my response:

OF COURSE the tool is flawed.

All tools are flawed. All of these statistical projection systems are flawed. Ours, and everyone else’s. When I throw 25 variables in there to project a quarterback’s performance, I can guarantee you there are at least 25 variables that would improve the projection but either a) I have not even thought of them yet, or b) we have no way of collecting the data in an orderly, objective fashion.

Every year, we seem to project the St. Louis offense too low. I’ve tried a ton of variables to fix that. Nothing has worked yet. It’s possible St. Louis has just enjoyed a couple of lucky years, and there’s nothing wrong with the projection systems. It is also possible we just haven’t figured out what is the missing “X” variable that would solve our St. Louis offensive issue.

(Note: This doesn’t change the fact that the Rams defense sucks.)

The team projections and the KUBIAK projections both improve each year as we gather new data and consider new ideas. That’s also why you will see FO people, including me, say things like “yeah, that number is weird, we don’t quite agree with that one.” A good example of this: I know the KUBIAK projections have Frank Gore ahead of LaDainian Tomlinson, but if I had the first pick in a fantasy draft, I would take LT, just like everyone else.

It’s something to remember when you see a projection that doesn’t seem to make sense — like the projection above for the Atlanta defense vs. fantasy running backs. It’s possible that the projection system is picking up something most people don’t understand. It’s also possible that we’re missing a variable that would improve things. If you think there’s a pattern of weird projections and you can figure out what variable they all have in common, feel free to e-mail me and suggest something for me to try in the projection systems. I’m all ears.

Sounds to me like Football Outsiders operates in exactly the opposite way that CalBear is accusing them of operating.

 

[CalBear]

Sounds to me like Football Outsiders operates in exactly the opposite way that CalBear is accusing them of operating.

They still list Frank Gore ahead of Ladanian Tomlinson, apparently. They can say "we don't agree with that, really," but that doesn't change the fact that the projection has it that way. What is the likelihood that Gore outscores a healthy Tomlinson in 2007? I'll say 10% and take the under. It fails the smell test.In their defense, they're trying to do something which is impossible; come up with a system which can project stats based solely on quantitative measures. There simply aren't enough data points to do it.

 

[SSOG]

Sounds to me like Football Outsiders operates in exactly the opposite way that CalBear is accusing them of operating.

They still list Frank Gore ahead of Ladanian Tomlinson, apparently. They can say "we don't agree with that, really," but that doesn't change the fact that the projection has it that way. What is the likelihood that Gore outscores a healthy Tomlinson in 2007? I'll say 10% and take the under. It fails the smell test.In their defense, they're trying to do something which is impossible; come up with a system which can project stats based solely on quantitative measures. There simply aren't enough data points to do it.

Like I said, I don't know about their success in fantasy projections yet, but their DVOA projections have been accused of failing to pass the "smell test" for years now, and yet they've soundly beaten every single "expert" power ranking in terms of predicting wins. When they predicted that San Diego would make the playoffs in 2004, that one didn't pass the "smell test", either, until the Chargers wound up going 12-4 and winning the AFC West. The rating system has been wrong before, and they identify where they think it might be wrong now, but they don't alter the numbers that the projection system spits out because the point is that it's been RIGHT before, too.

 

[AB in DC]

If you listen the the FBG podcast interview with Aaron Schatz of FO & the author of PFP, he says all of this and elaborates that their main goal is to predict team performance and why a team wins or loses. The projections sound like mainly a byproduct of that research, as it is not their main goal.

This is exactly why you don't want to put too much weight into their KUBIAK numbers -- the methodology was originally designed for other purposes, and sometimes the analysis doesn't translate well to fantasy projections.

 

[the_sig]

The stuff they provide about each defense's effectiveness against #1WR,#2WR,#3WR, TE etc is well worth a look too. Second table on the link below.

http://www.footballoutsiders.com/stats/teamdef.php

No surprise that Denver's defense was meanest against the opposition's #1WR, for instance.

Agreed, I used this for the first time last year and thought it was great. I didn't let it rule my lineup decisions, but it certainly helped. I'm sure I'll be reviewing that every week this year.

 

[NoFBinLA]

What I like about FO is they admit when they are wrong and are willing to go back to the drawing board to make things better. I rely heavily on KUBIAK now because, I think in the next 5 years, they will end up with the best projection system. There will be some growing pains, but I want to get on board the train early.

I think it's important for any site (or expert) to stand up an admit when they biff. It happens. And it can make a great article when you examine why it went wrong and what to look out for next time.

 

[lawrence phillips]

Anyone find the Kubiak IDP projections to be a little weird? Three Redskins corners in the top 10 in INTs (including David Macklin who might not even make the team)? Only 2 players reaching double digit sacks?

 

[hephner]

If you listen the the FBG podcast interview with Aaron Schatz of FO & the author of PFP, he says all of this and elaborates that their main goal is to predict team performance and why a team wins or loses. The projections sound like mainly a byproduct of that research, as it is not their main goal.

This is exactly why you don't want to put too much weight into their KUBIAK numbers -- the methodology was originally designed for other purposes, and sometimes the analysis doesn't translate well to fantasy projections.

Not going to disagree with you there. I'm not considering them to be the end all, be all. But I certainly will take their data and projections into consideration when tweaking my own. At the very least, they may help me break some ties when I have two or 3 players grouped closely together in a given tier, but I envision that I will be giving them a little more weight than just that even.By the way, there are a few sites out there that rate the FF website projections after the season, and last year's Kubiak projections didn't top the list, mainly because their QB projections were so off, which dragged down their overall ranking, but they were consistently ranked #1 in their RB projections. A few have mentioned that they re-assessed their system of projections with regard to QBs and receivers, so hopefully, those projections will be improved this year.

 

[Hear-the-Footsteps]

I swear I must be blind.

Where on their site do you even go to purchase their projections?

 

[AcousticNut]

I swear I must be blind.

Where on their site do you even go to purchase their projections?

http://www.footballoutsiders.com Front Page - Center Column - Top Features - Blue Box that says, "2007 Kubiak Fantasy Football Projectons Now Available For Download!"

Here's a direct link:

http://www.footballoutsiders.com/store/ind...p;products_id=5

 

[phthalatemagic]

I sure wish there were an easy way to plug them into the DD

 

[Willowdoc]

I sure wish there were an easy way to plug them into the DD

Well there might be. If you go to the FO boards there is this post:http://footballoutsiders.com/discussion/vi...4fbb28bc79dc335

The second post from Dr. Pepper says he'll send a way to do it for those who've paid for the Kubiak projections. I sent him an email, haven't heard back yet.

 

[phthalatemagic]

I sure wish there were an easy way to plug them into the DD

Well there might be. If you go to the FO boards there is this post:http://footballoutsiders.com/discussion/vi...4fbb28bc79dc335

The second post from Dr. Pepper says he'll send a way to do it for those who've paid for the Kubiak projections. I sent him an email, haven't heard back yet.

cool! I wish buying their PFP meant you didn't have to buy the digital version of the data too, but it's well worth it IMO.

 

[phthalatemagic]

Seems that email was bad.... please pm me if you manage to get the info....

 

[Josh the FunkDOC]

In case you guys don't know, you can change players' risk levels if you feel they're inaccurate. Pretty much all of the difference between LT and Gore, for instance, is Gore's "Green" rating; I think most of us here can agree that's overly optimistic. If you change it to "Yellow", then he'll either get pushed behind LT or rank only a couple points ahead.

But this actually is kind of a moot point now. Why?

They just put Eric Parker's injury into the system, and it bumped LT back up to #1 regardless of risk ratings or scoring system! Expect to see that in the next update. =)

Oh, and also, I actually *do* agree with CalBear on one specific thing FO's been pushing: The college QB "projection system". It uses exactly TWO variables: Games started and completion percentage. I will never in a million years believe that there aren't any other important factors. And another big problem here is that comp% is heavily dependent on the team's system.

Lewin knows this, and thinks that if he only applies it to 1st and 2nd-round picks, it will weed out the system guys. But there are some that fool the scouts; think Tim Couch, who had a pretty darn good projection thanks to his Mike Leach-inflated numbers. I see Colt Brennan as a good candidate to pull this off...

Basically, because the "system" got lucky with Rivers last year, people are already crowning its a**, so to speak. But I see it as more of a good start, and not a truly complete tool yet.

Other than this specific area, though, I love FO's work and am impressed by their attitude toward it.

-Josh

 

[phthalatemagic]

In case you guys don't know, you can change players' risk levels if you feel they're inaccurate. Pretty much all of the difference between LT and Gore, for instance, is Gore's "Green" rating; I think most of us here can agree that's overly optimistic. If you change it to "Yellow", then he'll either get pushed behind LT or rank only a couple points ahead.But this actually is kind of a moot point now. Why?They just put Eric Parker's injury into the system, and it bumped LT back up to #1 regardless of risk ratings or scoring system! Expect to see that in the next update. =)-Josh

 

[Maurile Tremblay]

Basically, because the "system" got lucky with Rivers last year, people are already crowning its a**, so to speak. But I see it as more of a good start, and not a truly complete tool yet.

I agree, and I'm sure Lewin does as well (as you suggested). The system already looks impressive when back-tested against Peyton Manning & Ryan Leaf (and scores of others), but back-testing isn't the same as testing. The true test is predicting future performance, and Rivers seems to have been the first major forward-looking prediction. It's passed with flying colors so far on Rivers, but obviously more evaluation is needed. Let's see how it does with Brodie Croyle, Jason Campbell, Matt Leinart, Vince Young, Jay Cutler, JaMarcus Russell, et al.It does look like Lewin may be onto something, though.

 

[oso diablo]

is this the same crew as Baseball Prospectus (BP)? And so football's KUBIAK = baseball's PECOTA?

 

[AB in DC]

is this the same crew as Baseball Prospectus (BP)? And so football's KUBIAK = baseball's PECOTA?

They're affiliated with each other, but it's a different group of people.

 

[hephner]

I sure wish there were an easy way to plug them into the DD

Well there might be. If you go to the FO boards there is this post:http://footballoutsiders.com/discussion/vi...4fbb28bc79dc335

The second post from Dr. Pepper says he'll send a way to do it for those who've paid for the Kubiak projections. I sent him an email, haven't heard back yet.

Seems that email was bad.... please pm me if you manage to get the info....

Dr. Pepper sent me the zip file for this yesterday. For anyone interested, feel free to PM me your email and I will forward it to you.EDIT - 8/14: I've sent this to about 8 or 9 people, and so far we've all had problems loading this into Draft Dominator. There's no problem getting the VBD file itself to work, which is useful, but when you try to import that into draft dominator, there are all kinds of errors that pop up and data that doesn't transfer right. I haven't had a chance to dig into the file to see what the problem is, but there are a few that were going to try to see if it could be fixed too. If anyone finds the issue and how to fix it, let me know, and I can send instructions out to all those that received this. I'm guessing that it has to do with something Dr. Pepper did when altering the VBD file, maybe a circular reference or something, but I'll let all those interested know if we find a solution.

Nonetheless, this is a nice tool to at least see the VBD worksheet with the Kubiak data, and it still allows you to plug in customized scoring.

 

[SSOG]

is this the same crew as Baseball Prospectus (BP)? And so football's KUBIAK = baseball's PECOTA?

Football Outsiders is an independent group of people who have been around for several years. Prior to the 2005 season, the guys who run the Baseball Prospectus approached them about the FO guys taking over PFP. Every Pro-Football-Prospectus written since 2005 was done by the Football Outsiders, but there was at least one (possibly more) done in years prior- those were done by someone else.

 

[phthalatemagic]

is this the same crew as Baseball Prospectus (BP)? And so football's KUBIAK = baseball's PECOTA?

Football Outsiders is an independent group of people who have been around for several years. Prior to the 2005 season, the guys who run the Baseball Prospectus approached them about the FO guys taking over PFP. Every Pro-Football-Prospectus written since 2005 was done by the Football Outsiders, but there was at least one (possibly more) done in years prior- those were done by someone else.

I thought some writers did 2004, but just weren't the editors for it?edit: either way, that's nitpicking, PFP as we know it started in 2005, you are right. Best football mag on the planet, if you can call it a magazine.

 

[guderian]

Oh, and also, I actually *do* agree with CalBear on one specific thing FO's been pushing: The college QB "projection system". It uses exactly TWO variables: Games started and completion percentage. I will never in a million years believe that there aren't any other important factors. And another big problem here is that comp% is heavily dependent on the team's system.Lewin knows this, and thinks that if he only applies it to 1st and 2nd-round picks, it will weed out the system guys. But there are some that fool the scouts; think Tim Couch, who had a pretty darn good projection thanks to his Mike Leach-inflated numbers. I see Colt Brennan as a good candidate to pull this off...Basically, because the "system" got lucky with Rivers last year, people are already crowning its a**, so to speak. But I see it as more of a good start, and not a truly complete tool yet.Other than this specific area, though, I love FO's work and am impressed by their attitude toward it.-Josh

They actually use three variables--completion percentage, games started and NFL draft round. You implied that they limit it to rounds 1 and 2 to weed out "system guys", but their theory is that if you start a lot of games AND are still drafted high that you would have generated enough game film for the NFL scouts to really get your draft grade right. I think that's a critical distinction, because they haven't pushed it as a system that predicts QBs as much as a system that will predict when the NFL guys get THEIR predictions right. So while their system doesn't explicitly consider a lot of variables, it implicitly considers the variables that NFL scouts consider.

 

[oso diablo]

is this the same crew as Baseball Prospectus (BP)? And so football's KUBIAK = baseball's PECOTA?

Football Outsiders is an independent group of people who have been around for several years. Prior to the 2005 season, the guys who run the Baseball Prospectus approached them about the FO guys taking over PFP. Every Pro-Football-Prospectus written since 2005 was done by the Football Outsiders, but there was at least one (possibly more) done in years prior- those were done by someone else.

thanks. key question was the 2nd one... is the concept of KUBIAK the same of that of baseball's PECOTA? Hoping someone is familiar with both.

 

[AB in DC]

Pretty much.

 

[Sabertooth]

just paid for the 2007 projections yesterday. I like to get a different opinion than just here.

I paid 15 bucks for all of those projections plus a huge amount more when ordering their 2007 book. That saves you money, plus it's 300+ pages of statistical goodness!

Doh! Have to remember that next year.

But aren't the downloadable projections still 20 even with the book? I mean it's like buying a fantasy mag if you can't get updated lists right up to the season no? I bought the book, but the projections aren't much good when there are guys getting injured in camp. They are months old.

 

[ConstruxBoy]

just paid for the 2007 projections yesterday. I like to get a different opinion than just here.

I paid 15 bucks for all of those projections plus a huge amount more when ordering their 2007 book. That saves you money, plus it's 300+ pages of statistical goodness!

Doh! Have to remember that next year.

But aren't the downloadable projections still 20 even with the book? I mean it's like buying a fantasy mag if you can't get updated lists right up to the season no? I bought the book, but the projections aren't much good when there are guys getting injured in camp. They are months old.

I paid for both separate and did pay $20 for the projections, which are updated every couple of days or so. I thought someone said there was a deal on their website to get both for cheaper.

 

[BusterTBronco]

Well, after the first six weeks of the season, it don't look like the KUBIAK's are holding up too well.

 

[The Man with the Plan]

How so?

 

[BusterTBronco]

How so?

Oh, like how they projected Frank Gore to be the most valuable player in fantasy football this year. They also had SJax, Maroney, Harrison, and MJD in their "top ten".I'll give them credit on one thing, though. They did have Tom Brady rated higher (12th overall) than most all other sites. But then they missed badly on Randy Moss (56th overall)

 

[CalBear]

How so?

Oh, like how they projected Frank Gore to be the most valuable player in fantasy football this year. They also had SJax, Maroney, Harrison, and MJD in their "top ten".I'll give them credit on one thing, though. They did have Tom Brady rated higher (12th overall) than most all other sites. But then they missed badly on Randy Moss (56th overall)

You can't really fault them for missing on people who got injured (SJax, Maroney, Harrison). But, MJD and Gore are misses, at least so far (and, to me, obvious ones going into the season).

 

[AnonymousBob]

How so?

Oh, like how they projected Frank Gore to be the most valuable player in fantasy football this year. They also had SJax, Maroney, Harrison, and MJD in their "top ten".I'll give them credit on one thing, though. They did have Tom Brady rated higher (12th overall) than most all other sites. But then they missed badly on Randy Moss (56th overall)

You can't really fault them for missing on people who got injured (SJax, Maroney, Harrison). But, MJD and Gore are misses, at least so far (and, to me, obvious ones going into the season).

If you want to argue MJD is a miss thus far because he hasn't been consistent, I'll give you that. MJD may have flopped the first couple of weeks, but as things stand he is a top 10 rb.

 

[WarRedbirds]

How so?

Oh, like how they projected Frank Gore to be the most valuable player in fantasy football this year. They also had SJax, Maroney, Harrison, and MJD in their "top ten".I'll give them credit on one thing, though. They did have Tom Brady rated higher (12th overall) than most all other sites. But then they missed badly on Randy Moss (56th overall)

You can't really fault them for missing on people who got injured (SJax, Maroney, Harrison). But, MJD and Gore are misses, at least so far (and, to me, obvious ones going into the season).

I thought they actually predicted an injury to SJax???I can't totally recall...I threw my book away cause they were right about Shaun Alexander and I drafted him in spite of this...

 

[WarRedbirds]

Well, after the first six weeks of the season, it don't look like the KUBIAK's are holding up too well.

From my recollection:HITS (so far...):Shaun Alexander :( Larry JohnsonMISSES (so far...):Tony Romo (they were down on him bigtime...and he's putting up big numbers)Frank GoreRandy Moss (Whiffed on this one.)DJ Hackett (even when healthy)Jerious NorwoodLee Evans (they had him pretty high)

 

[gheemony]

It's still early, but my reactions (off the top of my head):

QBs: They appear to have whiffed again. I will certainly be wary of their projections for QBs next season. In a trade, I took Hasselbeck instead of Romo based on their projections. Of course, "Bad Romo" may be back. It's still early. The McNabb pick looks bad. They have been right about Bulger. They have been right about Hasselbeck. Right about Brady.

RBs: I think they are pretty darn good. Owners using KUBIAK avoided LJ and SA, which I think is worth a lot. The book even cautioned about SJax and the Rams in general. They were right about Addai (including that he had to prove that he could stay healthy). They were too optimistic about Maroney's health. In general, I find that they tend to ignore injuries too much (I got burned a few years back when they said Eric Johnson was not injury prone and he missed the season). But then again, the may be better than the careless "injury-prone" labeling that happens among the FFers. Not sure which is worse. But I certainly will add my own injury risk factors when using KUBIAK.

WRs: Burress and Andre Johnson look bad (take away AJ's injury), although they did later indicate that they thought AJ was the most likely to surpass the KUBIAK projections, so the FO humans thought the computer was probably wrong. But I traded away AJ based on their projection. They guessed wrong on the NE WR situation, though they had Welker higher than most. They did well with Ronald Curry. Evans was a huge whiff. Hackett is an injury issue - I'll give them that one.

K: They did well with Gostkowski and generally predicting that the NE offense would be unstoppable.

D: I am really pleased with their recommendation of GB. I picked them up late on all of my teams.

In general, they did project the demise of the Rams and the Saints and the rise of the Bucs.

Overall, pretty pleased. I don't think they've been worse than anyone else, except with respect to QBs.

But it is early. I look forward to comparing them to others when all is said and done.

 

[GDogg]

How so?

Oh, like how they projected Frank Gore to be the most valuable player in fantasy football this year. They also had SJax, Maroney, Harrison, and MJD in their "top ten".I'll give them credit on one thing, though. They did have Tom Brady rated higher (12th overall) than most all other sites. But then they missed badly on Randy Moss (56th overall)

You can't really fault them for missing on people who got injured (SJax, Maroney, Harrison). But, MJD and Gore are misses, at least so far (and, to me, obvious ones going into the season).

Maurice Drew is a top 10 RB right now. Where did they have him ranked?

 

[WarRedbirds]

WRs: Burress and Andre Johnson look bad (take away AJ's injury), although they did later indicate that they thought AJ was the most likely to surpass the KUBIAK projections, so the FO humans thought the computer was probably wrong. But I traded away AJ based on their projection. They guessed wrong on the NE WR situation, though they had Welker higher than most. They did well with Ronald Curry. Evans was a huge whiff. Hackett is an injury issue - I'll give them that one.

Hackett was not performing well pre-injury...he was being pushed by Burleson for playing time. Even without injury, it doesn't look like he was going to break out like they were predicting.

 

[NCPanthersFan]

I think the biggest wiff is on Benson. With that said however, I have been happy with the Kubiack projections.

 

[Sam Quentin]

just shooting from the hip here, but it seems like football is, by far, the hardest game to predict based on statistical models.

I could go on for days about all of the factors where it is difficult to impossible to gather data.

I will say this, the data models we do have give us an object starting point for the discussion, but that's really all it is. You can't really apply the models blindly, you really need to understand how the projections are derived, so that you can weigh in other factors that may be more specific or difficult or imposssible to measure.

 

[mrspree15]

WRs: Burress and Andre Johnson look bad (take away AJ's injury), although they did later indicate that they thought AJ was the most likely to surpass the KUBIAK projections, so the FO humans thought the computer was probably wrong. But I traded away AJ based on their projection. They guessed wrong on the NE WR situation, though they had Welker higher than most. They did well with Ronald Curry. Evans was a huge whiff. Hackett is an injury issue - I'll give them that one.

I don't really fault them for the AJ projection. Based on his numbers, he looked like a short yardage possession WR. But the FO humans and fans everywhere realize how talented the guy is and could be if he got a real QB. With limited information about Matt Schaub and if he was actually a real QB, it's easy to see how KUBIAK could undervalue him.

 

[oso diablo]

what's a "FO human"?

 

[SSOG]

what's a "FO human"?

The writers and editors at Football Outsiders are "the humans", while the projection formula is generally referred to as "the computers". KUBIAK is compiled entirely by "the computers", although "the humans" weighed in with their thoughts on it once "the computers" had spit out the results.

 

[SSOG]

How so?

Oh, like how they projected Frank Gore to be the most valuable player in fantasy football this year. They also had SJax, Maroney, Harrison, and MJD in their "top ten".I'll give them credit on one thing, though. They did have Tom Brady rated higher (12th overall) than most all other sites. But then they missed badly on Randy Moss (56th overall)

Ummm... first off, it's still early, and second off, EVERYONE whiffed on a lot of those players (such as SJax, the consensus #2 fantasy player in the universe). If everyone thought SJax was the 2nd best fantasy player in the world, and FO thought he was the 3rd best, and he wound up being a total bust, then how does this somehow reflect more negatively on FO than it does on everyone else?Edit: Likewise, everyone had Harrison and Maroney in their top 10, and MJD isn't a whiff, since he's currently the 11th ranked RB in the NFL according to FBGs scoring, and he's just hitting his stride. Projecting a guy to be in the low end of the top 10 and having him wind up 11th is hardly a miss.