I can’t stand this argument.
Even if a guy picks up the wrong assignment, the better question is did they still make a good block? If an effective block is still made but on the wrong defender it should be a neutral or “0” value play. If an ineffective block is made, it should still be a negative or -1 play. It seems that the “bad” is still being captured and the good is being just slightly over estimated.
Is that how PFF grades players? I honestly don't know.
I have seen grades by them but I do not know what is behind those grades. What are they derived from?
Is it as you say a +1 0 or -1 for the lineman on one particular play, that does seem like one way to grade the players. This is what I have been trying to practice with my charting of RB traits. Each play if the player demonstrates power for example, I give the player a +1 in that trait. If the demonstrate bad power, I give them a -1. If power wasn't demonstrated either way it is a 0. This is a somewhat subjective process though, I am just trying to make it as simple as I can by grading in this way. On some plays for example the player maybe deserves a +2 or +3 if they are demonstrating the trait multiple times on the same play. No single play is the same. I can definitely see some plays as more impressive than others in regards to the trait being demonstrated, but they are all getting the same grade from me.
This is something I am still experimenting with. I am not sure how best to assign grades. No one has taught me how to do it. The thought certainly has crossed my mind that not all plays are equal, and that perhaps some plays should get more than a plus one, or conversely some plays may deserve more than a -1.
The way I have been doing this also is that I am more reluctant to give a -1 on a play than I am a +1. The player has to do some very bad to get a -1 from me on a play, while a player getting a +1 does not have to be that amazing for me to give the player credit for demonstrating the trait on that play. I am doing this purposefully, because I want to focus more on what the player does well than to focus on what they do not do well.
It is a subjective thing. I endeavor to be consistent in how I am grading the players, but there are a lot of variables on every play including variables in myself on any particular day. and I cannot say that I am always being consistent, even though I am trying to be.
I don't have a set of rules and guidelines to follow when assigning my grades to plays. Maybe PFF does? I think having some sort of structure could help my grading to be more consistent. I am just not sure how to set up those rules and guidelines right now.
I realize this is somewhat off topic, although i hope related to talking about how someone on PFF is giving out their grades. They are not any different than you or me. Everyone grades a bit differently and PFF has a lot of different people grading plays and players. If they had the same person always doing the grades for specific players, at least there would be some consistency in the grader, but to my understanding they are not doing that.
IIRC you took some sort of a class or tutorial on how to watch and grade plays. I have asked you about this before, but you didn't really tell me that much that you learned from doing this. I would be interested to hear more about what you learned, and perhaps more pointedly how I could set some rules and guidelines to make my charting more consistent.
Anyhow if I better understood PFF's process for grading, their grades would be more meaningful from that context.
If it is as you say a +1 - or -1 that would be helpful to know. I do not know if this is how they do it, or if they use some other method.
But the more important question is: How many “missed” evaluations are there in a season because of not knowing the exact protection? If a lineman plays the whole year, that’s roughly a 1000 snaps. For their model to be even just 90% correct, said lineman would have to miss 100 assignments a year or 6.25 per game. That seems like a very high number for a pro to not know what he is supposed to do on a play.
It is very difficult to know this. Only the coaches of the team and perhaps the players of the team actually know this.
If the protection is called and the player does their job, what they were asked to do, yet they still give up pressure, is that the players fault for not going off script? Are offensive linemen supposed to go off script? What are their rules and guidelines for this? If the TE was supposed to help to the outside and the lineman was supposed play the other side of the defender, is it the TEs fault for not redirecting to the tackle?
This is something that Zimmer hints at with why the PFF grades are wrong, because the grader does not know what the call was, but the coaches do.
90% accuracy of their grades seems really optimistic to me, when you consider all of the different variables that can go into a pressure and who is responsible, or who made the mistake. Some times the pressure is the QBs fault not the lineman, but PFF likely gives a -1 (or whatever it is they do) to the lineman for not making the block.
I am not sure how we could ever know this, but it would not surprise me if players are missing more assignments than what you are saying. Players will miss assignments or fail to execute a lot of times that does not end up affecting the execution of the play that is away from them for example.
How often are the PFF graders missing on their evaluations?
I don't know, but it would not surprise me if it a lot more than 10% of the time.
Outside of missed assignments what else would be a reason that an OL would go to a defender that they don’t plan on blocking? Sure there are certain plays (like a screen pass) but those are obviously not graded the same. And again what percentage of plays are like this?
A lot of times. Defenses will set up situations where the offensive linemen cannot be right. They flood defenders and make the lineman make a choice which one to block. If two defenders are attacking the same lineman responsibility om zone blocking, how does PFF grade that? Does the lineman get charted for giving up pressure for the defender they did not block? Even if they did block the other defender?
I don't know how PFF handles this but i can tell you that defenses are designed to make it difficult and to provide no win situations to linemen as much as possible.
The reason that PFF grades are the best is because A) they can be used for each individual lineman and B) their grades are from watching each snap. These to things are very important because it helps see how things will be going forward. Unless an entire OL stays together from 1 season to the next, unit grades are going to be a lot more skewed from year to year. If even 1 starter changes on an OL that’s a 20% unknown on projections/impact. Every model will be subject to player improvement and decline, coaching and scheme changes, etc. With PFF grades you can make a lot more reasonable projections with changes to the OL.
Edit: This isn’t to say that I throw out all other metrics and rankings and such. But I think that PFF is the strongest base to build off of.
While this is your opinion, I do not really agree that PFF is the best and I don't think your examples prove that it is.
I do agree with having a grade for each lineman is useful, because players can change teams, as I already mentioned above.
I already talked about several things that can go wrong with the evaluation on every snap earlier. The FBO outsiders adjusted line yards is also measuring every snap. So in my view it does not make the PFF grade any better or worse than what FBO is doing. However what FBO is doing has set guidelines and structure for how they assign the grade, which I quoted above, and to me that is better than what PFF is doing, because it is tied to actual yards, instead of a more subjective number, however PFF does this, again I do not know how they do it. If it is a +1 0 or -1 or if they are doing something else than this.
I am not sure if it does help us see things going forward, because players change over time. A player might have a bad game for example and that game goes into their grade, but the very next game might be terrific, so knowing what they did in the previous game or even on a play by play basis does not really tell me they are going to be that effective on the next play. Now obviously once you have a lot of plays and seasons of data to draw on, it does make how they might perform on the next play more predictable than a player who hasn't played much yet.
Although you made several statements after point B I think most of those things are related to point A.
I respect your opinion, but I don't really share it. Mostly because I do not really know what goes into the PFF grades. If I did they could be more useful to me.
Here are some articles where Zimmer and others are criticizing PFF grades:
Mike Zimmer (Awesomely) Addresses PFF Ratings
I know the people that are grading our games and our defenses and our offenses, they don’t know if the tackle gets beat inside he wasn’t sliding out to the nickel, or who our guys are supposed to cover,” Zimmer told the Star Tribune. “I guarantee they don’t know who’s in our blitz package and what they’re supposed to do. I would just ask that everybody take that with a grain of salt, including our fans. We as coaches get paid a whole bunch of money to do the jobs that we do, evaluate the players that we evaluate and grade them how we grade them, not based on something else.”
Vikings coach Mike Zimmer skeptical of Pro Football Focus grades
Zimmer is not the only one who disagrees with PFF grades. Many of the coaches do as do the players. The amount of disagreements are numerous.
In fairness ZImmer made these comments about PFF in 2016 when the Vikings offensive line was terrible and I don't need anyone to tell me that. It was obviously so. I agree with PFF calling out Matt Kalil for the numerous poor blocking attempts he made. I still can't say I agree with how they quantified that however, because I don't understand how they did that or what it means in context.
TJ Clmmings in particular missed a ton of blocks and PFF also graded him very poorly. So I was/am pretty much in agreement with that. Zimmer is making his comments against PFF in this context, because he was getting sick of journalists using PFF grades in the way they were framing their questions about the poor offensive line play.
There are other times where I very much disagree with how PFF is grading players that I have watched a lot. For example they did not grade Xavier Rhodes as well as I think they should have. While Rhodes was hurt and I don't think he played as well in 2017 as he did in 2016. He still shut down top WRs like Mike Evans and others, and PFF did not give him enough credit in my opinion.
There are many more examples than this of me disagreeing with their grades on players. There are also plenty of times that they warp their stats completely out of context in the form of a hot take. I think Matt Waldman is alluding to this in regards to the yards after contact stat for example. I think Matt is wrong a lot of the time too but I do not think the PFF graders have watched or understand more about football than Waldman does. Both have watched a lot. Hell I disagree with Gred Cossell a lot too, but he has likely watched more football than most.
