|
In this essay, reprinted from Pro Basketball Prospectus 2010-11, now available in paperback format from Amazon.com, we look at our changes to the WARP rating system. See here for a detailed explanation of WARP.
How do you rate a rating system? That thorny question has been problematic for statistical analysts. The oldest method is the "laugh test," which simply compares the results to conventional wisdom. In the early 2000s, the "Shaq test" held that any good rating system would rank Shaquille O'Neal as the league's best player. Something similar could be applied now with LeBron James. However, there is nothing scientific about this method, which is nothing more than a first cut.
Dave Berri has argued in favor of his Win Score metric on the basis of how successfully it explains team performance. While this might be a necessary condition for a good rating system, it is not a sufficient one. Any rating system that utilizes a "team adjustment" to account for the aspects of defense that are not captured by individual statistics will equate to team wins. At the extreme, team wins could simply be allocated on the basis of minutes played. This easy rating system correlates perfectly with team performance, but it does a poor job of crediting individual players for their contributions.
Dan Rosenbaum first came up with the notion of using adjusted plus-minus to evaluate rating systems. In this role, adjusted plus-minus--traditional plus-minus adjusted for a player's teammates and opponents--is objective and unbiased. The noise that makes adjusted plus-minus problematic at the individual level evens out over the sample of hundreds of players, leaving adjusted plus-minus as a capable measure of how different types of players help their teams win.
To calibrate the WARP rating system, we compared results from the 2007-08 season to adjusted plus-minus data from that season. Why 2007-08? For that season alone, Eli Witus of CountTheBasket.com compiled adjusted plus-minus data split up into its offensive and defensive components. This allowed us to focus specifically on improving the Offensive and Defensive Ratings that go into player win percentages and WARP.
Among regular players, here are the correlations between adjusted plus-minus and WARP components:
Win% ORtg DRtg
Net +/- .450 .369 .351
Off +/- .487 .567 -
Def +/- .081 - .413
Correlation measures the strength of a relationship between two variables, with 1 or -1 indicating a perfect relationship and 0 indicating none at all. In this case, we've made all the correlations positive. Again, the noisiness in adjusted plus-minus means the correlations inevitably are not very high. What stands out here is that Offensive Rating measures offensive adjusted plus-minus much better than Defensive Rating measures defensive adjusted plus-minus. Given the limited number of individual defensive statistics we have to work with, this is no surprise.
The bigger question is where WARP has room for improvement. Here, we focused on the differences between how WARP and adjusted plus-minus rated each player on offense and defense. For example, Kobe Bryant's adjusted offensive plus-minus in 2007-08 was +10.4 points per 100 possessions. His Offensive Rating was +6.4 points per 100 possessions compared to league average. Thus, adjusted plus-minus rated Bryant as 4.0 points better on offense. Are there shared traits among players who are overrated or underrated by WARP?
OFFENSE
2P% 3P% FT% TS% OR% Ast% Usage 2A% 3A% FTA% TO% PF%
-.087 .269 .153 .006 -.236 .059 .054 -.277 .318 -.185 -.139 -.124
DEFENSE
Stl% Blk% DR% PF%
-.091 .094 .147 .198
For the most part, these correlations are close to zero, indicating that WARP is properly valuing each statistic. The most notable differences share a common theme--three-point shooting. The correlations indicate that players with higher three-point percentages and especially three-attempt percentages tend to rate better by adjusted plus-minus than by WARP. Essentially, there appears to be a value to spacing the floor that is not captured by the individual statistics of three-point shooters.
The correlation between Offensive Rating and adjusted offensive plus-minus is maximized by adding a term that multiplies the difference between the player's three attempt percentage and the league three attempt percentage by seven. Here are the correlations between various offensive statistics and this new Offensive Rating:
2P% 3P% FT% TS% OR% Ast% Usage 2A% 3A% FTA% TO% PF%
.009 .061 .013 .001 -.032 -.008 .049 .015 .003 -.014 -.045 -.016
Now, it looks like Offensive Rating accurately captures what is helping teams win. This change increases the correlation with adjusted offensive plus-minus from .567 to .625. It also does a better job of valuing players at the wing positions, who previously lagged behind their peers at point guard and in the frontcourt.
WARP WARP2
Pos ORtg Win% ORtg Win%
------------------------------------
PG 105.7 .467 106.1 .479
SG 104.3 .431 105.0 .453
SF 104.1 .437 104.6 .454
PF 105.3 .487 104.7 .470
C 104.9 .508 103.9 .476
For whatever reason, wings still have less value statistically than other positions, but we're closer to seeing descending offensive value by position, which is what adjusted plus-minus shows. Defensive value increases the closer players get to the basket.
Almost all of the players who benefit the most from the change are wing players. Most of them were near or below replacement level despite helping their teams with their shooting.
Player Team WARP WARP2 Change
----------------------------------------------------
Quentin Richardson mia 1.8 5.6 3.8
Jason Kidd dal 11.8 15.4 3.5
Shane Battier hou 0.7 4.2 3.5
Danilo Gallinari nyk 3.2 6.4 3.2
Rasual Butler lac -3.7 -0.4 3.2
James Posey noh -2.2 1.0 3.2
Channing Frye phx 4.1 7.3 3.1
Anthony Parker cle -1.8 1.2 3.0
Rashard Lewis orl 2.5 5.4 2.9
Steve Blake lac 0.6 3.4 2.8
Channing Frye is a good poster boy for WARP2. The Suns' offense was 6.8 points better per 100 possessions with Frye on the floor. Under the old WARP, he rated as below average on offense. His new rating is much more appropriate.
Player Team WARP WARP2 Change
----------------------------------------------------
Brook Lopez njn 11.1 8.7 -2.4
Al Horford atl 11.0 8.8 -2.2
LaMarcus Aldridge por 7.3 5.0 -2.2
David Lee nyk 13.8 11.6 -2.2
Luis Scola hou 5.0 2.9 -2.2
David West noh 7.5 5.4 -2.2
Amar'e Stoudemire phx 11.7 9.5 -2.2
Nenê Hilario den 11.0 8.8 -2.1
Chris Kaman lac 2.7 0.6 -2.1
Carlos Boozer uta 11.6 9.5 -2.1
No single player loses as much value as the most frequent three-point shooters gain from WARP2. Since many players attempted no threes all season, the biggest losers are simply the big men who played the most minutes and generally had the most value.
Going forward, we will use WARP2 everywhere you are used to seeing WARP, including throughout this book as part of the past and projected player stat lines. Despite the potential for Star Trek puns, we will only refer to WARP2 as part of discussions comparing the two incarnations. Generally speaking, the term WARP will mean the updated version.
Follow Kevin on Twitter at @kpelton.
Kevin Pelton is an author of Basketball Prospectus.
You can contact Kevin by clicking here or click here to see Kevin's other articles.
| 
0 comments have been left for this article. 
Bubbles of their Own Making: Villanova, Temp...