Introducing HBox: Predicting Wins Based on Historical Data and Trends

A few years into the modern “databall era”, plus/minus analysis has been the basis for a new generation of metrics and stats. It’s not just about terms like APM or RAPM, but the more recent like basketball-reference’s BPM[1. Box score Plus/Minus, from the family of “Statistical Plus/Minus” metrics like our DRE, which assign weights to box score contributions as a way of estimating a player’s RAPM value without requiring the computing power of sample size of actually building and running such a model.] also use plus/minus data to construct the weights for their variables. That can work really well, as they’re an improvement over the old metrics like PER, but it can be dangerous if all our stats rely on the same component. If there’s a systematic error in plus/minus data, it is being propagated to a host of public metrics. Thus, I thought I’d build something that uses an entirely different method, and there’s the added benefit in that it can use decades of NBA data, not just everything since play-by-play logs have been available.

The origin of HBox[1. “Historical Boxscore” in my clever naming convention.] is that I feared applying modern data to the 70’s and 80’s when the game was played differently on offense and defense due to rule changes and dramatically different styles and schemes.  What was the value of spacing in an era before three-pointers were a major weapon? How important are drive-and-dish guards before the hand-checking rules were changed? Which old-school players will be penalized, judged by the standards of 21st century basketball?[2. Ed. I’ve long thought Orginal Isiah Thomas gets a short shrift historically because of his pre-modern game in a semi-modern league. One would think a player with Zeke’s overall skill level would have found a way to become a passable or better three point shooter if he came up in an era when deep shooting was an important trait for guards. But I digress.] HBox is box-score metric on the 100 possession scale (i.e. +5 means a player is, basically, adding five points in point differential to the average team) based on team-level results.

Methodology

With some ingenuity, I figured out a way to use regular season data going back to 1978 in predicting team-level performance with individual stats. Essentially, all I’m doing is reducing the error in predicting a team’s rating, and consequently team wins, from player performance in adjacent seasons. While there’s a lot of noise there, with enough seasons and by giving more weight to teams with less roster continuity there’s enough data for useful results.

The point of this exercise is to derive an historical metric usable for any season in which turnovers were recorded. You can actually project this back to 1974 with a turnover estimate[3. Team turnovers were recorded starting during the 1974 season in the NBA, but individual turnovers were not until 1978. The ABA recorded turnovers individually in every season, which goes back to 1968, but like the NBA steals and blocks weren’t recorded until 1974.], but any further back and you’re in the statistical dark without steals, blocks, or even rebounds split into offensive and defensive boards.

The final model was completed using the glmnet function in R, which uses a ridge penalty (like you see in RAPM) and an alpha parameter that drops insignificant variables. As the last step, ratings are adjusted so they sum to every team’s particular rating like what was done here, which greatly improved the prediction power. This is, however, an exploratory version of the metric as I work out the final issues and continue tweaking.

Variables

Since I’m in the beta version of HBox, essentially, I don’t want to provide full results, but there were a few discoveries that I found interesting. The first finding is the majority of the stats relevant to the modern era show up as impactful in the past as well. High assist, high usage players are the most valuable in today’s drive-and-dish game just as they were 80’s. Spacing[4. This is measured by three-point attempts per 100 possessions, but position was also a factor. Centers get the biggest boost for position and point guards the lest] was valuable and measurable on offense — even years before the seven-seconds-or-less Suns put pace-and-space on the map.

One variable that doesn’t show up, however, is offensive rebounding. Even with a low threshold in variable selection, offensive rebounding has very little impact. I used an interaction term between assists and rebounding, meaning offensive rebounds do matter to some extent, but I repeated the analysis without that term and offensive rebounding still didn’t show up. In fact, for the model to include it I either had to force the model not to drop it or exclude seasons 2001 to 2015, but the coefficient was so small it was nearly negligible. For example, during Rodman’s best season rebounding on that end of the court, the boost he’d receive to his rating would only be 0.2 points per 100 possessions. That’s a small effect, and Rodman’s at the extreme end of rebounding. Meanwhile, even without the assist-rebounding interaction term, during Rodman’s best season with defensive rebounding, he’d receive a boost of 2.3 points per 100 possessions compared to the average player.

Basically, according to those results, offensive rebounders don’t lead to wins and are entirely replaceable, which runs contrary to previous metrics like Win Shares or even basketball-reference’s new BPM. Most systems assume defensive rebounds are a lot less valuable. The traditional view is that an offensive rebound is an added possession, which is very valuable, and the credit is entirely given to the rebounder. But this is a radically different view of rebounding value. Actually, this effect is well-known at the team-level, as the best teams often rebound less on offense. This is probably why offensive boards don’t show up in my model, as the model relies on team results.

This might shed some light as to why previous models possibly overrating offensive rebounding. Obviously, an offensive rebound can’t be worth an entire possession on the individual level, because offensive rebounds at at least partially replaceable. A team which loses its best offensive rebounder, doesn’t “lose” every one of his rebounds because the player taking his place will at least grab a few of those. Plus/minus data seemingly should deal with his issue.

The following graph shows the value of offensive and defensive rebounding in concert with assists. Over a realistic range of offensive rebound percentage with the defensive rebound rate set at league average, there is only a modest change in value. But defensive rebounding has a huge influence. And interestingly, the model only finds substantial value in offensive rebounding with high assist players.

However, there is a distinction between what makes a lineup successful,[5. The basis of plus/minus models.] and what makes a team successful. Perhaps teams do better with more offensive rebounding, but it’s more replaceable when  year-to-year changes are considered because so much of offensive rebounding is about what position and how close to the basket one tends to be. Another theory might be that truly impactful offensive rebounding might be correlated with other variables, like defensive rebounding or other stats.

Diving the philosophy and tactics of basketball, the problem with offensive rebounders is that they occupy a valuable physical space that can clog the lane and offensive rebounding is associated with having fewer skills. If you’re a high level basketball player on offense, you’ll either have the ball, show your skills on the perimeter or in an active play, not fighting for the miss off-ball. There are exceptions, of course, including certain MVPs, and this might just be an issue with the method I’ve chosen, but it’s something to consider.

Another surprise is that shot usage has a negative sign — being a high usage player hurts your rating in that respect. Of course, I have interaction terms[6. An interaction term is when multiple variables are multiplied against each other so that each one has an effect on the other variable. For instance, shot volume*assists is a popular interaction term where the players helped most are high volume shot creators who also assist teammates, as opposed to the black holes who shoot often while rarely passing.], so it’s not straightforward. Overall, a player with healthy rates of assists will see an increase in value when the shot rate rises. In other words, a player like Michael Jordan will receive a benefit for shooting more often because he always had an above average rate of assists, but that benefit would be larger with more assists. I tried different terms and combinations, but the best results led to the negative coefficient.

Finally, the only other surprise was that steals were not as valuable as you see in other modern models. Again, my metric is new and it’s not necessary the Word of God, and I’m using a conservative method that shrinks coefficients, but it is an interesting result. Plus/minus models love steals, but why is that? They end a possession and lead to a more efficient possession on offense often due to fast breaks, but is there something about how lineup data is structured that the players getting credited with steals are being overrated?[7. Ed. Perhaps in the same way certain models overcredit individuals for defensive rebounds, the error is wholly crediting what is in many instances a team defensive event to only one player in its entirety?] I’m not entirely sure, but there might be some truth here.

Results

For a look at what the metric produces, I included a table showing the top three players every season since 1978 using some rough guide of value[8. This is shown in the VAL column, which is simply rating*MP/3000.] with every MVP noted. It reflects popular consensus fairly well for an advanced metric, where 18 out of the possible 38 MVP winners had the highest “VAL” for the respective season with another six behind by a slim margin. It’s not supposed to replicate what MVP voters think, of course, but it passes the laugh test — generally, the best players having higher ratings. As I was producing the results and testing new things, I always checked 1978 and how Kareem was rated versus Walton. That remains one of the most interesting MVP races. It was a high-scoring legend versus a team-first defensive newcomer, and I think it’s a good sign that HBox values Walton so highly.

The metric doesn’t agree with the choices of Iverson and Nash for MVP, but they were fairly controversial anyway. Moses Malone captured three MVPs in reality, but the numbers don’t back him — he has a tiny rate of assists for a high scorer, and assists are an influential part of the model, plus he doesn’t have the defensive box-score power that many other big men MVPs have. If you’re wondering why a beloved player doesn’t show up, it might have to do with the limitations of the box score.

If you looked at the table closely, you probably noticed something surprising: Magic Johnson and Michael Jordan have apparently switched places in the pantheon. Jordan doesn’t look like some insurmountable legend statistically, but “just another” superstar. Yet it’s Magic Johnson with some of the highest rated seasons according to the metric. The numbers value versatile box-score stuffers, and Magic and his 138 triple doubles certainly qualify. There’s also the previously discussed penalty to shot usage, which stifles Jordan’s stats.

No metric is perfect, of course, and this one certainly rewards a specific type a player, one who racks up a lot of assists, rebounds, and other counting stats, while perhaps not accounting for poor defense[9. Always a weakness of box score-based analysis]. This includes Magic as well as Charles Barkley, Kevin Love, and a few others. The worst offender is probably Troy Murphy, who had an all-star rating one season due to his outside shooting and defensive rebounding, but his porous defense isn’t easy to identify with box score stats.

Yet we should not dismiss any odd result and rely only on conventional wisdom; otherwise we wouldn’t learn anything. In 1994, the highest rated players were David Robinson, Shaquille O’Neal, Scottie Pippen, Karl Malone, John Stockton, Shawn Kemp, Charles Barkley, Hakeem Olajuwon, and … Oliver Miller, who actually had the third highest rating that season. Oliver Miller is a semi-obscure player now and only known for his weight issues and being arrested, but back in the mid-90’s he had the steal and block numbers of Olajuwon with sweet passing and rebounding with a highly efficient season in 1994.[10. Think Boris Diaw, but bigger including a huge wingspan, stronger and possibly even more skilled.] Miller’s weight problems didn’t surface until joining a terrible Detroit squad the next season, reportedly showing up to camp out of shape. He then bounced around the NBA and leagues abroad, his weight ballooning and his interest declining. The list of players with at least 4 blocks and 6 assists per 100 possessions is just him, Kirilenko, Kareem, and David Robinson. It suggested high potential and a fascinating career, but we really only got one peak season.

Finally, the below table shows the top ratings for the 2015 season. There are really no surprises here, and less renowned players are substantiated with great numbers from different metrics like the two Green’s. For what it’s worth, the correlation coefficient between the ratings below and the ratings from ESPN’s Real Plus/Minus is 0.75. Basically, the results are very similar with the exceptions of Marc Gasol, Pau Gasol, Love, and Gobert — and those guys are no slouches, depending on your personal opinion of Pau’s impact in Chicago.

Conclusion

Due to the nature of HBox, which is alone in its own family of metrics right now[11. I do want to stress the metric’s uniqueness. Statistical plus/minus metrics are quite popular now, and this one was derived completely independently.], certain actions might be over- or undervalued. It might not rate offensive rebounding highly since it’s negatively correlated to team wins. But every method has its strengths and weaknesses, and it’s good to use a wide range of tools to minimize glaring systematic errors. Nonetheless, the results are remarkably close to the ones given by the popular plus/minus models. And most importantly, pre-2001 data was used to build the model. The numbers don’t explain everything, and the box score is a small slice of what happens on the basketball court, but if we’re going to use statistical metrics we may as well use the best options available and attack it from all angles.