Nylon Calculus: Real Plus-Minus is a little less real when players change teams

By the end of July most of the big deals in the NBA are done, fans of every team other than the Knicks and Bulls have mostly talked themselves into whatever it was their front office did between draft night and today. There’s a near universal, “We gotta a squad,” feeling as NBA fans head to the beach — despite what the national media or number crunchers adding up all of those negative Real Plus-Minus numbers might say.

ESPN’s Real Plus-Minus (RPM) is one of the most cited single number metrics when trying to estimate player talent and impact. It gets used often among the analytics crowd, in large part, because it’s one of the better metrics for building a pre-season projection in terms of accuracy.

But a couple of cautions should be used, especially when applied to the one-year version available on ESPN. The most accurate projections do not rely on just one year of data, instead they use multiple years of data weighted toward the more recent seasons. Other adjustments like age, player comps, games started, outside shooting percentages and turnovers can also help improve the out of sample prediction accuracy of RPM in the upcoming season.

Another big factor appears to be simply whether or not the player changed teams or is returning to his old squad. Pretty much all stats have a lower correlation year to year for players on a new team than those returning. This is true in varying degrees for box score stats, minutes per game, synergy play types, tracking data, plus-minus, or ringz. So it is not too surprising that this is also the case with RPM, though it might not be understood how much.

I linked up every player from last season to 2015 with the RPM estimate of their prior year, then filtered for only players who played at least 800 minutes in both years, which gave me a little over 950 cases. The year to year RPM R^2  for players remaining on the same team was .46, while for players changing teams the R^2 was only .26, a pretty significant drop.

It’s also clear from the scatterplot above that the relationship is somewhat non-linear for players changing teams with the stars at +5 RPM maintaining their level of play more than role players (the cases in top right corner). That non-linear shape to the relationship is less striking for players remaining on the same team year to year.

As I noted above, stars seem to regress less on average than other players, but this has held primarily just for true stars. Even players that meet a +3 RPM threshold in the prior year, generally viewed as players with some impact, showed significantly more regression when they changed teams the next year. The +3 changers regressed -1.4 more on their new team than the +3 remainers did, statistically significant at a 99 percent level, for what it’s worth. There were many more +3 RPM players staying than changing teams, 28 changers and 99 stayers, a higher percentage than the players with lower RPM scores.  

On the other hand, for those few players with a +5 or higher RPM, the gap in regression is  smaller, with stars going to new teams regressing just .4 more than those staying on average.

Separating the returning players and players changing teams in a scatterplot, the contrast in the fit slope is shown below, with returning players in green and changers in blue:

It should also be noted that the gap in correlation for Defensive RPM is much less dramatic than gap in Offensive RPM correlation when changing teams. That may be the result of the height prior for DRPM, the one stat that doesn’t fluctuate with team or role. The greater separation in the slope of the trend lines for ORPM shows that fact along with the higher R^2:

There are a number of explanations for why RPM is less stable for most players year to year when they change teams: they are on the floor with different players, their role may different, or the evaluation of the teams there may be some selection bias based on the asymmetric information possessed by the team that employs them and their decision to try to retain them or not. In any case, cool your expectations on any new non-LeBron your favorite team signed this summer, at least a bit.