Team KOBE vs Team SHAC Defensive Edition

One of the things a number of the contributors here at Nylon Calculus, myself included, like to research is the consistency of different basketball metrics over time and their ability to predict outcomes out of sample. It is especially of interest for some of the newer metrics coming out of SportVU, where no one has much experience with the numbers, making contextualizing and interpreting findings somewhat difficult.

Last year I took a look at individual rim protection consistency and contested three point consistency for individual players. In short I both found that while rim protection numbers were somewhat noisy, they represented a real measure of defensive production that has measurable validity going forward. Contested opponent three point percentages, on the other hand, appeared to be just noise with a random relationship to future percentages.[1. This isn’t meant to imply that there isn’t value in contesting three point shots, there is a clear pattern that there is. But the individual results between players on their contests of opponent threes is likely basically noise.]

In the meantime Krishna Narsu developed an expected point per shot (PPS) model that incorporated shot distance, defender distance, catch and shoot vs pull up, and other measures called KOBE[2. Krishna’s Objective Basket Estimator ]. KOBE, the metric, can be calculated for both offense and defense. Along with forcing turnovers, minimizing free throws and securing defensive rebounds defensive teams want to “force” tougher shots. Using defensive KOBE we can get a decent measure of how well different teams do that.[3. Interpreting defensive KOBE for individual players is trickier given that players who defend the basket will face higher expected point per shot by their opponents and players that completely blow their assignment aren’t penalized in the current public data] Over the last two years defensive KOBE has explained 55% of the variance in opponent’s points per shot.

The corollary to KOBE is, of course, SHAC[4. SHot Adjusted Capabilities], which is the residual between opponent’s expected PPS and actual PPS. Looking at team defensive KOBE and team defensive SHAC year to year for the only two full years that are available[5. The 2013-2014 and 2014-2015 seasons] we can get a rough test of the degree of stability in each measure relative to the amount of luck and noise[6. The team laundry doesn’t play the games, roster turnover, injuries and coaching changes happen, so these measures are likely to understate the stability of KOBE and SHAC]. Between the two years there was fairly robust correlation between team defensive KOBE year to year, with an R2 of .57.

Unsurprisingly, defensive SHAC, a measure that is basically “things not accounted for in the model” is noisier year to year than defensive KOBE. However, there was some relationship with a year to year R2 of .26. That indicates that there is some shot discouraging skill not accounted for in the KOBE model, which could be things from the quality of the shot contest not captured by SportVU or funneling opponent shots to less capable shot makers.

Finally, looking at team defensive PPS there was a also a decent relationship between defensive KOBE in 2014 and PPS in 2015, a measure that was only modestly improved by including SHAC. Again this suggests that KOBE is capturing a good part of the work defenses do in affect opponent shots, with some measure of defensive shot contesting skill still not captured.