More About Defense, Variance and Luck

January 14, 2015; Oakland, CA, USA; Miami Heat center Hassan Whiteside (21, right) dribbles around Golden State Warriors center Andrew Bogut (12) during the first quarter at Oracle Arena. Mandatory Credit: Kyle Terada-USA TODAY Sports

Defensive performance is always a difficult topic to really get a handle on. Amidst Ethan Sherwood Strauss’s excellent discourse on Golden State’s transformation into a defensive juggernaut, there was this aside, which I endorse wholeheartedly:

"Part of the reason is big man defense is so outrageously pricey. Shot-blocking is a scarce, expensive resource in the NBA, as evidenced by [Andrew] Bogut’s handsome, $12.9 million salary. With few notable exceptions…you’re not getting that on the cheap. Wing defense is another matter. It’s more abstract, harder to measure and harder to price."

Evaluating and quantifying defense is difficult because we’re looking for evidence based on absence. The absence of scoring, made shots, trips to the foul line, and so forth. How much of a missed shot is a result of defensive attention? Was a turnover caused by poor execution or defensive pressure? Though with better, more granular data it is somewhat easier to start to sort these things out. Still why a shot was made or missed is an area where, from the perspective of our knowledge, there is a huge element of random chance. [1. We talked about variance in the abstract earlier this week.]

The variance of shooting isn’t constant. Defensive pressure on shooters is always a positive on the aggregate, though again the effects of this pressure vary greatly by shot type and location:

Intuitively, a defense’s skill at bothering close shots is easier to measure than longer ones. First of all, as the above chart shows there is a wider variation between an open and a highly contested shot at the rim than the variation found in longer shots. Moreover, we actually have data, both numerical and visual[1. Blocked shots of course are a highly imperfect proxy for rim protection, but we can still “scout” evidence of solid rim protection, such as found in the discussion of Marreese Speights in Strauss’s article.] to measure this skill. As Andrew Johnson discussed earlier this week, these numbers are measuring something real and relatively stable over time, a decent indicator that there is a measure of individual skill being observed.

With longer shots, this is harder to tease out. I and others have talked about the amount of sheer chance involved in opponents making or missing jump shots. Looking just at what I’ve described as “wide open” threes[2. No defender within 6 feet of the shooter on release, per SportVU data], there is wide spread between teams on the percentages shot by opponents. As of Tuesday, there was a gap of over 8 percentage points between Portland’s NBA best “defense” on these shots (34.2%) as compared with Miami’s league worst 42.5% allowed. This may not seem like much, but over the course of the season, this difference would add up to just over two points per game and a spread of six expected wins. As Phoenix has found out this year, being on the short end of near-term variance can be crippling to a team in a tight race.

Now, it’s possible this variation reflects something going on the data isn’t capturing. Perhaps teams which defend these “open” shots better close out in a more distracting fashion, such as with longer athletes. Perhaps they do a good job scheming to only allow open threes by poor three-point shooters and so on. Perhaps teams with the highest allowed percentages were especially bad at giving up “alone in the gym” threes, where the shooter has time and space to brew a pt of coffee before launching.[3. Having no defender within 10 feet seems to be worth approximately two percentage points of accuracy for three-point shooters over just being open by 6 feet.] When I looked in depth into the situation in Houston earlier this year, none of these appeared to be the case — it was a group of league average shooters taking the expected number of super wide open shots, without anything apparently special about the defenders closing out. Guys were just missing.

In fact, opponent’s shooting on open threes is incredibly volatile. Each line on the following chart is the percentage allowed for each team across the first forty games of the season in ten game increments:

There is really no apparent rhyme nor reason to these variations. Some of it is sample size. Over 10 games, we’re talking somewhere between 60 and 100 wide-open threes allowed for each team in each window. Some of it is opponent — giving up an open look to shooters from Atlanta or Golden State is on average going to be more problematic than when facing Charlotte or Philly’s merry casts of bricklayers. It does not appear to be any sort of “comfort factor, as though there are some hills and dips in terms of teams’ ability to contest threes, teams which are good at not allowing wide open threes tend to stay good and vice versa:

Philly and Minny have been bad at being close to shooters, Phoenix and Memphis have been consistently good. Further, it does not appear that teams’ ability to contest shooters has much bearing on the number of threes attempted by opponents[4. There are many possible confounding factors here, most notably score and game situations which might cause opponents to jack up more highly contested threes because of attempts to mount a late comeback and so on.]:

Each dot represents one 10-game sample of one team’s defensive performance. There is a slight increase in the proportion of threes taken as the openness of threes increases, but it’s minor and swallowed whole by the overall variance. So while leaving opposing shooters open is bad on the aggregate, what happens then is open to chance. And the difference on the scoreboard between a team going 1-8 and 5-8 on these shots is very often larger than the difference between winning and losing with no real change in a team’s defensive performance.

With so much randomness in just the open shots, it’s plain to see the difficulty in measuring the effect of defense. With such a wide spread in outcomes with no defense, how can we separate the signal of what might be real effects of defensive skill from the background noise of a make or miss league? As a final demonstration, this last chart  shows teams’ “Expected 3PT% Allowed[5. Using the same method as XeFG% but only for three point shots.]” over these same 10 game increments as well as the amount by which opponents either over or underperformed that expectation:

Green numbers equal relatively well-defended (in terms of distance from the shooter) mixes of shots or periods of holding opponents below those expectations while reds equal more open shots and higher actual percentages. While I don’t want to make sweeping claims about the proportional impact of luck and skill, a lot of what constitutes a slump or hot streak is the ball going in or not. It’s perhaps no mistake that San Antonio’s early season foibles coincided with opponents hitting over 40% from three for a 20 game period[6. San Antonio was 12-8 over this stretch as compared to 19-10 in the rest of their games.] despite defensive pressure in line with the Spurs’ season long defensive performance. On the other hand, maybe Portland IS doing something right in terms over their overall ability to contest threes. It should be noted Blazers’ opponents are shooting 46.2% from three over Portland’s last 9 games, so maybe the coin finally came up tails.