Jun 15, 2014; San Antonio, TX, USA; San Antonio Spurs guard Patty Mills (8) shoots a three point basket against the Miami Heat in game five of the 2014 NBA Finals at AT&T Center. Mandatory Credit: Brendan Maloney-USA TODAY Sports
Yesterday, I wrote an article for FiveThirtyEight about the conventional wisdom of teams needing stars to win championships. Using Statistical Plus-Minus, an estimated measure of a player’s value in points per 100 possessions relative to the league average, I tried to show that championship-level teams usually do have several elite players. The follow-up implication is that SPM reveals the definition of “elite players” to be a little broader than it is generally thought of. For example, the production of Kyle Lowry, Joakim Noah, DeMarcus Cousins, Goran Dragic and Paul Millsap last season would all have compared favorably to the production of players who have historically been the best player on a team reaching the NBA Finals.
I received a lot of comments about the article. Many were about how I handled the Warriors and Stephen Curry, an oversight I tried to explain in a stream of tweets this morning. There was one other comment that I wanted to address directly because I thought it created a nice moment of insight for me, both about SPM models and analytic work in general. The comment was left on the bottom of the article:
"This article takes a really thought-provoking statistical finding and asks a relatively boring question about it. Yes, teams need great players to be great. What I’d like to read an article about it whether we should believe Patty Mills is actually a better point guard than John Wall, Kyrie Irving and Damian Lillard, or at least on the same level. If so, what is it about their respective games that the “eye test” misses so completely? If not, what factors does SPM miss and what types of players is it likely to over/undervalue?"
If you haven’t read the article, it includes a table listing players from last season ordered by SPM percentile rank. Mills came in at 88th, ahead of the three point guards mentioned above and just outside the cut for what appears to be the window of production for the best player on a finals team.
My first thought on this comment was that I’d love to read that article as well. So let me try to answer some of those questions. SPM is based on box score—points, assists, blocks, steals, true shot attempts (field goal attempts and free throw attempts). Mills rated incredibly high in measures of shooting efficiency and, thus, rates very high in SPM. However, in the raw, true, pure, whatever, sense, I don’t think Mills was better than any of those three players last season. That’s besides the point though. Raw, true, pure, whatever, talent is instantly changed by team context. Per-100 possessions, I don’t find it hard to believe at all that Mills was more effective in his constrained role with the Spurs than Wall, Irving or Lillard were in their dramatically larger ones. SPM is leveling the playing field, accounting for context that is important for judging which players are better in a vacuum.
So if I don’t believe that Mills was better than those three players, how can I, in good faith, use a metric that implies that he was?
The answer is that SPM doesn’t say Mills was better. It says that, per-100 possessions, he was slightly more productive than those other point guards, last season. That’s a narrow distinction, but an incredibly important one. I wrote earlier this season that the power of a statistic is not inherent, it is derived from the specific question it is being used to answer. In this case, I was looking for production within a team context. It didn’t matter which player was better in a vacuum, because I was looking at a question that involved a scenario within competition. Chances are Mills would have been nowhere near the 85 percentile in SPM if he played for the Pistons last season. But he didn’t play for the Pistons. He played for the Spurs. I wanted to see which players, within their team contexts, produced similarly to how players from previous finals teams did, within their team contexts.
I know those may not be the exact answers the commenter was looking for, but from the synthesis of the article and his response, it struck me as the most important idea. There is no perfect metric. You pick the best one to answer your question and try to be aware of the flaws. Sometimes bringing a different question to that process can create confusion, requiring a different metric to be sorted out.