Guest Post: First Round Upsets in 5 & 7 Game Series

As six of the eight first round series saw the favored team take a 2-0 lead, many in rather convincing fashion, there have been some calls for the NBA to return to a best-of-five first round format so that we can “sim ahead” to more intriguing and closely matched march ups. Thankfully, only two series ended up in sweeps and there have been compelling games in recent days. Still, it is worth examining just what the effect might be of a return to shorter series. To address the topic, we have a guest post from Ryan Chen. Ryan is a doctoral student in Management Science & Engineering at Stanford University and a member of the Stanford Sports Analytics Club.  Follow him on Twitter @TheRChenemy and at www.ryandchen.net.

Conventional wisdom holds that lengthening NBA 1st round playoff series from best-of-five to best-of-seven favors the better team. If this is true, it stands to reason the move to best-of-seven 2003 would mean fewer upsets as compared to the period from 1984[1. The first year of the 16-team playoff format.] through 2002.  Certainly this notion makes intuitive sense – a worse team has a higher chance of beating a better team in one game than in two of three and so on, as there is more variability to the result of one game than to the result of three games.  In stats speak this result aligns the standard error of the mean scaling with 1/sqrt(n), with n representing the number of trials/games.  Nevertheless, I set out to test whether or not the conventional wisdom actually holds up to statistical scrutiny.

What I found was that the rate of upsets has indeed decreased since the shift to the best-of-seven system from 25% to 23.1%. However, the magnitude of the decrease was not statistically significant.[2. p = 0.36. A lower p value implies that you’d be less justified in saying something like, “Well, a change of this size could’ve happened randomly.”  Without inserting my opinion on the controversies of bad p value practices, the generally accepted standard in the scientific community is to set a p = 0.05 threshold for statistical significance.] If the rule change has had no effect on the rate of upsets, there would be a 36% chance we would have see a shift in percentages this large regardless.

In any case, we also know that playoff seeding can be a flawed way of measuring how truly good a team is – winners of weak divisions who are themselves weak relative to the rest of the conference may garner a 4 seed while not being the 4th-best team in their conference.  So, we’d like to be able to draw conclusions about whether or not the rule change has led to the truly better team winning series at a higher rate.  To get a more accurate feel for this, I repeated much of the above analysis, but defined “underdog” not as the lower-seeded team, but rather the worse team according to FiveThirtyEight’s NBA Elo.  These values were collected for each playoff team in each season, evaluated at the end of the regular season.  For example, in 2015, the 3rd place Clippers played against the 6th-seeded Spurs in the first round, but the Clippers were the “Elo underdogs” because their Elo score at the end of the regular season was 1705 compared to the Spurs’ Elo score of 1733.  With this structure in place, I found that the rate of “Elo upsets” has actually gone up slightly since the change to best-of-seven, but again not to a statistically significant degree.[1. Test statistic t = 0.132, critical t = 1.652.]

To try to approach the problem from a different angle, I also tried to answer the question: are the teams which do pull off upsets closer in ability than in the past? In other words, has the Elo differential between teams in upsets grown smaller since the rule change?  The results show:

Not really!  In fact, the Elo gap between teams involved in upsets has widened, though the difference is again not statistically significant.[2. Test statistic t = 0.225, critical t = 1.677.]

For a deeper dive into seed matchups, I plotted the series win percentages for all seeds during the best-of-five era (1984-2002) and the best-of-seven era (2003-2015):

The first observation from this plot is that 4 seeds won less than half the time in the best-of-five era, but this fact is easily explained by a combination of natural variability and the fact that the 4-5 matchup is the most balanced matchup in the 1^st round.  The shift to the best-of-seven system brought 4 seeds’ win percentage above 50%, which accords with our intuition that lengthening the series increases the chances that the higher seed wins.  However, there’s a huge caveat – again, none of these shifts in series win percentages are statistically significant.[3. Test statistic t values for the matchups: 0.852, -1.053, 0.053, -0.613 for 1 vs. 8, 2 vs. 7, 3 vs. 6, and 4 vs. 5, respectively, while the critical t values were approximately 1.67 for each.]

So, what to make of all of this?  I think the best summarizing conclusion is that we can’t say with any real certainty that the change to best-of-seven 1^st rounds has had any effect on upset rates. Further, pulling off an upset these days doesn’t require a team to be better in comparison to their opponent than in the best-of-five era.  These results are a lot of statistical insignificance, which may seem, well, insignificant, but the important thing to remember is that they contrast directly with the conventional wisdom that lengthening the 1^st round favors the higher seed and better team.