Is Playoff Experience a Prerequisite for Playoff Success?

[Ed – The following is a guest post from Harrison Chase (@aabsblog) of the Harvard Sports Analytics Collective.]

Before the playoffs began, LeBron James was quoted multiple times about his concern over the Cavs’ lack of postseason experience. Thought it hasn’t ended up mattering much, with Cleveland rolling through the East dropping only two games, was LeBron right o be to worried beforehand? Have teams with limited postseason experience struggled more than otherwise expected?

In order to measure a team’s postseason experience, I looked at the percentage of minutes that were played by a player who had never played in the playoffs before. The reason for using percentage of minutes and not percentage of players is based on the logic it matters more where a starter like Kyrie Irving has never played in the postseason than if this was true for a benchwarmer like Kendrick Perkins. By this measure,  60.8% percent of the Cavaliers’ playoff have gone to players for whom this was not their first trip to the playoffs. The other 39.2% have gone to playoff rookies[1. Mostly Irving, Tristan Thompson, Kevin Love and Matthew Dellavedova.] By this measure, the Cavs were fourth most inexperienced team in these playoffs, ahead of only the Celtics, Bucks, and Pelicans.

It also makes them the most inexperienced team to make the Finals since at least 1981, And by a wide margin. Below is table of the five most inexperienced teams to make the finals over that time period – there is no real precedent for a team as inexperienced as this year’s Cavs being in the finals:

But does this inexperience actually matter? I first attempted to quantify this by doing a logistic regression with the outcome being whether a team won the championship. I looked only at playoff teams from 1981 on, considering several variables like a team’s winning percentage, SRS, pace, OSRS, DSRS, and the percentage of minutes played by playoff veterans. The best individual predictor of whether a team would win a title was their regular season winning percentage, even more so than their SRS[2. (In itself this is an interesting finding as most studies tend to point the other way, that is, point differential is a better predictor of playoff success than record – Ed.)]. This makes sense as their winning percentage would encapsulate not only their skill but also how hard their path to the final would be. Adding in a team’s proportion of minutes played by playoff veterans improved the fit of the model,[1. This variable was not statistically significant at the .05 level, but with a p-value of 0.0724 it was significant enough to improve the AIC,  suggesting that it is a good addition.] Additionally, the sign of the coefficient was positive, indicating the more model predicts that the experienced a team was entering the playoffs.

This all suggests, a lack of experience could in fact hurt a team. LeBron had some reason to be a little worried.

However, with the Cavs having made the Finals, is their lack of playoff veterans still worrisome? Have three rounds of playoff action perhaps eliminated whatever tends to cause teams with playoff rookies to falter?

To test this I looked at every NBA Finals game since the 1981 Finals, and once again did a logistic regression to try to predict whether the home team won the game. Some variables I included were the difference between the two teams’ winning percentages, SRS, OSRS, DSRS, and percentage of minutes played by playoff veterans, all from the home team’s perspective[4. so home team winning percentage minus away team winning percentage, etc.]. Note that I used the percentage of minutes played by playoff veterans for the team’s whole playoff run, not the individual game, as doing game-by-game could lead to some confounding effects – a team is up they would play their bench players more, who are worse and therefore more likely to not have playoff experience.[5. Though the Cavaliers deep bench players in large part are their playoff veterans – ed.]

Doing this, the most significant predictor was again (somewhat surprisingly) the difference in the two teams’ winning percentages. Furthermore, including the difference between the percentage of minutes played by playoffs veterans for each team did not improve the model. It appears that once a team makes the finals, a general lack of playoff experience is no longer cause for alarm.

So if playoff experience is significantly correlated with winning the Finals once you’ve made the playoffs but not once you make the Finals, when does it affect your chances? I ran three more regressions – one for each round of the playoffs – to determine this. I summarized the results in a graph below, where I plotted the 95% confidence interval for the coefficient for each round (Round 1 is the first round, Round 4 is the NBA finals).

The lack of playoff experience appears to have the most impact in the second and third rounds. In both those cases, the coefficient is positive, meaning once again that having a larger percentage of minutes played by postseason veterans increases a team’s chance of victory. As noted above, the effect fades in the Finals. There two explanations for this that I can think of. Perhaps, by the time a team makes the Finals, they have gotten rid of the playoff jitters. Or maybe there just isn’t enough games/spread of percentage of postseason experience to show up as significant. In either case, the a lack of postseason experience most closely correlates to underperforming in the playoffs in the conference semifinals and conference finals.

As is often the case, statistically demonstrating causation from lack of experience to lack of success is difficult. Alternative explanations for the observed relationships include theories such as teams with a lot of playoff rookies simply over performed during the regular season[7. Playing over their heads perhaps explains lack of previous playoff experience.] and then they experience regression to the mean in the postseason. Despite the casual difficulties, the strength of the correlation plus fact that players themselves seem to think it is relevant, lends some credence to the idea of causality.

As a final note, when looking at the Finals specifically, previous Finals experience might be a boon. The percentage of minutes played by players whom have played in the Finals before is significantly correlated with winning in the Finals. Testing this percentage as a variable in the models for the other rounds, however, showed that it wasn’t as significant as just the percentage of players who had played in the playoffs before. As it applies to this year’s Finals, just over 25% of the Cavs’ rotation has played in the Finals before. Meanwhile, the Warriors are attempting to be the first team since the ‘91 Bulls to win the title when it is all of their team’s first trip to the Finals.