Nylon Calculus: How will we know if the tanking reforms work?

Just before noon UTC on Sept. 28, 2017, Adrian Wojnarowski broke the news that the NBA’s Board of Directors passed draft lottery reform.

The intent of the reform is to reduce the incentives behind tanking — the purposeful losing of games in order to improve the odds of getting a good pick in the draft. The draft was originally intended to better allocate talent to weak teams and improve competitive balance. However, that intent created the incentive to lose.

The 2018 NBA Draft will be the last one where the team with the worst record is guaranteed to pick in the top four. Beginning in 2019 the team with the worst record will only be guaranteed a pick in the top five, a harrowing fall, I know. The teams with the three worst records at the end of the season will all have a 14.0 percent chance at the first pick in the draft as opposed to the tiered 25.0 percent, 19.9 percent and 15.6 percent respectively. The biggest change will be to the team with the eighth-worst record, whose odds of getting the first pick will more than double, from 2.8 percent to 6.0 percent.

There will still be an incentive to be bad, but just less incentive to be awful.

Read More:

The college prospect growth curve

So, how will we know if the reforms work? It is unlikely that the entire landscape of the NBA changes with the reforms. It is far more likely that there are only marginal changes.

In 2002, Dr. Beck Taylor and Dr. Justin Trogdon wrote a paper introducing a method to quantify tanking for the Journal of Labor Economics, titled Losing to Win. In it, the authors use a logistic regression to study the effect that being eliminated from playoff contention has on the probability of winning. A logistic regression is one where the dependent variable is binary — in this case, one for a win and zero for a loss. The coefficients assigned to the independent variables are the log-likelihoods that correspond to a unit change in the variable.

That might sound complicated, but it is simply a way of determining how certain variables affect the likelihood of a certain event occurring.

In the paper, the authors study three separate seasons: 1983-84, 1984-85 and 1989-90. Those three seasons were chosen for their differing lottery rules. In 1983-84 draft position was decided solely on record; where the worst team was given the first pick, the second-worst was given the second pick and so on. In 1985 the lottery was established, and all non-playoff teams had equal odds in the lottery. Finally, in 1990 the lottery was altered to the system we have today; where teams with worse records have a higher probability of getting a better draft pick.

The incentive to lose is strongest for both the 1984 and 1990 seasons, as better draft position is tied to having a worse record. The expectation, then, is that once teams are eliminated from the playoffs, they will be less likely to win in the 1984 and 1990 season. However, there will be either no effect, or a much smaller one for the 1985 season. Indeed, the results bare this out. In 1984, a team was approximately 22.0 percent less likely to win a game once they had been eliminated from playoff contention, they were only 7.0 percent less likely in 1985 and 20.0 percent less likely in 1990.

To figure out how things might work in the future, we should first examine the status quo. I will be looking at the 2017-18 season in order to estimate the magnitude of tanking. All game data comes from Basketball-Reference. To determine when a team was eliminated from playoff contention I relied on a combination of articles and reddit posts.

My study closely follows that of Dr. Taylor and Dr. Trogdon’s, with some key changes. In order to control for team quality the authors used running win percentage, where each team’s win percentage is listed as their win percentage going into a given game. In general, net rating conveys more information than simply win percentage. There is a difference between winning a game by 20 or by only one point, but win percentage treats the two as the same. Instead, I will be using running net rating.

Another problem arises when using running totals. A team’s net rating after the first game is less likely to be representative of the team’s quality than the net rating after the 81st game. Each subsequent game provides more information, but the model treats net-rating after the first game as equally representative as the net rating after 81st game. The issue is best seen visually.

The lines tend to look haphazard at the beginning of the season, but stabilize near the 25th game. The trend is even more clear when highlighting one team.

Cleveland had a — to put it politely — turbulent season. In the first 10 games, their net rating bounced above-and-below zero; peaking around plus-10 and bottoming just below minus-5. Clearly the team did not change drastically in that time — although the roster did see its share of change this season — instead there simply was not enough information to stabilize against a few large differential games.

A solution to reducing the early season variance is to use a statistical prior. The prior’s main purpose is to provide more information to the early part of the season in order to reduce the effect of outliers.

For my prior I will be using Jacob Goldstein’s projected margin of victory.  The formula for net-rating with a prior is set up as such: [(A+Running Differential)/(1000+Running Possessions)]*100. A is Jacob’s expected margin of victory*10 in order to estimate the possible margin of victory after 10 games. Similarly,1000 was chosen for the denominator to estimate the number of possessions over a 10 game-span. The rating is updated after each game, so a team’s net-rating going into the first game will simply be equal to Jacob’s projection. I chose to have the prior be roughly equal to 10 games so that the net-rating will stabilize early in the season, but its effect will be largely quieted later in the season as the actual results convey more information.

It is immediately apparent that, although there is still some early-season variance, the lines are much more stable with the prior. Let’s take a closer look at Cleveland now.

Just as expected, the net rating starts closer to zero and has both a lower peak and higher floor. The net rating at the beginning of the season ends up being much closer to the net-rating at the end.

Using roughly 10 games worth of information as a prior seems to have the desired effect as well. Looking at the difference between net rating with a prior and the actual net rating after 10, 41 and 82 games shows the distribution of the differences condenses as the season goes on. The median difference between the two net ratings approaches zero as well. The median difference after 10 games was .06, -.02 after 41 games and -.008 at the end of the season.

The independent variables in my regression were:

Net-Rtg: Running net-rating

O Net-Rtg: The opponents running net-rating

Home: Whether or not it was a home game

Elim: Whether or not the team had been eliminated from playoff contention

O Elim: Whether or not the opponent had been eliminated from playoff contention

Again, the dependent variable was whether or not the given team won the game. The results without applying the prior to net-rating are listed below.

Direct your attention to the third column. Controlling for team quality, being eliminated from the playoffs was associated with a 19.0 percent decrease in the probability of winning the game. A result similar to that of Dr. Taylor and Trogdon’s findings for the 1990 season.

When including the prior:

The probability of losing the game decreases to 14.0 percent. Using expected point-differential is not a perfect answer. However, this may suggest that the model without the prior was in-fact impacted by early season noise. The pseudo R-squared, a measure of how well the model fits the results, improves when using net-rating with a prior.

This methodology may not be perfect. A mixed-effects logistic regression may net more precise result. However, the findings align with common knowledge regarding the NBA.

Using elimination from playoff contention as the point in which a team decides to tank probably does not align with the extent to which teams are willing to tank. The Phoenix Suns began the season losing their first three games by a combined million points (if memory serves), sitting their best player, then trading said player. Using an indicator variable for the point in which a team’s odds of making the playoffs dips below a certain point (30.0 percent?) may give more robust results, as it likely better reflects the point in which most teams decide to tank.

In general, though, teams tend to be less likely to win a game once they have been eliminated from playoff contention.

Next: Grayson Allen, draft stock and quantifying the decision to go pro

Since reading Losing to Win, I have wondered what the effect the draft’s talent may have on the incentive to tank. I can think of no better season and draft to examine than 2002-03. Again, the data comes from Basketball-Reference. I did not have prior to apply to the running net-ratings for teams. Instead of relying on Reddit to determine when teams are eliminated from playoff contention, I simply used when a team’s possible wins (current wins + # of games left) was less than the total wins the eighth seed had. There may be some error due to tying rules with regard to elimination from playoff contention.

Being eliminated from playoff contention was associated with a 16.0 percent reduction in the probability of winning a given game. The reduction is just less than the associated probability for the 2017-18 season. This may be due to a relatively noncompetitive tanking race. Cleveland and Denver were both so objectively terrible that there was no chance of any other team catching them. Simply replacing an indicator variable for Cleveland — who would go on to draft LeBron James — in place of elimination from playoff contention nets a result suggesting that simply being the Cavaliers was associated with an 11.0 percent reduction in the likelihood of winning — again, even while correcting for team quality.

That means that a team with exact same net-rating as Cleveland, playing a team that has a net-rating equal to Cleveland’s opponent was still more likely to win than Cleveland.