Nylon Calculus: Regression to the mean can even fool NBA coaches

Regression to the mean has the potential to fool everyone, even NBA coaches building systems of positive and negative reinforcement.

These are challenging times, but one thing I’ve enjoyed about sheltering-in-place has been having some extra time to read. I find it really satisfying to learn about new subjects and, of course, to think about how these new subjects relate to my old favorite subject, basketball. My latest read is the book Thinking Fast and Slow, by the Nobel Prize-winning psychologist and behavioral economist, Daniel Kahneman. It’s an autobiographical look at Kahneman’s life spent studying how our brains work.

In the book, Kahneman relates a eureka moment that he once had while speaking to a group of flight instructors from the Israeli Air Force. He was explaining how rewards for improved performance work better than punishments of mistakes when one of the senior instructors in the group interrupted him with an objection:

"“On many occasions I have praised flight cadets for clean execution of some aerobatic maneuver. The next time they try the same maneuver they usually do worse. On the other hand, I have often screamed into a cadet’s earphone for bad execution, and in general he does better on his next try. So please don’t tell us that reward works and punishment does not, because the opposite is the case.”"

What the instructor had chalked up to the power of punishment, Kahneman recognized as an example of regression to the mean due to random fluctuations in the quality of the pilots’ performances from flight to flight. Those cadets who executed perfect aerobatic maneuvers had likely been a bit lucky. Those cadets who executed exceptionally poor maneuvers had likely been a bit unlucky. Kahneman deduced that — whether he was praised or screamed at — any cadet who turned in a truly exceptional performance was likely to be more average during his next flight.

Basketball coaches can be fooled by regression to the mean too and, just like Kahneman’s flight instructors, coaches can become enamored with the power of punishment as a result.

Take my high school coach. He was a smart, rational guy; but when our team screwed up — by missing too many free throws, for example — we were punished in practice the next day. There would be lots of running and yelling, possibly some puking. Ostensibly, the wind sprints were meant to improve our conditioning and mental toughness so that we could toe the charity stripe without feeling our knees wobble; but there was always an undertone of retribution as well. Miraculously, our foul shooting would usually improve in the next game and that’s probably why our coach continued to favor this corrective approach.

We can use a really simple simulation to illustrate how regression to the mean could make this kind of punishment look like a panacea (and make a reward look like a jinx). For example, let’s consider the distribution of single-game free-throw percentages of the Miami Heat, the worst foul shooting team in the league during the 2018-19 season. They averaged 69 percent from the line during that year with single-game free-throw percentages as low as 26 percent and as high as 94 percent (reflecting a standard deviation of 11 percentage points). We can sample at random from a normal distribution with a mean of 69 percent and a standard deviation of 11 percentage points to simulate how such a team’s foul shooting might vary from game to game.

Now, let’s imagine that coach Erik Spoelstra has a policy of rewarding the Heat players with ice cream whenever they convert more than 90 percent of their foul shots in a game and of reprimanding them with screaming and yelling whenever they shoot below 50 percent from the line. You can see below how random fluctuations in performance could be misconstrued as a validation of this punishment.

In this simulated season, the Heat surpassed 90 percent from the free-throw line on five different occasions. Each time, Spoelstra handed out ice cream cones, but all that sugar must have gone to the players’ heads because their shooting suffered every time. On the other hand, whenever the team’s foul shooting dropped below 50 percent for a game, Spo became irate and lit into the players. And — holy moly! — that yelling really worked; the team improved every time.

Now, let’s extend this thought experiment by simulating the careers of 1,000 coaches, for 20 seasons of 82 games each.

After running the simulation 20,000 times, we find that our imaginary teams faltered after earning ice cream 98.7 percent of the time and they improved in response to being chewed out 98.0 percent of the time. It’s pretty convincing evidence, right? Think of how these experiences might shape a coach’s perspective. Imagine the coaching tree that would descend from our thousand digital gurus — all the books, clinics, and YouTube videos they could produce extolling the virtues of punishing poor foul shooting.

Of course, there wasn’t really any crying or any ice cream in my computer simulations (neither one of those things is any good for the motherboard). The ups and downs of these shooting performances were not really proof of the power of punishing mistakes or the dangers of rewarding success; they were the direct result of our process of randomly sampling from a normal distribution. It was exactly as expected that the outlier shooting performances were almost always followed by a regression to the mean. It was math, not motivation.

These imaginary Miami ice cream parties are completely contrived scenarios, obviously; but there are very real risks associated with having a dogmatic belief in the power of punishment.

Remember when coach Tom Izzo berated freshman Aaron Henry during Michigan State’s first-round NCAA tournament win last year? “What’s wrong with challenging a kid that makes some mistakes?” Izzo wondered after the game.

"“I did get after him and he did respond and he did make a couple of big buckets…”"

In justifying his aggressive behavior, Izzo ended up sounding a lot like Kahneman’s flight instructor, didn’t he? He screamed in his guy’s ear about some bad execution and on the next try, Henry did a lot better. He responded.

Except Henry didn’t really respond. At least not immediately. He turned the ball over five times in the next ten minutes. He missed a dunk. He looked confused. He was shaken, not motivated.

"“I was just like, ‘Man, I just can’t do nothing right,’ ” Henry said."

To his credit, Henry eventually calmed down; sinking a jumper and a pair of free throws to help his team close out the game. In the end, he regressed back to his (very talented) mean level. But did Izzo’s tirade really help? Or was he like our imaginary Spoelstra, yelling at random fluctuations in performance?

To be fair, Izzo’s situation was more nuanced than our silly simulation. He didn’t go off on Henry for missing shots; he was upset about what he perceived as a lack of effort.

The quality of a basketball performance is the result of a combination of several factors including talent, preparation, and effort; but also luck. If being talented and well prepared are the prerequisites that define a team’s mean level of performance, then game-to-game fluctuations in performance must be determined by changes in effort or luck. In-game motivation from a coach could presumably impact a player’s effort level but not his or her luck. We could think about trying to position all of the individual elements that comprise a basketball performance along a spectrum from effort to luck. For example, game-to-game fluctuations in foul shooting are mostly due to luck, whereas inconsistency in the speed with which a player runs back on transition defense is assumed to be more about effort level.

And yet, even something like defense — which at first blush is entirely dependent on effort — can be influenced by randomness. A bad bounce, a convincing ball fake, a wet spot on the floor — any of these unlucky coincidences could leave a defender flat-footed and looking lackadaisical. The truth is that effort (and perceived effort level) fluctuate from moment-to-moment and from game-to-game so that even the elements of the game that are most likely to benefit from in-your-face discipline are also subject to regression to the mean.

Kahneman firmly believed that rewards for improved performance work better than punishments of mistakes. While I’m definitely more ambivalent about the power of negative reinforcement, I do believe that negativity can become counterproductive in basketball. Moreover, because players and teams tend to regress towards their mean levels of performance, coaches are likely to overestimate the effectiveness of any past efforts to punish players for their mistakes.