Character Evaluation and the NBA Draft

A proper measure of what might broadly be termed “character” is the missing variable in most NBA draft models. After the fact, the stories told about dramatic draft busts generally point to poor work ethics or immaturity as reasons the players’ failures should have been predictable. Looking backwards, these are the tails told about the likes of Michael Beasley, Joe Alexander, or Javaris Crittenden[1. Though those stories don’t always match up with the contemporaneous evaluations prior to their respective selections.]. Meanwhile, the narratives about players who far outperform their “draft stock” such as Most Improved Player Jimmy Butler 0or Defensive Player of the Year Kawhi Leonard focus on their tireless work ethic, ability to overcome adversity and terrific character as “winners”.

These things may all have been true and observable before the fact. But the question for a team or draft analyst evaluating  a player is to what degree can these traits be seen beforehand? How many middling performers are out there putting in just as much work but not getting the results or the glowing media attention that comes with it?

I have been on the search for draft contemporary scouting and character evaluations for a while, with no luck so far. Absent that data, I decided to do some broader research on the topic looking at what prior academic work had to say about character and performance projections.

The main work horse model for this type of thing in psychology is the five personality factor model. With the five factors being Openness, Neuroticism, Extroversion, Conscientiousness, and Agreeableness. There has been extensive research into the relationship between the five personality traits and academic achievement, a fair amount of study on professional achievement, but less research directly on sports.

Conscientiousness

In all three areas the most consistent predictor of future achievement among the factors was conscientiousness. Conscientiousness is closely related to the currently popular notion of “grit.” In the Wikipedia definition it is described as follows:

"Conscientiousness is the personality trait of being thorough, careful, or vigilant. Conscientiousness implies a desire to do a task well. Conscientious people are efficient and organized as opposed to easy-going and disorderly. They exhibit a tendency to show self-discipline, act dutifully, and aim for achievement; they display planned rather than spontaneous behavior; and they are generally organized and dependable. It is manifested in characteristic behaviors such as being neat and systematic; also including such elements as carefulness, thoroughness, and deliberation (the tendency to think carefully before acting.)"

In an extensive meta-analysis of the existing research of academic achievement and personality traits German researchers Sabrina Trapmann and Benedikt Hell found that conscientiousness was the only personality trait consistently associated with higher academic achievement. Their conclusion was that in academic settings conscientiousness explained 7.2% of the variation in achievement, and that number could drop to 2% if grades from high school was included given that conscientiousness was also strongly related to high school success.

There have been a few smaller studies of personality traits and the relationship to sports achievement[2. And many more studies looking for distinctive personality traits of athletes compared to non-athletes. Spoiler alert: these investigations seem to find little evidence of a specific athlete personality traits. One study looked at personality measures and position in basketball also found no differences, not surprising given how extensively position correlates with height.] I did not find enough to draw a separate consensus about conscientiousness and any independent relationship to success in sports. Though there were some interesting studies:

• One study of American football players indicating both higher conscientiousness and neuroticism positively predicted football rank.
• Another study of Australian rules football players found tremendous success with a very small sample size using only an athleticism rating and personality evaluation.
• One of Canadian Hockey players that used a normative all in one personality test, though the analysis seemed overly simple.
• A study of Iranian Football and Futsall players indicating that conscientiousness had a positive relationship with performance.
• A study of women NCAA soccer players found that conscientiousness explained 8% of statistical performance.

The overall trend of the direct sports related studies does seem to line up with the academic achievement data, conscientiousness has some independent relationship to future success by athletes as it does for students.

The Conscientious Bayesian

As a thought exercise, I decided to use the parameters from the larger literature on academic success as a simplified adjustment to my Quality Player Odds (QPO) model using Bayes theorem. The QPO model is ideal for this exercise as the output is a simple estimation of the player’s odds of becoming at least a quality role player/borderline starter.

The range of influence for conscientiousness is estimated at 7.2%, from high to low, applying a simple three point scale of High, Medium and Low conscientiousness indicates a +3.6% factor for a high conscientiousness (HC) player and a -3.6% for a low conscientiousness (LC) player, and no adjustment in the middle range (MC)[3. This is actually a higher end adjustment from the academic success, but I think it is justified as college athletes may be able to succeed based on athletic advantages they won’t have in the pros, so there maybe less correlation to college success with conscientiousness].

Taking a few players in the middle of the model’s estimate range I will apply the adjustment based on a hypothetical conscientiousness rating. We start out with the QPO model as our prior, or starting estimate;

• J.P. Tokoto- 45%
• R.J. Hunter- 43%
• Dakari Johnson- 42%

Let’s say, just as a hypothetical example[3. Ed note: to be absolutely clear, we don’t have any indication one way or another about the personality types of these three players. Andrew is merely using a set of roughly similar prospects as a demonstration of a theoretical magnitude of a “character” variable.], that our ratings indicate that J.P. Tokoto has a medium conscientiousness (MC + 0) score, R.J. Hunter high (HC +3.2) and Dakari Johnson low (LC -3.2). Updating our model using Bayes theorem we get these updated odds of reaching the quality player minimum:

• R.J. Hunter- 46.8%
• J.P. Tokoto- 45%
•  Dakari Johnson- 38.9%

Hardly a game changer. But, some useful information if a team is truly torn between closely rated prospects.

Part of a literature review like this is actually to cool one’s jets when applying the kind of ad hoc adjustments that are inevitable dealing with both voluminous and incomplete information like the draft. The sum total of research gives us a better grasp of the direction and magnitude of an adjustment we should be applying.

I should note that I do not mean these scale adjustments to be applied to prospects with significant red flags, like Robert Upshaw who was removed from two different college teams. Those cases need to be considered separately based on the particular merits of the prospects’ history and similar past cases.

The study results for academic achievement relied primarily on responses to written personality tests, while the sports achievement studies relied on combinations of written tests and judgement by coaches[4. In the sports studies players scoring high on neuroticism were rated less well be their coaches, but this seemed to have little effect on performance judged by objective on field performance]. Those methods are unlikely to pick up the intricacies or magnitude of a prospects who’s had real concerning past legal, drug or behavioral problems. Those tests are also unlikely to be directly available to drafting teams, and even if they did administer personality tests to all of the prospects they were interested in drafting, the prospects would have every incentive to game the tests and, in all likelihood, coaching from their agents to help them do so.

But, be that as it may, teams looking at prospects need to evaluate the whole player and adjust for the differences in work ethic, perseverance and character. In doing so they should have as sound a model as possible and realistic expectations of their ability to do so.