Grading the 2016 NBA Mock Drafts

We’re all used to grading teams that just drafted a bunch of teenagers and other college-aged kids, but it’s important to grade and analyze the process too. Without that, we’re driving blind and relying on a system we know nothing about. NBA mock drafts are big draws on the internet and television, but how consistent are they and which ones perform the best? I’ve graded last year’s draft too, and looking at this year there’s a clear winner.

More from Nylon Calculus

I’ve graded several of the most well-known mock drafts below. For legitimacy, I grabbed all the data right before the actual draft started to eliminate any chance of chicanery. I included all the writers and websites from last year for a comparison. There’s Chad Ford of ESPN; the preeminent DraftExpress; CBS has Gary Parrish and Sam Vecenie; the NBA site has their own mock draft via Scott Howard-Cooper; Sports Illustrated has Andrew Sharp; the salary site Hoopshype has one too; Bleacher Report has one from Jonathan Wasserman; and lastly NBAdraft.net has one plus another created by user consensus.

The first method is to catalogue the number of hits. This is simply the number of draft picks successfully predicted. Chad Ford wins here with 11, followed by Draft Express and Andrew Sharp of Sports Illustrated, respectively, with eight. However, both Sharp and Hoopshype only had predictions for the first round, which undervalues their abilities. You’ll note there are a few second round hits, which are usually quite rare. Both Wasserman of Bleacher Report and NBADraft.net had three such hits, which are like holes-in-one. The greatest hit was probably Thon Maker, who many people had going in the second round; Ford got him exactly right at a surprising 10th spot.

2016 draft hits /

The above method ignores all other draft picks, however; a better one is to rate them using the root-mean squared error method. If a player is drafted 5th and the mock draft said 7th, then you square that error (i.e. 2^2=4), sum that for every pick, divide by the number of picks, and take the square root. This heavily penalizes picks that are way off. Using that, as seen in the table below, Draft Express dominated the lottery and had the best score overall. Chad Ford was hurt by the early Taurean Prince and Georgios Papagiannis selections, as well as Pascal Siakam in the first round, who Ford didn’t even have drafted in either round. There was a near three-way tie for second place, while NBADraft.net stood out thanks to an excellent first round derailed by a shaky second round.

2016 draft rmse /

The root-mean squared error method isn’t ideal, however. An error in the second round isn’t the same as in the lottery. To correct for this, I devised a simple method: take the absolute error and divide by the draft slot number. Thus, getting the first pick has a heavy penalty while you’d have to be way off for the 60th pick to make a dent in your rating. Draft Express is the winner there followed by Chad Ford and then Draft Site, which is purportedly the first mock draft site. A top performer from last year, Sam Vecenie, did not fare well this time. But Draft Express “won” for the second year in a row. Looking at the overall values this year, it was a more unpredictable draft than in 2015.

2016 draft rating /

For the best value, Draft Express is the gold standard in mock drafts. They do last-minute corrections, they have a wealth of data and analysis on their site, their mock drafts are usually the most accurate, and importantly their site is actually free. No other mock draft besides Chad Ford’s is a close competitor if you look at last year too.

My work ends here, but theirs doesn’t — the major sites already have 2017 mock drafts. And now we know which sites to follow and how accurate we can expect them to be.

Next: Average position measurements: 2016

Home/Nylon Calculus