Nylon Calculus: How Second Spectrum is redesigning the NBA

As kids, the basketball obsessives among us acted out the game’s legendary moments using little but our imaginations. We counted down the final seconds of the big game in our head while pounding the driveway blacktop, transforming for those few seconds into a Jordan, a Bryant, an Iverson, ready to shatter a helpless opponent’s hopes and dreams.

For those fleeting moments, it was like we were right there on the court with them. Like we were watching the game through their eyes, seeing the choices they made just as they did. Imagine how cool it would be to recreate that dream-like experience as an adult.

What if we didn’t have to imagine? What if your kids could impersonate LeBron James in the Finals on the hoop outside, then come in and watch a real Finals game through his eyes?

Welcome to the future, folks.

More specifically, welcome to the world of Second Spectrum, the company that took over all licensed optical tracking operations for the NBA starting with the 2017-18 season.

Read More:

Psychological testing is the last frontier of NBA Draft analysis

On a day-to-day basis, Second Spectrum is the NBA’s successor to SportVU, a name that burst into prominence around a half-decade ago. Articles like this one from ESPN’s Zach Lowe introduced fans to a camera-fed, digitized version of the game that converted players on the court (plus the ball) into moving dots on a diagram — dots from which machine learning and researchers could cull a nearly endless amount of information never before available to league decision-makers.

These early revelations were just the infancy of optical tracking in the NBA, with companies like Second Spectrum now at the forefront of their continuing rise. It’s an NBA geek’s paradise; all the best parts of public stat and video sites rolled into one, then put on steroids.

But while basketball nerds could die happy after just a few hours of access, it’s Second Spectrum’s other innovations in the works that could really raise eyebrows.

They’re already in the working stages of several entertainment and media platforms that could change the landscape. Some, such as derived statistics like qSQ (Quantified Shot Quality) and qSI (Quantified Shooter Impact), are actively being incorporated into ESPN broadcasts and written analysis.

Among the more ambitious goals of the company is to extend its scope back across the history of the game, using broadcast-based technologies to enable the LeBron vs. Jordan or Bird vs. Curry debates to be powered by the information being used in the NBA today.

Pretty cool, right? We’re barely getting started. Sooner than anyone might expect, Second Spectrum could begin to completely change the way fans around the globe consume NBA basketball.

Are you a fantasy buff looking for a simpler way to keep track of your players? You’ll soon likely have access to a broadcast with overlaying graphics showing your guys’ stats right underneath them as they run around on the court, a theme that can be extended to a virtually limitless array of graphical additions. Sick of a given play-by-play announcer? Get ready for a future where you choose your own broadcasters for every game, whether they’re actually calling it or not — the power of machine-generated intelligence right at your fingertips.

On top of all that, Second Spectrum wants to create technologies that enable the data professionals now use to be available for every pickup and backyard game of basketball around the world. And yes, that opener wasn’t some pipe dream — the company is already using this very same technology to develop a program where one can view the game from the eyeline of an actual NBA player, with a 360-degree range of view. The capability is already there.

“The reason I know we can do it is, we’ve already done it,” Second Spectrum CEO Rajiv Maheswaran told Nylon Calculus.

How did all of this come to be? What do people inside the league and with specific teams think about the company, both in general and as it compares to SportVU and previous optical tracking efforts? Here’s an unfiltered look inside the universe of Second Spectrum.

Second Spectrum was founded by Maheswaran, Yu-Han Chang (both former professors in the computer science department at the University of Southern California) and Jeff Su, a staff software engineer at a data startup. Maheswaran, the current CEO, and Chang, the current COO, were in charge of a USC research group aimed at helping average citizens make better decisions using data, machine learning and similar concepts.

In 2011, two years before Second Spectrum would form, Maheswaran attended the MIT Sloan Sports Analytics Conference in Boston just as an observer — “the first conference I went to without a paper,” he says today with a chuckle. He learned about tracking data in the NBA, which was just in its infancy; from here, he, Chang and some other members of their team went to work in this area, and ended up coming back to the conference with an award-winning paper the very next year (they’d win another in 2014).

Their research quickly made waves, and the outlines of a company grew from there. Second Spectrum launched in February 2013.

The basis of Second Spectrum was and is a science known as “spatiotemporal pattern recognition.” Maheswaran and his team were far from the first to pioneer the idea of optical tracking in sports, which has been on some tech radars since the mid-2000s or earlier; the idea of using cameras to track movements on a microsecond-by-microsecond basis has obvious appeal in several sports. Where they truly were innovators, though, was in the collection, recognition and useful application of the mountains of resulting data this tracking would produce.

With this theme in mind, Second Spectrum “trained” their machine learning programs to incorporate the outputs from cameras already in place in the rafters of NBA arenas. These aren’t six-figure priced cameras with proprietary designs, either; “If they had 10 GoPros in NBA arenas, they’d lose very little accuracy,” says one source with knowledge of the platform. The real value of what they produce is found in how efficiently machine learning programs can process and interpret the outputs.

The gap between this and prior levels of detail available to NBA decision-makers and analysts was staggering, to say the least. Even for talented team-level analytics experts, a generally confident group who had worked with optical tracking data already, Second Spectrum was a revelation. Some had to virtually beg their bosses to invest.

“This is what we can do [on our own],” says a source who worked in analytics for an Eastern Conference team during the early parts of the company’s rise to prominence, miming a demonstration he gave to his GM. “This is what Second Spectrum can do. It obviously isn’t close.”

Second Spectrum recruited heavily from places like MIT, USC, Cal Tech and others — they’ve employed no fewer than five former MIT basketball captains. For those select few who have an elite understanding of both machine learning/programming and the game of basketball, Second Spectrum is a crossroads of both worlds. The company now has over 100 employees, many of whom boast this kind of background.

And then, over the 2017 offseason, their efforts were rewarded in one of the most public ways possible: an agreement with the NBA kicked in, making them the official optical tracking data provider, replacing SportVU after a side-by-side comparison — one that also included other unnamed entities in the tracking world, per league sources — proved their superiority as a service.

An analogy from that same former Eastern Conference analytics staffer sums up the gap many in the field see between Second Spectrum and others in the space: “If Second Spectrum is the Warriors, everyone else is playing high school basketball.”

Pinpoint accuracy is of paramount importance in the optical tracking world for multiple reasons: The legitimacy of the data is the obvious one, but acceptance across the world of basketball is another huge sticking point.

Remember, this stuff is still very new in the grand scheme of things. Some franchises are already all-in, but others, particularly those with old-school folks in decision-making positions, are still hesitant to put all their eggs in the tracking basket. Even a single pick-and-roll that’s incorrectly logged as a hand-off, as inconsequential as it may seem, could severely damage an entire franchise’s trust in the company and its ability to provide real insights if it happened to be noticed by a GM or coach demoing the platform.

Through this lens, it became clear that STATS LLC, the operators of the SportVU platform, didn’t have the capability to move the entire NBA forward in the tracking realm. As initiators of this technology in the NBA realm, their data struggled to weed out inaccuracies. Per sources independent of both companies but who have worked with each separately; STATS LLC had plateaued in their ability to improve their system. Second Spectrum engineers were even regularly forced to correct flawed raw SportVU data before applying it to their own machine learning during the changeover, these same external sources say.

Second Spectrum helped rapidly improve raw data feeds, removing a large percentage of these errors. The NBA’s league-wide contract with Second Spectrum provides this raw data and their Eagle platform to all 30 teams. From here, though, 26 of the 30 pay additional fees to the company for a premium version of the machine learning Eagle platform.

Put simply, Eagle offers teams a virtually limitless ability to organize, rank and otherwise analyze every piece of data tracked by the platform. Even beyond that, though, the program learns as it receives more data.

Say a coach wants to know how efficient his point guard is at running the pick-and-roll with various different big men on the roster. He’d simply select that play type within his team’s Breakdown page, then be given the option to identify that specific point guard as the ball-handler. From there, Eagle will show every screener he ran pick-and-roll with in the desired time period, how often they ran it, their average points scored per possession, and numerous other insights.

What if that same coach wants to see how that point guard stacks up to other top guards in the league in that category or one of numerous others? Simple: Eagle also contains a Rankings tab with virtually unlimited filtering capabilities.

The big kicker, though, and the one that’s won over even some of the staunchest old-school coaches and GMs in the game, is the video tool.

Every single area we just went over is tied directly to video — this is compared to about 50 percent on the old SportVU platform, sources say. And unlike a service like Synergy Sports, which only tracks possession-ending actions such as shots or turnovers, Second Spectrum can provide video for any segment of the game.

One source who has used both platforms reported finding over 10 times the number of plays logged for the average player in the Eagle database compared to Synergy over the same sample period. The video player is also far more streamlined than Synergy’s, eliminating useless lead-up chunks that cost video staffs hours in the long run as they wait for the important stuff in the play to happen.

Again: An NBA geek’s paradise. One current Eastern Conference analytics staffer put it best:

“It would take a long time to get back to this level without it.”

Now, none of this is to say the platform is perfect. It’s only been available to the full league for under a year, certainly not enough time to iron out every concern coming from 30 different team staffs.

One noted issue, both from Second Spectrum staff and team personnel, is turnaround time. They’ve made major advances here from what used to be a six-hour delay with SportVU, but even the five-to-seven-minute range they’re now able to deliver live data in is a bit too long for certain in-game insights. This delay is mostly precautionary, though, per Second Spectrum sources – by the start of the 2018-19 season, data is expected to be available to teams at a much faster rate, and teams are allowed to have personnel on the bench using iPads, laptops or other systems that can run the company’s tech.

The other big issue? Privacy. NBA analytics staffs guard their insights like precious diamonds. Even with Second Spectrum’s excellent track record of discretion here – “We have an organizational DNA of not talking about things,” Maheswaran tells Nylon Calculus – multiple sources in team analytics departments report a general caginess when they make requests of Second Spectrum.

No one wants to be the first to think of a new innovation, only for it to be shared with the rest of the league and lose its value. This issue is likely just a simple result of how new the service is in many corners of the league; as trust builds here and teams see their privacy concerns met properly over a longer period of time, these reservations should dissipate.

And really, most of the other “concerns” you hear from league people are things Second Spectrum is already working on. One research area within the company involves acquiring a full skeletal picture of players, rather than just a single central torso point for tracking. They’re also working on tracking jumping in several areas. This could improve the accuracy of several metrics, particularly shooting-related ones.

From a fan perspective, everything we’ve discussed up to this point is just a tasty appetizer. Second Spectrum holds the potential to completely revolutionize the field of basketball entertainment, from the way you consume broadcasts to how you help your teenage aspiring NBA player refine his game in the back yard.

Consider an app they’ve been piloting in one large NBA market for most of the past year, one that can overlay various Second Spectrum insights in close to real time on top of a game broadcast — this is where a potential fantasy companion comes in, or numerous other potential applications. Want to know what play types a given team is running? That’s an option, along with several similar applications.

Interested in the probability of a given shot going in as the player enters his motion to shoot it, or at any point on the court? It can be displayed right underneath him as he shoots, all based on the same data powering the Eagle platform. This technology was already on display during the NBA Finals:

Hell, do you have kids who want to see the backboard explode every time someone dunks the ball? It’s all easily possible. Here’s another trial game the company released in partnership with the NBA, a bit more of a fun option:

The potential uses here are almost never-ending, including from an educational and coaching standpoint.

“That is going to be out in the world very soon,” Maheswaran tells Nylon Calculus.

Down the line, Second Spectrum holds the potential to offer a personalized broadcast experience where fans truly dictate everything. We’re talking a fully augmented reality, not a megacast with several separate pre-set options; fans could literally choose their broadcasters for a given game, with machine-generated personalities creating a life-like experience. Combine that with the potential ability to watch full games from a player’s vantage point, and you’re looking at a completely different broadcast product than anyone has even considered until now.

The benefits go beyond public broadcasts, too. Second Spectrum employees regularly play pickup games that are tracked by their own cameras and software; soon fans could be doing this in their own homes, or coaches at every level could be using the technology to improve their approach.

Whether you’re dreaming into the future or marveling at what they’ve already accomplished, there’s no doubt Second Spectrum is a name to know in the NBA moving forward.

Next: Luka Doncic is ready to connect with basketball history

“We have stories about how insights from our platform have helped more than one team win championships,” Maheswaran says. “We have convinced a league with coaches who’ve been around since before video was a thing to use AI-based advice to help win championships. That’s something that we are incredibly proud of.”

Their next step? Convincing millions of NBA fans that there’s a whole new world out there when it comes to the way they consume basketball.