Numbers aren’t the problem (people are)

A specter is haunting soccer — the specter of analytics.

It has already overtaken baseball. We know this because Brad Pitt was nominated for an Oscar for playing Oakland A’s general manager Billy Beane, valiantly trying to exploit market inefficiencies in MLB. Jonah Hill was also there.

Basketball is gone, too, lost in a flood of eFG% and PER and VORP and VLEEP and VLOOP. Football and hockey aren’t far behind, emboldened, in the way of things, to search for their own acronyms, to have their own debate about the value of new and old, numbers and eyes.

Soccer, then, has come increasingly to seem like the final frontier of the analytics movement. If soccer falls, there will be no worlds left to conquer. The analysts may sit, and weep.

There are good reasons for this: soccer is low-scoring, fluid, recordable actions are relatively scarce, there seems occasionally to be a direct correlation in certain positions, mostly center-back, between being good and not doing anything (in the words of the great Paolo Maldini, “If I have to make a tackle then I have already made a mistake”) and, above all, stats other than goals have only really been recorded for the past 25 years. There are also bad reasons for this, like that some people blame good analysts for bad analysis, and other people are afraid to admit the shortcomings of their own, non-statistical expertise.

But the game is changing. Not only do more and more professional clubs employ analytics departments, stats like pass completion percentage and key passes are now embedded into the vocabulary of the average fan.

The value of this development to professional clubs is, or should be, obvious. The proper employment of analytics gives them a real, tangible advantage on the pitch, resulting in more wins. But what is the value this development to everyone else, and what might a mainstream embrace of analytics look like?

First, what is analytics, where does it come from, what language does it speak and why on earth is it taking our jobs?

***

Alan Shearer*, who has scored 60 more goals than any player in Premier League history, is not a popular man. The former Newcastle and Blackburn striker is now one of the mainstay pundits on England’s most popular soccer highlights show, Match of the Day, on which much of his analysis consists of his saying things like, “he should have scored” and “he’s got to score that” and “that has to be a goal,” often (maybe) all in the same sentence.

(*Alan Shearer is not, to my knowledge, the reason soccer analytics exists, but he is a helpful explanatory tool.)

These sorts of comments never seemed particularly insightful, but Shearer always benefited from the fact you, the viewer, could say to yourself pretty confidently, “to be fair, he probably would’ve scored.” But as punditry became more sophisticated, and as fans sought out more, and more in-depth information, he’s-got-to-score punditry came to appear increasingly quaint. Or to use another, less polite word, stupid.

For a certain kind of fan, the kind who likes cold, hard facts more than warm, soft feelings, a broad acknowledgement of the silliness of Shearer-ism wasn’t enough. This kind of fan wanted to know, once and for all, what was the right thing to say when not-Alan Shearer fluffed a shot from inside the box. And so it was that, for those versed in the early phases of soccer’s analytics movement, it became possible to say with conviction not only that Shearer wasn’t insightful, but also that he was wrong.

This was, and is, because of expected goals (xG for short), the most popular advanced stat the sport so far has to offer. xG tells us the likelihood each shot in a match will result in a goal based on the outcome of similar shots taken in the past. You think Harry Kane should have scored that first-time volley from the top of the 18-yard box? xG tells us the probability of him scoring there is, in fact, about 0.1, i.e., not good.

The value of xG over a single match is limited, but over longer periods, it’s one of the most accurate indicators we have of whether a team is “good,” that is, whether they’re creating good chances and preventing the opposition from doing the same. Players don’t always convert good chances, but the fact a team is creating them is an indicator of success in the long run, and so xG is a much better way to predict future outcomes than goals (which are often lucky) or shots (which are often bad) or most anything else.

As is the case with many advanced stats, xG is a way of assigning an objective numerical value to concepts already employed in even the most Neanderthalic argument in the pub. Liverpool deserved to win. Bollocks, United were better. If Rashford hadn’t missed that chance, we would’ve won. xG gives us, among other things, a way to objectively evaluate such claims.

Many at the forefront of soccer analytics have shifted their focus away from xG — to the role of passing, the nature and shape of possessions and, most elusive of all, defensive metrics — but it’s worth dwelling on, for a couple reasons. First, it’s relatively easily digestible, the sort of thing someone like Shearer, if he were so inclined, could discuss in a 30-second segment on Match of the Day.

Second, and a little paradoxically, it’s really complicated. As James Yorke, managing editor of the soccer analytics website StatsBomb, wrote to me in an email, “xG is an interesting line in the sand, because its adoption represented a before and after between which anyone could get interested, pull some numbers and get really actively involved [in analytics], and the not so trivial exercise of sourcing data and model building that came with xG.”

To build an xG model, you need real expertise; almost certainly a lot more than whatever math/statistics you were taught in school. Even just to understand the theory behind creating an xG model involves time and a level effort most fans either aren’t willing or don’t have the time to exert. For the average fan, then, buying into xG means grappling with a lot of unfamiliar, and often non-intuitive, concepts.

How can you compare a chance for Lionel Messi to a chance for Connor Wickham? Philippe Coutinho scores from distance all the time; it’s not just luck. It doesn’t matter whether a team creates good chances; what matters is whether they score. Good teams find a way to win.

The problem with these sorts of criticisms isn’t that they aren’t valid (though some of them aren’t); it’s the idea, all too common, that even the smallest shortcoming of a stat like xG should persuade us to scrap the whole thing. This is stupid for many reasons, but the most schadenfreudean is this: if we treated pundits the same way, we’d have none left.

The larger point is that not even the most devoted analytics-type has ever claimed the numbers — let alone one, single, specific number — can tell you everything there is to know about a sport. They are one tool for analysis among many, and must therefore be interpreted with skepticism. That’s part of the point. Good statistical analysis involves acknowledging what the numbers don’t tell you as much as it involves acknowledging what they do.

And so, for all its interesting and valuable applications, xG is perhaps best viewed as a sort of gateway stat, its ultimate worth to fans more philosophical than analytical. Indeed, it’s possible the most important work xG will do, when we come to look back at the early days of soccer analytics, will be to teach people to appreciate the value of these numbers, or at least to respect the time and devotion and knowledge required to calculate them.

xG may be stuck, for now, on the fringes of the mainstream, but as it and other advanced stats have fought for acceptance, there’s been a huge growth in the use of not-quite-advanced stats* in soccer media overall. It’s rare to watch a match now and not hear the commentators at some point reference possession percentages or shot totals (seemingly always with that gleeful, dumb caveat: the only number that counts is the one in the top-left corner of your screen). If you follow matches on Twitter, you will almost certainly see a touch map or heat map, pass completion percentages, duels won and more.

(*When does a stat become advanced? There’s no set rule, but there’s a distinction between basic counting stats [goals, assists, passes, etc.], stats that require either interpretation [what constitutes a dribble?] or simple calculations [pass completion percentage] and stats that depend on models [xG]. These all exist on a spectrum, with goals on the simple end, expected goals on the advanced end, and the rest scattered in between.)

The increased visibility of these numbers owes much to the work of Opta, the first company to start seriously collecting stats beyond goals scored, and now the dominant data company in the sport. Opta, which was founded in the mid-90s, has played a hugely significant role in the development of soccer analytics, but its data aren’t freely available, and so fans who want access to this information often end up at sites like WhoScored, Squawka and Transfermarkt, which offer it for free.

This marks progress, in its way. The problem with stats is their interpretation requires expertise, and the media outlets best exploiting the stats boom are often not the ones with that expertise. It’s not uncommon, for example, to see Sky Sports tweet a graphic listing, say, the five “best” passers in the Premier League in 2016-17, who as it turns out are the same five players who completed the most passes in the Premier League in 2016-17. But of course you don’t have to be Pep Guardiola to figure out the number of passes a player completes does not a good passer make.

Perhaps the most notorious examples of bad data are the rating systems at WhoScored and Squawka. Both sites use an algorithm to rank players based on their raw statistical output (the WhoScored Rating is, mercifully, out of 10; Squawka’s “Performance Score” doesn’t even make that limited amount of sense).

These numbers are regularly cited, including in such auspicious outlets as the Guardian, as if they provide evidence of something. And yet interpreting them is impossible, because neither site explains how its scores are calculated, and the numbers themselves often tell a totally different story to the eye test.

(One big misconception about stats is that they tell us anything more than what they literally tell us. The fact Paul Pogba completed 85 percent of his passes last season doesn’t mean Paul Pogba is a good passer or a bad passer or somewhere in between. It means he completed 85 percent of his passes last season. Anything beyond that is interpretation. In other words, there are no bad numbers; only bad analysts.)

And so the bigger problem is that these scores are presented, with no small amount of pride, as definitive, catch-all reflections of player performance. For all the good work WhoScored and Squawka do, the presentation of these numbers is awful, not only because the numbers themselves tell us little, but also because they misrepresent what good statistical analysis looks like, and therefore only serve to increase skepticism toward the important work being done in the field.

Talking about these sites, Mike Goodman, who wrote about soccer analytics for Grantland and ESPN and is a co-host of the analytics-minded Double Pivot podcast, told me, “The quality of that content can vary dramatically … I worry that you risk recreating the problems that baseball had where for, you know, 80 or 90 years, they were looking at statistics that were not the best statistics. But they’re sort of easy ones for a broader audience to grasp, even if they’re not the best ones in terms of the things that you might value for making decisions, for projecting, those kinds of things.”

ESPN senior writer and Times columnist Gabriele Marcotti told me this sort of bad data is also making it harder for good analysts to make an impact at the professional level.

“I’ve been around a while. I’ve gotten to know many, if not most, managers in the top leagues and I can tell you from speaking privately to them their familiarity or awareness with even very basic things like expected goals is extremely limited,” he said. “And I think in some ways there’s a reason for that. I think when … data first became available there was a lot of what I consider bad data or meaningless decontextualized data, like, you know, distance covered or passing percentage or possession percentage and I think a lot of the managers looked at this and quite clearly, quite soon realized that this is kind of nonsense on its own. And so I think a lot of the more inquisitive ones, you know, they became naturally more skeptical towards some of the, what I would consider, more meaningful analytics that came in later.”

Any meaningful mainstream embrace of analytics must, therefore, involve some sort of statistical education. Unfortunately, because of all the bad stats work currently, and so visibly, out there, the challenge facing analytics experts involves more than finding a platform and explaining their research; it involves re-educating people about fundamental statistical concepts. Even more unfortunately, that effort will depend largely on the same mainstream outlets that seem either not to care or not to understand that their own statistical analysis is so bad.

And, as Marcotti pointed out, there’s no immediate incentive for mainstream outlets to change their approach. “There’s so much media out there I think this type of [statistical] analysis is there for people who want it and who choose to seek it out,” he said. “But I don’t think that, you know, apart from a very small group of analytics types, I don’t think people sit around, you know, football fans, and during a game start talking about passes per defensive action or anything like that. So I mean until that enters the mainstream vernacular, I don’t think mainstream media is necessarily going to react to it in a big way.”

Still, Yorke is optimistic: “Maybe I’m being hopeful that organizations will see the value in smart application of stats analysis and move towards better quality content, but it’s simply not good enough to put up a basic list of totals and draw conclusions. If organizations want genuinely informative stats usage in their content, they need to talk to people who have experience in this work (hello!) not just glue stats onto narratives or build bad narratives from misuse. So what will cause the gap to close? Hopefully demand. StatsBomb itself has grown year on year, more and more people see our stuff and engage with it, there’s no reason why that audience can’t translate to the wider media.”

Given these challenges, it’s important to ask what the value of a statistically literate fanbase is. The obvious answer is that it would be more informed and intelligent than a statistically illiterate fanbase, and that’s a good thing, because being more informed and intelligent is better than being less informed and intelligent. That seems reasonable enough. But, alas, fandom is not the province of the reasonable.

As Goodman told me, while pundits have a certain obligation to try to get things right, “ … fans are under no obligations to get things right at all. They can believe whatever the heck they want to about their team, they can root however they want, they can follow whatever they want … there’s no sort of obligation there.”

This point is particularly relevant when you consider the anger numbers often seem to elicit, from fans and pundits alike. Goodman, however, doesn’t think this is a response to the numbers specifically. “People often ask me, do people respond negatively to me because I write about analytics and my answer is generally no,” he said. “My answer is generally people respond negatively to me when I say something mean about their team. When I say that this team is getting lucky or this team played badly. It’s just that the way they then respond to me because they’re angry about what I said about their team is to disregard the field of analytics because that is the thing they can latch onto.”

He’s right of course — it’s amazing how quickly fans embrace analytics (or anything, for that matter) when it tells them things they want to hear — but the unique character of the anger these numbers evoke is nonetheless interesting, and often seems to run deeper, or perhaps wider, than club loyalty.

The most curious evidence of this is the regularity with which analytics types are dismissed on the grounds they don’t really, when it comes right down to it, like the games they’re analyzing. What they like is math or statistics. A wedge is driven between passion — foaming at the mouth, swearing at the ref, staring at the top-left corner of the screen — and statistical insight.

As Yorke wrote, “It’s a natural by-product, a hell of a lot of people hated maths at school! One traditional view of football is the experience: go to the match, sing the songs, shout at the ref and so on. If that’s your relationship with football, you might not be impressed if I tell you your striker is rubbish because he doesn’t shoot enough when you remember his last minute winner in a derby game and the feeling it evoked.”

Marcotti made the same point: “You can come and tell me that so and so is not a good finisher … but you know, anecdotally, if I see it happen twice in the course of a game that somebody finishes coolly then I will think that he’s a cool finisher. I think that’s part of human nature.”

Humans, then. A terrible set of people to introduce statistics to. Granted, there are different kinds of fans. Some like numbers, and will take the time to understand and appreciate good statistical analysis. Others don’t, and won’t. And it’s tempting to say that’s fine — different strokes and all that — and to move on. But that won’t suffice. Because, first, there’s just something deeply flawed about the idea of dismissing analytics types merely as one niche interest group among many on the grounds their research pisses off some people who don’t understand it. If you’re interested in maximizing your understanding of the sport (and it’s fine not to be), analytics can’t simply be ignored.

More importantly, as analytics begins to influence decision making at the professional level more, it won’t be possible to dismiss it as a fringe discipline, even if you want to. In general, as clubs go, so go the fans, and when the time comes that analytics is fully integrated into the running of these clubs, fans are going to want to know what it is, and to understand the ways it’s influencing their team’s decision making. This sort of wholesale shift may still be a decade or a more a way, but when it comes (and there are few good reasons to think it won’t), demand among fans for analytics content will increase rapidly.

Still, no matter how publicly professional teams embrace analytics, it’s simply a fact that clubs and media outlets have different motivations for using it. Indeed, there’s an argument — I’m not sure how compelling it is, but it’s there — that media outlets have an incentive to mis-analyze data, or at least to interpret them, let’s say, liberally. Media outlets are in the business of telling stories, and the numbers don’t always direct you toward the most compelling story.

But haven’t you heard? Statistics can be used to prove anything. This of course is true only in the same way it’s true a hammer can be used to fix anything, which is to say it’s true only if by “anything” you mean “anything incorrectly,” which is to say it’s false. But it’s an engaging lie. People want things to talk about, and statistics are much easier to talk about when you think about them as if they are number-shaped tropical birds rather than facts.

And so it’s important to acknowledge the very real possibility of a world (possibly the one we’re living in) in which advanced stats are both embedded into the mainstream vernacular and no one knows their expected assist from their elbow. But if “embracing analytics” means nothing more than, say, being willing to use the words “expected goals” in the pub, what’s the point?

Perhaps such a line of thought is too defeatist. It’s unlikely even the most devoted statto has any illusions they can turn every fan into a rational, big-picture thinker (and what a boring world it would be if they did). That doesn’t mean progress can’t be, or hasn’t been, made, that the fan conversation can’t become more informed and intelligent on average.

The point is simply that a world where analytics has been embraced is not necessarily a world where the spirit in which analytics was conceived has pervaded conventional wisdom. That’s a much more difficult project, though one writers like Yorke and Goodman are at least tangentially committed to.

In regards to StatsBomb, Yorke said, “One of the oldest tenets of the site was to shake up the old order, to challenge lazy punditry or analysis and fight back with facts. We still do that plenty enough and I think everyone who contributes does so with a view to bringing detail and intelligence to subjects, but most of all to tell stories and inform.”

Goodman, who can often be found debating analytics on Twitter with his various, um, well wishers, felt similarly, “Fans are allowed to experience the game any way they want. That said, I feel, you know, if I’m offering up an opinion, it’s a reasoned opinion. … And so you know I, I don’t always do this, but I do feel that if somebody is going to respond in a way that indicated they clearly have no idea the depth of sort of work and thought that goes into sort of developing these analytic ideas and concepts, then I think it’s useful to demonstrate how much sort of process there is behind it, how much sort of work and thought and consideration goes into developing these ideas and these thoughts. And how that’s married too to a sort of deep understanding of the game itself and enjoyment of the game itself. That it’s not an either or.”

Since advanced stats are, in general, more complicated than straightforward ones, and therefore easier to misuse, this willingness to explain and educate and challenge lazy punditry is only going to become more important — if, that is, the goal is a statistically literate fanbase. Because what is at stake is not an expanded vocabulary, but a mode of thought. To truly appreciate the good work of the analytics community, and to be able to differentiate the good from the bad, is, at least to some non-negligible degree, to adopt their way of thinking: thoughtful, objective, big-picture.

And so finally, the most compelling argument against embracing analytics isn’t that it’s not useful or interesting or insightful; it’s that it offers precision to an argument that is imprecise by design.

Let’s check in on the Neanderthals: Fuck off. We should have had a penalty, and we hit the post twice. Yeah, off a deflection. And the ref fucked us, too. xG gives us, among other things, a way to objectively evaluate such claims. The question is why we’d want to. These two pub-goers aren’t talking about the match to figure out who, objectively, deserved to win; they’re talking about the match because they’re at the pub, and what else is there to talk about at the pub?

It’s not so different with a show like Match of the Day, on which pundits have the deeply unenviable task of analyzing an entire match not only in 90 seconds but also in such a way that an entire country understands at once. What Alan Shearer does is incredibly hard, and that seems worth acknowledging whether you think he’s good at it or not. A highlights show such as Match of the Day, especially in this digital age, is just about the worst possible place to go for hard-hitting, in-depth analysis. So: don’t. Go instead for the highlights, and the soothing Geordie patter of a man who speaks a language that might, if some analysts get their way, soon be dead.

Besides, there’s this strange thing that happens when you’re watching a match, especially your own team. When you’re 1-0 down in the 90th minute and the ball drops to your best player 12 yards out with only the keeper to beat, and he misses, you’re furious, because he has to score that, absolutely has to, no question, not even if the ball was bouncing at a strange angle and it was on his wrong foot and he’s under pressure from a defender and 50,000 home fans are telling him exactly where he should fuck off to and, as it turns out, only two out of 10 similar shots in the past ended up in the back of the net. None of that really matters, does it, even if it’s true? Because he just simply, absolutely, 100 percent has to score.

The right kind of missed chance can turn the sharpest mind into the giddiest idiot in a fraction of a second, and thank goodness for that. And so perhaps the really interesting question about analytics in the long run isn’t whether it’ll change the way clubs operate and the game is played and managed (it will, given enough time), but whether it’ll change the way we, as fans, think while we’re watching it.

It is customary during discussions of analytics to mention mutual exclusivity, and this seems as good a time as any, so here goes: being interested in stats doesn’t prevent you from being interested in any of the many other interesting things about soccer. Nor, if you’re using stats well, should it.

As Marcotti said, “I think everybody could benefit, at least from understanding it [analytics]. I think it’s also quite grotesque that, at the high end, there are football clubs who employ whole teams of analysts, but they’re kind of there as kind of like window dressing, you know, managers don’t listen to them. And again I don’t want to generalize, but there are a lot of situations like that, and I speak anecdotally.

“I think if you’re intelligent and you’re open minded about it and most of all you understand the process behind those numbers then I think it’s another tool, it’s another vantage point from which to evaluate your team’s performance, certainly, and I think that makes you a better manager. It’s an additional piece to the puzzle, along with what you see on the training pitch, along with cutting video, along with your own experience, you know, along with all the other qualities that make a good manager.”

This, I think, is the right way to think about analytics, whoever you are. It doesn’t prevent you from taking the eye test or from analyzing tactics or from laughing when a player misses an open goal from five yards or from cheering when he scores an impossible one from 40. It’s possible to understand statistics deeply and still to engage in the sort of vague cliché-ry that has probably characterized arguments in the pub for centuries. It’s even possible to acknowledge the value of analytics while at the same time wishing it didn’t exist. This seems, if not always strictly logical, certainly very human.

However, the way we analyze the game when we’re not watching it clearly has an effect on the way we watch it. If you spend all morning reading about Chelsea’s tactics, you’re probably going to pay extra close attention to their team shape when you watch them play in the afternoon. In the same way, if you understand that, in the long run, shot quality is more important than goals, you might come away from a 3-0 win feeling uneasy, knowing your team got lucky, despite failing to really break down the opposition for the fourth match in a row.

I asked Yorke whether his work in analytics has changed the way he consumes the sport. “The post match rituals of checking the shot count or expected goals, looking at how each player performed and comparing it to my perception aren’t very different to reading a match report, they just replace opinion with facts,” he wrote. “And it’s quite nice to detach myself from ‘scoreboard journalism.’ Being able to look beyond the league table or recent results and understand if a team is doing the right things is great fun and probably allows me to be less emotional about what happens. That could just be a sign of age, though.”

For many, of course, being emotional about what happens is the whole deal. Which is not to suggest a more analytical approach to fandom is better or worse than any other, but rather to point out that a tension does exist between Yorke’s style of big-picture thinking and a more immediate, emotional approach to fandom. Because while it’s true that being interested in analytics doesn’t prevent you from being interested in any of the many other interesting things about soccer, fully adopting the analytics mindset is likely to result, at least, in the arrival of a little voice in your head telling you to take a deep breath, look at the big picture, the numbers say …

As Marcotti said in answer to the same question, “I was won over to the fact that performance really, really matters and I think I developed a newfound appreciation for the randomness of football. You know, we all watch a game and we understand it and we can think of big chances missed or scored and I’ve just long been fascinated by how so much, so much rides on often random events, you know, guys shooting from 30 yards out, you know, a couple inches one way, a couple inches the other way … people win World Cups and Champions Leagues based on this kind of stuff.”

For fans, this is probably the most confounding lesson of soccer’s analytics movement, such as it is. The difference between winning and losing a single match is, almost disturbingly often, luck. This has always been true, of course, but it’s becoming much harder to explain away with appeals to desire, fighting spirit, that ever-reliable sign of champions: the ability to find a way to win.

If, say, your team have just been relegated because of one perfect 40-yard volley from the opposition left-back, who’s scored three goals in the past eight seasons, is bad luck more of a comfort than bad execution? After all, there’s nothing you can do about a strike like that, from a player like him, is there? Then again, as they say, the only stat that matters is the one in the top-left corner of your screen. Well, except for all the others.