We need to talk about stats. Or more specifically, how not to use them. The rampant misuse of statistics in football does much to discredit them as a whole, and it’s important to be able to distinguish between the ‘good’ and ‘bad’ ones. I just wanted to jot down some thoughts about the use of stats and the ways in which we can (and should) interpret them better.
WhoScored and Squawka
Let’s start off with two serial criminal offenders, WhoScored and Squawka. These two sites can be hugely useful in terms of looking up players’ and teams’ statistics, so we shouldn’t disregard them entirely, but each have flaws. WhoScored’s oft-retweeted Teams of the Week/Season consistently throw up quite strange suggestions, for example, often favouring midfielders who make a high number of passes, regardless of whether or not these passes have impacted a game, meaning that the player rating that they come up with is not necessarily a good indicator of how well they have played. As the formula they use to weight the stats they use to come up with these teams is at present unknown to the public, take them with a pinch of salt.
Squawka’s Comparison Matrix is so often misused (producing monstrosities like the one below) that I thought it worth making a few basic points on how to get better results from it.
First things first, always adjust your stats to per90. Always. And preferably with a decent sample size. This is a fairly basic concept that people often ignore. Adjusting your stats like this better allows you to compare two players’ outputs because they ignore whether a player has played more minutes than another, which often accounts for higher numbers of raw actions (eg. passes and shots). However, as with almost everything, there should be an appreciation of context. Players who often come on as substitutes are often involved in more goals p90 because goals are more frequent later on in games, thanks to fatigue and teams throwing the kitchen sink at the opposition (which in turn leaves them open at the back), so it’s important to understand that sub effects exist. This is a good piece on that subject.
Making sure that you use non-penalty goals p90 when evaluating goalscorers (if you don’t have any xG data) is another concept that is generally ignored by the mainstream media. However, the winning of penalties is not generally a repeatable skill and therefore the goals coming from them are not necessarily indicative of a wider scoring ability. Furthermore, a striker being given a ~80% chance doesn’t necessarily show any ability to get into good positions, which has been proven to be key to defining a goalscorer’s quality.
It’s also important to understand whether the stats you’re comparing players with are indicators of quality or of style, but more on that later.
Defensive Stats Are Different
You often see defenders’ individual stats bandied about like badges of honour after games, but in truth, they don’t really mean much. Defensive stats such as tackles or interceptions made are often a better indicator of a defender’s style rather than their actually quality. Mark (@ETNAR_uk) has done a tonne of work on evaluating defenders using statistics and he hasn’t really been able to find defensive statistics that correlate well with how good he thinks defenders are at preventing shots.
This leads on to another point about individual statistics, which is that they’re often more a description of style than anything else. Take dribbling for example. If a player X dribbles twice as much as a player Y, does that make him a better player? Not really. Maybe he’s a better dribbler? Maybe the other player has a style of play that revolves more around passing than moving with the ball? Even then, you should still try to break these statistics down to percentage success rates to try to work out which player is a better dribbler. Does player X or player Y bypass more players on average when dribbling? You can almost always go into more detail and context, and it’s important to do so.
For individual defensive stats such as tackles made and interceptions, it’s also helpful to try and adjust them based on the player’s side’s possession, as Ted Knutson (@mixedknuts) explains in this piece:
“If your team has possession of the ball, you can’t rack up defensive rate stats. Teams that have a ton of possession don’t give their opponent the ball very often, and thus can’t accumulate defensive stats. What do you do when you know the basic rate stats are meaningless? You adjust them.”
And if you’re interested, the formula that Ted uses to adjust defensive stats for possession is at the bottom of the previously-linked piece.
Team stats have to correlate to winning in order to be relevant in determining teams’ quality, otherwise they’re measures of style rather than anything else. If you take more shots than the opposition, you’ll generally win. If you run more than the opposition, there’s no guarantee you’ll win. There’s no correlation between distance run over a season and finishing position (contrary to what Sky Sport’s Distance Covered graphics would have you believe) as shown in Soccernomics. The sides that run more tend to press more heavily, but that doesn’t in itself make them a better team, it just makes their style different to opponents who are more willing to play a low block. It’s always important to ensure your statistics are relevant.
Unsustainability and Sample Size
Stats companies such as Opta often tweet little gobbets like the above tweet concerning England’s rising star, Marcus Rashford. It gets them those sweet, sweet numbers, but often these stats are fairly useless for two reasons. Firstly, the 8 goals from the 14 shots on target Rashford had taken by the 21st May represents such a small sample size that it’s going to be pretty much wholly unrepresentative of his career finishing ability. Secondly, this is plainly unsustainable. No professional footballer alive can sustain a conversion rate of anywhere near (or above) 50% as Rashford has done here, so this stat clearly doesn’t represent his true ability. Rather, it’s not repeatable, an important subject I touch upon here. Conversion rate have been proven to fluctuate a lot, and the English media for one seem rather taken with stats that are clearly unsustainable. Sure, it’s a fact and you can’t dispute what the numbers say in this case, but is it helpful in evaluating how good the player is? Not really. For me, there’s a difference between facts and stats. I see facts as little gobbets, like the Rashford tweet above, but they don’t have any predictive or evaluative value, unlike stats, which are much better predictors of the future.
Despite the weirdly prevalent idea that those who are into stats don’t actually watch football, there’s a lot to be said for applying the ‘eye test’ to football matches to validate statistical theories, although one should also be aware of the inherent biases that come with watching sport.
One of the more frustrating misconceptions surrounding stats is the idea among non-statisticians that one stat or number can unequivocally prove everything, something now self-respecting statistician would ever claim. The idea that football can be reduced to one single number is silly in such a fluid and random sport, and it’s vital to always remember that one stat doesn’t necessarily prove something else to be true. Stats don’t lie, they can only be misinterpreted. I would say that applying context to the stats that you find is the most important thing to do when presenting or evaluating them. There’s always an extra level of detail or explanation you can, and should, go into. Nuance is vital in increasing the chance of you producing a valid conclusion.
Thanks for reading.
You can follow me here @OneShortCorner
And you can find the rest of this series below: