How to Lie with Statistics

Did you know ice cream causes drowning?

Well, not really – but if you trust statistics blindly then it might appear that way. On hot days, ice cream sales soar, and so do swimming-related accidents. It’s the kind of statistical mischief Darrell Huff warned about more than half a century ago in his best-selling classic How to Lie with Statistics.

First published in 1954, Huff’s guide teaches us how to spot some of the numerical nonsense that still fills headlines, social media feeds and even courtrooms.

As he put it back then: 

The secret language of statistics, so appealing in a fact-minded culture, is employed to sensationalise, inflate, confuse and oversimplify.

In today’s data-driven world, politicians, advertisers and influencers can bamboozle us with numbers that often aren’t quite what they seem.

How do they do it? Here are some of the main tricks Huff warns us to watch out for.

The self-selecting sham

Back in Huff’s day, newspapers often conducted opinion surveys by post – relying entirely on people who were a) readers and b) motivated enough to cut out a form and mail their responses back. It’s a bit like polling fans at a football match to establish the nation’s favourite sport. Don’t be surprised if the answer that comes back is “football”.

Today, online polls echo the same flaw. Clickbait news stories scream about results that reflect the views of a passionate, self-selecting few – and dress them up as the opinion of the majority. 

Small numbers, big problem

Big claims backed up by small studies are another textbook example of how statistics can mislead. A few years ago, many UK national newspapers sounded the warning that “driving dehydrated is as dangerous as drunk driving.” Alarming stuff. Until you discover that this was drawn from a study that analysed the performance of just 11 men. 

It’s a fairly typical example of how many studies get reported (often omitting to mention the tiny sample sizes used to draw those headline-grabbing conclusions). Harmless enough, you might think. After all, it’s no bad thing if a few more people are prompted to pack a bottle of water for the road.

But the consequences can be much more serious. In 1998 Andrew Wakefield’s infamous flawed study linked the MRR (measles, mumps and rubella) vaccine to autism. Among the many issues with Wakefield’s research was the fact it was based on just 12 children. Wakefield selectively used data and omitted crucial information. The panic that ensued saw vaccination rates fall and preventable diseases rise. Wakefield was subsequently found guilty of dishonesty and banned from practising medicine in the UK.

Lies, damned lies and averages

Huff warned us to be particularly wary of averages. The mean (sum divided by count), median (the middle value), and mode (the most common number) are frequently swapped around or cherry-picked to spin a narrative. When you see an “average” used in a news story or ad, you can bet it’s the version that tells the most convenient tale.

Huff reveals:

My trick was to use a different kind of average each time, the word ‘average’ having a very loose meaning. It is a trick commonly used, sometimes in innocence but often in guilt, by fellows wishing to influence public opinion or sell advertising space. When you are told that something is the average you still don't know very much about it unless you can find out which of the common kinds of average it is – mean, median, or mode.

Consider US warehouse retailer Costco’s recent claim that it paid its workers an average hourly wage of $31. Sounds impressive. But it masked the fact that many of the firm’s workers were earning less than $20 per hour, with the average inflated by a small number of mega-salaries – like the $12.2 million reportedly paid to their CEO.

Half-truths and hidden context

Sometimes statistics aren’t technically wrong, just conveniently incomplete. In 2023, Florida Governor Ron DeSantis boasted that crime in the state had reached a record low. A phenomenal statistic that he hoped would boost his run for the presidency. 

But the claim quickly unravelled. Yes, the number of crimes recorded at state level was at a 50-year low – but only because nearly half of Florida’s law enforcement agencies had recently stopped sharing their data.

Graphs that tell tall tales

Graphs clarify data – or do they? Huff warned:

[Graphs] are not always what they seem. There may be more in them than meets the eye, and there may be a good deal less.

A favourite trick Huff identified was the truncating axis – not starting from zero – to dramatise minor changes. If the scale on a graph begins at 98%, then a potentially harmless 1% drop can suddenly look catastrophic. 

It’s the kind of thing you’ll often spot in campaign leaflets around election time: a bar chart where one bar towers over another despite only a small percentage difference. That’s not a proper graph – it’s the Huff playbook in action.

The trouble with correlation and causation

“Correlation is not causation” is one of the golden rules of statistical analysis. Huff showed how easy it is to mislead by conflating the two. He calls this the “post hoc fallacy” – just because one event follows another, it doesn’t mean the former caused the latter.

Nutritional studies are notorious for this (something I explored in more detail here). It partly explains why one day we’re told coffee is good for our brains and the next that it might lead to cognitive decline. Sometimes the results can border on the ridiculous: The Guardian once actually ran a story with the headline “Diet of fish can prevent teen violence.” Often, these claims turn out to be little more than curious coincidences.

As the Spurious Correlations website wryly demonstrates, you can find associations between almost anything if you look hard enough. Does this mean that a decline in the number of pirates is causing global warming? Margarine consumption is linked to divorce? Or Google searches for Nicolas Cage are driving up the price of gold?

Finding the truth with statistics

Next time you hear someone claiming a causation that sounds a little fishy, remember that, well, it probably is.

Huff reminds us that the numbers might not lie, but people sometimes do. That’s why the insights in How to Lie with Statistics still matter today. As Huff puts it:

The crooks already know these tricks; honest men must learn them in self-defence.

Good statistics help us to make sense of the world. By better understanding how they can be manipulated, we’re better equipped to cut through the tricks – and get closer to the truth.