Last Friday, the Today Programme on BBC Radio 4 did a “Fact Check” on the US Ambassador’s recent claim that US had some of the lowest food poisoning rates in the world.
Presenter Justin Webb interviewed BBC Reality Check correspondent, Chris Morris, who reported that they had investigated the issue and that the statistics were clear: rates of Campylobacter illnesses in the US were 4 times higher than those in the UK, whilst Salmonella rates were 20 times higher. The Ambassador was, we were told “flat wrong” and no room for doubt was left. Apart from the embarrassment to Woody Johnson, the piece was clearly designed to cast doubt on the desirability of the post-Brexit US-UK trade deal.
The trouble is that virtually every element of Chris Morris’s ‘Reality Check’ was either flat out false or based on a seriously incompetent use of statistics.
Let’s look at his claims in a bit more detail.
US-UK foodborne illnesses
Chris Morris stated:
“The Center for Disease Control in the US estimated there are about 1.2m illness from Salmonella in the country every year and 1.3m from Campylobacter. Now that works out about 0.4% of the population for each of them. And if you compare that with the UK figures for 2017, it means you are about 4 times as likely to get an illness caused by Campylobacter in the US than the UK and roughly 20 times more likely to suffer from Salmonella poisoning.”
Unfortunately Morris is making the statistical rookie error of comparing two statistics measuring completely different things. For the US, he reports estimates of total illnesses whilst for the UK he uses recorded lab reports. The actual number of illnesses in any country are unknown as many will not be diagnosed or reported. We do know for sure that the number will be far higher than lab reports of known, reported cases.
And in fact, the lab report data are available for both countries and could have made a valid comparison. The US reported 46,623 salmonella lab cases in 2016, a rate of 14.5 per 100,000 people and a similar rate for Campylobacter. The latest UK figures (reported on the Reality Check article) are 10,089 for Salmonella (around 17 per 100,000 people) and 63,946 for Campylobacter (over 100 per 100,000 people). It might justifiably be queried whether lab reports are collected on the same basis in the US and UK but on the basis of what we have, rates are actually higher in the UK than the US.
Trying to compare total illnesses in both countries might have been valid but you need to take care. There exist a number of different methodologies for estimating illnesses from known data and these can give rise to huge differences in estimates. Look, for example, at the UK evidence:
Haagma et al (2012) estimate 2.5 million Campylobacter cases and 563,000 Salmonella cases per year in the UK. In contrast, O’Brien et al (2009) estimate 286,000 Campylobacter cases but only 33,640 Salmonella cases. Compared to the CDC estimates for the US, the evidence is mixed regarding Salmonella but both sources indicate that Campylobacter rates are higher in the UK. However, given that all methodologies from all three sources differ, it is probably not appropriate to rely on them for comparisons.
There is, however, another study which can help: a WHO investigation of the foodborne disease burden across the world which compares rates in different regions but using a common methodology. This is the source given by US Ambassador in support of his claim. Chris Morris dismissed it on the grounds that it gave statistics only for total foodborne diseases and not the specific pathogens (Salmonella and Campylobacter) relevant to chlorine-washed chicken. Well, arguably we should indeed be interested in total illnesses but, as it happens, Morris is wrong again. Table 4 of the WHO study gives rates for every single foodborne agent including Campylobacter and Salmonella.
There is one drawback with the WHO study in that it gives estimates by region rather than country. So the AMR 1 region covers the USA, Canada and Cuba. This, though, is a minor problem as the relative size of the US means the AMR 1 region rate will almost certainly correspond very closely to that for the USA.
The comparisons with Western Europe region (EUR A) are interesting. North America (and by extension the US) is estimated to have a lower burden of Total Foodborne Disease as well as lower burdens for All Diarrheal Diseases for Campylobacter and for the main Salmonella agent (S. enterica).
In sum there is absolutely no justification from any source for Chris Morris’s claim that Campylobacter and Salmonella rates are 4 and 20 times respectively higher in the US. There is no perfect cross-country comparison, but the available data are consistent with the US Ambassador’s claim that US has lower rates of foodborne illnesses than in Europe.
Deaths from Salmonella
Chris Morris further claims that food-related death rates are much higher in the US than the UK. He says:
“Look at the number of deaths, for example. The CDC reckon there are about 450 deaths from Salmonella every year whereas in the UK fatalities are extremely rare.”
Wrong again, I am afraid. As before, Morris compares two completely different measures: estimates of total deaths in the US with known, officially recorded deaths in the UK. As with illnesses, estimating actual deaths from foodborne disease is highly complex and depends on numerous assumptions.
The most comparable UK study by Adak et al (2002) estimates 119 UK Salmonella deaths in a year, a higher rate per population than the estimate given by the CDC for the US. Morris doesn’t mention Campylobacter deaths but Adak report 67 deaths in the UK, a much higher rate per population than the 119 reported for the US.
Again, the estimates of deaths from both pathogens are quite consistent with the US Ambassador’s claim and the exact opposite of what is reported by Chris Morris. In fact both the UK and US estimates are somewhat out of date and I am not aware of more recent evidence. It might have been reasonable simply to conclude that there is insufficient recent evidence to come to firm conclusions regarding death rates. What is clear, though, is that Chris Morris’s conclusion that death rates in the US are much higher than in the UK is unjustifiable.
Taken together, the Today programme piece was wrong from start to finish. The reporting failed even the most minimum requirements of journalistic checks and scrutiny. Worst of all it mislead listeners on an issue of political significance and sensitivity.
I have no interest in promoting chlorine-washed chicken and whether we wish to accept such products from the US will no doubt be a matter of lively debate post-Brexit. We do, however, need to be clear that in contrast to the misinformation peddled by the Today programme, there is little evidence that US food-safety outcomes are worse than in Europe.
I have raised these issues with Chris Morris and the Today programme and also with one of the authors of the BBC Reality Check article on which Chris Morris seems to have based most of his reporting. The author has acknowledged in an email to me that the article contained errors and has agreed to do some re-writes. That is welcome but does not address the fact that viewers of the Today programme were given a completely misleading impression of the facts as well as that the US Ambassador’s reputation was unjustifiably damaged.
To date, I have not had a response from Chris Morris or Today and so I have now submitted an official complaint. Let’s see how the BBC respond.
Professor David Paton holds the Chair of Industrial Economics at Nottingham University Business School and is member of Economists for Free Trade
