The McKentucky-Hut review of fast food zoning evidence

Zoning bans for fast food outlets have been tried before. And failed

So says the IEA

Given the source, I’d guess it’s been funded by KFC, maybe the Colonel himself.

It’s a helpful source of references. I’ve no way of knowing whether it’s systematic as little included on methods. Maybe the killer smoking gun reference has been missed. Maybe not.

Obviously we all have prior beliefs and ideologies, and IEA is no exception. Obviously their corporate sponsors will also have a view and that will influence how what’s there is interpreted.

Bluntly, I can’t be bothered to go through it line my line. It seems of pretty poor quality academically speaking – a bit opaque re methods, selective quoting, flawed lines of argument. Of course maybe it doesn’t present to be an academic work, but it will still be presented as evidence to decision makers – who may or may not be able to pick through the flaws.

I will lay odds the evidence put together will lead to a conclusion of mixed bag (reflecting the different research questions posed by different studies), no smoking gun and a lot of contradictory stuff. Easy enough to spin the conclusions multiple ways depending on ideology and prior belief etc.

A few thoughts.

Extracts – selective interpretation.

Some extracts of the studies reviewed are given. Obviously there’s no way of knowing whether the extract actually represents the conclusion of the original study.

I recall a submission to a Planning Authority opposing a zoning restriction. The McDonald’s review made great stall of citing a review by Prof Rayner, giving the Rayner study considerable weight and inferring from an extract of that study there was evidence zoning was pointless. When you actually take the time to read the whole study that was not the conclusion.

So…… buyer beware – extracts may not represent what the study actually says.

There will never be a smoking gun

We will never have evidence in the strictest sense that plain packaging, sugar taxes or takeaways bans do want we hope that they do.

We will always be reliant on observational evidence, and thus have to be careful how we use it. I’d encourage you to look through this thread on the use of evidence in this space.

Complex systems, ecological evidence rarely is able to be linked to individual level outcomes. Ecology of systems vs isolated interventions.

Cumulative impact

For me the cumulative impact of more places selling us fast food – predominantly high in fat, salt and/or sugar – is the key thing. Zoning will never be retrospectively applied, so it will never do much about the existing density, just help not make a problem worse.

It is important we don’t become our own worst enemy in the pursuit of evidence.

As I said in a prior blog, Lack of evidence is frequently cited as barrier for not doing something. This is fine, but that must be considered in terms of counterfactual – what’s the evidence for the counterfactual or maintaining the status quo. Is doing noting an option. Establishing a level of burden of evidence proof in a complex system is different. Some say we need to move to a decision logic framework, not hypothesis testing – ie not a criminal burden of proof, but balance of probability and what is happening in the background.

It’s certainly a review of evidence. Whether it’s any good seems debatable.

Perhaps the IEA might seek a peer review of the study?

There will always be those who are opposed to a policy proposition, sometimes on the basis of evidence, sometimes on pure ideological or commercial grounds.

Whether the Planning Authority should pay any attention to it is also debatable

The Authority might ask who sponsored the IEA to do this review, and why.

Postscript

A few people have been giving me grief on this. How dare you write off the cuff thoughts / just conjecture / all you grumble about is who funds the IEA….you are not fit to hold office.

My blog has upset some. And I apologise if it has. These blogs are written in my own time, some of them are quick and dirty, some are more tricky. This one was off the cuff and similar to what I’d write as rapid response to something in BMJ

In response to the critics, inc Chris, I thought I’d better think more critically. It hasn’t changed my overall view.

And of note, as I originally said I broadly agree with some of the conclusions – it’s a mixed bag of research with no smoking gun. I draw other conclusions, again based on my ideology and prior beliefs.

Thoughts by way of critique to methods:

• Reasonably clear research question, could have done with more setting out the context and the methodological issues of research in this space (ecological design trying to draw conclusions about individuals, complex system – one intervention in a set, as two eg)
• Don’t know what search terms were used
• Only pubmed? May have missed a bunch of important studies.
• No clarity re dates papers were searched for
• No clarity re what was, wasn’t excluded. Or why. As I said, may have been the key dealbreaker study (on either side of the argument) that was missed. We don’t know.
• Don’t know how the quality of the individual papers were judged.
• Don’t know how data were extracted from the studies
• Can’t see much of attempt to deal with the doubtless heterogenous nature of research papers, different research designs, research questions, populations, contexts.
• No substantial critique of the individual studies.
• No mention of funding sources. We’ve all read Bad Pharma, we’ve all read about industry influence of research. Why would this be any different.
• No peer review. Submit it to an academic journal.

Other thoughts

There was a really interesting statement along lines of – ‘with all this evidence in the public domain why is it still widely assumed that the prevalence of fast food outlets has a direct and causal impact on obesity rates – and in childhood obesity in particular?’

Erm……..who is claiming a ‘direct and causal’ relation. Im not, I’m not aware of anyone who is.

The fundamental point is that obesity os a complex systems problem, and the location of fast food outlets can’t be causally disaggregated from all the other factors.

The central claim is that there is no evidence of a link, but I would flip it round and ask for evidence that disproves the highly plausible hypothesis that increased availability of FF is part of the problem…