The McKentucky-Hut review of fast food zoning evidence

Zoning bans for fast food outlets have been tried before. And failed

So says the IEA

Given the source, I’d guess it’s been funded by KFC, maybe the Colonel himself.

It’s a helpful source of references. I’ve no way of knowing whether it’s systematic as little included on methods. Maybe the killer smoking gun reference has been missed. Maybe not.

Obviously we all have prior beliefs and ideologies, and IEA is no exception. Obviously their corporate sponsors will also have a view and that will influence how what’s there is interpreted.

Bluntly, I can’t be bothered to go through it line my line. It seems of pretty poor quality academically speaking – a bit opaque re methods, selective quoting, flawed lines of argument. Of course maybe it doesn’t present to be an academic work, but it will still be presented as evidence to decision makers – who may or may not be able to pick through the flaws.

I will lay odds the evidence put together will lead to a conclusion of mixed bag (reflecting the different research questions posed by different studies), no smoking gun and a lot of contradictory stuff. Easy enough to spin the conclusions multiple ways depending on ideology and prior belief etc.

A few thoughts.

Extracts – selective interpretation.

Some extracts of the studies reviewed are given. Obviously there’s no way of knowing whether the extract actually represents the conclusion of the original study.

I recall a submission to a Planning Authority opposing a zoning restriction. The McDonald’s review made great stall of citing a review by Prof Rayner, giving the Rayner study considerable weight and inferring from an extract of that study there was evidence zoning was pointless. When you actually take the time to read the whole study that was not the conclusion.

So…… buyer beware – extracts may not represent what the study actually says.

There will never be a smoking gun

We will never have evidence in the strictest sense that plain packaging, sugar taxes or takeaways bans do want we hope that they do.

We will always be reliant on observational evidence, and thus have to be careful how we use it. I’d encourage you to look through this thread on the use of evidence in this space.

Complex systems, ecological evidence rarely is able to be linked to individual level outcomes. Ecology of systems vs isolated interventions.

Cumulative impact

For me the cumulative impact of more places selling us fast food – predominantly high in fat, salt and/or sugar – is the key thing. Zoning will never be retrospectively applied, so it will never do much about the existing density, just help not make a problem worse.

It is important we don’t become our own worst enemy in the pursuit of evidence.

As I said in a prior blog, Lack of evidence is frequently cited as barrier for not doing something. This is fine, but that must be considered in terms of counterfactual – what’s the evidence for the counterfactual or maintaining the status quo. Is doing noting an option. Establishing a level of burden of evidence proof in a complex system is different. Some say we need to move to a decision logic framework, not hypothesis testing – ie not a criminal burden of proof, but balance of probability and what is happening in the background.

It’s certainly a review of evidence. Whether it’s any good seems debatable.

Perhaps the IEA might seek a peer review of the study?

There will always be those who are opposed to a policy proposition, sometimes on the basis of evidence, sometimes on pure ideological or commercial grounds.

Whether the Planning Authority should pay any attention to it is also debatable

The Authority might ask who sponsored the IEA to do this review, and why.

Postscript

A few people have been giving me grief on this. How dare you write off the cuff thoughts / just conjecture / all you grumble about is who funds the IEA….you are not fit to hold office.

My blog has upset some. And I apologise if it has. These blogs are written in my own time, some of them are quick and dirty, some are more tricky. This one was off the cuff and similar to what I’d write as rapid response to something in BMJ

In response to the critics, inc Chris, I thought I’d better think more critically. It hasn’t changed my overall view.

And of note, as I originally said I broadly agree with some of the conclusions – it’s a mixed bag of research with no smoking gun. I draw other conclusions, again based on my ideology and prior beliefs.

Thoughts by way of critique to methods:

  • Reasonably clear research question, could have done with more setting out the context and the methodological issues of research in this space (ecological design trying to draw conclusions about individuals, complex system – one intervention in a set, as two eg)
  • Don’t know what search terms were used
  • Only pubmed? May have missed a bunch of important studies.
  • No clarity re dates papers were searched for
  • No clarity re what was, wasn’t excluded. Or why. As I said, may have been the key dealbreaker study (on either side of the argument) that was missed. We don’t know.
  • Don’t know how the quality of the individual papers were judged.
  • Don’t know how data were extracted from the studies
  • Can’t see much of attempt to deal with the doubtless heterogenous nature of research papers, different research designs, research questions, populations, contexts.
  • No substantial critique of the individual studies.
  • No mention of funding sources. We’ve all read Bad Pharma, we’ve all read about industry influence of research. Why would this be any different.
  • No peer review. Submit it to an academic journal.

Other thoughts

There was a really interesting statement along lines of – ‘with all this evidence in the public domain why is it still widely assumed that the prevalence of fast food outlets has a direct and causal impact on obesity rates – and in childhood obesity in particular?’

Erm……..who is claiming a ‘direct and causal’ relation. Im not, I’m not aware of anyone who is.

The fundamental point is that obesity os a complex systems problem, and the location of fast food outlets can’t be causally disaggregated from all the other factors.

The central claim is that there is no evidence of a link, but I would flip it round and ask for evidence that disproves the highly plausible hypothesis that increased availability of FF is part of the problem…

Advertisements

11 comments

  1. When I read the IEA report I was surprised by the conclusion which was not what I assumed. My position, I guess, is similar to yours.

    Now I lack to time (not the expertise) to put that report under scrutiny. I rely on professionals whose day job it is to apply such scrutiny. So I looked for responses and found this post, written by a Director Of Public Health no less. I was excited to read a response, however I am very sadly
    quite appalled at your riposte.

    There is nothing of substance in your post I could but just conjecture and supposition, that is either irrelevant or of very dubious argumentative merit. The most grievous, apart from your hubris in not reading the report comprehensively, was twofold.

    First, saying ” It seems of pretty poor quality academically speaking – a bit opaque re methods, selective quoting, flawed lines of argument.” with zero arguments presented to substantiate your “seemings”. If it was that poor, it should have been easy to provide some examples, yet you provided none!

    Second, and most seriously, you said ” Obviously there’s no way of knowing whether the extract actually represents the conclusion of the original study.” Huh? In your professional role either you or your colleagues should have such knowledge already, and, if not, someone in Public Health should be able to do so and needs to do so, and, anyway it must be part of your remit, especially if your are making a response, even on a personal blog, that has authority in virtue of your day job. This is exactly what I was looking for, someone with expertise, who is familiar with the literature and can point out cherry picking and misquotations in the report. You have completely let me down.

    Indeed I find your response so poor that it is difficult to conclude this post as additional evidence that the IEA report is more likely correct. Quite the opposite of your intention nor what I expected!

    I think you owe the author, yourself, your audience and the public for whom the operation of your department affects, a proper honest, rational and empirically based response. This does not have to be done by you personally, but in your role, you should know who has or can write a proper, considered response and post the link here in your blog when (if not yet) it is available.

    Like

  2. And there will always be those who are opposed to research on pure ideological grounds.

    Perhaps you need another holiday, it must be tiring with all those research papers piling up that you can’t be bothered to read.

    Like

  3. I endorse martinfreedman’s commemts. Your response to the IEA paper was unprofessional and unworthy of a person in public office. You have a duty to be objective.

    Like

    • I’ve had a bit of this stuff…..
      How dare you write off the cuff thoughts / just conjecture / all you grumble about is who funds the IEA….you are not fit to hold office.
      My blog has upset some. And I apologise if it has. These blogs are written in my own time, some of them are quick and dirty, some are more tricky. This one was off the cuff and similar to what I’d write as rapid response to something in BMJ

      I updated the blog

      Like

      • These posts might be written in your own time but you make clear that you are in the DPH on your blog and you provide no disclaimer that these are your personal opinions and not those of your employer. Given your position, I would find this problematic even if you made it clear this was your personal ( i.e. unpaid time) blog and with such a disclaimer. Either way ,you are a public official with responsibilities and authority and so your personal blog carries a veneer of respectability that some no name non-professional does not have.

        Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s