In a recent blog post, Allyson Pollock, Alison Macfarlane and Ian Greener misrepresented work on NHS competition done with Zack Cooper, Simon Jones and Alistair McGuire, and built straw man arguments designed to undermine our findings. This might pass as political discourse, but is it academic debate?
Perhaps these objections arose because of the publicity that our work has received, rather than the detail of the research. But our goal as academics is to produce the most rigorous research possible, with the further goal of promoting evidence-based policies. When we produce evidence that can have an impact on policy, we present this work and make it available to policy-makers, press and the public. Researchers in publicly funded universities are expected to publicise their work in a timely manner to policy makers. How else should good policy be formed or research funding justified?
Pollock and her co-authors decried ‘The drip feed of pro-competition studies’ we have produced. In fact, there are just two studies. Our first study in the Economic Journal (EJ) looked at the impact of competition - by which we mean a move to less monopolistic local markets - on quality. This work illustrated that competition between NHS providers in a market with fixed prices led to better outcomes. Our findings were consistent with what economic theory would predict and they mirror precisely the academic literature from empirical research in the US. More than that, since our research came out, two subsequent studies by separate research teams (Gaynor et al and Bloom et al) have found nearly identical conclusions about the positive impact that fixed price competition has had in the NHS.
Our second paper looked at the impact of this competition on patients’ length of stay (and was an expansion of an earlier paper). It showed that NHS providers in competitive environments shortened their pre-surgical and overall length of hospital stays (which we regard as evidence of improvements in efficiency). In contrast, the net effect of the introduction of private providers into the market was to increase the average length of stay in NHS hospitals and is potentially suggestive of cream-skimming. This latter finding is not overtly pro-competitive. These studies provide precisely the kind of evidence that policy makers look for, so that they can learn about what has worked and not worked in the past, in order to chart a sensible path forward. This is why they have had a significant impact. Of course, wide-reaching policy should not be set on the basis of one study. However, as a body of evidence grows, the case for policy action becomes more persuasive.
What about the counter-arguments? So far, critics have not articulated a theory as to how fixed price competition could undermine quality in the NHS. They have presented no evidence of their own that competition has harmed patient outcomes.
To be fair, Professor Greener has done work in this area. For example, in 2009, he published an article in a journal called Public Money And Management titled, “Patient Choice in the NHS: What is the Effect of Choice Policies on Patients andRelationships in Health Economies”. This ethnographic study presents insights into the attitudes of hospital managers and staff in one NHS trust. But is it ‘good science’ in contrast to our work which he and Pollock call ‘bad science’ and which they have criticized in the Lancet?
Greener’s piece drew on 60 semi-structured interviews of NHS staff at a single NHS hospital; no patients were interviewed during the course of the research. From these interviews, he concludes that: ‘The case presented suggests that patient choice policies fall short on all of the conditions that are necessary for them to work. Patients in the case study were reluctant to exercise choice decisions’. Here, Greener is happy to use a qualitative style of research (interviews at a single hospital) to draw conclusions of his own against a national policy.
However, when other researchers use qualitative research together with quantitative evidence to show that competition can have positive effects, there is less tolerance. In their co-authored Lancet comment piece attacking our research competition, Pollock et al. dismissed work by Nick Bloom, Carol Propper, John Van Reenen and Stephen Seiler, stating disparagingly that in their study: “An association with management quality is based on interviews with 161 senior staff that did not take account of relevant causal factors’. Bloom et al. involved interviews at 100 hospitals and integrated quantitative work with advanced econometrics. If this is ‘bad science’, what are we to make of the critic’s own work, which adopts a related approach?
My colleague Henry Overman has kicked off a good conversation on this blog about what constitutes sensible blogging - we hope this discussion continues. We hope such a debate, plus our reply here, will provide a teachable moment to pause and reflect on how academics discuss evidence, consider the casual use of phrases like ‘bad science’ and begin a thoughtful discussion of the role of blogs in academic and policy debates in the social sciences.
Elsewhere on the LSE site we give a further point-by-point rebuttal to the criticisms of our work. It is worth noting that Professor Pollock has raised these points before and we responded to her points twice, both in a Lancet letter, and in freely accessible online 8-page document (a detailed response that Pollock et al. do not mention) posted online, also included as a linked appendix to our Lancet reply.
A version of this piece was originally published on the LSE British Politics and Policy blog.
