Posted by Steve Gibbons, SERC and LSE
You know your research has hit a nerve when it gets described as 'seriously flawed'. The last time this happened to me was when the Church of England complained about my finding that the apparent performance gap between faith and secular primary schools is due simply to the fact that they enrol higher-ability children. This time, it's some medical/public health researchers and campaigners complaining about my research on the effects of the 2006 policy to expand choice and improve competition between NHS providers in England (Cooper, Gibbons, Jones and McGuire 2011, an earlier version of which was published by SERC here). A letter appeared in the Lancet last week, and there have been previous rounds of lambasting in the media.
This research (and a related body of evidence from other teams) has been cited a lot by politicians to justify the current round of NHS reforms. This use of the evidence is what has motivated the quite vitriolic attacks to which the research has been subjected. These criticisms generally arise from ideological positions, prior beliefs, and dislike of the findings - not on any alternative evidence that the findings are wrong, nor on a serious evaluation of the methods we used or the evidence we have provided. The criticisms amount to assertions and opinions, based on a misreading or misunderstanding of the research. This is a pretty sad state of affairs, and disappointing for those of us who value scientific evidence and the importance of evidence-based policy making.
A more balanced reading of the research and serious engagement with what we actually did and wrote would, I hope, lead the reader to a more interesting finding. Allowing patients more choice over where they received elective treatment for hip replacements, cataracts and the like, had consequences for quality of care more generally – in our study, evidenced by improvements from survival rates from heart attacks. Our conjecture (drawing on other theoretical and empirical literature in the field) is that these effects occurred through general improvements in hospital management, for which there were sharper incentives in more competitive places.
Of course no empirical study is perfect, or can incontrovertibly establish causality – although we go a lot further than most to try to demonstrate causality. It is also quite right that our evidence should be subject to scrutiny, and we support peer review and open science. However, for those who don't believe our findings, the way forward should be to objectively look to see what is driving those findings, rather than dismissing our results out of hand.
For those interested we have published a detailed response to the criticisms in the Lancet article here.
