Do health reforms work?

The world of health policy isn’t short of opinions, ideas or hunches. But knowing what succeeds isn’t easy. We might examine our health system and decide that it isn’t working, for example, basing our conclusions on quality outcomes, patient satisfaction surveys or process measures. A snapshot of performance, however, tells us little about what works and what doesn’t. When the gold standard for evaluating an intervention is a randomised controlled trial, the dearth of this methodology in analysing health reforms leaves us with insufficient data to act upon.

The reasons for the relative absence of randomised controlled trials in health policy are several and hard to overcome. Health reforms tend to be complex interventions, making it difficult to isolate the intervention, blind the participants and the investigators to the intervention, and find a suitable control group. Importantly, trials take time and do not fit the life cycle of an elected government. Politicians want to make decisions at a time of maximum benefit, and both the timing of trial results and the findings themselves may not suit a political agenda.

Trials are expensive too. Why waste time and money when, a policy maker might argue, a modelling study will provide good enough information? Then the design, analysis and interpretation of trials are themselves controversial and open to disagreement. Which approach do you prefer? Frequentist, Bayesian, or a mixture of the two? ¹

Yet, these reservations might be more of a hindrance in richer countries with established health systems. Emerging economies around the world are using randomised controlled trials to evaluate health reforms and decide on policy. Mexico is the best example, where the willingness to evaluate is helped by a separation of politics and technical health advice. Brazil, China and India are others attempting rigorous evaluation. Perhaps we have invested so much emotion in health reforms that we are afraid to ask questions? Instead, we drive down a road of incessant innovation, innovation that itself is inadequately evaluated, and, when demonstrably successful, is hard to ‘scale up’.

In the absence of randomised controlled trials, Alice Lale and Jonathan Temple use routinely published data to determine if NHS reforms affect population mortality. Interestingly, or perhaps depressingly, the researchers find that centrally led NHS reorganisation has never had any detectable effect on mortality and should be considered ineffective for this purpose. ² Increased funding, however, may be beneficial.

Rather than focusing on central reforms, the NHS might better serve its population by tackling medicalisation, although the drivers of medicalisation are so strong that it is unlikely to diminish. ³ The Choosing Wisely campaign, recently launched in the UK, is one attempt to respond to the challenge of medicalisation. Ahmed Rashid argues that doctors in training, the newest members of each specialty tribe, can be agents of change, which does sound like a hunch worth evaluating. ⁴