Abstract
Introduction to the informed health choices essays
The explanatory essays in The James Lind Library were written to promote wider understanding of why fair tests of treatments are needed, and what they have come to consist of.1,2 The informed health choices essays complement the explanatory essays by focusing on the use of information from fair tests of treatments to inform decisions.
The world is awash in health information, including an abundance of false or inaccurate information – misinformation. Believing and acting on misinformation can result in wasted resources and harm. Not believing and acting on reliable information can also result in waste, harm and unnecessary suffering.
Assessing the reliability or trustworthiness of information about treatments requires understanding and application of key concepts – general principles that are ‘applicable in a great variety of different instances in spite of their difference’, and can ‘serve as points of reference by which to get our bearings when we are plunged into the strange and unknown’. 3
The informed health choices essays present three sets of concepts that can help people to assess claims about the effects of treatments and to make informed health choices:
concepts that can help you recognise when a claim about the effects of treatments has an untrustworthy basis, concepts that can help you recognise when evidence from comparisons (tests) of treatments is trustworthy and when it is not, and concepts that can help you to make well-informed choices about treatments.
The concepts within each set are grouped under high-level concepts (Table 1). These essays provide an overview of the key concepts in each of those groups, and then address why and how it is important to teach these concepts from primary school onwards. They can help anyone, not just researchers, to think critically about whether to believe a treatment claim and what to do.
High-level concepts addressed in the informed health choices essays.
In these essays, we use the term ‘treatment’ to include any intervention (action) intended to improve health, including preventive, therapeutic and rehabilitative interventions, and public health or health system interventions. Although the focus of these essays is on treatment claims, people in other disciplines have found most of the same concepts to be relevant for assessing claims about the effect of actions taken to improve, for example, agriculture, education or the environment. 4
We use the term ‘fair comparison’ rather than ‘fair test’ to avoid confusion with diagnostic tests, and to emphasise that fair tests of treatments always involve a comparison with some other treatment (or withholding a treatment).
The basis for the concepts that are explained in these essays is described elsewhere, 5 as is their development and revisions between 2013 and 2022.6–9
This is the first of four essays in this series explaining the key concepts that can help you avoid being misled by untrustworthy treatment claims.
Claims about the effects of treatments are made all the time – in print, broadcasting and digital media, and in personal communications by friends, family, colleagues, healthcare providers, shamans and others. Often the basis for those claims is unclear. Sometimes they are based on assumptions that treatments are safe or effective.
Trust in claims about treatment effects may be based on seemingly logical assumptions about research or about treatments. Or it may be based on the assumption that the source of a claim alone is a sufficient basis for trust. These assumptions can be misleading and the consequences of being misled by these assumptions can be deadly.
For example, in many countries, parents and health professionals were led to believe that babies should be put to sleep on their tummies so that they would not choke if they puked. The basis for this claim was the seemingly logical assumption that babies might choke if they puked while lying on their backs. Trust in this claim was often based on the source of the claim. Health professionals, including Benjamin Spock, an influential American child health expert, included this claim in his bestselling book Baby and Child Care, and advised his millions of readers to put babies to sleep on their tummies.
However, researchers looking into the possible causes of unexplained deaths during infancy found that babies who had died were more likely to have been put to sleep on their tummies than babies who had survived infancy. Three times as many babies died suddenly and unexpectedly if they had been put to sleep on their tummies. Earlier recognition of the risks of putting babies to sleep on their tummies might have prevented such deaths. It has been estimated that over 10,000 infant deaths in the UK and at least 50,000 in Europe, the USA and Australasia 10 are likely to have been attributable to front sleeping.
This example illustrates that it can be dangerously misleading to simply assume that:
treatments are safe, a plausible explanation of how or why a treatment works is a sufficient basis for a claim about treatment effects, a treatment is helpful or safe based on how widely used it is or has been, and the source of a claim alone is a sufficient basis for believing a claim about treatment effects.
In this and the next three essays in this series we explain concepts such as these and how they can help you to avoid believing and acting on unreliable claims.
Assumptions that treatments are safe or effective can be misleading
In this essay, we explain five assumptions about the safety and effectiveness of treatments that can be misleading. These are assuming that:
treatments are safe, treatments have large, dramatic effects, treatment effects are certain, it is possible to know who will benefit and who will be harmed, or comparisons are not needed.
The basis for these concepts is described elsewhere. 5
Do not assume that treatments are safe
People often exaggerate the benefits of treatments and ignore or downplay potential harms. However, few effective treatments are 100% safe. Similarly, people in need or desperation hope that treatments will work, and they may ignore potential harms – especially when reliable evidence of treatment effects is lacking. As a result, they may waste time, money or both on treatments that have never been shown to be useful and may cause harm. The harm that is caused may be minor, but treatments also sometimes cause serious, irreversible harms, including death.
Even simple advice can sometimes cause serious harm, such as advice to put babies to sleep on their tummies, which resulted in tens of thousands of deaths that could have been prevented. 10
Do not assume that treatments have large, dramatic effects
Large effects (where everyone or nearly everyone who is treated experiences a benefit or a harm) are easy to detect without fair comparisons. However, few treatments have effects that are so large that fair comparisons (designed to minimise the risk of being misled by systematic errors [biases] or the play of chance) are not needed. Treatments that do not have large, dramatic effects may be helpful, but fair comparisons are needed to determine how safe and helpful they are.
Some treatments have obvious effects. For example, if someone is bleeding and losing lots of blood, it is obvious that it is a good idea to stop the bleeding. However, most effective treatments do not have such obvious effects. For example, any effects of exercise or changes in diet on heart disease or cancer may occur only after many years. Some medicines and medical procedures have immediate and obvious effects, such as giving adrenaline to someone with a severe allergic reaction, transfusing blood to someone who has lost a lot of blood or draining pus from a painful abscess. However, like changes in exercise or diet, any effects of most medicines and medical procedures do not have such easily observed or experienced effects by everyone who receives the medicine or procedure. This includes common medications used to prevent heart disease or strokes, such as medicines for high blood pressure or high cholesterol, which help some people but not everyone who takes them. 11 It also includes treatments for cancer and pain, and complementary and alternative medicines, such as herbal remedies, public health measures (such as closing schools to reduce the spread of COVID-19) and changes in the ways healthcare is delivered or financed.
Do not assume that treatment effects are certain
Fair comparisons of treatments can provide a basis for confidence about the probability of beneficial and harmful effects of treatments. However, it is rarely, if ever, possible to be 100% certain about the size of treatment effects, or to predict exactly what will happen if a treatment is used. This is especially true for treatments that are intended to prevent adverse effects happening a long time in the future. Fair comparisons of such treatments are difficult because they entail following people up for a very long time and it is rarely possible to ensure that people adhere to the advice they are given. Consequently, claims about the effects of such treatments are often based on associations and belief in explanations of how the treatments work. Some people argue that there should be different standards for judgements about the trustworthiness of claims when fair comparisons are difficult. However, it can be lethal not to acknowledge and reduce important uncertainties, even when there is limited potential for doing so. It is also important to recognise that certainty about treatment effects can change as new information becomes available. This is especially true for new problems and treatments, such as treatments for COVID-19.
For example, at the start of the COVID-19 pandemic, little was known about the effects of measures to control it. However, in less than a year, over 2000 randomised trials addressing these uncertainties were registered. 12 Dexamethasone – an inexpensive and widely used medicine – was shown to reduce mortality among patients with severe COVID-19 disease. 13 On the other hand, no evidence was found to justify the use of another inexpensive and widely used medicine – hydroxychloroquine – and it was found to have harmful effects. 14 At the same time, there have been very few reports of fair comparisons of measures to reduce the spread of COVID-19 (such as closing schools), and major uncertainties exist about the effects – wanted and unwanted – of these measures. 15
Do not assume that it is possible to know who will benefit and who will be harmed
For some kinds of health problems, fair treatment comparisons can be made by giving different treatments to a patient at different times, and then comparing the outcomes associated with each of the different treatment periods. These are called n-of-1 trials because they compare the effects of alternative treatments in one patient. For example, n-of-1 trials have compared paracetamol to non-steroidal anti-inflammatory drugs (NSAIDs) in patients with osteoarthritis of the hip or knee. 17 The results varied across patients. Most uncertainties about the effects of treatments cannot be compared in this way, however. For example, this person-specific approach cannot usually be used to compare a surgical treatment with a drug treatment.
Most treatment comparisons involve comparing similar groups of patients assigned to one of two or more alternative treatments. Fair comparisons of treatments usually tell us what happened, on average, in groups of similar people. Usually, in a group of people who have used a treatment, some people benefit, some do not and some may even be harmed. For example, the proportion of people who benefit from common pharmacological treatments varies from 1.5% – for aspirin to prevent serious vascular events (myocardial infarction, stroke or vascular death) in people at high risk – to 58% – for proton pump inhibitors for relief of reflux oesophagitis. 11 It is rarely possible to know in advance who will benefit from which treatment among alternatives, who will not benefit or who will be harmed. Paradoxically, there is only one way to know whether ‘personalised medicine’ – customising treatment for individuals – works. This entails testing it in fair comparisons. Unless individual treatment customisation is 100% effective and 100% safe, it is still not possible to know in advance who will benefit from ‘personalised care’ and who will not. Beyond n-of-1 trials, ‘personalised medicine’ is not really personalised; it is simply an effort to identify subgroups of people who are most likely to benefit from specific treatments.
For example, HER2-positive breast cancer exists when breast cancer cells have a protein receptor called human epidermal growth factor receptor 2 (HER2). About 20% of breast cancers are HER2-positive. Trastuzumab (Herceptin) and other monoclonal antibodies that block HER2 receptors to keep cancer cells from growing are used to treat HER2-positive breast cancer. Accordingly, those medicines are given to women with HER2-positive breast cancer and not to other women with breast cancer. However, not all women with HER2-positive breast cancer benefit from the medicine and some will experience serious harmful effects, such as congestive heart failure (CHF). For example, for women with breast cancer detected at an early stage, who have a moderate risk of cancer recurrence or death in the next three years (30%) and a moderate risk of CHF (2%), only about 10% more women who take the medicine will benefit (experience disease-free survival) and about 8% more will be harmed (experience CHF). 18 It is not possible to predict who among those women will benefit and which will be harmed.
Do not assume that comparisons are not needed
Unless a treatment is compared to something else, it is not possible to know what would happen without the treatment. This makes it difficult to attribute outcomes to the treatment. Whenever comparative terms are used to describe a treatment – for example, ‘faster relief’ or ‘better’ – ask ‘compared to what?’ Sometimes people argue that a fair treatment comparison is impossible because the treatment is ‘holistic’, ‘individualised’ or ‘complex’. However, as with any other treatment, claims about the effects of such treatments depend on the results of comparing them with one or more alternatives. How trustworthy those claims are depends on how fair the comparisons are.
For example, a television commentator in the USA reported that ‘Between late-December of 2020 and last month [April 2021], a total of 3,362 people apparently died after getting the Covid vaccine in the United States.’ He exclaimed: ‘That is an average of roughly 30 people every day,’ and he went on to suggest that the vaccine was killing people. 19 There are many problems with that claim, including the lack of a comparison – how many similar people who had not been vaccinated died or would have been expected to die? Given that over 250 million doses of COVID-19 vaccines had been administered at that time, 20 and that old people and others with a high chance of dying were prioritised for vaccination, it would be surprising if some of those people did not die after receiving the vaccine. That does not mean the vaccine caused them to die. The U.S. Centers for Disease Control and Prevention (CDC) reported that there were 17 reported deaths per million vaccinated people (up to 10 May 2021). 20 The proportion of Americans who died from any cause in 2019 was 8697 per million. That corresponds to an average of 7821 people dying every day. Most of them probably drank some water before dying. So, you could say that thousands of Americans apparently died every day after drinking water. That does not mean that drinking water caused them to die.
Implications
Always consider the possibility that a treatment may have harmful effects. Claims of large treatment effects are likely to be wrong. Expect treatments to have moderate, small or trivial effects (wanted or unwanted), rather than dramatic effects. If estimates of treatment effects are not based on systematic reviews of fair comparisons of treatments, be sceptical about claims of small or moderate effects of treatments. It is important to recognise that there is some uncertainty about the effects of all treatments, and that there is likely to be more uncertainty about some types of treatments. Choices are still required but it is preferable to acknowledge, accept and take account of uncertainty than to deny it and make misinformed and potentially dangerous decisions. Fair treatment comparisons provide the best basis for making well-informed decisions about treatments, but there is almost always some uncertainty about who will benefit, who will not and who will be harmed. Always ask which comparisons provide the basis for claims about the effects of treatments. Claims that are not based on fair comparisons are not reliable.