Better to be in The Placebo Arm for Trials of Neurological Therapies?

Abstract

Patients with progressive neurodegenerative diseases often pursue trial entry seeking to access cutting edge therapies. However, cutting edge therapies for neurodegenerative diseases tend to have higher adverse event rates and underperform placebo. This essay argues that patients seeking trial entry are probably better off, medically, by being assigned to the placebo arm. Because trials involve extra clinic visits and research procedures, patients may be still better off medically by skipping trial participation altogether. I close by arguing that the Neurology research community might better honor the contributions of research subjects by pressing sponsors to promptly publish the results of non-positive trials, minimizing the use of uneven randomization ratios that favor assignment to the investigational treatment, and by fostering systematic collection of data on the risk/benefit balance of trial participation.

Introduction

Patients with advanced or progressive neurological illnesses are often eager to access novel experimental treatments, whether through clinical trial participation or compassionate use. One might think that the rational patient seeking direct medical benefit would maintain a hierarchy of preferences when it comes to accessing experimental therapies, where the top choice is access to a treatment that is in late stages of development, followed by access to treatments in early phases of drug development.

In this essay, I want to advance two somewhat contrarian claims: that the first best option is neither to be in the experimental arm nor in the placebo arm of a trial, but rather to be skipping trial participation altogether; for patients with advanced or progressive neurological illnesses who do, nevertheless, opt for trial participation, the best place to be is in the placebo arm of a trial. Since I am an advocate for more and better research, I will close by explaining how the neurology research community can tilt the balance towards making trial participation a more sensible option for patients.

Better to be in the Placebo Arm?

Drugs that reach late-phase testing are generally supported by a body of early-phase clinical evidence suggesting a drug has activity against a disease. Late-phase studies should meet conditions of clinical equipoise, whereby there is genuine uncertainty in the expert community as to whether the experimental treatment is better or worse than what it is compared with in the trial ¹ . How, then, can it be true that accessing treatment in late-phase studies is worse than allocation to a placebo?

The answer has to do with two issues. The first concerns prior probabilities that a drug will show meaningful activity in a late-phase trial. Neurology has one of the highest attrition rates in drug development, with approximately 9 of 100 drugs entering clinical testing ultimately fulfilling their promise by getting an United States Food and Drug Administration (US FDA) approval ² . However poor these odds, this number hides the fact that there are large areas of neurological drug development, in particular disease-modifying treatments for neurodegenerative disease, where success rates are much lower. Consider that scores of disease-modifying candidates have been tested in Parkinson’s and Alzheimer’s disease, with no success. Where there have been successes, gains have generally been very modest. For Amyotrophic Lateral Sclerosis (ALS), there are two approved drugs, both of which have marginal activity, if any at all ^3,4 . This means that for disease-modifying treatments of neurodegenerative disease, the odds are heavily stacked against investigational treatments showing superiority over placebo.

However, the odds are stacked in favor of encountering toxicities in late-phase studies. Late-phase trials deliver biologically active doses of drugs over prolonged periods (if they involve cell transplantation to the brain, the treatment arm also involves a highly invasive delivery procedure). Together, the cumulative and sometimes more invasive dosing mean that probabilities of harm due to side effects can be substantial in late-phase studies ⁵ . Thus, it should be no surprise that for three high profile anti-Alzheimer’s drug candidates, serious adverse events were generally lower in placebo arms than in arms assigning semegecestat ⁶ , bapineuzumab ⁷ , and solenuzumab ⁸ .

Skeptics of this line of argumentation might appeal to the concept of clinical equipoise and point out that if conditions of clinical equipoise hold, new treatments should outperform comparators in roughly equal proportion as comparators outperforming new treatments. Such an argument has been made in the context of cancer trials ⁹ . If this view were correct, then patients in the placebo should be no better or worse off than patients in the treatment arm.

However, this misunderstands the concept and implications of clinical equipoise. Clinical equipoise does not require that expert communities be evenly divided about the comparative advantage of two arms in a study. Neither does it entail that half of all trials have outcomes favoring experimental drugs. Conditions of clinical equipoise only require that there is a reasonably sized minority of well-informed experts who favor the new treatment. Even better for a stable state of clinical equipoise if that minority believes, on credible evidence, that a therapy could have a very large benefit. The fact that comparators almost always outperform new treatments in randomized trials of disease-modifying neurological treatments is not inconsistent with a state of clinical equipoise.

Are patients who receive active experimental treatment in late-phase studies at least better off than their peers in early-phase studies? I am not aware of any data quantifying risk and benefit for patients in trials testing treatments for neurodegenerative disease. But it seems reasonable to posit that overall risk/benefit for participating in late-phase studies is roughly comparable with that for early-phase studies. Compared with early-phase studies, late-phase trials involve prolonged and cumulative dose exposure to probably ineffective treatments. For example, the phase III trial of solanezumab involved 18 months of drug exposure ⁸ . The two earlier phase studies preceding it involved a single dose ¹⁰ , and a 12-week exposure ¹¹ . Late-phase trials also often enroll patients with less advanced disease. But these disadvantages for late-phase studies are offset by the possibility of receiving still higher doses of drug in early phases (for solanezumab, two of the four cohorts of patients enrolled in the phase II study received doses that were higher than those used in phase III). They are also offset by the less intensive use of research procedures in late-phase studies (whereas in the phase III study of solanezumab, about 2% of patients submitted to lumbar punctures ⁸ , in the phase II study these lumbar punctures were pursued in all patients ¹¹ ). And of course, drugs in late-phase trials have greater relative (but perhaps only marginally greater absolute) probability of being effective, since some ineffective ones have been screened out in earlier phases, and likewise lower risk of toxicity given that very toxic drugs can be ruled out in earlier stages. This means that, all things considered, it is difficult to say for sure that patients receiving drug in late-phase trials are better off than patients receiving drug in early-phase trials. If all of the above is correct, then the best place to be if you do enroll in research is the placebo arm.

Better to Skip Trials Altogether?

If placebo is the best place to be, is it possible that skipping trials altogether is even better? There is a plausible case to be made that the answer is ‘yes.’ Patients assigned to the placebo arm are still inconvenienced by study procedures like blood draws and lumbar punctures. Add to this the hardship and labor of transporting patients with declining capacity and/or mobility to clinics for trial participation and follow up. True, many patients may derive emotional comfort from the greater caregiver contact afforded by trial participation. But pursuit of caregiver contact outside a trial, or comparable time spent in standard-of-care palliation and psychological support, would arguably be more effective.

A strong case can be made that patients derive meaning by contributing to science by volunteering for trials. Patients often describe altruism as a major factor motivating research participation ¹² . Unfortunately in neurology, however, the degree to which patient sacrifices in trials are transformed into scientific insights is unclear. This is because companies make a habit of not sharing and disseminating trial reports. Hakala et al. documented that most trials testing neurological treatments are not published within 5 years of study completion if the treatments never receive US FDA approval ¹³ . There is some evidence suggesting that nonpublication is more prevalent in neurology than cancer or cardiovascular disease ¹⁴ . Nonpublication seems to be about as common for late-phase studies as it is for early-phase studies. If you are a patient and you participate in a clinical trial of a drug that does not go on to receive US FDA approval, the probability that results from your trial will be shared with other scientists who might build on insights is two in five ¹³ .

Limits and Implications

The preceding arguments have limits, of course. There are areas of neurology, like relapse–remitting multiple sclerosis, where success rates are higher and major advances seem to be accelerating. Here, the case for being in a placebo arm (or skipping a trial altogether) may not be so strong. In stroke too, there have been major advances in surgical treatment approaches ^15,16 . Base rates of success could also be higher where surgical approaches are tested (though I am not aware of any data that would support this). Perhaps one day, other neurodegenerative disease areas will turn a corner and benefits for trial participation will increase. It should also be acknowledged that there are some patients who, after proper counselling and being informed of risks and benefits, place so much value on survival that the slimmest odds or magnitude of direct medical benefit might favor accessing an experimental drug. However, some evidence suggests that advanced disease patients who opt for palliation have lower decisional regret than those who choose curative treatment ¹⁷ . It is important to keep in mind that even if the odds of benefit are very low, research is always digging into the unknown, and if we do not try, we will never know and never make progress. This analysis is not meant to discourage trial participation but rather to readjust potential misconceptions about risk/benefit among various options.

One can also point out that there are areas of neurological therapeutic development, like cell and gene therapy, where ‘placebo’ arms (properly called ‘sham’ or ‘invasive placebo’”) are a lot less benign than sugar pills. Yet even here, there are grounds for maintaining the claim that sham arms have a better risk–benefit balance. Whereas patients assigned to the active arm potentially endure nontrivial risk of needle trajectories to deep brain structures ¹⁸ , patients in sham arms generally receive only partial burr holes to the cranium (there have been exceptions where sham groups have received vehicle injections in the brain ^19,20 ).

The above analysis has several implications. A first is that neurology research communities should consider how they might get more mileage from unsuccessful translation trajectories, thus swinging the balance of risk and benefit towards more favorability for altruistic patients. Negative trials, if designed and reported well, provide valuable feedback on the validity of pathophysiological theories driving drug development programs ²¹ . They also offer insights into the relationship between clinical and surrogate endpoints, or the predictive value of preclinical models. For this reason, prompt and complete reporting of negative trials is critical, alongside reporting of underlying preclinical evidence (studies suggest that much preclinical data go unpublished in neurology ^22,23 ). Also important is unbiased uptake and integration of these negative findings into the design and interpretation of other trials. Given that there are sound reasons for believing patients are not made better off medically by trial participation, neurologists should refuse to recruit patients into trials unless a study protocol specifies a commitment to publish results in full within a year of completing collection of primary outcome data (alongside preclinical evidence). Indeed, academic medical centers should refuse to host such studies without a contractual agreement to publish results. Researchers can also contribute their voices to supervised initiatives ²⁴ that would make individual patient data from trials accessible for re-analysis.

A second implication is that neurology drug developers should avoid study designs that maximize patient exposure to an unproven drug, or that minimize the number of patients receiving placebo in studies that are otherwise in clinical equipoise. Methods like 2:1 randomization to treatment versus placebo ^25,26 are often presented as if they were enhancing therapeutic opportunities for patients. If the analysis above is correct, then these studies are doubly problematic, first because they are less efficient ²⁷ , and second because they maximize assignment to treatment arms that are probably more harmful, an aspect that has been recently considered in oncology where the likelihood of positive outcome in randomized trials is of the same low magnitude as in neurology ²⁸ .

A third implication concerns ‘compassionate use’ and ‘right to try’ initiatives. If the foregoing analysis is correct, patients receiving investigational agents for neurodegenerative diseases are made worse off by accessing anything other than a placebo. It is hard to argue that it is ‘compassionate’ or fulfilling a ‘right’ for physicians to deliver treatments that, at least at a population level, are more likely to do more harm than good. This ‘right’ becomes far weaker when one considers the deleterious consequences such activities can have on trial enrolment or other drug development activities. In any event, the path to major advances in neurology is through well-designed and reported clinical trials, not through permissive scientific, regulatory, and care standards.

Last, we need better data on risk and benefit in neurological drug development research. Whereas in cancer, a lot is known about risk and benefit across the spectrum of clinical development (see, for example ^{29 –32} ), much less is known in neurology. As a result, it is difficult to counsel patients and their families about the risks and benefits of trial enrolment. It is also very difficult to track trends. Is trial enrolment getting more burdensome for patients because of added research procedures, or less because we are zeroing in on useful treatments? Are certain types of trials better for patients than others?

For now, the best approach to counselling patients and families is for neurologists to explain that the prospects of meaningful direct medical benefit are low and that trial participation is on balance more likely to cause harms, burdens and inconveniences than to directly benefit. As odd as it might sound, this sentiment can be succinctly conveyed to patients by telling them ‘if you join this trial you’re better off in the placebo arm!’