Who protects participants in non-inferiority trials when the outcome is death?

Abstract

A non-inferiority design accepts the possibility of some efficacy loss, as part of a “successful”, statistically significant result. That loss may be excessive when the non-inferiority threshold is lenient. However, even stringent significance thresholds and safety monitoring may fail to adequately protect study participants when the primary outcome is death. The OPTIMAAL trial, a large randomized clinical trial performed in high-risk patients, is discussed as an example, using the Belmont Report principles as an ethical frame of reference. OPTIMAAL compared losartan, a new drug, to captopril, a drug known to reduce the risk of death in patients with heart failure after a myocardial infarction. Serious ethical challenges occurred in that study. Firstly, subjects had to tolerate captopril to participate, meaning that participation implied the possibility of higher risk of death if randomized to the losartan arm, as compared to the standard of care. Additionally, the stopping rules had to ensure enough power to detect non-inferiority, and potentially tolerated additional participant deaths as the study went on. There may have been fifty additional deaths in the losartan arm by the end of the trial—deaths that perhaps could have been prevented if participants took captopril. Those features represent a challenge to the beneficence principle. Finally, it is unclear whether the consenting process adhered to the respect of persons principle, because consent forms were not published, and we cannot determine whether a clear description of the risk of death during participation was provided.

Non-inferiority randomized clinical trials are performed to assess new treatments when an efficacious treatment is available, and the new treatment is expected to preserve the efficacy of that available treatment to a given extent, while improving at least one of the following: safety, tolerability, convenience, acceptability, or cost. A treatment is considered non-inferior if it is “not significantly worse” than the standard of care. For non-inferiority, we are willing to accept at least a modicum of harm (loss in efficacy). But, who determines how much harm, if any, is acceptable? Individual investigators have used a number of approaches, with varying degrees of leniency in regard to harm (Kaul and Diamond, 2006). Lenient significance boundaries can result in a statistical finding of “non-inferiority” without showing substantive clinical non-inferiority (Flacco et al., 2016) and they almost invariably accept greater risk for trial participants, because trial stopping rules are designed to accommodate the non-inferiority hypothesis testing. However, even in the best-case scenario of a conservative (non-lenient) non-inferiority significance threshold, there remain important ethical issues, particularly when the outcome of interest is severe, such as death. This article examines those issues using a specific trial as a case study. Whenever applicable, the ethical discussion considers the Belmont Report principles (Department of Health et al., 2014)

OPTIMAAL (optimal trial in myocardial infarction with the angiotensin II antagonist losartan) was a multi-center randomized clinical trial approved by multiple institutional review boards, and its results were published in a prestigious medical journal (Dickstein et al., 2002). Angiotensin converting enzyme inhibitors (ACE-I) were already known to reduce mortality risk in patients after a myocardial infarction who had either clinical evidence of heart failure or decreased left ventricular ejection fraction. Angiotensin receptor blockers (ARB) were developed afterwards, and were hypothesized as possibly having a greater therapeutic effect. OPTIMAAL assessed the non-inferiority, and possible superiority, of losartan, an ARB, as compared to captopril, and ACE-I, for mortality post myocardial infarction.

The significance threshold for non-inferiority was pre-specified as a relative risk of 1.10. This stringent threshold ensured that a finding of non-inferiority would exclude, as a worst-case scenario, an increase in mortality of 10 percent. That pretty much guaranteed that the point estimate for the relative risk would be very close to, or below, 1—a finding compatible with true non-inferiority, or superiority. That ensured that a statistical finding of non-inferiority would translate into clinical non-inferiority. Of note, many other non-inferiority studies have been performed with far more lenient significance thresholds (Rehal et al., 2016). In other words, OPTIMAAL represented a best case ethical scenario in regard to the non-inferiority significance threshold. Let us examine whether that was sufficient to provide adequate protection to study participants.

Commentary

Beneficence

Beneficence means that “persons are treated in an ethical manner not only by respecting their decisions and protecting them from harm, but also by making efforts to secure their well-being.” At the core of this principle is the Golden Rule of conduct for all physicians: first, do no harm. But it also includes the concept of maximizing possible benefits, while minimizing possible harms. In OPTIMAAL, all participants had a compelling indication for an ACE-inhibitor to reduce their risk of death. ACE-inhibitors are sometimes not tolerated, because of side effects, such as cough, and angioedema, but those adverse effects are rarely fatal.

Trials are overseen by an independent group of experts, usually known as the data and safety monitoring board (DSMB), which monitors outcomes. The DSMB also sets up boundaries to trigger study termination in case of proven benefit, harm, or futility. Importantly, stopping rules that accommodate a non-inferiority design are more likely to have a boundary for harm that is more permissive, as compared to that for a superiority design. The planned sample size in OPTIMAAL consisted of 5,000 participants, who would be followed until 937 deaths occurred, in an event-driven design (Dickstein and Kjekshus, 1999). Two interim analyses were planned by the DSMB, at the time that 422 and 703 deaths were observed, respectively. A two-sided symmetric O’Brien-Fleming stopping boundary was created. The symmetric approach meant that the amount of evidence required to stop the trial because of harm from losartan was the same as that required to stop it because of benefit. The study was not stopped at either interim analysis. Rather, the lower than expected mortality in the study prompted an extended duration of follow-up than expected, and a total of 5,477 participants were randomized. At the end of the study the mortality rates were 18.2 percent in the losartan arm and 16.4 percent in the captopril arm. The estimated relative risk (95% confidence interval) for losartan participants was 1.13 (0.99; 1.28). The number needed to harm (NNH, the number of people you need to switch from captopril to losartan to cause an extra death) was 55 (26; –577); the negative number reflects the possibility of a large number needed to treat (NNT, the number of people you need to switch from captopril to losartan to prevent a death). Based on the point estimate of the NNH, it can be estimated that by randomizing 2,744 participants to losartan, study participation may have resulted in approximately 50 extra deaths in that arm—deaths that could have been prevented if those participants had taken captopril instead.

As previously noted, the OPTIMAAL trial is exemplary in that its threshold for non-inferiority was strict, and thus guaranteed that a finding of non-inferiority would have not accepted any substantial loss in efficacy, as compared to the standard of care. It should be a cause for concern that many other non-inferiority studies apply more lenient significance thresholds than OPTIMAAL, which translates into higher risk during participation (Rehal et al., 2016).

Cui bono?

The expected societal benefit from the use of losartan, if it were proven non-inferior for death, was its greater tolerability, without cough or angioedema. However, the design required that all participants had to be able to tolerate captopril. Therefore, OPTIMAAL participants, and our patients who share their characteristics, did not stand to benefit after a successful trial completion because they already tolerated captopril. On the other hand, the non-inferiority design meant that participants were potentially exposed to a higher risk of death if they were assigned to the losartan arm. Who, then, stood to benefit from the study? Potential beneficiaries from a successful outcome of the OPTIMAAL study included: (a) patients who cannot tolerate an ACE-I because of cough or angioedema; and (b) the pharmaceutical company and its shareholders. Of note, this ethical challenge is universal in non-inferiority trials, because implicit in the randomized design is the fact that participants must be able to tolerate the standard of care.

Respect for persons

Respect for persons entails the protection of their autonomy. When potential volunteers for a trial are offered participation, they should receive adequate information, so they can make an informed decision on whether to participate. Autonomy is not respected when the information provided is insufficient, or difficult to understand. Did participants in the OPTIMAAL trial understand what participation entailed? Did they know what non-inferiority meant? Were they aware of the fact that they might be given a medication that could, potentially, increase their risk of death during the study? Unfortunately, those questions cannot be answered because the consent forms for most clinical trials are not systematically published, preventing an open review of their contents.

In addition, it should be of concern that even volunteers with excellent literacy skills may not comprehend the nuances of non-inferiority trials. Even medical professionals experience substantial difficulty to understand the methodological aspects of non-inferiority trials, and “there is confusion about the purpose of NI trials, and in turn, about their design, application, and interpretation” (Powers and Fleming, 2013). As pointed out by Schumi and Wittes, “as difficult and counterintuitive as classical statistics may be, they are simple compared to the problems of inference in non-inferiority trials” (Schumi and Wittes, 2011). Participants may believe that they are volunteering for a superiority study, or that non-inferiority means therapeutic equality between the new intervention and standard treatments (Powers and Fleming, 2013).

Conclusions

A non-inferiority trial may create serious ethical problems, when the main outcome is death and the potential added treatment benefits are not strikingly important. A careful and transparent examination of those ethical problems appears warranted, on a case-by-case basis. Furthermore, truly informed consent is needed, and unambiguous language should be mandatory in the consent forms. Institutional review boards (IRB) should require that very clear and specific language be used in explaining the risks of participation in a non-inferiority trial. There should be a statement describing, in very simple terms, the specific additional risks a participant may be exposed to. If that risk is death, the consent form should unequivocally state so. There is currently no mandate in the United States from the Office for Human Research Protections, or the Food and Drug Administration, to use specific, unambiguous, language describing risks to participants in non-inferiority trials. This deficit should be addressed. Further, society at large is an interested party in protecting the rights of clinical research volunteers, and thus consent documents should be published.