A Novel N of 1 Trial Design and Proposed Utility in Complementary and Alternative Medicine Research

Abstract

An N of 1 trial is a multiple crossover study in a single participant. N of 1trials can combine the benefits of individualized patient practice and evidence-based medicine and are amenable to complementary and alternative medicine practice and research. This article will review the basic structure of N of 1trials, discuss how they are commonly used, and review their limitations and statistical considerations. The authors also propose a novel use of the N of 1 trial in the form of mixed-methodology add-on N of 1 trials targeted to a parent trial’s responders. This design can help uncover evidence of subgroup effects in small trials, address issues surrounding the small study effect, and explore the role of interparticipant variability and random chance in the parent trial.

An N of 1 trial is a single patient crossover trial where an active intervention is controlled for via an appropriate placebo (or active control). Generally, 3 pairs of active (A) and placebo (P) interventions are applied in random order (eg, PA/AP/PA). Washout periods are employed, both the patient and doctor are blinded, and often the services of a pharmacy are used to hold the randomization codes and prepare placebos. ¹ In such a trial, the patient serves as his/her own control, and statistical significance is reached by the selection of the appropriate amount of intervention pairs and magnitude of effect.^2,3 Originally used in psychological research, N of 1 trials were enthusiastically embraced in the biomedical field beginning in the late 1980s for use in clinical decision making. ⁴ Often credited for this enthusiasm was the trial design’s proposed ability to confirm an intervention’s effect for a specific patient, as opposed to relying on average effects from a sample population. ⁴ A decade and a half later Johnston and Mills suggested that N of 1 trials could be useful for complementary and alternative medicine as they could “allow for an individualized approach and incorporat[e] patient values.” ³ However, despite early enthusiasm, N of 1 trial activity has slowed. Kravitz et al have suggested that this might be due to physician perceptions that within a general clinical framework, the imposed inconvenience of an N of 1 trial is not worth the reduced clinical uncertainty. ⁴ Nevertheless, as will be discussed below, other scenarios have been proposed where such reduced uncertainty is more clinically valuable. In the discussion that follows, we will describe the ways N of 1 trials have been used in the past as well as discuss some limitations and statistical considerations for this type of trial. We then present a novel use for the N of 1 trial, which we suggest can be uniquely appropriate for complementary and alternative medicine research.

Past Uses, Limitations, and Statistical Considerations

N of 1 trial design has been used in many forms over the past 20 years. This design is most commonly used as an evidence-based clinical tool to determine the ideal treatment for an individual patient. ⁵ Hundreds of such N of 1 trials have been run since the 1990s, ⁶ and their usefulness in clinical decision making has been documented. ¹ However, since N of 1 trials can be time intensive, physicians cannot always find the additional clinical precision worth the effort. ⁴ Despite this, N of 1 trials can be uniquely appropriate in specific clinical and research scenarios. For example, Scuffham et al have recently published their experience using N of 1 trials to determine optimal treatment for patients in need of high-cost pharmaceuticals. ⁷ The N of 1 trials they conducted showed that in certain participants the drugs in question were not effective. When taking into account the cost savings from not choosing long-term treatment with those individuals, this approach exhibited net economic savings. ⁷ Kravitz et al have also attempted using N of 1 trials as a cost saving tool for expensive treatments and have suggested its use for managed care organizations. ⁸ From a research perspective, Avins et al suggest embedding N of 1 trials within population-level randomized controlled trials to increase adherence, ⁹ and Sung et al promote the N of 1 trial as an appropriate tool for researching rare disorders. ¹⁰

N of 1 trial services have even been created over the past 2 decades in multiple countries including the United States, Canada, and Australia.^2,5,11 Such services have provided logistics, pharmacy preparation, blinding, trial protocols, and analysis to assist researchers and clinicians in conducting N of 1 trials. Some of these services have planned scores of identical N of 1 trials prospectively with randomized participation. This approach theoretically allows for those N of 1trials to be combined in some way to estimate a more generalized population level effect.^{4,6,10,12–14} There have been attempts at such an analysis using a hierarchical Bayesian random effects model among other approaches.^12,14

N of 1 trials have unique limitations. Since an N of 1 trial is in essence a multiple crossover trial in a single patient, it is imperative that the treatment’s effect be washed out in a reasonable amount of time and that this washout period be chosen judiciously so that the effect of one period does not carry over into the measurements of the following period. If 2 active periods followed each other and the effect was additive, such carry over could artificially increase the effect estimate of the second period. Alternatively, if a placebo period followed an active period and carry over occurred, this might decrease the effect estimate by minimizing the difference between the 2 periods. While in theory one could model such carryover effects or minimize them by only taking outcome measurements toward the end of each intervention period, the concerns involving adequate washouts remain paramount when designing N of 1 trials. N of 1 trials can also be very lengthy. The “industrial standard” of 3 pairs of active and control periods necessitates that a single N of 1 trial can include 6 periods in addition to washout periods. Depending on the magnitude of effect, some N of 1 trials could require even more sets to reach statistical significance. ⁵ Because of these concerns it has been suggested that N of 1 trials are best suited to investigate interventions for symptomatic treatment with short washout periods whose action is not curative. ¹⁵ Also, while N of 1 trials can show efficacy of intervention with statistical significance in an individual, a single N of 1 trial’s results cannot be generalized to a larger population.

In a recently published focus group study of patient and physician views on N of 1 trials, some participants expressed concern about whether a study of one person could achieve statistically significant results. To address that concern, the study’s authors call for clearer explanations of the statistical foundation and analysis of N of 1 trials. ⁶ The following is such an attempt.

The common statistical tool used in most N of 1 trials is the parametric 1-tailed Student’s t test. Because of the multiple intervention periods, there are always time length concerns with N of 1trials. By using Student’s t test, statistical power is maximized in a shorter amount of time because the test takes into account not only the direction but also magnitude of effect.^3,6 For example, if one ignored the magnitude of effect and used the sign test, a total of 5 active/placebo intervention sets would be necessary to reach statistical significance of P ≤ .05 (½ × ½ × ½ × ½ × ½ = ½ ⁵ = .03125). If, however, we assume a normal distribution and incorporate magnitude of effect by using Student’s t test, statistical significance can often be achieved with 3 sets of active/placebo intervention pairs, which is generally considered to be the “industrial standard” N of 1 approach ⁵ (personal communication, Guyatt, G. August 2009). However, others have suggested alternative ways of determining how many intervention periods are necessary to reach significance,^4,8 including taking into account the physician’s a priori estimation that the treatment will be effective, as well as a comprehensive literature search. ² Additionally, in a clinical situation it is not always imperative to reach statistical significance in an N of 1 trial, as the patient/physician team can have other variables to consider, such as side effects. ⁵ Finally, using Student’s t test makes assumptions about the data that might or might not be valid. Of primary concern is that each intervention measurement is independent of another that might not always be true. A patient’s response can be higher on a sampling day later in an intervention period than on a sampling day earlier in that period because the treatment effect could be cumulative. While such autocorrelation can occur, there are statistical packages currently available that can take such patterns of correlation into account (eg, Simulation Modeling Analysis [computer program]. Version 8.3.3. Charleston, SC, USA, © 2006 Jeffery J. Borckardt).

A Novel N of 1 Trial Design

We suggest a novel N of 1 trial design, which is currently being piloted. We used primary outcome measures from an randomized controlled trial to select apparent intervention responders from the active arm of the trial. From these apparent responders we recruited potential N of 1 trial participants for a follow-up mixed-methodology add-on N of 1 trial. Specifically, we used the immunological endpoint measures from a parent placebo controlled clinical trial designed to study the immunomodulatory effects of the mushroom Trametes versicolor. Because it is a mixed-methodology design, outcomes collected in this study represent a spectrum of participant experience, both quantitative and qualitative, including laboratory-based immunological assays, modified short-form health surveys (SF-36), as well as open-ended qualitative questionnaires that include inquiry about the emotional and spiritual elements of the therapy. A mixed-methodology add-on N of 1 trial such as this can be uniquely appropriate for complementary and alternative medicine research and address many of the concerns with both conventional randomized controlled trial and N of 1 trials in the following ways, all further delineated below. First, the addition of mixed-methodology and N of 1 elements help address the needs for holistic and individualized research, respectively. Second, in the complementary and alternative medicine field many interventions can only have a few smaller research trials associated with them. Small studies can make identifying subgroup effects difficult, complicate evidence-based decision making due to what has been called “the small study effect,” and make results more vulnerable to interparticipant variability and random effects.^16,17 A mixed-methodology add-on N of 1 trial design can help address many of these concerns as explained further below. Third, it has been suggested that the major impedance to widespread adoption of N of 1s may be due to the clinical inconvenience of the additional trial periods and subsequent length of N of 1 trials. ⁴ By using the N of 1 trial in a research setting, mixed-methodology add-on N of 1 trial design can avoid this clinical inconvenience factor. To our knowledge, such a combination of a conventional randomized controlled trial and subsequent N of 1 trials targeted to responders has not been performed before. A more in-depth description of these 3 benefits follows.

Holistic and Individualized Research

Although the evidence-based medicine movement has championed the randomized controlled trial as the gold standard in biomedical research and the Evidence-Based Medicine Working Group has placed the N of 1 study in the paramount position on the evidence hierarchy for individualized research, ¹⁵ others have questioned the hierarchical structure of evidence accumulation itself.^18,19 Conventional randomized controlled trials, which occupy a high position on the standard evidence-based medicine hierarchy of evidence, often ignore the “meaning” of an intervention to a patient. ²⁰ There are growing voices promoting the additional value of qualitative research in biomedical studies, which can help “give due emphasis to the meanings, experiences, and views of all the participants.” ²⁰ Additionally, qualitative research can help generate new endpoints or outcomes for future study, which were not originally considered by researchers. ²¹ In response to these concerns, alternative evidence models such as the evidence circle and evidence house have been proposed.^18,19 In such models of evidence utilization, qualitative evidence occupies positions different than, but not subordinate to, quantitative randomized controlled trials. While perhaps not always feasible within larger randomized controlled trials for logistical reasons, incorporation of qualitative research may be more possible within the context of the smaller N of 1 trial and is often included in such studies. ³ For complementary and alternative medicine researchers who employ qualitative information gathering and consider it integral to a holistic research model, N of 1 trials can serve as an appropriate venue. By augmenting a conventional randomized controlled trial with mixed-methodology add-on N of 1 trials, trialists can address the desires of complementary and alternative medicine practitioners for both individualized and holistic research.

Smaller and Fewer Studies

Subgroups

One theoretical concern with smaller randomized controlled trials is that if a general population is made up of responders and nonresponders, a small number of responders in the active arm of the trial, or low magnitude of effect in responders, can dilute the effect within the larger cohort to such an extent as to be no longer statistically significant overall. Subgroup analysis can correct for the former, but this is difficult, if not impossible, to determine from smaller randomized controlled trials and is usually not detected with statistical significance until meta-analyses are performed. If there seem to be some responders in the intervention arm but not enough to reach statistical significance overall, the general assumption in a randomized controlled trial analysis is that the supposed effect observed in certain individuals was due to random chance. However, subsequent N of 1 analysis of those individuals would show, with statistical significance, if they were indeed responding to the active intervention or not and if, therefore, the intervention is active in a certain subset of patients. If the subsequent N of 1s uncovered evidence of active intervention effect, the randomized controlled trial could be reexamined with a focus toward subgroups with similar characteristics to the N of 1–confirmed responders, and perhaps a new conventional randomized controlled trial planned.

Small study effect

If the parent randomized controlled trial did show a significant intervention effect, follow-up N of 1 trials focused on responders could further confirm the effect estimate. In underresearched fields, practitioners interested in practicing in an evidence-based manner can often only identify a single small trial on an intervention of interest. This may present difficulties in assessing the evidence level of that intervention since smaller studies tend to overestimate intervention effects. ¹⁷ Small trials are associated with selection bias, lower quality, and between-trial heterogeneity. ¹⁷ Additionally, because smaller trials require larger treatment effects for the effect to be statistically significant and those studies that find a significant intervention effect tend to be published more often, ²² the literature may suggest an unrealistic presentation of large effect estimates and positive trials. ¹⁷ Unfortunately, even when there are enough small trials to conduct a meta-analysis, they are poorly predictive of treatment effect when compared with subsequent very large multicenter randomized controlled trials. ²³ Furthermore, while small studies comparatively tend to be of lower quality, ¹⁷ even in the medical journals of highest impact where one would expect the highest quality studies, smaller randomized controlled trials were more likely to later be contradicted or their results shown to be overestimates. ²⁴ Considering these concerns with small trials, any further validation of a small trial’s effect estimate could be of great service to clinicians in fields where the research activity and finances cannot support large, multicenter randomized controlled trials. In an mixed-methodology add-on N of 1 trial design, by further subjecting the apparent randomized controlled trial responders to follow-up N of 1 trial analysis, their response to the study intervention can be confirmed on an individual level and clinicians can put further weight on the study’s effect estimate. By using the initial randomized controlled trial as a screen of sorts for responders, N of 1 trials could be efficiently targeted to appropriate subjects, thereby maximizing the useful information gathered and not necessitating the addition of unnecessary and expensive intervention periods for all the parent randomized controlled trial’s participants.

Interparticipant variability and effects of random chance

Interparticipant variability and random chance can have significant effects on smaller randomized controlled trials. ¹⁶ High outcome values for an individual in an randomized controlled trial arising due to his or her own response variability or to random chance would be expected to have a larger skewing potential in smaller trials than larger ones just as 3 heads in 3 flips of a coin would be less surprising than 300 heads out of 300 flips of a coin. This is true even for trials that perform power analyses. Indeed, “powering trials for statistical significance may not be enough”; in some cases it has been calculated that when considering clinically important intervention effects estimated by numbers needed to treat, the desired n may be 10 times that estimated solely for statistical significance. ¹⁶ With interparticipant variability and random chance of higher concern in small trials, the added certainty of treatment effect gained with mixed-methodology add-on N of 1 trial design can be very valuable to clinicians attempting to weigh evidence based on a single small trial. While it is the results of the parent randomized controlled trial that can be generalized, not the individual N of 1 trials, confirming or rejecting apparent responders from the parent randomized controlled trial, especially if it was a small trial, can lend added confidence to the parent trial’s effect estimate.

The Convenience Issue

Kravitz et al suggest that physicians are not using N of 1 trials more often in clinical practice because the added clinical precision is not worth the additional time and effort. ⁴ This convenience issue is relevant not only for the physician but perhaps even more so for the patient. As discussed above, there are certain situations where clinical N of 1 trials could still be worth the added effort, such as with high-cost pharmaceuticals. However, while N of 1 trials have traditionally been used in a clinical setting, mixed-methodology add-on N of 1 trials use them in a research environment and therefore use participants who are being compensated for their time and effort and researcher-physicians who are invested in discovering a true and precise intervention effect.

Conclusion

The N of 1 trial can be useful to both the clinician and researcher as well as invaluable to patients. Whether used in evidence-based medicine clinical decision making, pharmaceutical dosing, rare diseases, study adherence, or confirming and clarifying effects in smaller randomized controlled trials, the N of 1 trial is a versatile tool that can combine the benefits of evidence-based medicine and holistic, personalized patient care so integral to complementary and alternative medicine practice. Additionally, the combination of N of 1 trials and a conventional randomized controlled trial in a mixed-methodology add-on N of 1 trial, with all of their respective advantages, can augment the utility of either methodology used on its own and present a novel and uniquely appropriate tool to study complementary and alternative medicine interventions.