It's not either-or: Why migraine care needs both real-world evidence studies and randomized controlled trials

The editorial by Peres et al. (1) reviews a real-world evidence (RWE) study on the importance of treating migraine attacks early when using remote electrical neuromodulation (REN), including in youth (2).

This editorial discusses the original study (2) encompassing data from 55,261 patients who performed 586,981 migraine treatments with the wearable REN, 8886 of whom were youths who performed 82,040 treatments. It initially affirms the value of Ailani et al.'s (2) methodological approach especially for timing research, but raises several biases, most of which are described as inherent to RWE studies. Peres et al. then pose an important question: “Is real-world data speaking louder than clinical trials?” (1). Here, we probe some of the biases raised in the editorial in the context of available research, and challenge the binary approach presented, to highlight the value of RWE research alongside that of randomized controlled trials (RCTs).

Selection bias: Selection bias comes in many forms. All studies, including RCTs, may be characteristically influenced by selection bias since patients participating in trials are often unsatisfied by their current level of care (3). Thus, one can argue that individuals participating in RCTs of novel therapies are more highly motivated than those included in RWE studies. To mitigate the issue of motivation, Ailani et al. (2) included data of any participant who logged timing for one or more treatments regardless of whether they treated early or late, in addition to looking at individuals who were “frequent loggers”. Another potential selection bias could arise from the ‘match’ between specific patients and their treatment type. While neuromodulation users may be relatively more technologically savvy, participants with injectable treatment studies may be more pain tolerant, etc. Another potential aspect of technology savvy bias could be the use of an e-diary for RWE data collection, yet clinical guidelines for migraine therapy trials, including RCTs, also recommend an easy-to-use e-diary over paper diary (3). It is therefore unclear how notably different selection bias may exist in this RWE study compared to RCTs following international guidelines.

Recall bias: Recall bias refers to the possibility of over-reporting or under-reporting past events based on recent experiences and the time since the original events. Retrospective studies are therefore highly susceptible to recall bias. In Ailani at al.'s (2) prospective study, participants logged headache onset time at the time of treatment initiation, not after treatment success or failure. The majority of treatments (56.5%) were initiated within 1 h from attack onset. Given this design and data, it is unclear how recall bias plays a role here.

Contextual effects and subjective and expectation biases: The importance of early treatment is specifically mentioned in the guidelines for the acute treatment of migraine (4). Thus, a particular concern in Ailani et al.'s (2) study might be an expectation bias for higher efficacy following early treatment during the course of a migraine attack, and a nocebo effect as later treatment may be expected to result in less efficacy (5). However, this effect is not specific to neuromodulation and is seen in both adults and adolescents using pharmacological treatments.

Absence of a randomized control group or blinding: While RCTs, by definition, include a control group, a pre-defined control group is often not relevant to RWE studies. Nevertheless, Ailani et al. (2) compared two groups: the test group included early treatments and the control group included late treatments; in the patient-level analysis, patients who more regularly treated late were the control for those who more regularly treated early. Regarding blinding, many RCTs attempt to blind participants to the treatment they receive (active or placebo), yet “blinding is often compromised in pharmacological trials because patients can correctly guess their treatment assignment based on the occurrence of side effects they were warned about in the informed consent process” (6). While patients cannot be blinded in RWE studies, REN has already undergone extensive double-blinded, sham-controlled, multi-centered RCTs prior to its regulatory clearances (7–9), and acute efficacy was shown to be higher when used early (9). In essence, Ailani et al.'s (2) RWE study shows the potential consequences of treating late.

In addition to the biases raised, Peres et al. (1) elaborates on the question mentioned above and asks: “what carries more weight, real-world outcomes in tens of thousands of patients or smaller but rigorously controlled clinical trials?” The initial trial protocols and current FDA and CE labels of REN call for initiation of therapy within 1 h of headache onset. The RWE study by Ailani, et al. (2) simply provides over half a million data points of additional support for early treatment. Support for early treatment, as Peres et al. (1) point out, “is well-established for pharmacological interventions,” yet, “extending this concept to neuromodulation requires more rigorous investigation” and “until such evidence emerges, early treatment recommendations for neuromodulation should be considered provisional”. While a sham-controlled RCT to determine timing could certainly be conceived, early-treatment recommendation for REN was already established in RCTs (7,8). Given this evidence, the ethical dilemma of actively recruiting subjects, including children, to risk halving the efficacy of treatment by ignoring well-established RCT-based treatment protocols and large RWE supporting it, is at best questionable.

We thank Peres et al. (1) for raising issues related to RWE studies. In our view it is not a question of “either-or”. Different clinical study types have different advantages and limitations, and each type is prone to biases. We agree that RWE studies are not meant to replace RCTs, but rather to complement them by providing post-marketing data from large and diverse patient populations in real world settings. Thus, combination of RWE studies and RCTs may represent the best method to equip clinicians with the knowledge needed for informing practice recommendations and optimize outcomes in migraine care.