Abstract
Background
In 2008, the International Headache Society published guidelines on the “evaluation and registration of adverse events in clinical drug trials on migraine”. They listed seven recommendations for reporting adverse events in randomized controlled trials on migraine. The present study aimed to evaluate adherence to these recommendations, and based on the results, to recommend improvements.
Methods
We searched the PubMed/MEDLINE database to identify controlled trials on migraine drugs published from 2010 to 2015. For each trial, we noted whether five of the recommended parameters were presented. In addition, we noted whether adverse events were reported in abstracts.
Results
We identified 73 trials; 51 studied acutely administered drugs and 22 studied prophylactic drugs for migraine. The number of patients with any adverse events were reported in 74% of acute-administration and 86% of prophylactic drug trials. Only 30 (41%) of the 73 studies reported adverse events with data in the abstracts, and 27 (37%) abstracts did not mention adverse events.
Conclusion
Adverse events, both frequency and symptoms, should be reported to allow a fair judgement of benefit/tolerability ratio when randomized controlled trials in migraine treatment are published. Clinically significant adverse events should be included in the abstract of every randomized controlled trial in migraine treatment.
Introduction
Guidelines have been developed on measurements of efficacy endpoints in clinical trials of acute and prophylactic drug treatment of migraine to ensure uniform reporting of efficacy (1,2). However, in the guidelines for clinical trials on migraine drug therapies, published by the International Headache Society (IHS) from 1990, 2000, and 2012 (1,2), adverse events (AEs) are only briefly mentioned in one paragraph. In 2001, Ioannidis and Lau stated that “the quality of safety reporting varies across medical areas, study, design, and settings, but they are largely inadequate. Current standards for safety reporting in randomized trials should be revised to address this inadequacy” (3).
Subsequently, in 2004, the CONSORT group elaborated and published a document on “better reporting of harms in randomized trials” (4). They included 10 recommendations for reporting harmful drug effects. Some of those recommendations and comments, however, were concerned with repeated administrations of drugs for long periods in randomized controlled trials (RCTs) (4). Migraines are treated with both acute and prophylactic drugs; thus, for a single dose administration of an acute drug, some of the recommendations and debated points in the CONSORT statement are irrelevant. For example, withdrawal from an RCT due to AEs is a highly relevant parameter for prophylactic drugs tested for treating migraines, but not for the acute-administration drugs, which are typically administered as a single dose (5).
In 2008, the IHS published guidelines on the “evaluation and registration of adverse events in clinical drug trials on migraine” (6). These guidelines listed six simple recommendations for reporting AEs, which could be used in RCTs for both prophylactic and acute-administration migraine drugs (6).
This review aimed to evaluate articles published in 2010 and later (6) to determine how the current practice of reporting AEs in trials on migraine drugs compared to the 2008 IHS guidelines. In addition, we aimed to suggest improvements for reporting AEs associated with both acute-administration and prophylactic migraine drugs.
Methods
We searched the PubMed/MEDLINE database to identify controlled trials on migraine drugs published from 1 January 2010 to 31 December 2015. The search string was “MIGRAINE RANDOMISED CONTROLLED TRIAL”. Two authors (PTH, TPD) screened the titles and abstracts independently. The database search was conducted in May 2017. In addition, one author (JKL) manually searched the six annual volumes (2010–2015) of the three official headache journals: Headache, Cephalalgia, and the Journal of Headache and Pain, to identify double-blind RCTs on acute-administration and prophylactic migraine drugs. The manual search was conducted in August 2016. Only articles in English were included. Studies with children and adolescents were excluded. We also excluded post hoc analyses of efficacy and tolerability in patients with, for example, menstrual migraines, severely intense headaches, and allodynia. Any disagreement was resolved by consensus between the authors. We entered all of the data in a table (online appendix). The data was doubled-checked by PTH and TPD independently. The PRISMA four-phase diagram was followed for the selection process (7).
We noted that each RCT presented five of the seven parameters recommended in 2008 by the Drug Trial Committee of IHS (6). These parameters were:
For each treatment arm, the number of patients with any AEs. For each treatment arm, the number of patients with any serious AEs. For each treatment arm, the number of patients withdrawn because of AEs. (This was only relevant for RCTs on prophylactic treatments). For each RCT, the intensity of AEs. For each RCT, whether details of individual AEs were reported, in either the text or the tables.
In addition, we noted whether all or no AE parameters were reported in the paper.
The two omitted parameters were “number of patients with individual/specific AEs that are of prespecified interest or are common or otherwise likely to affect tolerability” and “details of all serious AEs, including possible causation”. We did not note the details of all serious AEs, including the possible causes, because these details were unlikely to be reported when so few serious AEs were reported (see Results). Finally, we evaluated AE reporting in abstracts, as follows: not mentioned; mentioned, but no data; mentioned with data.
Results
The study selection process is illustrated with the PRISMA flow diagram (Figure 1).
PRISMA 2009 flow diagram. From: Moher D, Liberati A, Tetzlaff J, et al., The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. For more information, visit www.prisma-statement.org.
The manual search of the three official headache journals (Headache, Cephalalgia, and the Journal of Headache and Pain) did not identify any new RCTs that were not found in the PubMed/MEDLINE database search.
Adverse Event reporting in randomized controlled trials of acute and prophylactic migraine treatments
Nineteen of 51 (37%) studies on acute migraine treatments reported four of the recommended parameters on AEs. Five of 22 (23%) studies on prophylactic treatments reported all five parameters on AEs.
Number of patients with any adverse events
The number of patients with any AEs in each treatment arm was reported in 75% of RCTs on acute migraine therapies and 91% of studies on prophylactic treatments. In 20 RCTs on acute drugs, one or more doses were compared with placebo (see online appendix). In only seven (35%) RCTs was the drug superior to placebo (p < 0.05, Fisher’s exact test). The placebo response for number of patients with any AEs varied from 0% to 60%, with a median of 12%.
Number of patients with any serious adverse events
Serious AEs occurred in 1–8% of patients per acute-administration drug. Among the studies on acute drug therapies, 11 (22%) studies reported the occurrence of serious AEs. Serious AEs occurred in 1–3% of patients per prophylactic drug. Among the studies on prophylactic drugs, 10 (45%) studies reported the occurrence of serious AEs.
Number of patients withdrawn due to adverse events
Among prophylactic studies, 15 (68%) studies reported withdrawals due to AEs. The withdrawal frequency varied from 0% to 21%. No withdrawals were reported when a proprietary supplement, riboflavin, magnesium, or Q10 (61) was investigated. In contrast, 25/188 patients withdrew (13%, 95% CI: 8–18%) due to AEs with topiramate (100 mg) (79,74).
Intensity of adverse events
For acute drug therapies, AE intensities were reported as mild in 10% of RCTs, in mild to moderate in 27% of RCTs, and mild to moderate plus severe in 8% of RCTs. For prophylactic treatment, AE intensities were reported as mild in 9% of RCTs, and mild to moderate in 32% of RCTs.
Details of adverse events
Among 51 studies on acute drug therapies, the details of individual AEs were reported in 42 (82%) RCTs. Among 22 RCTs on prophylactic treatments, details of individual AEs were reported in 21 (95%) RCTs.
Reporting adverse events in abstracts (Table 2)
We examined whether abstracts reported on AEs in the 51 acute drug therapy RCTs and 22 prophylactic treatment RCTs. When all studies were combined, only 30 (41%) abstracts reported AEs with data, and 27 (37%) abstracts did not mention AEs at all.
Discussion
Limitations of this review
This study aimed to estimate whether the five most clinically relevant out of seven IHS recommendations on AE registration and reporting in migraine trials (6) from 2008 were currently being followed. One could argue that, by limiting the time period for collecting trials to the years 2010–2015, we missed a considerable number of previous acute-administration RCTs on triptans; for example, the 53 RCTs reviewed by Ferrari et al, in 2001 (80). However, this review did not endeavor to estimate AE frequency among migraine patients in RCTs, which would require a systematic review (81–84); instead, we endeavored to determine the quality of reporting of AEs. Thus, our current inclusion of 73 acute-administration and prophylactic-administration RCTs was most likely sufficiently large to disclose problems related to reporting AEs.
Another limitation of this review was the lack of definitions for the AE parameters reported in the reviewed studies. Thus, some RCTs stated that the AEs were mild, mild-to-moderate, and in a few cases severe; but the reader remained unaware of how these terms were defined, and the terms are most likely used differently in different RCTs. The time interval after drug administration during which patients are asked to register AEs was also variable. In a few RCTs, AEs that occurred up to 7 days after drug administration were reported (85–87).
Comments about reporting adverse events in randomized controlled trials on migraine drugs
The most important goal in RCTs on migraine is typically to demonstrate efficacy, and RCTs are designed accordingly. Consequently, RCTs in the therapeutic area of migraine are not designed to demonstrate conclusively whether a treatment shows superiority or inferiority on tolerability outcomes (84). Thus, the primary efficacy parameter is always reported, but the present review found that the accepted most important measure of tolerability, the number of patients with any AEs in the RCT, was only reported in 75% of acute-administration RCTs and 91% of prophylactic RCTs.
In a single RCT, tolerability can be considered a measure of any AEs, but in the majority of current RCTs, the acute-administration drugs were no different from placebo. One reason for the lack of difference could be that the drug is very well tolerated or could be due to a very high frequency of any AEs in placebo group (e.g. 46% [60] to 60% [25]). Another probable reason for no difference despite an established tendency for AEs of the drug is a relatively small size of RCT. The solution to this problem is to perform a systematic review with a meta-analysis of RCTs on one drug (81,82,88–91), or a group of drugs (83,84). Those systematic reviews showed that many drugs caused more AEs than placebo. However, the prerequisite for a systematic review is that the large majority of the RCTs must report patients with any AEs with the drug and placebo treatments. Reporting the number of patients with any AEs in only 79% of RCTs, as in the present review, could result in bias.
The number of patients with serious AEs, although reported in only 22% of acute-administration and 45% of prophylactic RCTs, did not seem to be a great quantitative problem since serious AEs occurred only in 1–3% of patients (see Results).
In prophylactic RCTs, withdrawal due to AEs was reported in 15 (68%) of 22 studies. This parameter is clinically very important, because it depends on the patient’s evaluation of the benefit/tolerability ratio. In three pivotal RCTs on topiramate, 24% of patients withdrew due to AEs during treatment with the 100 mg dose (92–94). However, a high AE frequency does not automatically result in many withdrawals, when the AEs are mild to moderate. For example, in the three RCTs on topiramate (92–94), topiramate (100 mg) caused paresthesia in 51% (195/386) of patients, but only 8% (31/386) withdrew due to paresthesia (95). This result suggested that topiramate had some prophylactic effect in those patients with this AE.
In the studies reviewed here, most papers that reported AE intensity used vague terms, like mild to moderate, which provided little guidance to AE intensity. In addition, approximately 50% of studies on acute-administration or prophylactic drugs did not report on the intensity of AEs.
In the present review, the details of individual AEs were reported in 80% of acute-administration RCTs and 96% of prophylactic RCTs. Similarly, in a systematic review of AEs in placebo groups in RCTs of drug treatments in migraine description of AEs were presented in 81% of papers (96). Some of the more frequent AEs could be evaluated in a meta-analysis, as performed for AEs in the CNS and chest-tightness in RCTs on triptan (81–83).
Clear, detailed abstracts are important for journal articles that report RCT results, because readers often base their initial assessment of a trial solely on information in the abstract (97). “They may then use this information to decide whether or not to seek more knowledge about the trial, such as reading the full report if available” (97). If clinically relevant, the frequencies of AEs in each treatment arm should be presented (97). This can be relevant when an established drug with known tolerability profile when used for another indication (e.g. epilepsy or hypertension) is introduced for prophylactic treatment of migraine. Thus, two studies (98,99) have shown that migraine patients are more likely than epileptic patients to experience AEs with prophylactic use of topiramate.
The distribution of abstracts that reported adverse events (AEs) among randomized controlled trials (RCTs) on migraine treatments.
Two different reporting systems for adverse events in randomized controlled trials, sponsored by two different medical firms. The AEs after eletriptan 40 mg and placebo are presented. The original titles for tables of AE data are shown in italic.
Notes: aTotal of 15 AEs are listed; bno severe AEs were reported.
Difficulties in reporting adverse events in randomized controlled trials on migraine drug treatments
Results from two RCTs (26,45) identified in the current review illustrated some difficulties associated with reporting AEs in RCTs on migraine treatments. Table 3 shows two different ways to report AEs in RCTs. These data illustrate problems that are currently encountered when AEs are reported in studies on acute-administration migraine treatments. In Diener et al. (45), the RCT was too small to show any difference in the incidence of any AEs between eletriptan 40 mg and placebo (p = 0.23, Fisher’s exact test). In Almas et al. (26), it was likely that the active drug (again eletriptan 40 mg) caused more AEs than placebo; however, that analysis was omitted, because they did not report data on the number of patients with any AEs. In addition, there was no definition for severe AEs in that study.
Methods for evaluating benefit/tolerability in randomized controlled trials on acute-administration migraine drugs
The importance of AEs should be evaluated in the context of the benefit/tolerability ratio. This ratio should be based on the patient ratings.
In crossover RCTs, patients can be asked their preference between two drugs; for example, two triptans. We found three crossover RCTs, and patients always expressed a preference for frovatriptan over almotriptan, rizatriptan, or zolmitriptan (51,55,42). The reasons given for patient preference were only briefly mentioned, but 50–80% of patients stated that the rapid effect was the main reason for the preference; in contrast, improved tolerability was the reason in 40% of cases (51,55,42).
Acute-administration RCTs typically have a parallel group design, and the benefit/tolerability ratio can be evaluated subjectively, by asking patients whether they are willing to use the drug again. Seven placebo-controlled eletriptan RCTs (85–87,101–104) found that 30% (95% CI: 27–32%) of patients rated placebo as an acceptable treatment, and 70% (95% CI: 68–72%) of patients rated eletriptan 40 mg as an acceptable treatment. As expected, the active drug, eletriptan, was rated higher than placebo. In addition, eletriptan was rated higher than naratriptan (50%, 95% CI: 43–57%) (104). In both cases, the higher rating of eletriptan was most likely due to the higher efficacy of eletriptan. To date, there is no information on the contribution of tolerability to the rating, but this issue should be investigated further.
The question, “would you take the drug again?” is clinically relevant, because it mimics a frequent clinical situation, when a physician renews a prescription for an acute migraine drug.
Conclusion
There is a need for improved reporting of the number of patients with any AE, types of, and impacts of any AEs in each treatment arm of RCTs on acutely administered migraine drugs. This information is important, both for the readers when judging the benefit/tolerability ration and for researchers that wish to perform analyses of the tolerability of the drug in systematic reviews; as shown in the present review, this is currently not always the case.
Readers of RCT studies in migraine treatment should be aware that, in most cases, RCTs are powered to demonstrate efficacy, not tolerability. Thus, in the present review, 65% (20/31) of the comparisons performed to reveal AEs showed no statistically significant difference between drugs and placebo. The main problem was the relatively small sizes of RCT cohorts.
Readers often base an initial assessment of a trial solely on information in the abstract (97). Thus, a clear, detailed abstract should be presented for RCTs. If relevant (see above), quantitative data for any AEs should be presented for each treatment arm. Currently, only 41% of abstracts report AEs with data (Table 2).
It should be a joint responsibility of investigators, reviewers, and editors to improve the quality of reporting on AEs in abstracts.
Article highlights
The main measure of tolerability in reports on RCTs of migraine drug treatment should be the number of patients with any adverse events in each treatment arm. When the frequency of adverse events is equivalent with a drug and placebo, the randomized controlled trial size may be too small. The quality of reporting of adverse events in abstracts should generally be improved because readers often judge the results, efficacy vs. tolerability, solely on the abstract.
Supplemental Material
Supplemental Material for Evaluating the reporting of adverse events in controlled clinical trials conducted in 2010–2015 on migraine drug treatments
Supplemental Material for Evaluating the reporting of adverse events in controlled clinical trials conducted in 2010–2015 on migraine drug treatments in Cephalalgia
Footnotes
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
