Abstract
Introduction
Since the definition of chronic migraine as a new disease entity in 2004, numerous clinical trials have examined the efficacy of preventive treatments in chronic migraine. Our aim was to assess the adherence of these trials to the Guidelines of the International Headache Society published in 2008.
Methods
We searched PubMed for controlled clinical trials investigating preventive treatment for chronic migraine in adults designed after the release of the Guidelines and published until December 2017. Trial quality was evaluated with a 13-item scoring system enlisting essential recommendations adapted from the Guidelines.
Results
Out of 3352 retrieved records, we included 16 papers in the analysis dealing with pharmacological treatment of chronic migraine. The median score was 6.5 (range 2–13). All trials were randomized, the large majority (81.25%) were placebo-controlled and double-blinded (87.5%). Adherence was lowest on i) a priori definition of outcomes (31.25%), ii) primary endpoint definition (37.5%%) and iii) trial registration (37.5%).
Discussion
Most clinical trials adhered to the recommendations of the IHS, whereas adherence to migraine-specific recommendations was lower. Greater awareness and adherence to the guidelines are essential to improve the quality of clinical trials, validity of publications and the generalizability of the results.
Introduction
Clinical trials are expensive and resource demanding. This is especially true for CNS disorders, such as migraine, which often impose higher development costs and have a lower success rate compared to drugs targeting other disorders (1). To enhance the quality and validity of clinical trials, and to enable a comparison between different compounds, it is important to ensure reliability and consistency in trial design and outcome parameters. For this purpose, a set of recommendations for controlled trials on preventive treatment of chronic migraine (CM) (according to the ICHD-2 chronic migraine revised version (CM-R) criteria (2)) in adults was published by the Task Force of the International Headache Society (IHS) in 2008 (3). Similar but separate guidelines had previously been published for episodic migraine, tension-type headache and cluster headache (4–6) – all with the purpose of strengthening the quality of the trials in order to generate evidence-based treatment recommendations for different headache disorders.
The 2008 Guidelines for trials on preventive treatment of CM offered recommendations on subject selection, trial design, outcome evaluation, statistical analysis, and publication. Recently, an updated edition of the Guidelines was published (7). These were guided by a critical analysis of RCTs published since 2008. The information and knowledge gathered during this critical analysis are here illustrated for the reader’s perusal. More precisely, the objective of this study was to quantitatively assess the adherence of the published trials to the Guidelines of the International Headache Society published in 2008 with the specific aim of identifying critical areas to be addressed.
Methods
Eligibility criteria
Clinical trials on preventive treatments for CM published between May 2008 and December 2017.
Specific inclusion criteria were: a) Trials on adult CM patients; b) investigation of preventive treatments: oral or injected drugs, including nerve blocks; c) controlled trials; d) study design developed after the publication of the 2008 Guidelines and enrolment started consequently.
Trials published after 2008 but started anteriorly in terms of design and/or enrolment were excluded. Trials on acupuncture, behavioral interventions and devices were excluded.
Search method
We searched the PubMed database for articles on clinical trials and clinical studies on the preventive treatments for CM published between 1 May 2008 and 31 December 2017 without language restrictions, using the terms “((migraine) OR (headache)) AND ((randomized controlled trial) OR (randomized) OR (placebo) OR (drug therapy) OR (randomly) OR (trial) OR (preventive) OR (group) NOT (animals) NOT ((humans) and (animals))”.
Screening of papers
Titles and abstracts of studies identified by the literature search were screened for eligibility. If title and abstract evaluation was deemed not sufficient to determine eligibility (or exclusion), the manuscript was evaluated in full.
Data analysis
Scoring frame used for evaluating clinical trials on chronic migraine: For each item, we assigned a score of 1 if the recommendation was followed, 0 if it was not.
Results
Inclusion of papers
The search was performed on 31 December 2017 and yielded 3352 PubMed records. Out of the 3352 titles and abstracts screened, we identified 61 papers that satisfied the eligibility criteria and were therefore evaluated in full. Of these, we excluded 30: 26 because they evaluated the efficacy of interventions not covered in this review (devices in 16 papers, acupuncture in five, behavioral therapy in two, primary care intervention in two and stem cells in one case report); four studies were excluded because they dealt with chronic daily headache and not specifically with CM. Two additional studies were not retrievable or withdrawn by the editor. The remaining 29 papers were included in the full review. Of these, 11 studies reported data of trials that were devised and conducted before the publication of the 2008 Guidelines; in two additional studies it was not possible to define the temporal sequence with respect to the Guidelines. The final set that was included in the analysis was therefore formed by 16 studies (Figure 1).
Flowchart describing literature evaluation and paper selection.
Evaluation of trials
Of the total of 16 reviewed studies, 13 (81%) were randomized, double blinded, placebo-controlled parallel-arm trials; none had a cross-over design. Six studies evaluated the effect of oral medications (9–14), five the effect of injected medications (CGRP monoclonal antibodies in four cases (15–18) and botulinum toxin in one (19)), and five trials dealt with nerve blocks (20–24).
The scores attributed to the trials included in the evaluation are illustrated in Table 2. The median score was 6.5, with a range of 2–13. It is worth noting that the mean score of the studies dealing with oral and/or injected drugs was higher (10, with a range of 3–13) than the score of trials dealing with nerve blocks (5, with a range of 2–6) (Figure 2).
Median score (with range) of all papers evaluated (black bar), the score of papers dealing with oral or injected drugs and the score of drugs administered via nerve block. Total scores attributed to the trials. Add on to trial in episodic migraine. Registered under the same NCT number. GON: Greater Occipital Nerve.
The recommendations that were most frequently followed were: i) randomization (100%), ii) double-blind design (87.5%), and iii) placebo-controlled design (81.25%). The three studies that were not placebo-controlled used an active comparator. Fifty percent of the trials applied the ICHD-2 CM-R diagnostic criteria as recommended. The recommendations that fewest trials followed were: i) The a priori definition of outcomes (31.25% of the trials); ii) the primary endpoints being either number of headache days with moderate or severe intensity, number of migraine or probable migraine days or number of migraine episodes (37.5% of the trials); iii) NCT registration before the recruitment (37.5% of the trials); iv) duration of treatment period being three months (37.5% of the trials); v) inclusion of healthcare outcomes/Quality of Life assessments as secondary endpoints (37.5% of the trials).
As regards ii), primary endpoints, we found a wide variety of definitions. For example, several trials reported the number of all headache days instead of the number of moderate-to-severe headache days; in other cases, the intensity of pain was considered, but it was rated according to a visual analogue scale instead of using the recommended categorical scale ranging from 0 to 4.
The recommendation regarding the 3-month duration of treatment was lowered by the nerve block studies (n = 5), which generally had a shorter treatment duration (1–4 weeks). However, it is worth noting that even after excluding these studies, the percentage of trials using a 3-month treatment period only increased to 45.5%.
More than half of the studies (62.5%) followed the recommendation on concomitant drug use (i.e. monotherapy with a 28-day washout period or add-on therapy with stable doses for other preventives for at least 3 months prior to baseline). The most common reasons for scoring 0 for this specific item in our scoring system were inadequate information on wash-out period or the period of being on stable doses before inclusion being too short (e.g. two months instead of three). It is worth noting that acute medication overuse (MO) was either not allowed or allowed and stratified for in only two thirds (62.5%) of the trials.
Adherence of the studies to the 2008 Guidelines.
We attempted an evaluation of the effect size of the medications under investigation based on the level of adherence to the Guidelines, but this exercise proved uninformative due to the disparate adoption of primary endpoints in the various trials, the different classes of drugs tested, and the different routes/modalities of administration.
Discussion
Adherence to recommendations for the planning and conduct of clinical trials enhances the validity of the trials while the low adherence to migraine-specific criteria leads to heterogeneity across trials, which hampers a direct comparison. In this review of the literature, we found that most studies indeed followed the trial recommendations regarding the study design (randomization, double-blind, placebo-controlled), but this satisfying level of adherence was of course influenced by the search strategy that we adopted, which included the terms “randomized” and “controlled”. When examining instead the specific recommendations for migraine trials or the recommendations regarding the end points, the adherence was much lower.
Only half of the studies used the recommended ICHD-2 CM-R criteria for the definition of CM. According to these, CM is defined as ≥ 15 headache days per month of which ≥ 8 fulfill criteria for migraine (25). This definition was an evolution of the original ICHD-2 definition of CM (≥15 migraine days/month) (26), which was deemed insufficient to correctly identify CM within the larger group of chronic headaches. Consequently, the results of the trials using ICHD-2 cannot be fully generalized to the CM population.
The definition of primary endpoints is crucial for proper comparison of efficacy between migraine preventives. In our analysis, the primary endpoints varied greatly among studies and only one third followed the guideline recommendations. Furthermore, some trials failed to clearly specify primary versus secondary endpoints; the primary endpoint was often not clearly defined. This diverse selection and description of endpoints prevents the comparison of efficacy of different drugs across trials, which is exactly our experience in this analysis. The possibility of calculating the effect size of interventions is extremely important, since efficacy has been rated by migraine patients as the most important feature of a preventive drug (27). Single trials are essential for defining efficacy and tolerability of new therapeutic options. However, because of their nature, RCTs have intrinsically a low generalizability since they are conducted on a very selected population and according to strict rules (28). The comparison of multiple trials, conducted on diverse CM populations (e.g. patients refractory or naïve to treatments, with or without psychiatric comorbidity) provides a higher degree of generalizability, if the studies are comparable. Full adherence to the recommendations is likely the best way to ensure methodological consistency and therefore comparability.
When testing a new drug candidate, both randomization and blinding are essential components of the design of the clinical trial. Here, all trials were randomized and almost 90% were double-blinded. However, in most studies the randomization method was not described and, more importantly, it was not reported whether blinding was successful. Assessment of blinding success is mandatory in order to evaluate the true efficacy of the drug; for example, both the placebo effect and treatment effect may be more trustworthy if patients were sufficiently blinded. Since the placebo response is generally high in migraine patients (29), it is especially relevant to include placebo, and to preserve blinding in clinical trials in migraine. In this frame, it is surprising that some trials were not placebo controlled: three trials compared two active drugs used for preventive treatment in CM. This approach appears correct for the comparison of the effect size of drugs with proven efficacy but it may not be the best choice for evaluating the efficacy of a new drug candidate, especially in a field where prophylactic drugs with proven efficacy are limited, blinding can be unmasked by the side effects, and the drugs’ effect size is limited.
Chronic migraine patients are often difficult to treat, and some may even have refractory headaches (30). Thus, it may be necessary or appropriate to include patients who already take a preventive medication. The adherence to the recommendations on concomitant preventive drugs was quite high, though far from the ideal 100%, as the majority of trials (62.5%) either applied monotherapy or applied add-on therapy with a timing congruous with the Guideline recommendations. When evaluating efficacy of drugs and choosing the right treatment for the patients, it is important to distinguish between these two kinds of trials since add-on therapy may provide a therapeutic gain for some patient groups but not for all.
Another important aspect to consider when dealing with CM patients is the presence of medication overuse (MO). MO plays an important role in the development of CM and removal of MO improves headache in a large proportion of patients (31–33). Stratification for MO, or, at very least, a specified approach to MO during the trial, is very important since patients may become aware of the importance of reducing acute medications during the trial, which will ultimately prevent disentangling of the effect of the reduction in the intake of acute medications from the effect of the preventive drug.
It is not clear why certain trials followed some parts of the Guidelines but not others. This may depend on logistics, financing, or practical issues. For example, nerve block studies had a shorter duration of treatment (less than the recommended 3 months) compared to other studies. It may also depend on priority issues of the trial sponsor. Geography of regulatory requirement may be another source of variation (34).
The findings of this review have limitations. Firstly, we used a simplified version of the Guidelines to assess adherence. We based our scoring on 13 recommendations, which were considered to be of the highest importance for the evaluation of the quality of the trials. The hierarchy was discussed and agreed upon within the group of authors, building on a previous similar experience conducted on trials for episodic migraine (8). We acknowledge that other recommendations of the Guidelines could have been included as well. Secondly, we gave equal weight to each of the 13 items of the scoring system. We acknowledge that this can lead to two studies having similar scores but being of completely different quality depending on which criteria they followed. However, depending on the objective of the trial, which items should be considered more important may differ, and due to the large variation between studies we did not find it plausible to differentiate between the recommendations in our study. Therefore, we also chose not to determine a cut-off for an acceptable score. Thirdly, the trials were evaluated based on the description of the trial in the published paper, or in its supplementary materials, or on the relative page at clinicaltrials.gov. Some papers were unclear in the description of endpoints or period of observation, or other parts. Whenever a clear description was missing, we scored 0 for the corresponding item. This may have resulted in lower scores for some papers and directly impacted the validity and reliability of inferences drawn from the scores. In addition, if the findings of a single trial are split over different papers, separately reporting different outcomes, this will result in a lower score for one or more of the single papers. This underlines the importance of transparency and thorough descriptions when publishing papers on trials.
It is worth noting that, though imperfectly following the Guidelines, the RCTs conducted in the decade 2008–2017 have been very helpful and informative for elaborating the new set of Guidelines (7). These latter built on the 2008 edition to improve and expand some pivotal areas (Supplemental Table 1): This is the case for primary and secondary end-points, which are more clearly listed and defined, and the duration of disease, which has been extended to 12 months to minimize the occurrence of spontaneous reduction of attacks. In addition, the 2018 Guidelines provide precise indications for the management of critical issues, such as MO.
Conclusion
We assessed the adherence of CM clinical trials conducted since 2008 to the Guidelines for controlled trials of preventive treatment of chronic migraine in adults published in the same year and found that, while most studies followed the general recommendations for study design, the migraine-specific and end-point-related recommendations were followed to a lesser extent. This diversity hampers comparison of results across trials for CM preventive treatments.
Guidelines on clinical trials are an extremely important tool to ensure not only the uniformity and quality of clinical trials, but also the subsequent steps: Availability on the market of effective drugs, scientific advances, optimized management, change in healthcare approaches, and so on. This is the reason why IHS undertook the effort to release in 2018 a new version of the Guidelines (7), improved and guided by the experience gathered in 10 years of RCTs. It is expected that investigators will carefully follow these new guidelines to the greatest possible extent.
Future studies on preventive treatments in CM should carefully follow the newly published updated migraine clinical trial guidelines. This will enhance the external validity and quality of the trials, as well as improve the evidence-based treatment for CM patients.
Article highlights
Recently published clinical trials showed limited adherence to the 2008 Guidelines of the International Headache Society (IHS) for controlled trials on preventive treatment of chronic migraine in adults. Most studies followed the general IHS recommendations for study design, while the migraine-specific and end-point-related recommendations were followed to a lesser extent. This diversity hampers comparison of results across trials. Evidence and experience accumulated support the need for the publication of an updated edition of Guidelines and for a greater adherence to these.
Supplemental Material
Supplemental material for Adherence to the 2008 IHS guidelines for controlled trials of drugs for the preventive treatment of chronic migraine in adults
Supplemental Material for Adherence to the 2008 IHS guidelines for controlled trials of drugs for the preventive treatment of chronic migraine in adults by Marie Deen, Daniele Martinelli, Judith Pijpers, Hans-Christoph Diener, Stephen Silberstein, Michel D Ferrari, Messoud Ashina, Cristina Tassorelli and Hsiangkuo Yuan in Cephalalgia
Footnotes
Declaration of conflicting interests
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Marie Deen, Daniele Martinelli and Judith Pijpers have no conflicts of interest. HC Diener received honoraria for participation in clinical trials, contribution to advisory boards or oral presentations from: Addex Pharma, Alder, Allergan, Almirall, Amgen, Autonomic Technology, AstraZeneca, Bayer Vital, Berlin Chemie, Böhringer Ingelheim, Bristol-Myers Squibb, Chordate, Coherex, CoLucid, Electrocore, GlaxoSmithKline, Grünenthal, Ipsen, Janssen-Cilag, Labrys Biologicals, Lilly, La Roche, 3M Medica , Medtronic, Menerini, Minster, MSD, Neuroscore, Novartis, Johnson & Johnson, Pierre Fabre, Pfizer, Schaper and Brümmer, Sanofi, St. Jude, Teva and Weber & Weber. Financial support for research projects was provided by Allergan, Almirall, AstraZeneca, Bayer, Electrocore, GSK, Janssen-Cilag, MSD and Pfizer. Headache research at the Department of Neurology in Essen is supported by the German Research Council (DFG), the German Ministry of Education and Research (BMBF) and the European Union. H.C. Diener has no ownership interest and does not own stocks of any pharmaceutical company. HCD serves on the editorial boards of Cephalalgia and Lancet Neurology. HCD chairs the Clinical Guidelines Committee of the German Society of Neurology and is a member of the Clinical Trials Committee of the IHS. Stephen Silberstein has received honoraria from Alder Biopharmaceuticals; Allergan, Inc.; Amgen; Avanir Pharmaceuticals, Inc.; Cefaly Technology (unrelated to this manuscript); Depomed; Dr. Reddy’s Laboratories; eNeura Inc.; electroCore Medical, LLC; Ipsen Biopharmaceuticals; Medscape, LLC; Medtronic, Inc.; Mitsubishi Tanabe Pharma America, Inc.; NINDS; St. Jude Medical; Supernus Pharmaceuticals, Inc.; Teva Pharmaceuticals and Trigemina, Inc. Michel D Ferrari reports grants and consultancy or industry support from Medtronic and independent support from the European Community, NWO, NIH and the Dutch Heart Foundation. Messoud Ashina is a consultant, speaker or scientific advisor for Allergan, Amgen, Alder, ATI, Eli Lilly, Novartis, and Teva, primary investigator for Amgen 20120178 (Phase 2), 20120295 (Phase 2), 20130255 (OLE), 20120297 (Phase 3), Alder ALD403-CLIN-001 (Phase 3), Amgen PAC1 20150308 (Phase 2 a), and GM-11 gamma-Core-R trials. Cristina Tassorelli has received consulting fees from Allergan, Eli-Lilly, ElectroCore, Novartis and Teva. She is PI in clinical trials sponsored by Allergan, Eli-Lilly, ElectroCore, Novartis and Teva. Dr. Yuan received honoraria from Supernus Pharmaceuticals, Inc.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
