Abstract
Objective
To evaluate the association between the degree of response to placebo in migraine studies and the observed difference between drug and placebo across studies of preventative treatments for migraine.
Methods
A systematic review was performed using MEDLINE and the Cochrane Central Register of Controlled Clinical Trials from January 1988 to June 2019. Randomized, double-blind, parallel-group, placebo-controlled trials on oral or injection preventative treatments for migraine were included. Single- and multi-variable linear regression analyses were performed on the placebo-subtracted response rate (i.e. placebo responders subtracted from active responders), and the proportion of placebo responders. Fisher’s exact tests were performed on the level of placebo response and the success in meeting the study’s primary endpoint.
Results
After adjusting for route of administration and number of randomized subjects, there was a statistically significant association between the proportion of patients who were placebo responders and the placebo-subtracted response rate (b = −0.27, p = 0.02). There was a statistically significant difference in trial success rate (60%) between studies with ≤20% placebo responders and studies with > 30% placebo responders (p = 0.03).
Conclusion
Considering the detrimental impact that high placebo response can have on clinical trials, it is imperative to find effective solutions to decrease the placebo response and increase assay sensitivity.
Introduction
One of the many challenges that must be met before providing new, effective treatments to patients in need is the completion of successful clinical trials. The success of clinical trials depends on assay sensitivity; that is, the ability to distinguish an effective treatment from a placebo or other non-effective treatment (1). One commonly proposed cause of poor assay sensitivity in clinical trials is a high placebo response, or large improvement in the placebo group (1). There are many factors that can contribute to a high placebo response such as the placebo effect itself, natural improvement in the population, poor research or study design, and regression to the mean (1). For neurological disorders such as neuropathic pain and Parkinson’s disease, a high placebo response has indeed been shown to have an impact on the separation between placebo and active treatment (2–4).
Despite mounting evidence of an association between placebo response and assay sensitivity, this relationship has not been fully investigated in many indications, including migraine. Migraine is a painful and disabling condition with a worldwide burden estimated at 1.04 billion and an estimated prevalence of 14.4% according to the 2016 Global Burden of Disease (GBD) study (5). There is a range of validated outcome measures used to assess migraine both in the physician’s office and in clinical trials. The objective of this study is to evaluate the association between the degree of response to placebo in clinical trials on migraine and the observed difference between drug and placebo in such studies across a range of preventative migraine treatments.
Methods
Systematic literature review
A systematic review was performed using MEDLINE (through Pubmed) and the Cochrane Central Register of Controlled Clinical Trials from January 1988 to June 2019; the review followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (6). Additional studies were identified through previously published systematic reviews and meta-analyses on migraine treatments (7,8). The protocol was submitted to the International Prospective Register of Systematic Reviews (PROSPERO), protocol registration number CRD42019139041 (9).
Searches were performed using a combination of the terms “migraine”, “placebo”, “sham”, “preventative”, “prophylactic”, “randomized”, and “clinical trial” (Supplemental Material D). Studies were included if they were randomized, double-blind, parallel-group, placebo-controlled trials. Subjects were required to have migraine diagnoses based on the International Headache Society Criteria (IHSC) and be at least 18 years of age. The treatment must have been prophylactic and pharmacological in nature, administered orally or via subcutaneous, intramuscular, or intravenous injection. The studies were required to have a double-blind, randomized treatment period of at least 8 weeks. The studies must also be published in the English language and report the proportion of 50% responders, defined as ≥50% reduction in migraine days/month, for both active and placebo groups. The threshold for proportion of responders (50%) was chosen as the outcome measure of interest because of the high prevalence of this measure used as a primary or secondary outcome in most migraine studies (8). Studies were excluded if they included children, adolescents, or did not clearly define migraine diagnosis according to the IHSC. Treatment for acute migraine, herbal or homeopathic remedies, or treatment for specialized groups or subgroups such as menstrual migraine patients or placebo non-responders, were excluded. Cross-over, open-label, or non-randomized studies were also excluded.
Articles identified in the systematic search went through title and abstract screening with duplicate articles removed. Relevant full-text articles were extracted for another round of screening and data extraction. The initial screening process was performed by KE (MS, MPH). Discrepancies and issues, n < 10, were discussed in detail between NK (MD, MS) and KE. Final decisions were based on agreement between KE and NK. Relevant data were extracted from the remaining articles using a standardized form. In addition, the articles were evaluated for bias using the Jadad Scale (10) and Cochrane Collaboration Risk of Bias tool (11) (Supplemental Material B and Supplemental Material C).
Data analysis
A responder was defined as a subject who experienced 50% or more reduction in migraine days per month. The placebo-subtracted response rate was calculated by subtracting the proportion of placebo responders in each study from the corresponding proportion of active responders. Single- and multi-variable linear regression analyses were performed to determine if an association between variables was present, with the dependent variable being the placebo-subtracted response rate and the independent variable being the proportion of placebo responders. Outliers were examined graphically using 95% prediction limits. Route of medication administration, length of the double-blind, randomized treatment period of the study, baseline migraine days per month, and number of subjects were examined to determine if they were covariates or confounders and should be included in the regression model. This assessment was determined based on testing for a relationship with the dependent and independent variables, as well as examining the R2 value in the regression models.
An exploratory analysis was performed to determine if success in meeting the primary endpoint (i.e. meeting statistical significance of the primary endpoint) was associated with placebo response, the latter was categorized into one of three levels: ≤20% proportion responders, 21–30% proportion responders, and >30% proportion responders. Fisher’s exact tests were performed on the level of placebo response and the success in meeting the primary endpoint.
A post hoc analysis was performed to determine sample size requirements for statistical significance between proportion of responders based on the regression model. Sample sizes were displayed graphically based on the proportion of placebo responders at an alpha level of 0.05 and 80% power.
Results
A total of 816 articles were identified through Pubmed and Cochrane Central Registrar of Controlled Clinical Trials and an additional 86 were identified through other meta-analyses and systematic reviews, for a total of 790 articles after duplicates were removed (see Figure 1). After title and abstract screening, 195 articles remained. After full-text article screening, 27 articles remained that encompassed 28 studies (one article had two studies).

PRISMA diagram.
Of the 28 included studies, 15 were with oral medications and 13 were with medications administered by injection (Supplemental Material A). The primary endpoint for three of the studies was the proportion of patients with a 50% reduction in migraine days per month; the rest of the studies chose variations of change in migraine attacks per month (e.g. change in migraine attack frequency per month, change in migraine headache days per month). All studies had a prospective baseline period of 4–6 weeks to monitor migraine frequency before randomization, except for Buettner et al., which had a baseline period of 12 weeks (12). The mean proportion of responders was 30.3% ± 15.9% (Standard Deviation; SD) for the placebo group, 42.9% ± 14.7% for the active group, and the mean placebo-subtracted response rate was 12.6% ± 10.8%. Studies in which the medication was administered by injection (i.e. subcutaneously, intramuscularly, or intravenously) had a numerically but not statistically higher proportion of placebo responders (33.5% ± 15.9%) than studies with an oral route of administration (27.5% ± 15.9%; p = 0.327). Baseline migraine days per month, length of the double-blind, randomized treatment period and the year of the study were not found to be confounders or covariates; therefore, they were not included in the regression model (data not shown). Route (i.e. oral or injection) and sample size of randomized subjects were found to be covariates and were included in the model. Sample size was represented categorically as large and small sample sized based on the median split of number of randomized subjects (n = 216). Two studies were identified as outliers, falling outside the 95% prediction limits in the linear regression, and were therefore removed from the main analysis.
Once these outliers were removed, 25 articles and 26 studies remained. The proportion of responders remained relatively unchanged with the mean proportion of responders at 30.4% ± 16.3% for the placebo group, 42.7% ± 14.4% for the active group, and the placebo-subtracted response rate at 11.9% ± 8.4% (see Table 1).
Mean proportion of responders across all studies.
Note: Responder defined as a patient with ≥ 50% reduction in migraines/month.
N: number of studies; SD: standard deviation.
After adjusting for the route of administration and sample size, there was a statistically significant association between the proportion of patients who were considered placebo responders and the placebo-subtracted response rate (b = −0.27, p = 0.022) (Figure 2). Further, 35% of the placebo-subtracted response rate is explained by the proportion of placebo responders. Based on the linear model for small sample sizes (n < 216), reducing the proportion of placebo responders from 50% to 20% would be predicted to almost double the placebo-subtracted response rate for injection treatments and would triple it for oral treatments.

Difference in placebo-subtracted response rate by proportion of placebo responders. The placebo-subtracted response rate decreases with increasing proportions of placebo responders. Responder defined as a patient with ≥50% reduction in migraine days/month. Variable xa is the proportion of placebo responders; variable xb is route where oral = 0 and injection = 1; variable xc is sample size where small = 0 and large = 1. Large N indicates studies with large sample sizes. Small N indicates studies with small sample sizes.
In the exploratory analysis, a total of 11 (42.3%) of the included trials failed to meet statistical significance of the primary endpoint, whereas 15 (57.7%) of trials were successful (results were statistically significant). There was a statistically significant difference in proportion of successful trials between studies with ≤20% placebo responders (100%), 21–30% placebo responders (50%), and >30% placebo responders (40%; p = 0.048). This statistically significant difference was driven by the 60% difference in trial success rate between studies with ≤20% placebo responders and studies with >30% placebo responders (p = 0.034). However, there was a strong relationship between successful trials and baseline migraine days. Therefore, the relationship between the proportion of successful trials and proportion of placebo responders was stratified by high and low baseline migraine days based on the median split (8.24 migraine days a month) – this stratification also led to some cells having less than three studies to represent that category. In the high baseline migraine group, there was not a statistically significant difference in proportion of successful trials between studies with ≤20% placebo responders (100%), 21–30% placebo responders (80%), and >30% placebo responders (75%; p = 1.0). In the low baseline migraine group, there was not a statistically significant difference in proportion of successful trials between studies with ≤20% placebo responders (100%), 21–30% placebo responders (20%), and >30% placebo responders (17%; p = 0.38) (Figure 3).

Proportion of studies that met their primary endpoint by placebo response and baseline migraine frequency. A trial is considered successful if it meets statistical significance of its primary endpoint; see Methods. Numbers on the top of each bar indicate the number of studies. High baseline migraine frequency was considered ≥8.24 migraines per month. Low baseline migraine frequency was considered <8.24 migraines per month.
Sample size estimates were calculated at an alpha level of 0.05 and 80% power for oral and injection treatments using the regression model in Figure 2 for small sample sizes (n < 216). A study with a placebo response rate of 40% would need approximately 830 subjects for oral treatments and 221 for injection treatments to meet statistical significance; however, a study with a placebo response rate of 20% would need only 197 subjects for oral treatments and 91 for injection treatments to reach statistical significance (Figure 4).

Sample size estimates by proportion of placebo responders. Sample size requirements increase as proportion of placebo responders increase. Responder defined as a patient with ≥50% reduction in migraines/month.
Discussion
We found that in clinical trials of preventative treatments for migraine, the greater the placebo response, the smaller the placebo-subtracted response rate and the higher the sample size requirements. These results are consistent with other meta-analyses on the association between placebo response rates and the success of chronic pain trials, such as neuropathic pain (4,13). Other studies have also found an increasing placebo response over time (4,13). Our study, however, found no significant relation with year of study and placebo response when study year was added to the regression model. We did find route of treatment administration and sample size to be covariates in this model. Studies with an oral treatment had slightly lower levels of mean proportion of placebo responders than studies with an injection treatment. Other studies have also found an association between route of administration and placebo response, suggesting that the more invasive the treatment, the higher the placebo response (7). One study found baseline severity of depression to be associated with differences between drug and placebo (14); however, we did not find a similar trend in our analysis. We did find baseline migraine to be associated with success of meeting the primary endpoint in the exploratory analysis on the relationship between trial success and proportion of placebo responders. As many factors influence the ability to meet the primary endpoint – difference between arms, variability, and sample size – further research is warranted to determine which factors, or combination of factors, baseline migraine may influence.
This analysis had several limitations. First, only published articles were included in the analysis and successful trials are much more likely to be published than unsuccessful ones (15). This means that the already-high proportion of clinical trials failing to meet their primary endpoint in this study is likely an underestimate of the true proportion. Second, as this study examined the association of placebo response to study endpoints across preventative migraine treatments, there is a high level of heterogeneity across studies with known and unknown variables that could confound the results such as protocol deviations, the extent of patients’ accurate pain reporting, number of sites (16), and differences in study conduct between sites. For example, one of the two outliers (17) had a very high difference in responders between groups (42%), but only a little over half of the subjects remained by the end of the study. The analysis was based on the population who remained in the study and the majority (74%) of patients who withdrew from the treatment arm did so due to lack of efficacy. Therefore, this study likely has an inflated difference in responders between groups due to bias caused by excluding patients who dropped out of the study. To reduce the risk of bias caused by possible covariates and confounders, several variables were tested to be included in the regression model.
It is important to determine the root cause for the strong association between placebo response and the success of clinical trials. One possible explanation is expectation bias causing a high placebo effect. While the additivity theory of clinical trials suggests that expectation should increase the response in both treatment and placebo groups equally, our study as well as others indicate the opposite: Factors that influence the placebo response in clinical trials appear to affect the different groups asymmetrically (3,18,19). Another possible cause for these results is that patients who are not good at discerning and reporting their own symptom states demonstrate both higher placebo responses and reduced differences between drug and placebo, attributable to inaccurate reporting of symptom intensity (3). If the root cause is expectation bias, one solution would be to minimize expectation through trainings, such as the Placebo Response Reduction Training (20). If the root cause is inaccurate symptom reporting, one solution to improve subject reporting would be to train subjects in accurate symptom reporting (18). If both expectation bias and poor reporting skills contribute, then both types of training would be beneficial. In general, training can reduce protocol deviations, address patients’ expectations of benefit, and increase accurate symptom reporting, thus increasing assay sensitivity (1).
Another method to minimize the placebo response is to utilize a placebo run-in phase where placebo non-responders are dropped from the study. However, this solution requires a larger sample size, an additional phase in the study, and its generalizability and clinical usefulness and effectiveness has been debated (21). In one preventative migraine study that used a placebo run-in phase in a clinical trial on Botox®, the placebo-subtracted response rate was not affected by this method (22). Sequential parallel comparative design (SPCD) studies consist of two phases where subjects are first randomized to placebo or active treatment, with more subjects randomized to placebo. In the second phase, the placebo non-responders are then randomized again to placebo or active treatment (2). This method may eliminate high placebo responders; however, it is a more complex study design that requires more time to complete and may have similar limitations in generalizability and effectiveness as placebo run-in studies.
It is important to note that in a therapeutic setting, maximizing the placebo effect while treating patients can potentially optimize patients’ therapeutic benefit (22); however, in the clinical trial setting, there is mounting evidence of the detrimental impact that high placebo responses can have on the success of clinical trials. For this reason, it is important to find solutions to decrease placebo response and increase assay sensitivity. Truly effective drugs should not fail to meet their primary endpoint because of the potentially modifiable issue of a high placebo response, or its possible root cause of inaccurate symptom reporting or expectation bias.
Key findings
In clinical trials of preventative treatments for migraine, higher placebo responses were associated with smaller placebo-subtracted response rates, and higher sample size requirements than trials with lower placebo responses. With mounting evidence of the detrimental impact that high placebo responses can have on the success of clinical trials, it is important to find solutions to decrease placebo response and increase assay sensitivity.
Supplemental Material
sj-pdf-1-cep-10.1177_0333102420960020 - Supplemental material for The relation between the placebo response, observed treatment effect, and failure to meet primary endpoint: A systematic review of clinical trials of preventative pharmacological migraine treatments
Supplemental material, sj-pdf-1-cep-10.1177_0333102420960020 for The relation between the placebo response, observed treatment effect, and failure to meet primary endpoint: A systematic review of clinical trials of preventative pharmacological migraine treatments by Kathryn Evans, Heather Romero, Egilius LH Spierings and Nathaniel Katz in Cephalalgia
Supplemental Material
sj-pdf-2-cep-10.1177_0333102420960020 - Supplemental material for The relation between the placebo response, observed treatment effect, and failure to meet primary endpoint: A systematic review of clinical trials of preventative pharmacological migraine treatments
Supplemental material, sj-pdf-2-cep-10.1177_0333102420960020 for The relation between the placebo response, observed treatment effect, and failure to meet primary endpoint: A systematic review of clinical trials of preventative pharmacological migraine treatments by Kathryn Evans, Heather Romero, Egilius LH Spierings and Nathaniel Katz in Cephalalgia
Supplemental Material
sj-pdf-3-cep-10.1177_0333102420960020 - Supplemental material for The relation between the placebo response, observed treatment effect, and failure to meet primary endpoint: A systematic review of clinical trials of preventative pharmacological migraine treatments
Supplemental material, sj-pdf-3-cep-10.1177_0333102420960020 for The relation between the placebo response, observed treatment effect, and failure to meet primary endpoint: A systematic review of clinical trials of preventative pharmacological migraine treatments by Kathryn Evans, Heather Romero, Egilius LH Spierings and Nathaniel Katz in Cephalalgia
Supplemental Material
sj-pdf-4-cep-10.1177_0333102420960020 - Supplemental material for The relation between the placebo response, observed treatment effect, and failure to meet primary endpoint: A systematic review of clinical trials of preventative pharmacological migraine treatments
Supplemental material, sj-pdf-4-cep-10.1177_0333102420960020 for The relation between the placebo response, observed treatment effect, and failure to meet primary endpoint: A systematic review of clinical trials of preventative pharmacological migraine treatments by Kathryn Evans, Heather Romero, Egilius LH Spierings and Nathaniel Katz in Cephalalgia
Footnotes
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
