Abstract
To quantify the placebo response of prophylactic therapy in migraine, a meta-analysis of prophylactic, double-blind, placebo controlled migraine studies was performed. The total analysis included 22 studies testing 19 different products, including 2013 patients, of which 828 were treated with placebo. A reduction in migraine attacks of 50% or more (responders) was seen in 23.5% ± 8.0 (95% C.I. 18.3-28.8%) of the patients in the placebo groups and 45.5% ± 15.5 (95% C.I. 37.4-53.6%) in the active groups. A reduction in migraine attacks of 16.8% ± 12.7 (95% C.I. 10.9-22.6%) was observed in the placebo groups and 41.8% ± 11.7 (95% C.I. 36.9-46.6%) in the active groups. We propose that if the percentage of responders in an open-label prophylactic trial in migraine is above 35-40%, or if a reduction in migraine attack frequency is found of 40% or more, further studies are indicated to determine the prophylactic activity of the drug. In all studies included in this analysis, no placebo response was seen above these limits.
Introduction
In general, drug trials in migraine should be designed double-blind, randomized and placebo-controlled because of the subjective nature of migraine and the high placebo effect, both for trials dealing with the treatment of migraine attacks and with migraine prophylaxis (1). However, clinical observations and small open-label trials may be hypothesis generating for possible acute or prophylactic effects of drugs in migraine. Therefore, it is acceptable to perform an open-label trial in a small number of patients to get a first impression of the efficacy of a drug. If the response of a drug in such pilot studies is well above the response of a placebo, further double-blind, placebo-controlled studies are warranted.
However, the placebo response in an open-label trial is unknown and can only be estimated from placebo-controlled trials in comparable study populations. In migraine attack studies the placebo response varies from 6% (2) to 44% (3), depending on patient characteristics and the design of the trial. To our knowledge the placebo response in prophylactic trials has never been studied systematically. Therefore, we performed a meta-analysis to quantify the placebo response in prophylactic trials in migraine. The ultimate goal was to develop a frame of reference for new drugs to be tested for prophylactic activity in migraine.
Methods
Literature search
Three different search strategies were used to identify potentially relevant trials. First, a search was performed in Medline for the years 1984–98. Secondly, the electronic version of the headache dedicated journals Cephalalgia (1982–97) and Headache (1961–97) were searched for full papers and abstracts on drug trials dealing with migraine prophylaxis. Thirdly, Cephalalgia (1997–98) and Headache (1997–98) were searched by hand as these were not yet available in electronic versions. Also the references of these publications were searched for additional prophylactic studies. No attempts were undertaken to identify unpublished studies.
Study selection
Criteria for including a trial in the analysis were:
A double blind setup.
A placebo-controlled design.
A run-in period after which the trial medication was started.
Results expressed (or could be calculated from the data presented) as at least one of the following parameters compared with baseline: percentage of patients with 50% or more reduction in attack frequency during treatment compared with the baseline period (referred to as ‘responders’); and/or percentage of reduction in migraine attack frequency (MAF).
Prophylactic studies reporting only other parameters, such as headache index, were excluded from the analysis. Trials of migraine in children (under the age of 18), or in very specific types of migraine, e.g. menstrual migraine, were excluded as well.
Analysis
All studies were reviewed independently by two persons, who checked whether the study complied with all inclusion criteria and, if so, extracted the data needed. Disagreement between the reviewers was settled by consensus.
Studies with a crossover design were analysed only for the first part, as a parallel study, unless these data were not given separately.
Two parameters were extracted from the publications: percentage responders and percentage reduction in frequency of migraine attacks. The third parameter, therapeutic gain, was calculated for each study as the difference between the active group and the placebo group for both aforementioned parameters.
If more treatments were compared within one trial, each active group was included separately; the placebo group, however, was included only once. In the first analysis, we determined straightforward for each group, i.e. ‘placebo’ or ‘active’, the mean of the three parameters with the corresponding standard deviation and 95% confidence interval (C.I.) for all trials, regardless of the number of patients included in the trial. The highest value, the lowest value and the total number of studies were determined as well.
To study the robustness of the data three further analyses were performed on the percentage responders and the percentage reduction in frequency of migraine attacks. In the second analysis, the data from all studies in which the efficacy of the test compound was regarded as not different from placebo were joined with the placebo data. In the third analysis, all data from the first analysis were re-analysed by weighing for the number of patients per trial. Weighing for the number of patients per trial was done to exclude any bias that might be introduced when small studies are weighed equally as large studies. A fourth analysis was performed in which only studies were included reporting both parameters, the percentage responders as well as the percentage of reduction in migraine attack frequency.
The standard deviation and 95% confidence interval were calculated by confidence intervals for proportions and differences between two means (4). Data are presented as mean±standard deviation, unless stated otherwise. Where possible the two-sided 95% confidence intervals are given as well.
Results
A total of 22 studies were reviewed in full text as they complied with all inclusion criteria (5–26). The oldest study dated from 1978 (5), eight studies were published in the eighties, while the other studies appeared in journals in the last decade of the 20th century.
The 22 studies represented 28 ‘active regimens’. In three studies different dosages of one compound were compared with placebo (6, 8, 25), while in two studies two different compounds were compared with each other and placebo (7, 22). In total 18 different chemical entities and a homeopathic drug were compared with placebo, of which eight are licensed for the prophylactic treatment of migraine (acebutolol, bisoprolol, fenoprofen, indobrufen, metoprolol, propranolol, timolol and valproic acid).
Four studies had a crossover design (5, 10, 11, 18), while the others applied a parallel methodology.
The total analysis included 2013 patients, of which 828 (30%) were treated with placebo.
The results of the studies are shown in Tables 1 and 2 for the separate parameters.
Studies reporting their results in percentage responders (patients with a reduction of attack frequency of 50% or more)
N-placebo=number of patients in the group treated with placebo; N-active=number of patients in the group treated with aforementioned active compound; P=parallel; CO=crossover.
Studies reporting their results as reduction of migraine attack frequency (MAF)
N-placebo=number of patients in the group treated with placebo; N-active=number of patients in the group treated with aforementioned active compound; P=parallel; CO=crossover.
Five studies were excluded because they described results in paediatric subjects (27–31). In these studies the reduction in migraine attack frequency for the placebo group ranged from 16% (27) to 55% (30). In four of these paediatric studies the placebo response was higher than the response after active medication (27–30). Only flunarizine showed a clinically relevant difference between placebo and the active compound in a paediatric trial (30).
Percentage of patients responding
The data from the studies reporting the percentage of patients showing a response (reduction of attack frequency of 50% or more) are given in Table 1.
The placebo groups of nine studies were included. The mean percentage responders was 23.5%±8.0. The lowest value observed was 14% (26) and the highest value 34.4% (17). The aggregated data are shown in Table 3.
Aggregated data of the four analyses (see text): comparison of the two parameters for the placebo group and the active group. Results expressed as mean±standard deviation and 95% confidence interval
Fourteen ‘active’ regimens (nine studies) were included (7, 8, 11, 17, 22–26). The mean percentage responders was 45.5%±15.5. The lowest value found was 22% (fenoprofen 200 mg) (8), while the highest value encountered was 77.8% (timolol) (22).
Frequency of attacks
The data from the studies reporting the reduction of attack frequency are given in Table 2.
For the placebo arm 18 studies were included. The mean frequency reduction observed after treatment with placebo is 16.8%±12.7. The lowest value appeared to be a negative response of −6.8%, i.e. an increase in the migraine frequency of 6.8% (21). The highest value noted after placebo treatment was 38.6% (9). These results are summarized in Table 3.
From 18 publications 22 different ‘active’ regimens were extracted for the analysis of the total reduction in migraine frequency (5, 6, 9–19, 20–22, 25, 26). The mean reduction found in these studies was 41.8%±11.7. The lowest value was 11% (lamotrigine) (15), the highest value 57.9% (timolol) (22).
Robustness
The results from the four analyses are given in Table 3 and Fig. 1. In general, no significant changes were seen for the means of the placebo group in the four different analyses. With respect to the active group, it is seen that the second analysis gives a slightly higher result only with respect to the percentage responders. As this was not seen in the placebo group, it can be concluded that the data from the placebo group are fairly robust. With respect to the third analysis, it can be concluded that no bias is introduced by weighing small studies as equal with large studies. The fourth analysis included five studies, reporting their results both as percentage of responders and as reduction of the migraine attack frequency (11, 17, 22, 25, 26). The means of the placebo group were similar compared with the other analyses, but the standard deviations and 95% confidence intervals were higher (Table 3).
Graphical display of the four analyses (see text): comparison of the two parameters for the placebo group and the active group. Results are expressed as mean±standard deviation.
Discussion
In order to interpret the results of drugs tested for prophylactic activity in migraine in open studies, we analysed the (placebo) responses in double-blind studies. It should be emphasized that the placebo response in double-blind studies may differ from the response in open studies. It can even be argued that the placebo response in open trials may be higher. These limitations should be taken into account when interpreting these results.
Because paediatric studies were excluded the results of this study do not apply for studies reporting results in children. In general, placebo responses in these studies are considerably higher than in adult studies. In this study the results of 22 different trials, published in a time range from 1978 to 1996, have been combined.
In the older studies IHS diagnostic criteria for migraine were not used, possibly leading to differences in study populations. Also other factors may contribute to the heterogeneity of patient populations. This should be accounted for in interpreting the results of this study.
As one of the parameters to study, we haven chosen the percentage of patients showing a reduction of attack frequency of 50% or more, because this parameter is widely used in these types of trials. These limits are strictly arbitrary, however. Many subtle changes might be made in this approach. Certain patients may welcome even a reduction of 30% in their migraine frequency. Steiner and colleagues have suggested that a difference between treatment groups of 1.5 attacks per month is clinically significant (15).
Data were re-analysed by weighing for the number of patients per trial to exclude bias introduced by small studies (third analysis). However, one should realize that large studies are not by definition the best trials. Instead, one could argue that the quality of the trial should be assessed by simple criteria. Consequently, a weight factor could have been introduced to correct for the quality of the trial (32). This only seems necessary if a large variability is seen in the results of the different trials. The variability of the placebo response in this analysis was, however, relatively small.
This analysis was performed to create a frame of reference for new drugs tested for prophylactic activity.
The first question to address is which parameter seems the most suitable to distinguish between true prophylactic activity and placebo response. The parameter should have a high discriminating power and therefore the coefficient of variation should be low. As can be seen from Table 3 (first analysis), the coefficients of variation (placebo group) are 34% and 76% for, respectively, the percentage responders and the percentage reduction in migraine frequency. If, however, only those studies are used that present both parameters (fourth analysis), the difference in coefficient of variation is small (44% and 52%, respectively). We conclude that both parameters can be used to distinguish between true prophylactic activity and placebo response.
The second question to address is what threshold should be used to determine whether a drug is effective enough for further studies or not. This threshold depends on the ultimate goal of a research project. If one wants certainty on the efficacy of a drug, the threshold may be set high. However, it should be realized that the definite evidence can only be obtained in double-blind, placebo-controlled, randomized studies.
If one looks for a new kind of drug, the threshold may be set lower. The development of a new drug is a costly and time-consuming matter, therefore one should not reject a new drug unnecessarily. There may always be a subpopulation that displays good results, which deserves further attention. We propose that the high threshold should be set at the mean plus two times the standard deviation, while the lower threshold may be set at the mean value itself. If the percentage of responders in an open-label prophylactic trial in migraine is above 35–40%, or if a reduction in migraine attack frequency of 40% or more is observed, further studies are indicated to determine the activity of the drug as a prophylactic anti-migraine drug. If the percentage of responders is 23.5% or lower or if the reduction in migraine attack frequency is 16% or lower, the drug probably has no prophylactic activity of any clinical relevance. In all the studies included in this analysis, no placebo response was seen above 35% for the percentage of responders and 40% for the reduction in migraine attack frequency. This suggests that the chance that a promising effect appears to be a placebo response is small.