Abstract
Patients present with specific problems against a background of their particular situation. The applicability of a treatment to a specific case may be difficult to assess in the light of the information available on that treatment. Evidence-based medicine is a discipline developed to achieve the best informed treatment decisions in clinical situations. In this paper, we illustrate a logical approach to reaching a decision regarding which of two migraine treatments should be used in a clearly defined patient. The information evaluated is a comparative trial between sumatriptan (100 mg), rizatriptan (5 and 10 mg) and placebo. The use of the analytical tools of evidence-based medicine is demonstrated in this exercise. The example was chosen because this is an area of great current interest in the field of neurology.
Introduction
The symptomatic treatment of migraine has been transformed by the introduction of a class of compounds known as the triptans. These are agonists at the 5HT1B/1D receptor. All currently marketed triptans cause vasoconstriction in the extracerebral intracranial arteries, reduce neurogenic inflammation by blocking release of vasoactive peptides and, in most cases, they also reduce firing in second-order trigeminal neurones. The first triptan to be marketed, sumatriptan, was subjected to extremely rigorous clinical trials. In fact, more clinical trials were performed with this drug than with all previous drugs for migraine put together. Sumatriptan is available as an oral preparation at a dose of 100, 50 or 25 mg, as a nasal spray at a dose of 5 and 20 mg, and as a subcutaneous injection at a dose of 6 mg (although not all preparations may be available in all countries). Sumatriptan, given by subcutaneous injection, is by far the most effective triptan available for migraine at the present time. However, because the other triptans are not marketed in an injectable form, it cannot be directly compared to them.
Rizatriptan (Maxalt) was developed shortly after sumatriptan but came to market in Canada only in 1999, although it has been available in the USA and some other countries for a longer time period. The question most clinicians have at the present time is which particular triptan should be used in which particular case? In a sense, this is not a very useful question because, as patient responses vary so widely; a general answer may not be individually applicable. Nevertheless, it is important to try and arrive at some sort of consensus concerning the relative merits of the different available compounds.
In this paper, we examine the use of evidence-based medicine principles to try and make sense of the information available to the medical profession.
A recent editorial entitled Medicine Based Evidence: A Prerequisite For Evidence-based Medicine (1) starts off stating ‘seeking an evidence base for medicine is as old as medicine itself, but in the past decade the concept of evidence-based medicine has done a good job in focusing explicit attention on the application of evidence from valid clinical research to clinical practice’. According to this editorial, evidence-based medicine has been defined as ‘conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients’ (2). It then goes on to state that the problems of criteria for internal and external validity, i.e. clinical applicability, may conflict. They then discuss the fact that there may be a selection bias in clinical studies that may not match the type of patients for whom the studied intervention will be considered (1).
Our paper will focus on a particular problem in managing a specific patient in the light of the evidence available from ‘evidence-based medicine.’
Methods
To practice evidence-based medicine clinicians need to follow five main steps: (1) converting the need for information into clinically relevant, answerable questions; (2) efficiently finding the best external evidence with which to answer the question (3); critically appraising the evidence for its validity (closeness to the truth) and its usefulness (clinical applicability) (4); integrating the appraisal with clinical expertise and applying the results to clinical practice; and (5) evaluating your performance (3). Using the clinical scenario of a patient with migraine, we illustrate steps 1–4 of this process. Because step 5 (self-evaluation) is an ongoing exercise, resulting from the practice of evidence-based medicine principles, we will not address it here.
Process
The patient
An otherwise healthy, inquisitive, 35-year-old female accountant has suffered from migraine with aura since childhood. Her attacks have increased in frequency and severity during the past year, and are interfering with work and family life. They are often associated with nausea but she is able to take oral medication. In the past, she obtained partial relief with simple analgesics, ergotamine, nonsteroidal anti-inflammatory drugs and dihydroergotamine nasal spray. Your suggestion of prophylactic medication is not greeted with enthusiasm. After visiting several headache web sites, your patient is interested in trying acute migraine treatment with one of the triptans. She asks you specifically whether sumatriptan or rizatriptan should be tried first and what is the basis for choosing one over the other. You offer to review the subject and discuss it with her during her next appointment.
Formulating a focused question
In order to apply the best external evidence to a clinical problem, one must ask answerable clinical questions. These questions contain three main elements: (1) the patient; (2), the intervention(s); and (3) the outcome. Thus, an answerable question about our patient's problem may be formulated as follows: ‘In a healthy 35-year-old woman with migraine with aura (the patient), how effective is rizatriptan compared to sumatriptan (the intervention) in relieving headache and migraine symptoms acutely (the outcome)?’
Locating the best evidence
Locating the evidence can be timeconsuming, especially if there is abundant literature on the subject. Therefore, it is a good idea to start with the most efficient sources of evidence, such as the Best Evidence® CD-ROM (available from the Canadian Medical Association (http://www.cma.ca) and American College of Physicians (http://www.acponline.org/catalog/electronic)); and the Cochrane Library®. (available from the Canadian Medical Association (http://www.cma.ca) and Update Software Inc (http://www.cochranelibrary.com/)). If necessary, we can search more timeconsuming sources, such as MedLine® or other bibliographic databases. Typically, efficient sources of previously appraised evidence take no more than a few minutes to search. If they contain the information we are seeking, we need go no further. The Best Evidence® CD-ROM contains all the critically appraised topics published in the American College of Physicians (ACP) Journal Club and Evidence-based Medicine journals since 1991 and 1995, respectively. Topics are easy to search and come complete with a clinical comment. The Cochrane Library® CD-ROM contains the Cochrane database of systematic reviews, other critically appraised systematic reviews, and a large database of randomized, controlled trials. Searching is intuitive and efficient. Finally, the MedLine® database can be accessed freely through the PubMed website at http://www.ncbi.nlm.nih.gov/entrez/query.fcgi/. Its ‘Clinical Queries’ section contains validated filters that enhance the sensitivity and specificity of the search, making it more efficient.
The literature search leads us to an article specifically addressing our clinical question (4). We obtain a copy from the library and set out to critically appraise its validity and usefulness.
Critically appraising the evidence
Study overview
This is a double-blind, multicentre, parallel-group, randomized, controlled, therapeutic trial of a single migraine attack in 1268 adults, using either placebo, sumatriptan 100 mg p.o., rizatriptan 5 mg p.o. or rizatriptan 10 mg p.o., as a single dose.
By using published guides to appraise the clinical literature, we can assess the validity of evidence about therapy in individual clinical trials (5, 6). Optimum external evidence has two characteristics: (a) it is valid, i.e. close to the truth and we can trust it; and (b) it is useful and can be applied to our patients.
Is the evidence from this comparison of sumatriptan and rizatriptan valid?
Analysing the ‘methods’ section of the article allows us to answer the following checklist about its validity:
This is important, because nonrandomized trials, even if they have a control group, generally overestimate the efficacy and safety of experimental therapy by about 35% (7). In this study, treatment to sumatriptan or rizatriptan was allocated at random (p. 749).
This refers to taking adequate measures to conceal allocation to study groups from those responsible for assessing patients for entry in the trial. It is important to recognize the difference between biases that result from lack of allocation concealment and those arising from lack of blinding. Allocation concealment helps to prevent selection bias, protecting the randomization sequence until the interventions are given to study participants (7, 8). However, blinding helps protect the randomization sequence after allocation, and cannot always be implemented. Lack of concealment may overestimate benefit of an experimental therapy by about 40%, and sometimes may be more potent than lack of blinding of patients and caregivers (8). Indications of concealed randomization include statements such as ‘central randomization; numbered, opaque, sealed envelopes; numbered or coded containers; drugs prepared by the pharmacy.’ Tfelt-Hansen et al. (4) indicate they used a double-blind, computer-generated (concealed) method (p. 749).
This is important, because patients who are lost to follow-up often differ in important prognostic variables from those that stay in the trial. The larger the number of patients who enter the study and are not accounted for, the larger the chance for bias. Tfelt-Hansen et al. (4) randomized 1268 patients but included only the 1099 that took the medication in the analysis. Thus, 13% of patients are not accounted for. It is possible that the investigators included only patients who actually experienced at least one migraine attack during the study and thus, had an opportunity to try and evaluate the interventions. However, this is not clearly stated. If the latter is correct, then only a negligible proportion of patients (3/1099 or 0.003%) did not provide analysable data (p. 750). Independent inquiry revealed that the nonevaluated patients were indeed randomly distributed (Visser, personal communication)
Also known as ‘intention to treat analysis’ this is relevant, because study patients who do not take their assigned medications or drop out of the study are prognostically different (usually worse) from those that comply. In the study under appraisal, investigators analysed all 1096 patients by intention to treat (p. 750).
As stated earlier under ‘concealment’, blinding protects from bias in ascertaining outcomes, i.e. deciding which patients benefit or are harmed. This study used a triple-dummy method to preserve blinding to any of the four interventions (p. 749).
Clinicians need to be assured that patients were similar in all important prognostic aspects other than the treatment they received, which is allocated at random. Table 1 (p. 751) shows that many important characteristics are equally distributed, except for an average age difference of 2 years between the rizatriptan 10 mg (37 years) and sumatriptan 100 mg (39 years) groups. The clinical significance of this difference is unclear. One dissimilar feature that merits discussion concerns the eligibility criteria. Patients were excluded if they had been previously exposed to rizatriptan, but not if they had previously received sumatriptan. This is addressed in the Discussion section.
Comparison of rizatriptan 10 mg and sumatriptan 100 mg
Absolute risk difference.
Number needed to treat.
Not significantly different.
Number needed to treat with sumatriptan to produce an adverse event in one additional patient as compared to treatment with rizatriptan (number needed to harm). Modified from (4).
Differences in patient care, other than those under scrutiny, can occur in a number of ways and distort the results. This is minimized when the study is blinded, as in this study. In addition, the investigators carefully outlined additional interventions offered to all patients (p. 749).
What are the results and will they help me care for my patient?
In order to apply the evidence, clinicians need to make sense of the results. This requires assessing the outcomes that were measured, the size of the effects, and their precision.
In this study, researchers chose time to pain relief, a clinically relevant measure, as the main outcome. They also analysed a number of secondary outcomes (p. 750).
In addition to statistical significance, clinicians need a notion of the clinical significance of the results. This is often absent from clinical trials results. However, summary measures, such as the number of additional patients one needs to treat (NNT) with a drug to obtain one additional good outcome, are useful estimates of the magnitude of the effect. Precision of the estimate refers to the range of possible values within which the true effect is likely to be found. This is most commonly expressed as 95% confidence intervals (95% CI).
The unadjusted analysis showed no significant difference between sumatriptan and rizatriptan in time to pain relief, expressed as a hazard ratio. This can be understood as the ratio of the ratios of observed to expected numbers of patients reaching an event for rizatriptan and sumatriptan (i.e. a hazard ratio of 1 equals no difference). A secondary, age-adjusted analysis showed a shorter time to pain relief with rizatriptan 10 mg than with sumatriptan (hazard ratio = 1.21; P = 0.032). Unfortunately, there is no indication of how clinically relevant this difference is and 95% CI are not provided. Neither rizatriptan nor sumatriptan were significantly different from placebo during the first half-hour. Other results are shown in Table 1.
This refers to whether study patients are so different from ours that the results cannot be applied at all. Patient selection and Table 1 in the article (4) (pp. 749, 751) do not show any differences that would prevent us from applying the results to our 35-year-old female patient.
The clinically relevant outcomes considered were as follows: time to pain relief through 2 h; pain-free response through 2 h; reduction in functional disability; and relief of nausea at 2 h.
The hazard ratio for time to pain relief for rizatriptan 10 mg vs. sumatriptan 100 mg was 1.17 (95% CIs 0.98–1.39; P = 0.075), which is not statistically significant. However, confirmation that rizatriptan 10 mg achieved migraine relief faster than sumatriptan 100 mg was obtained in the prespecified non-age-adjusted per protocol analysis (P = 0.040; hazard ratio 1.20; 95% CI, 1.01–1.44.) Rizatriptan 10 mg showed a numerically greater response rate for pain relief over sumatriptan 100 mg at each time point up to 2 h, and this was statistically significant at 1 h (37% vs. 28%, P < 0.05). By 4 h, the response rates for rizatriptan 10 mg and sumatriptan 100 mg groups were similar and both were superior to rizatriptan 5 mg. However, escape medications were allowed after 2 h, and this may have confounded the results.
At 2 h, 42% of patients in the rizatriptan 10 mg group were able to ‘function normally’, vs. 33% in the sumatriptan 100 mg group (P < 0.05).
Other clinical outcomes included freedom from nausea at 2 h. More patients were free of nausea at 2 h in both the rizatriptan 10 mg and rizatriptan 5 mg groups, as compared to sumatriptan 100 mg (P < 0.05). The requirement for escape medication at 2 h was similar in the rizatriptan and the sumatriptan groups, and both were superior to placebo.
The 24-h total headache relief measure was not used. This has been recently introduced as an overall measurement of response to medication with headache relief at 2 h and without headache recurrence, or the need for rescue medication within 24 h. Most adverse events were comparable in the rizatriptan 10 mg and sumatriptan 100 mg groups. All adverse events (i.e. any drug-related adverse event) were significantly less frequent in the rizatriptan 10 mg group as compared with the sumatriptan 100 mg group (P < 0.05)
It would be fair to say that most clinically relevant outcomes were considered, although patient preference was not taken into account.
The costs of sumatriptan and rizatriptan are comparable. Both are fairly expensive. Whether they are deemed costeffective is not a question this study set out to answer.
The adverse event profile seemed to favour rizatriptan, as the incidence of drug-related adverse events in the rizatriptan 10 mg group (33%) was lower than in the sumatriptan 100 mg group (41%; P < 0.05). The headache recurrence rates were similar for the rizatriptan 10 mg (35%) and the sumatriptan 100 mg (32%) groups and somewhat higher in the rizatriptan 5 mg (48%) group, compared to 20% for the placebo group.
This paper concerns the evidence-based analysis of a migraine treatment drug comparison trial. We have used the trial by Tfelt-Hansen et al. (4) as an example to demonstrate how we perceive that such an analysis should proceed. Overall, the study stands up well to this sort of scrutiny. As demonstrated in the previous section, the specific questions asked were well answered, rendering the study applicable to our patients and generally applicable to the population at large.
Discussion
In summary, this is a well executed, largely valid study. It certainly satisfies the most important criteria of concealed randomization, blinding, and intention to treat analysis, and clinicians can consider its results as valid (or close to the truth). However, readers need to be aware of the uncertainty introduced by two aspects of the study. The first aspect concerns the inequality of eligibility criteria regarding previous exposure to sumatriptan. When faced with a potential confounder such as this, readers need to estimate in which direction the results would be biased. In this case, they could go in either direction. That is, they may favour rizatriptan if we assume that previous sumatriptan users who had responded poorly to sumatriptan in the past would have been more likely to enter this trial. Alternatively, they may favour sumatriptan if we assume that only those with a favourable response to sumatriptan agreed to participate in the trial, therefore preselecting ‘responders’. Finally, it is conceivable that this inequality may have had no impact on the results, but we do not know. Thus, the validity of this particular aspect of the study is questionable. Secondly, there is insufficient information regarding the number of patients randomized, but not included in the analyses. As discussed above, this may have been a negligible proportion, and these patients seemed to have been equally distributed.
Looking at these results one might assume that rizatriptan would be the treatment of choice for our particular patient. She fulfils the criteria for entry into the study that we are evaluating. If one looks at the results one can see that pain relief at 1 h is more often found in patients who take rizatriptan than sumatriptan while at 2 h the figures begin to even out. However, looking at the more rigid criterion of pain-free at 1 h there is no significant difference between the two. For pain-free at 2 h there does appear to be a statistically significant difference.
At first sight, this study would appear to support the contention that rizatriptan 10 mg was better than sumatriptan 100 mg. In conclusion, the authors state that the ‘increased efficacy of rizatriptan 10 mg does not appear to be accompanied by an increase in overall side-effects relative to sumatriptan 100 mg’ (4) In their comments on this paper, O'Quinn et al. however, contend that the authors' conclusions were not borne out by the data (9). They point out that patient preferences are an important clinical endpoint. Patient preference was apparently collected in the study by Peer Tfelt-Hanson et al. but the data was not reported in the original publication (4). It was subsequently reported that there were no statistically significant differences in the patient satisfaction data between rizatriptan 10 mg and sumatriptan 100 mg in the Tfelt-Hansen study (10). O'Quinn et al. (9) also quoted a study by Goldstein et al. (11) that reported a statistically significant difference in time to pain relief (within 2 h) between rizatriptan 10 mg and sumatriptan 50 mg with hazard ratios of 1.14 (95% CI = 1.00–1.29). They reworked the data of the study by Goldstein et al. (11) to show that the time to pain relief was the same for both drugs when one looked only at patients who actually had relief. Nevertheless, the study we quote here (4) found that rizatriptan had a shorter time to pain relief than sumatriptan in the prespecified non-age-adjusted per protocol analysis.
Overall it is clear that both rizatriptan and sumatriptan are very effective and that both drugs benefit many patients. The small statistical advantage of rizatriptan may be important in some patients, however, as was stated previously, patient preference data did not appear to show a statistically significant difference (10).
Comparing differences in response within one particular study, albeit a fairly well-powered study, has its dangers and pitfalls. For instance, patients may desire other characteristics in their acute migraine medication such as consistency, freedom from adverse events and more nebulous items such as a clear head following ingestion of the medication. It is impossible to use a single comparative study to make a definite statement concerning differences in the benefits of the two medications, given the size of the effects noted.
This study does demonstrate that both treatments are well-tolerated, effective and that further comparative studies using other endpoints, such as patient preference and quality of life, may be necessary in order to evaluate fully the comparative usefulness of each medication.