Validity and Reliability of the Migraine-Treatment Optimization Questionnaire

Abstract

Our aim was to establish the validity and reliability of a patient-rated Migraine Treatment Optimization Questionnaire (M-TOQ in primary care. Patients who met International Classification of Headache Disorders, 2nd edn criteria for migraine completed a 19-item questionnaire containing candidate items for the M-TOQ and three questionnaires designed to test convergent/construct validity [Migraine Disability Assessment Scale (MIDAS), Headache Impact Test (HIT)-6 and Migraine-Specific Quality of Life Scale (MSSQoL)]. A 15-item (M-TOQ15) and a five-item (M-TOQ-5) questionnaire were derived. Two hundred and fifty-three adult patients were recruited. Five treatment optimization domains were identified: functioning, rapid relief, consistency of relief, risk of recurrence and tolerability; with Cronbach alphas of 0.70-0.84. The Cronbach α for M-TOQ-15 was 0.85, and it correlated well with MIDAS, HIT-6 and MSQoL (r = 0.33-0.44). The Cronbach α for M-TOQ-5 was 0.66, and it also correlated well with the three questionnaires (r = 0.33-0.41). The utility of the M-TOQ for assessing treatment benefit in research (M-TOQ-5) and primary care (M-TOQ-5) should be further validated.

Introduction

Migraine is a highly prevalent neurological disorder affecting approximately 15–18% of women and 6% of men (1–3). Several classes of treatment are currently available to treat acute migraine attacks, including analgesics (e.g. paracetamol, aspirin, non-steroidal anti-inflammatory agents, combination products and opiates), ergot derivatives and triptans. Treatment guidelines generally recommend choosing an acute treatment based on attack-related impairment, with simple analgesics prescribed as initial treatment for less disabling headaches, and triptans for more severe or disabling migraines (4).

The gap between evidence-based treatment guidelines and actual clinical practice continues to be large (5). Most migraine sufferers take an over-the-counter (OTC) medication rather than prescription medication for their headaches (2). Those who take prescription medications may not receive optimal treatment. Physicians need to optimize treatment on an individual basis; a simple decision tool, based on treatment guidelines, might facilitate treatment optimization.

Determining whether treatment is optimized is complicated. Although headache specialists often employ detailed headache diaries, this approach is difficult in busy primary care settings (6). Treatment satisfaction has been put forward as a multidimensional global measure combining patient perception of efficacy, tolerability, ease of use and overall value into one simple index. However, treatment satisfaction is determined partly by patient expectations: if expectations are low, a patient may be satisfied with suboptimal treatment, and if expectations are too high patients may be dissatisfied with optimal treatment (7). Treatment optimization should, therefore, be defined by the achievement of realistic goals. A validated treatment optimization questionnaire could provide rapid and accurate assessment of migraine therapy. We sought to develop and validate such an instrument—the Migraine-Treatment Optimization Questionnaire (M-TOQ). Two previous instruments have been developed to facilitate treatment optimization. One was developed in French based on expert consensus without formal psychometric validation (8); the other was developed primarily in English using a clinic-based validation process and provided a foundation for the present study (9).

The current study was designed to establish the validity and reliability of the M-TOQ. The goal was to develop a long-form version for use in research and short-form version for use in clinical practice. We sought to validate the instrument in five languages (English, French, German, Italian and Spanish) and to assess the validity and reliability of the instrument.

Methods

Men and women aged ≥ 18 years making an individual out-patient visit with their primary care physician were eligible for study enrolment if they met International Classification of Headache Disorders, 2nd edn criteria for migraine headache with or without aura (10); reported at least one migraine headache per month in the past 3 months; and reported no change in treatment taken for their migraine in the past 3 months. Patients were excluded from the study if they suffered frequent non-migrainous headache (more than six per month); met Diagnostic and Statistical Manual of Mental Disorders, 4th edn criteria in the past year for alcohol or drug dependence or abuse; or reported any medical condition (including any neurological illness, except for migraine, or psychiatric illness) or had used concomitant medications in the previous 3 months that might interfere with the assessment of efficacy or tolerability of current migraine therapy.

Patients were recruited at a total of 25 primary care centres in Canada, France, Germany, Italy, Spain and the USA, and provided written informed consent. International Review Board approval was obtained at all sites, and the study was conducted in accordance with the Declaration of Helsinki.

Item selection for the M-TOQ

The item pool for the long-form version of the M-TOQ was developed through an iterative process. Two study authors reviewed published questionnaires focusing on treatment needs, disability and quality of life in migraine as well as patient preference and satisfaction. Questions from the published literature were organized according to candidate domains shown by previous research to be important for assessing the benefit of acute treatments for migraine headaches. These domains included: ability to function, rapid relief of headache pain, consistency of response, improvement in associated symptoms, prevention of headache recurrence, side-effects and global response. An Expert Panel including neurologists, headache specialists, primary care doctors, epidemiologists, psychiatrists and methodologists prioritized the domains and the candidate questions in each domain.

Study assessment

Patients completed four assessments at their validation visit: (i) the long-form M-TOQ (Fig. 1); (ii) the Migraine Disability Assessment Scale (MIDAS) (11, 12), a subject-rated five-item scale that measures headache-related disability over a 3-month period; (iii) the Headache Impact Test (HIT-6) (13), a subject-rated six-item scale that measures the overall functional and quality of life impact of migraine across multiple domains including pain, social and role functioning, energy level, and cognitive and mental health functioning; and (iv) the Migraine-Specific Quality of Life Scale (MSQoL) (14), a subject-rated 20-item scale that measures perceived quality of life across multiple domains, including physical, emotional and social functioning, general health, mental health, vitality and pain. All questionnaires were completed using a computer touch-screen system.

Figure 1

The M-TOQ-19. The five questions in bold represent the items on the M-TOQ-5.

Subjects from one site in each country participated in a test–retest reliability assessment. This subgroup completed a paper-and-pencil version of the M-TOQ in addition to the touch-screen version at the screening session, and then completed a second paper-and-pencil copy of the M-TOQ approximately 7–10 days after their initial visit. Subjects were also asked to note whether they experienced one or more additional migraine attacks or changed migraine treatment during the 7–10-day retest interval.

Language of the questionnaires

All questionnaires were administered in the primary language of the country where each study site was located. The M-TOQ was linguistically validated based on translation and back-translation. Ease of comprehension of the questionnaires were evaluated by focus groups conducted in each country. Based on the results of the focus groups, the language of individual questions was revised, as needed. Previously developed, linguistically validated versions of the MIDAS, HIT-6 and MSQoL were also used.

Statistical analysis

Data were prepared for statistical analysis by resolving any invalid codes, inconsistencies in response patterns, and outliers (identified based on examination of frequency distributions of all variables), and normalizing any non-normal continuous variables by log transformation. To guard against influential data points, non-parametric statistics were used to confirm parametric results. No differences between parametric and non-parametric analyses were observed, nor were any serious outliers identified.

Given the exploratory nature of the research and to reduce the item pool to a small scale useful for clinical practice, exploratory factor analysis was utilized. All items were examined for evidence of lack of variation, and the mean sampling adequacy (MSA) of each item was then examined. Scree plots were created to view graphically the number of likely factors. An Eigen value of 1.0 was set as the minimum to extract a factor. Two rotation methods were used: orthogonal and oblique. Values ≥ 0.40 were considered to represent a clear loading on a factor, and values ≥ 0.35 but < 0.40 borderline loading. Loadings of ≥ 0.40 on two or more factors were considered to represent straddling. Cronbach's alphas were calculated for identified factors.

For subscale development of the TOQ, whether the scale α increased with the omission of any single item was also examined. Decisions to omit or include specific items were based on the change in the α and weak item-to-total correlations along with theoretical considerations. The degree to which all subscales of the TOQ correlated with each other was examined by computing Pearson and Spearman correlations.

To provide external validation of the TOQ, convergent validity was tested by computing Pearson and Spearman correlations of the long-form M-TOQ with three widely used migraine scales: the MIDAS, HIT-6 and MSQoL. To evaluate test–retest reliability of the M-TOQ, κ statistics were computed for the individual dichotomous items of the TOQ (touch-screen vs. paper-and-pencil answers). Paired t-tests were also used to examine whether any shift in location of the means occurred over time due to method of administration. SAS Version 8.2 was used for all analyses (SAS Inc., Cary, NC, USA).

Results

Two hundred and fifty-three patients met study eligibility criteria and constituted the validation sample with approximately 50 patients in each of the five language groups (range 40–71). The demographic and clinical characteristics of the validation sample are summarized in Table 1. Patients were typically women (90.1%), with chronic headaches of moderate-to-severe intensity (92.7%).

Table 1

Demographic and clinical characteristics of the validation sample

	N = 253
Age, years, mean ±s.d.	43.1 ± 12.4
Women, %	90.1
Country, n (%)
Canada∗	20 (7.9)
France	40 (15.8)
Germany	60 (23.7)
Italy	41 (16.2)
Spain	41 (16.2)
USA∗	51 (20.1)
Marital status, %
Currently married/living as married	70.2
Separated or divorced	7.1
Widowed	3.2
Never married	19.4
Employment status, %
Full-time	60.3
Part-time	13.5
Homemaker	22.6
Student	3.6
Headache duration, %
< 5 years	15.5
5–10 years	15.9
> 10 years	68.5
Headache intensity (typical), %
Mild	7.3
Moderate	48.4
Severe	44.3
International Headache Society diagnosis, %
Migraine with aura	9.9
Migraine with or without aura	64.4
Migraine without aura	25.7
Duration of typical migraine, %
< 4 h	12.4
4–12 h	48.0
> 12 h	39.6
Missed work in past 3 months due to migraine, %	38.5
Visited emergency room in past month due to migraine, %	3.2
Previously diagnosed as having migraine, %	86.2

^∗The Canadian and US sites constituted the English language validation sample (n = 71).

Factor analysis

The overall MSA for the M-TOQ-19 was 0.84. The MSA of individual variables showed satisfactory coverage (MSA ≥ 0.65) for all variables except item 17 (Do you have troublesome side-effects after using your current migraine medication?; Fig. 1), which was borderline at 0.61. Using Scree plots and by the Eigen value criteria (≥ 1.0), we identified five factors (consistency, functioning, recurrence, side-effects and pain relief). Whereas the orthogonal and oblique rotations yielded very similar solutions, the oblique rotation yielded better separation of variables (clearer loadings). The five resulting factors closely resembled the original hypothesized factors (Table 2).

Table 2

Rotated factor pattern of the M-TOQ-19: results for oblique rotation (N = 253)

Original hypothesized factor	M-TOQ-19 Item no.	Factor 1: Consistency	Factor 2: Functioning	Factor 3: Recurrence	Factor 4: Side-effects	Factor 5: Pain relief
Functioning	1	−0.042	0.869	−0.069	−0.048	0.102
	2	−0.026	0.928	−0.100	−0.049	0.013
	3	−0.125	0.801	−0.014	−0.017	0.188
Rapid relief of headache	4	−0.176	0.065	0.030	0.172	0.706
	5	0.355	0.111	0.062	−0.024	0.539
	6	0.310	0.059	0.083	0.030	0.601
	7	0.598	0.009	0.009	−0.048	0.321
Consistency of response	8	0.486	0.151	0.122	−0.075	0.259
	9	0.701	0.092	0.058	0.080	−0.212
Associated symptoms	10	0.897	−0.217	−0.119	0.017	−0.055
	11	0.821	−0.004	−0.135	−0.093	0.159
Prevention of recurrence	12	−0.102	−0.053	0.927	−0.076	−0.025
	13	−0.022	−0.006	0.811	0.047	0.067
	14	−0.005	−0.078	0.903	−0.041	0.114
Side-effects	15	−0.098	−0.054	−0.009	0.863	0.041
	16	0.235	−0.016	0.027	0.736	0.065
	17	−0.079	−0.001	−0.077	0.897	0.112
	Global response	18	0.324	0.448	0.229	0.118	−0.378
19		0.368	0.330	0.196	0.160	−0.026

Item reduction

A primary goal of scale development was to reduce the item pool to a manageable number (three to five items) without sacrificing the validity of the scale. As a first step, items were identified that could be omitted without resulting in substantial reductions in Cronbach's α levels of the subscales. For domains of recurrence (α= 0.84), side-effects (α= 0.77) and rapid relief (α= 0.70), deleting any single item resulted in substantial decrements in Cronbach's α for that domain; thus, no items for these domains were deleted. For the ‘function’ domain the overall α with all four items was 0.79. Omitting item 18 (Are you comfortable enough with your migraine medication to be able to plan your daily activities?; Fig. 1) increased the α to 0.84. This item, originally conceptualized as a ‘global’ measure, fitted the factor less well than the others. The consistency factor comprised six items. Deleting three items resulted in a three-item factor with an α of 0.80, which is comparable to the α of 0.83 for the original six-item factor. Thus the three-item factor was preferable.

This analysis delineated a 15-item version of the M-TOQ (M-TOQ-15), consisting of the following factors and items: functioning (items 1–3), rapid relief of headache pain (items 4–6), consistency of response (items 7, 8 and 11), absence of headache recurrence (items 12–14) and side-effects (items 15–17). Although the intradomain correlations (Cronbach's alphas) were high, the interfactor correlations were low to moderate (r = 0.04–0.52), suggesting that the factors measured distinct clinical dimensions. Cronbach's α for the M-TOQ-19 was 0.88, and for the M-TOQ-15 was 0.85. Thus, elimination of the four items did not reduce the internal consistency of the overall scale or any of its factors.

Although M-TOQ-15 might prove useful as a research instrument, we consider it too long for use in primary care settings. To create a short form, domains were reviewed to identify the single item that best represented the entire domain; this was undertaken by examining item-to-total correlations within each domain. The result was a five-item scale with the following items for each factor: consistency (item 8), functioning (item 3), recurrence (item 14), side-effects (item 16) and rapid relief (item 6). Although item 6 (pain-free) and item 8 (pain relief) loaded on different factors, they are very similar. Therefore, item 6 (pain-free) was omitted from the five items, in part because it was redundant (M-TOQ-4) and was replaced with the global item 18 (M-TOQ-5).

The M-TOQ-4 had a Cronbach's α of 0.59, which was increased to 0.66 with the addition of the global item (M-TOQ-5). Since we selected the single items that are most representative of different domains, and as we chose these domains as they represent different dimensions of a broader concept, it is not suprising that the alphas for the M-TOQ-4 and M-TOQ-5 are modest. However, both shortened versions had high Pearson correlations with the M-TOQ-15 (r = 0.91 and 0.89 for M-TOQ-4 and M-TOQ-5, respectively; both P < 0.0001), suggesting that they were representative of the long-form scale.

Convergent/construct validity

The validity of the M-TOQ-5 was tested by examining Pearson correlations with previously validated scales that assess migraine-related impairment in functioning (MIDAS and HIT-6) and quality of life (MSQoL). Results demonstrating low-to-moderate levels of correlation were considered supportive of M-TOQ-5 validity, suggesting that treatment optimization is related to but independent of HRQoL and disability. Higher correlations (e.g. > 0.70) would suggest that the M-TOQ is not measuring a conceptually distinct domain and, thus, might be redundant.

Table 3 summarizes the mean scores and internal consistency (Cronbach's alphas) for the different versions of the M-TOQ and the three scales used in the validation. Mean M-TOQ-15 and M-TOQ-5 scores were similar for all countries, with no significant differences found using analysis of variance. Cronbach's alphas showed no overall country effect on internal consistency reliability (0.78–0.88 for M-TOQ-15; 0.59–0.76 for M-TOQ-5). These observations support pooling of data from all countries.

Table 3

Mean scores and internal consistency for M-TOQ, MIDAS, HIT-6 and MSQoL scores for the validation sample

Scale	Mean ±s.e. scores (n = 253)	Cronbach's α
MIDAS score	22.3 ± 1.6	0.84
HIT-6 score	62.8 ± 6.1	0.79
MSQoL total score	59.7 ± 1.4	0.93
M-TOQ scores
M-TOQ-19	11.5 ± 0.3	0.88
M-TOQ-15	8.9 ± 0.3	0.85
M-TOQ-15 3-item subscales
Consistency	1.9 ± 0.1	0.80
Recurrence	1.5 ± 0.1	0.84
Functioning	1.6 ± 0.1	0.84
Side-effects	2.4 ± 0.1	0.77
Rapid relief	1.4 ± 0.1	0.70
M-TOQ-5	3.1 ± 0.1	0.66
M-TOQ-4	2.4 ± 0.1	0.59

MSQoL is also known as the MQoL.

Table 4 summarizes the Pearson correlation results for both M-TOQ-15 and M-TOQ-5. Correlations with the MIDAS, HIT-6 and MSQoL range from 0.30 to 0.44, suggesting that the M-TOQ taps into an autonomous outcome domain that is related to functioning and quality of life, but is not redundant with it. Correlation coefficients with the external scales were similar for the M-TOQ-5 and the M-TOQ-15 scales, supporting the assertion that M-TOQ-5 is representative of M-TOQ-15.

Table 4

Convergent validity: Pearson correlations between the M-TOQ and other migraine-related functional and QoL assessments (N = 253)

	MIDAS	Log MIDAS	HIT-6	MSQoL
M-TOQ-15	r = 0.33	r =−0.33	r = 0.44	r = 0.39
M-TOQ-5	r = 0.33	r =−0.30	r = 0.41	r = 0.37

All P-values are significant at P < 0.0001.

Test–retest reliability

Sixty-four subjects completed the M-TOQ-19, twice, using paper and pencil, to evaluate test–retest reliability. The mean (±s.d.) number of days between administrations was 10.5 ± 4.7 (median 9; range 6–28). Test–retest reliability was high for the M-TOQ-19 (Spearman 0.88; Pearson 0.90; intraclass correlation 0.89), the M-TOQ-15 (Spearman 0.87; Pearson 0.88; intraclass correlation 0.88) and M-TOQ-5 (Spearman 0.88; Pearson 0.87; intraclass correlation 0.87, all P-values < 0.001). The test–retest reliability of individual items was also good, with κ values ranging from 0.61 to 0.87 (all P-values < 0.001).

Discussion

The objective of the current study was to develop a valid and reliable patient-centred instrument in several languages that can be used to determine if acute migraine treatment is optimal. We report an instrument with 15 items for use in research, the M-TOQ-15, and a shorter, more user-friendly instrument, the M-TOQ-5, for use in primary care.

Both M-TOQ-15 and M-TOQ-5 have excellent test–retest reliability. Correlations with MIDAS, HIT-6 and MSQoL were similar for the five-item and 15-item versions, and ranged from 0.30 to 0.44, confirming that M-TOQ was tapping into an autonomous outcome domain that was related to disability and health-related quality of life, but was distinct from them. The Pearson correlation between M-TOQ-5 and M-TOQ-15 was high (0.89), suggesting that M-TOQ-5 is representative of the longer scale. These results demonstrate that M-TOQ-15 and M-TOQ-5 are valid and reliable instruments that measure a multidimensional patient-centred outcome related to functioning and quality of life.

In a research setting, we recommend use of M-TOQ-15 as a measure of overall treatment benefit. One point is given for each response of ‘yes’, so a patient on a particular treatment regimen with a score close to 15 is unlikely to require a treatment change. We have not yet determined the magnitude of a clinically meaningful change in M-TOQ-15, although we hope to validate measures in future clinical trials.

In clinical practice, M-TOQ-5 is intended to help physicians assess the adequacy of acute headache treatment and to identify areas where improvements could be made. If all five questions are answered ‘yes’, treatment is satisfactory; an answer of ‘no’ to a single question suggests that a change in treatment should be considered. The change should address the specific limitation of current acute treatment; some examples of optimization strategies are shown in Table 5. These should be viewed as hypotheses for improving care, and should be tested in future studies, informed by the results of clinical trials, switch studies and meta-analyses (15–20).

Table 5

Suggested treatment in patients with a ‘no’ answer to individual M-TOQ-5 questions

M-TOQ item name/number	Question	Suggested treatment optimization strategies∗
Item-3: Functional response	Are you able to return quickly to your normal activities (i.e. work, family, leisure, social activities) after taking your migraine medication?	• Treat earlier
		• Increase dose
		• Switch to medications with greater efficacy (e.g. eletriptan or rizatriptan)
		• Combine medications from different classes (e.g. triptan and NSAID, triptan and metoclopramide)
Item-8: Consistency	Can you count on your migraine medication to relieve your pain within 2 h for most attacks?	• Treat earlier
		• Switch to a drug with evidence of higher consistency of response (e.g. rizatriptan)
Item-14: Recurrence	Does one dose of your migraine medication usually relieve your headache and keep it away for at least 24 h?	• Treat earlier
		• Increase dose
		• Consider a drug with a low recurrence rate or a high 24-h headache response (e.g. eletriptan, frovatriptan)
Item-16: Side-effects	Is your migraine medication well tolerated?	• Treat earlier
		• Reduce dose
		• Consider use of a drug with a more favourable tolerability profile (e.g. almotriptan, naratriptan)
Item-18: Global	Are you comfortable enough with your migraine medication to be able to plan your daily activities?	If patient is satisfied with all four of the above but not overall: discuss expectations and consider behavioural or preventive treatment

^∗Treatment optimization strategies should be based on published results (e.g. comparative studies, switch studies and meta-analyses) and treatment guidelines (15–20).

The present findings are limited by several factors. First, although the study included 253 subjects, there were only approximately 50 subjects per language. Thus, the sample size is relatively modest. We have not yet determined a clinically meaningful difference for clinical trials. Nor have we demonstrated that treatment modification based on the M-TOQ improves patient outcomes relative to usual care. A primary care-based naturalistic treatment study would, therefore, be useful to demonstrate the value and utility of this tool in defining the domains of optimal acute care and improving dialogue between doctors and patients. In the control arm, treatment decisions about medication changes and treatment optimization would be made based on usual clinical practice, whereas in the experimental arm the M-TOQ-5 would be administered, with the results used to identify patients requiring change in their acute treatment in accordance with guidelines. If M-TOQ can then be shown to improve patient outcomes, such a study may increase acceptance and use of M-TOQ in clinical practice. Based on the assessments reported here, we conclude that the utility of M-TOQ for assessing treatment benefit in research (M-TOQ-15)and primary care (M-TOQ-5) warrants further validation.

Competing interests

R.B.L. has acted as a paid consultant, received lecture honoraria from and conducted studies funded by Allergan Inc., Bristol-Myers Squibb, GlaxoSmithKline, Johnson & Johnson, Merck, NeuraLieve, Pfizer and ProEthics. K.K. was paid to conduct the analysis of these data. He has worked in the same capacity for Pozen. M.E.B. has consulted for and conducted studies funded by Advance PCS, Allergan, AstraZeneca, Boehringer Ingelheim, GSK, Merck, Ortho-McNeil, Pfizer and UCB. A.G. has consulted/spoken for Merk-Scharp & Dome, Pfizer, Johnson & Johnson and Almirall. D.V., M.J.A.L., J.P. and G.B. have no financial disclosures. N.I., K.A. and B.P. are employees of Pfizer.