Abstract
Objective: To determine the minimal clinically important change (MCIC) in Headache Impact Test-6 (HIT-6) score in patients with chronic tension-type headache (CTTH).
Methods: The HIT-6 was administered at baseline and at 8 weeks follow-up in a cohort of 186 participants with CTTH who received manual therapy or usual care by their general practitioner. An anchor-based method was used to determine the MCIC, with the external criterion (anchor) being based on general perceived improvement (‘much improved or very much improved’ on a 7-point Likert scale) in combination with 50% reduction in headache days. Using receiver operating characteristic (ROC) curve analysis we defined an optimal cut-off score discriminating between improved and not improved participants.
Results: The optimal cut-off point for the MCIC for the HIT-6 was −8 points, on a total scale range of 42 points.
Conclusion: A clinically relevant improvement in patients with CTTH is reflected by a decrease of at least 8 points on the HIT-6.
Introduction
In medical research and clinical practice questionnaires may be useful to monitor change over time in patients with frequent headaches. The Headache Impact Test-6 (HIT-6) is a regularly used and recommended questionnaire that consists of six items (pain intensity, social functioning, role functioning, vitality, cognitive functioning and psychological distress) (1). Among a heterogeneous population of headache sufferers the HIT-6 was found to be a reliable and valid instrument (2–6).
Estimating the minimal clinically important change (MCIC) of relevant outcome measures enables a comparison of outcome between interventions on a patient level and can contribute to the relevance and interpretability of change scores. The MCIC provides information about the required change in score on an instrument that reflects changes of health status within a person over time. This is important for the clinician to be able to interpret change scores on an instrument in individual patients. At this moment information about the MCIC on the HIT-6 is lacking.
The objective of this study was to use an anchor-based approach to determine the MCIC on the HIT-6 by comparing change in HIT-6 score with change on an external criterion (anchor), in this study consisting of a combination of a general measure of patient perceived improvement (GPI) and a headache-specific measure (50% reduction in headache days).
Methods
Data were obtained from a multicentre pragmatic randomised clinical trial (RCT) and parallel prospective cohort study of in total 186 participants with CTTH who received manual therapy (MT) or usual care (UC) by their general practitioner over an 8-week treatment period. The RCT is registered with the Dutch Trial Register (nr. TR 1074): its design as well as results have been published previously (7,8).
Participants
Recruitment of patients with CTTH for both studies was performed by 14 general practices in an urban area near Amsterdam, the Netherlands. CTTH was defined according to the classification of the International Headache Society (IHS) (9) as headache occurring on at least 15 days on average per month for a period of more than 3 months. Participants had to be between 18 and 65 years of age.
Exclusion criteria were rheumatoid arthritis, suspected malignancy or brain tumour, pregnancy; intake of triptans, ergotamines or opioids on ≥10 days/month or simple analgesics on ≥15 days/month for ≥3 months. Participants who received manual therapy in the 2 months before the study or were not able to read or write Dutch were also excluded. The study protocol was approved by the Medical Ethics Committee of the VU University Medical Center in Amsterdam, the Netherlands.
Baseline and follow-up measures were available from 182 participants and were used to determine the MCIC (Figure 1). Of these 182 participants, 142 participants received MT treatment, either as participants of a RCT (n = 40) or invited as participants of a prospective cohort study (n = 102) if they provided no consent to randomisation because of a strong preference for MT. The MT intervention consisted of on average seven sessions of 30 minutes including mobilisation of the thoracic and cervical spine, training of craniocervical muscle endurance and posture correction.
Flow diagram: recruitment and retention of participants.
Forty participants received usual care from their general practitioner (in the RCT, these participants served as controls). According to national guidelines for the management of headache the general practitioner provided reassurance and advice about lifestyle and, if necessary, prescribed or changed current medication for headache.
Outcome measurements
All measurements were administered at baseline and directly after the treatment period of 8 weeks, except for general perceived improvement (GPI), which by its nature could not be measured at baseline. The frequency of days with headache and hours with headache per day were measured with a 2-week headache diary (10). Participants were asked to complete this headache diary during a 2-week period before baseline and follow-up measurement.
The HIT-6 consists of six items (2,4). When the HIT-6 was developed these items were selected from headache questionnaires (the migraine-specific quality of life questionnaire Version II, the Headache Disability Inventory, the Headache Impact Questionnaire, the Migraine Disability Assessment Score) and 54 items were analysed in the HIT item pool by Item Response Theory (IRT) methods. Finally, a set of six items were selected representing the six main content areas covered in widely used surveys (pain, social functioning, role functioning, vitality, cognitive functioning, and psychological distress). These items covered more than 50% of the range of headache impact measured by the total item pool, and were among the most valid items in discriminant validity tests involving criterion measures of headache severity, frequency of migraine symptoms and work loss productivity (4).
All six items are scored on an ordinal scale with five response options: never −6 points, rarely −8 points, sometimes −10 points, very often −11 points, always −13 points, with a total score ranging from 36 to 78 points (total scale range 42 points). This score reflects the impact of headache on a patient’s life. A score lower than 49 indicates ‘no or little impact’, 50–55 points indicates ‘some impact’, 55–59 points ‘substantial impact’ and more than 60 points ‘severe impact’. Internal consistency (Cronbach’s alpha 0.89) and test-retest reliability (ICC ranging from 0.78 to 0.90) have been demonstrated to be good (2–4).
GPI was reported by the participant at 8 weeks follow-up on a 7-point Likert scale (0 = very much worse to 6 = very much improved). The frequency of headache days was recorded on the 2-week headache diary.
Definition of anchors
To determine the MCIC on the HIT-6 we used an anchor-based method. This method is similar to that applied in diagnostic studies. The anchor or external criterion is used as a reference standard and includes a clinically relevant cut-off point to differentiate between improved or non-improved patients. The change on the HIT-6 since baseline that corresponds best with the cut-off point on the anchor is calculated. ROC curves are used to visualise the correspondence between index test change scores and the cut-off point on the anchor.
As anchor we chose a combination of GPI (a score of ‘much improved’ or ‘very much improved’) and a headache-specific measure consisting of at least 50% reduction in headache days.
In addition to this combined anchor we used both elements of the anchor separately.
Statistical analysis
Changes in scores on the HIT-6 were calculated by subtracting the baseline values from the values at 8 weeks follow-up. A ROC curve was constructed to describe the performance of changes in scores of the HIT-6 to discriminate between patients who do or do not show important improvement. The area under the curve (AUC) was calculated as a measure of discrimination of change on the HIT-6. For each change score the percentage of false positive and false negative findings was calculated and summarised as total percentage misclassification. The optimal cut-off point (which represents the MCIC on the HIT-6) was estimated by the change score that represents lowest percentage misclassification.
The smallest detectable change (SDC) on the HIT-6 was calculated as 1.96 ×
To allow detection of change in scores over time the initial scores must allow improvement and deterioration in subsequent score. We defined the presence of ‘floor’ and ‘ceiling’ effects as more than 15% of respondent scores being within one SDC from the theoretical minimum or maximum range of the scale (12).
Results
Baseline characteristics participants (n = 186).
On the combined anchor of both GPI and ≥50% reduction in frequency of headache days, 66% of the participants were classified as improved. The optimal cut-off point on the ROC curve (AUC 0.83, 95% CI: 0.77 to 0.89) was −7.5 points. The lowest percentage misclassification was 46% at a cut-point of −8 points (MCIC).
On the second anchor, based only on a score of ‘much improved’ or ‘very much improved’ on the GPI, 70% of the participants (n = 182) were classified as improved. The ROC curve (AUC 0.83, 95% CI: 0.77 to 0.90) showed an optimal cut-point of −4.5 points on the HIT-6 score. The threshold for improvement with the lowest percentage misclassification (44%) on the HIT-6 was −5 points (MCIC).
On the third anchor, based only on ≥50% reduction in headache days, 74% were classified as improved. The ROC curve (AUC 0.82, 95% CI: 0.76 to 0.88) showed an optimal cut-point of −4.5 points on the HIT-6 score. The MCIC was set at −5 points for which the lowest percentage misclassification (45%) was found.
Discussion
Main finding
An anchor-based approach was used to determine the MCIC on the HIT-6 in primary care patients with CTTH. Improvement was defined as ‘much improved or very much improved’ on the GPI Likert scale and ≥ 50% reduction in headache days, which resulted in a MCIC of −8 points on the HIT-6.
Comparison with the literature
The applied combined anchor in our study is strict, requiring a considerable relevant change in headache frequency as well as overall patient perceived improvement in the condition. The type of external criterion and cut-point used in an anchor-based approach is still subject to debate in the literature, but a dichotomised score on a GPI is frequently used as an anchor (11). The drawback of a solitary use of this criterion to define improvement is that the cut-off point on this scale is arbitrary and may not directly relate to headache. On the other hand, using only a headache-specific anchor may not cover the subjective experience of (lack of) recovery. As a consensus reference standard is lacking we decided to use a combined anchor, which incorporated both headache-specific information and the subjective experience of recovery, which in our primary care population classified 66% as improved. A MCIC of 8 points means that improvement needs to occur in at least two domains of the HIT-6. The HIT-6 contains six domains (pain, social functioning, role functioning, vitality, cognitive functioning, and psychological distress) and each domain is represented by one item with a maximum score of 7 points. Hereby, a MCIC of 8 points has face validity: improvement in more than one domain can be seen as an important clinical improvement. As expected, separate analyses based only on the anchor for general improvement or on the headache-specific anchor showed a lower MCIC (5 points). This supports the conclusion of Hays et al. that different anchors can lead to different MCICs (13). On a total scale range of 42 points on the HIT-6 a decrease of 8 points represents a change of 20% or less. Data on the MCIC of other headache-specific questionnaires are not available, but this percentage equals previously reported percentages in change scores on other questionnaires for pain patients with musculoskeletal disorders (14,15). We calculated the SDC to assess whether the MCIC is larger than the measurement error of the HIT-6. The SDC partly depends on sample size (standard deviation of change scores) and in our study only a small sample size (n = 20) was available to calculate the SDC. Therefore we consider 5 points on the HIT-6 to be a conservative estimate of the SDC. The MCIC of 8 points is out of range of the measurement error of the instrument, which means that clinicians can be confident that real change (beyond random error) in impact of headache has occurred if the HIT-6 score improves by 8 points or more.
Strengths and limitations
This study was carried out in a primary care population of patients with CTTH who received either one of two commonly used interventions (manual therapy, usual care by the general practitioner), which means that the MCIC reported in this study should be applicable in primary care. The mean HIT-6 score of our participants at baseline was 62 points and lies in the middle of the scale range of 36 to 78 points. No floor or ceiling effects have occurred and this enhanced the accuracy of our findings. Although we used a strict anchor for improvement, the majority of participants reported improvement, which may reflect the good prognosis in this primary care population. Generalisation of the MCIC value on the HIT-6 to other types of headache is limited, because we restricted the inclusion of participants with headache to the IHS classification of CTTH.
Recommendations and conclusions
We recommend clinicians to consider a decrease of 8 or more points on the HIT-6 as an indication of clinically relevant improvement in patients with CTTH. A consensus reference standard for improvement in headache is not available. Therefore, more research using different anchors and other populations is needed to establish more robust estimates of the minimal important change on the HIT-6.
Footnotes
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Conflict of interest
The authors declare that they have no competing interest.