Abstract
We validated a German-language self-administered headache questionnaire for migraine (M), tension-type headache (TTH) and trigeminal autonomic cephalalgia (TAC) in a general population sample of people with headache. Randomly selected subjects (n = 240) diagnosed by the questionnaire as M (n = 60), TTH (n = 60), a combination of M and TTH (M+TTH, n = 60) and TAC (n = 60) were invited for examination by headache specialists. One hundred and ninety-three subjects (80%) were studied. Sensitivity and specificity for M were 0.85 and 0.85, for TTH 0.6 and 0.88, for M+TTH 0.82 and 0.87, respectively. Cohen's κ was 0.6 (95% confidence interval 0.50, 0.71). Of 45 patients with TAC according to the questionnaire, physicians diagnosed cluster headache in two patients only. We conclude: (i) the questionnaire can be used to diagnose M, TTH and M+TTH, but not TAC; (ii) screening questionnaires for epidemiological research should be validated in a general population sample but not in a tertiary headache clinic.
Introduction
Large-scale population-based studies on headache are important to gain information on prevalence and distribution of different headache syndromes. These investigations collect data from thousands of individuals and therefore exclude the possibility of a face-to-face examination. Self-administered questionnaires represent an attractive and inexpensive alternative. These kinds of questionnaires should be short and simple, while the requirements of the questionnaire are to achieve maximum sensitivity, specificity, and positive and negative values.
We recently introduced a self-administered questionnaire for the diagnosis of migraine (M), tension-type headache (TTH) and trigeminal autonomic cephalalgia (TAC) to employ in epidemiological research. For the purpose of validation we recruited 278 patients with idiopathic headache syndromes [M, TTH, a combination of M and TTH (M+TTH), and TAC] from our headache out-patient clinic and compared the questionnaire's diagnoses with neurological examination results of headache experts. We found fairly high sensitivity and specificity values for all four headache syndromes (1). We were aware, however, that patients recruited from a tertiary headache centre introduced a selection bias, and therefore findings could not be extrapolated to the general population.
In our recent study, we investigated people with headache in a general population sample to re-validate the screening questionnaire, comparing the diagnoses results with those of neurologists experienced in headache.
Methods
The study was approved by the ethics committee of the University Duisburg-Essen, Germany. Informed written consent was obtained from all subjects.
Construction of questionnaire
A detailed description of the questionnaire has been provided previously (1). Briefly, the questionnaire was based on the second version of the classification criteria of the International Headache Society [International Classification of Headache Disorders (ICHD)-2] (2). It first explained the principles and general rules for answering, followed by specific questions regarding M (seven items), TTH (seven items) and TAC (six items). The questions were to be answered with ‘yes’ or ‘no’. Furthermore, subjects were interviewed about the number of days associated with the different headache types and the number of intake days of acute pain or migraine drugs per month. The analysis algorithm corresponded to the ICHD-2 criteria as well.
Subjects
Subjects with headache were recruited during the first phase of a population-based survey of the German Headache Consortium, which aims to investigate the prevalence of idiopathic headache syndromes in the general population of Germany. One part of the study population comprised a random sample of 6000 inhabitants of the city of Essen, a town in the Region of North Rhine-Westphalia in the western part of Germany. The city covers an area of 210 360 km2 and has 585 481 inhabitants, 305 726 female and 279 755 male. Inclusion criteria were: age 18–65 years and German citizenship (to ensure proper knowledge of the German language). Subjects received a questionnaire via postal mail and in a case of non-response a reminder 2 weeks later. Individuals who did not respond were called and asked for an interview per phone performed by trained medical students based on the same questionnaire. After eight unsuccessful calls subjects were considered as non-responders. Individuals who refused the interview either by postal response or by phone were also considered as non-responders.
The population-based validation was performed during the first phase of the survey. The response rate was 69%. We selected a random sample of 240 responders who were diagnosed by the questionnaire as M (n = 60), TTH (n = 60), a combination of M and TTH (M+TTH, n = 60) and TAC (n = 60). These subjects were asked to undergo a neurological examination performed by one of authors (M-S.Y., M.O. or M.S.). Neurologists were blinded to the questionnaire's diagnosis.
Statistics
Sensitivity, specificity, positive and negative predictive values were calculated for M, TTH, M+TTH and TAC using physician medical diagnoses as a gold standard. Cohen's κ with 95% confidence interval (CI) was calculated for the overall agreement of physician and questionnaire diagnoses. Data analysis was performed by SPSS 13.0 (SPSS Inc., Chicago, IL, USA) and BiAS 8.0 (3).
Results
Of 240 invited subjects, 47 refused and therefore 193 subjects (80%) were studied, of whom 132 (68%) were women. Mean age was 45.5 ± 12.4 years.
Validity of all questionnaire diagnoses
The questionnaire had diagnosed M in 49 subjects, TTH in 46, M+TTH in 53, and TAC in 45 subjects. Headache experts diagnosed M in 71 cases, TTH in 68, M+TTH in 49, and cluster headache in two cases. In three subjects neurologists diagnosed post-traumatic headache. Table 1 shows the demographic characteristics and distribution of headache syndromes. Table 2 demonstrates the agreement between questionnaire and physician diagnoses for the entire study population as well as for patients with M, TTH and M+TTH, ignoring 45 subjects with the questionnaire diagnosis of TAC.
A and B: demographic and clinical characteristics of the study population
M, migraine; TAC, trigeminal autonomic cephalalgia; TTH, tension-type headache.
Agreement between physician and questionnaire diagnoses for the entire study population
M, migraine; TAC, trigeminal autonomic cephalalgia; TTH, tension-type headache.
An important finding was that of 45 questionnaire diagnoses of TAC, the headache experts confirmed only two. This fact clearly demonstrated that the questionnaire cannot be used for the diagnosis of TAC. We therefore performed a post hoc analysis ignoring 45 subjects with the questionnaire diagnosis of TAC.
Table 3 summarizes the sensitivity, specificity, positive and negative predictive values as well as the corresponding likelihood ratios. The sensitivities and specificities for M, TTH and M+TTH were fairly high. The Cohen's κ coefficient was 0.60 (95% CI 0.50, 0.71).
Sensitivity, specificity, positive and negative predictive values for migraine, TTH, and M+TTH
CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value; M, migraine; TTH, tension-type headache.
Discussion
We re-validated a self-administered questionnaire as a screening instrument for M, TTH and TAC in a population-based sample of patients.
The values for sensitivity, specificity, positive and negative prediction were fairly high for M and TTH, as well as for a combination of M and TTH. The values were quite similar to those observed in the first validation (1). The overall agreement between the questionnaire and physician diagnoses was 0.60, which should be considered as a ‘strong’ agreement level (4).
Overall, the quality of our questionnaire is comparable to the international literature (5–13). The values of sensitivity and specificity were usually higher if the instrument focused on migraine only. Lipton et al. presented a very short screening questionnaire for migraine with only three items and were able to achieve a sensitivity of 0.81 and specificity of 0.75 as well as a κ agreement of 0.68 (9). More detailed migraine questionnaires have been presented by Kallela et al. (7). This migraine-specific instrument obtained a sensitivity of 0.99 and a specificity of 0.96. The κ value for the comparison ‘telephone interview’ vs. ‘clinical examination’ was 0.85. Questionnaires seeking more than one diagnosis resulted in considerably lower agreement levels. Hagen et al. differentiated between migraine, non-migraine headache and chronic headache. κ agreements were 0.59, 0.43 and 0.44, respectively (14). The questionnaire suggested by Rasmussen et al. covered migraine, episodic and chronic TTH. Relatively low κ agreements of 0.43, 0.30 and 0.24, respectively, were reported (4).
Another important finding of the study has been the fact that in a population-based setting the questionnaire revealed a very low specificity in diagnosing TAC. The majority of misdiagnosed patients suffered from migraine with one or more cluster-like autonomic symptoms, which is not unusual (15). Some others had TTH. Future studies will have to show whether the questionnaire could be used as a first screening step to confine the scope on suspected headache cases to be followed by neurological examination. This approach has recently been used by an Italian group (16). The authors reported a high sensitivity of 100%, but a quite low specificity of 34%, and therefore examined all suspected cases personally.
Hence, we consider that the main goal of the study, to re-validate the questionnaire in a population-based setting, was achieved. The results clearly demonstrate that the self-administered questionnaire of the German Headache Consortium can be used in population-based epidemiological studies to assess the prevalence of M, TTH and a combination of M and TTH. At present, the questionnaire is used in a population-based epidemiological study on the prevalence of headache in Germany. Referring data will follow.
Finally, this study has clearly demonstrated that the validation of a self-administered headache questionnaire in patients of tertiary headache centres could be misleading, overestimating the questionnaire's quality even though this first validation step might be necessary for other reasons. The final validation should, however, be done in a population-based sample of headache patients.
Footnotes
Acknowledgements
The study was supported by the German Ministry for Education and Research, Heinemannstrasse 2, 53175 Bonn, Germany.