Abstract
Aim
The aim of this article is to compare 90-day and 30-day recall of Pediatric Migraine Disability Assessment (PedMIDAS) elements and headache frequency against daily entries from an Internet headache diary among pediatric patients and their parents.
Methods
In a prospective cohort study, patients aged 10–18 years with episodic migraine or probable migraine completed a 90-day Internet-based headache diary that incorporated PedMIDAS questions. Following the 90-day diary period, patients and parents completed modified PedMIDAS instruments to assess 90-day and 30-day recall. Intraclass correlation coefficients (ICC) were calculated to measure recall reliability. The Kruskal-Wallis and Jonckheere-Terpstra tests were used to explore recall accuracy as it relates to each participant’s self-reported confidence in recall and to patient age.
Results
Fifty-two subjects completed 90 consecutive diary entries. Comparing 30-day to 90-day recall of PedMIDAS elements, ICC scores improved by 26.2% (patients) and 17.5% (parents). Patients had better recall than their parents for all study measures. Self-reported confidence in recall and patient age had limited and inconsistent effects on recall accuracy.
Conclusion
The optimal recall interval to assess migraine disability must balance recall accuracy with generalizability across a range of headache frequencies. When compared to daily diary entries, recall accuracy of PedMIDAS elements and headache frequency improves at 30 days compared to 90 days. Parent report of migraine disability should not be used as a replacement for patient report.
Introduction
Migraine headaches are common in children and adolescents. The prevalence of pediatric migraine varies between studies and changes with patient age. As many as 10.6% of children aged 5–15 years and up to 28% of adolescents aged 15–19 years have migraines (1,2). When probable migraine is included, the prevalence among children increases considerably (3). The disability caused by migraines can lead to significant impairments in a child’s daily activities and can adversely affect quality of life (4).
The Migraine Disability Assessment (MIDAS) was developed to quantify adult headache-related disability over a three-month period (5,6). The Pediatric Migraine Disability Assessment (PedMIDAS) follows the three-month MIDAS format, but provides a developmentally appropriate clinical tool to estimate migraine disability in the school-aged child (7). The PedMIDAS captures information on disability using patient and parent reports of full and partial school days missed, decreased functioning during school, inability to participate in home and social/recreational activities, and decreased functioning during social/recreational activities. The original validation of the instrument involved parent and child completing the questionnaire together, regardless of the child’s age (7).
The PedMIDAS can be helpful in estimating migraine disability and in monitoring a treatment response in the outpatient clinical setting. Hershey and colleagues demonstrated that the instrument was sensitive to treatment effects in their original study population and suggested that it also may provide a useful outcome measure in pediatric migraine research (7). It has been used in multiple clinical trials since its introduction (8–13). While the MIDAS has been validated against daily entries from a 90-day diary (14), patient and parent recall accuracy of PedMIDAS responses has not been compared against a daily measure. Several factors can affect the reliability of an instrument that measures retrospective recall of episodic migraine disability among children. Factors that may affect the recall of PedMIDAS elements specifically include the three-month recall period, the assumption of patient-parent agreement, and the age of the pediatric patient.
The aim of our study was to compare patient and parent 30- and 90-day recall reliability of PedMIDAS elements and headache frequency against daily headache diary entries. To help accomplish our aim, we developed an Internet headache diary with multiple features including real-time monitoring of diary compliance, date and time stamping of all diary entries, and prevention of incomplete entry submission. In addition to our comparisons of recall at 30 and 90 days, we explored whether patient age or patient or parent self-reported confidence levels in recall helped to predict recall accuracy.
Methods
Overview
We conducted a prospective study of 52 children and adolescents with migraine or probable migraine headaches over an 11-month period of time from December 2011 to October 2012. All subjects completed a 90-day Internet headache diary. At the end of the 90-day period, each patient and parent reported headache frequencies and answered PedMIDAS questions revised to assess 90-day and 30-day recall. Each participant rated their confidence in recall of migraines and migraine disability on a scale ranging from 1 to 5 (1 = very low confidence, 5 = very high confidence).
This study was approved by the institutional review board at Nationwide Children’s Hospital. Informed consent (parents and subjects 18 years of age) and assent (subjects <18 years) were obtained in all cases.
Patient population
All children were referred to our headache clinic or to the outpatient general neurology clinic at Nationwide Children’s Hospital. Prior to recruitment, each patient was interviewed by one of the study investigators (G.L.H or S.C.A) to determine relevant medical history and headache characteristics including frequency, duration, location, quality, prodrome/aura, associated features, and current/prior treatments. A complete neurologic examination was performed by the study investigator or by the primary neurologist on the day of study recruitment. Study inclusion criteria included (1) patient age ranging from 10 to 18 years, (2) English language fluency, (3) home Internet access, (4) active parental email account, (5) a clinical history consistent with episodic migraine (with or without aura) or probable migraine, and (6) headache frequency ≥1 day and ≤15 days per month. Migraine diagnoses were based on International Classification of Headache Disorders, second edition (ICHD-2) criteria (15). Probable migraine was diagnosed when children met all but one migraine criterion and did not fulfill criteria for any other headache syndrome. Migraine patients who also had episodic tension-type headaches were included if migraines occurred ≥1 day per month and the frequency of all combined headaches remained ≤ 15 days per month. We excluded patients with very rare (<1 monthly) and very frequent (>15 monthly) headaches as these extremes could improve recall accuracy. The sample size was chosen empirically (goal of n = 50) to include adequate variations in patient age, headache frequency, and migraine disability.
Internet headache diary
Daily diary entries included 11 questions and a comment section. The first question asked “Did you have a headache today?” In the absence of a headache, headache disability questions 2–8 were automatically blocked and could not be answered. Questions 2–7 followed the PedMIDAS scale (7). Questions 2–4 addressed missing school (Q2), leaving school early or arriving to school late (Q3), and functioning at less than half ability in school (Q4) because of a headache. Question 5 asked if activities at home such as homework or chores were affected because of a headache. Questions 6 and 7 addressed missed participation in social or recreational activities (Q6) and functioning at less than half ability during activities (Q7) because of a headache. In keeping with the PedMIDAS structure, patients could not choose more than one form of disability related to school or recreational activities for any given headache day. For example, if Q1 (“missed school”) was selected, then Q2 (“left school early or arrived to school late”) and Q3 (“functioned at less than half ability in school”) were blocked for that day’s entry. The sum of diary disability reported from questions 2–7 over 90 days provided daily scores to compare against the respective PedMIDAS scores. Question 8 provided a headache intensity rating scale that ranged from 1 to 10. Questions 9–11 addressed medicine compliance. A comment section allowed participants to list specific headache features and to describe headache disability from missed diary entries. All questions needed to be completed (or blocked) before the daily entry could be submitted. Once submitted, entries could not be altered or reviewed by subjects. Diary entries could not be submitted after midnight for the given 24-hour calendar day. Study investigators had an administrative login feature that allowed review of all daily diary entries upon submission.
Study protocol
Each family received the headache diary URL (Internet address), an instructional packet, a study contact email, and a unique diary username and password. Parents were encouraged to remind their children to log on daily, but were asked not to complete any diary entries themselves. During the study period all diary entries were reviewed by a study investigator or coordinator on a daily basis, and email reminders were sent for each missed day. Patients were able to complete missed diary entries by responding directly to the email reminder or by using the comment section of the next diary entry. Families were contacted by telephone if the description of the missed entry was incomplete or if five consecutive entries were missed. Except in cases of planned family holiday or Internet absence, patients missing more than seven consecutive diary entries were removed from the study. Parents and patients (with parent copied) were allowed to email the study staff for any headache-related questions or concerns throughout the study period. This form of easy family-clinician communication served as our primary strategy for subject retention.
PedMIDAS testing
At the conclusion of the 90-day study period, each patient and at least one parent (for subjects <18 years) was asked to complete a study questionnaire. We made every effort to reduce the expected positive effect that diary participation would have on recall. We did not ask patients or parents to complete a PedMIDAS prior to starting the diary because the anticipation of a follow up PedMIDAS at study conclusion could lead to improved awareness of the daily disability elements entered in the diary and improved recall. Families were told during recruitment that they would be asked questions about “participation in an Internet headache diary” at the end of the 90-day period. The topics of the questionnaire and the focus on recall accuracy were not shared until diary completion. Additionally, patients and their parents were not able to review prior diary entries (described above) as this could improve recall.
The questionnaire included the standard PedMIDAS, a PedMIDAS questionnaire modified to assess 30-day recall, and questions about headache frequency over 90 and 30 days. Participants rated their confidence in recall of headache frequency and headache-related disability on a scale of 1–5 for each of the 90- and 30-day intervals. Patients and parents completed the forms on their own, in separate rooms. If two parents were present, their results were averaged. Forms were reviewed for completeness, and participants were asked to fill in missing data prior to collection. Notably, the original validation of the PedMIDAS allowed parent and child to complete the instrument together, regardless of the child’s age. Our methodology is a deviation from this and has not been validated.
Statistics
All statistical analyses were performed using SPSS Version 19 (SPSS Inc, Chicago, IL, USA). Descriptive statistics were calculated for the patient population and for Internet diary compliance. PedMIDAS scores and total reported headaches were summed over 90 days and 30 days from the diary and from the patient and parent questionnaires. We used the intraclass correlation coefficient (ICC), two-way mixed model with absolute agreement, to compare diary scores with patient/parent recall. The ICC is a statistical measure of reliability. Differences in reliability between 90- and 30-day results were calculated as the percentage change between ICC scores.
Recall error, calculated for each patient/parent dataset, was defined as the diary score minus the corresponding patient/parent questionnaire score. The nonparametric Kruskal-Wallis test (KWT) was used to establish whether the absolute values of errors differed across patient/parent confidence ratings or across subject ages. The nonparametric Jonckheere-Terpstra test (JTT) was used to explore trends in the error data based on confidence and age. Significance was set at the 5% level.
Results
Patient, headache, and diary compliance characteristics.
ICC scores quantify the agreement between diary results and patient (n = 52) and parent (n = 50) recall of 90-day and 30-day headache frequency and PedMIDAS elements (Table 2). Patients had better recall than their parents for all study measures. ICC scores for 90-day recall of PedMIDAS elements range from 0.65 (patients) to 0.57 (parents); 90-day recall of headache frequency ranges from 0.83 (patients) and 0.41 (parents). Recall reliability improved for patients and parents when comparing 30-day recall to 90-day recall. The percentage change in ICC scores for 30-day patient recall of PedMIDAS elements increased by 26.2% (Table 2). Similarly, parent recall of PedMIDAS elements improved by 17.5%. Bland-Altman plots (16) illustrate the reductions in PedMIDAS error scores at 30 days compared to 90 days (Figure 1). Recall of headache frequency also improved for patients and parents by 6% and 29.2%, respectively. Patients and parents generally had higher scores (greater disability) on the retrospective PedMIDAS questionnaires than the daily diary entries for both 90-day and 30-day recall periods. Mean headache frequency from diary entries was 0.2 headaches per day at 90 days (range one to 47 headaches over 90 days) and 0.17 headaches per day at 30 days (range 0–16 headaches over 30 days). The lower headache frequency at 30 days may reflect a treatment effect.
Bland-Altman plots comparing Pediatric Migraine Disability Assessment (PedMIDAS) error scores (diary scores minus questionnaire scores) across mean scores for each patient and parent participant at 90 days and 30 days. The dashed lines represent zero error. Solid lines represent 95% upper and lower limits of agreement. The 95% limits of agreement for patient scores at 90 days were 17 and −22.8; parent scores at 90 days were 23.6 and −33.6; patient scores at 30 days were 5.5 and −6.1; parent scores at 30 days were 7.3 and −9.3. Note that the range in error values (y axis) differs between graphs. Recall reliability at 90 days compared to 30 days. aPercentage change in ICC scores calculated as [ICC30−ICC90/ICC90] × 100 for 30-day data and [ICCgrouped−ICC90/ICC90] × 100 for grouped data. ICC: intraclass correlation coefficient; CI: confidence interval; PedMIDAS: Pediatric Migraine Disability Assessment.
The error between diary scores and questionnaire scores was converted to absolute values and compared across subject ages and, separately, across each participant’s reported level of confidence. Box-and-whisker plots (Figure 2) illustrate the magnitude of patient and parent errors relative to confidence ratings. Results from the KWT and the JTT are summarized in Table 3. The reported confidence levels among parents did not influence recall accuracy. The KWT showed a statistical difference between errors across the range of confidence levels only for the 30-day patient headache ratings (p = 0.04). The JTT results suggest a statistical trend of less error with greater self-reported confidence for patient recall of 90-day headache disability (p = 0.006) and 90-day headache frequency (p = 0.04). However, similar trends were not seen for 30-day patient recall or for any measure of parent recall. Recall errors statistically differed relative to patient age for 30-day patient recall of disability (p = 0.05). Recall accuracy was not influenced by patient age in any other patient or parent dataset.
Box-and-whisker plots comparing absolute values of Pediatric Migraine Disability Assessment (PedMIDAS) error scores across patient and parent self-reported levels of confidence in their recall of migraine disability for 90 days and for 30 days. Confidence levels ranged from 1 (very low confidence) to 5 (very high confidence). Only the 90-day patient data showed statistical trends toward less error with higher confidence. Comparison of diary error with confidence ratings and patient age. Significant, p ≤ 0.05. K-W: Kruskal-Wallis test; J-T: Jonckheere-Terpstra test; PedMIDAS: Pediatric Migraine Disability Assessment.
Discussion
We developed an Internet headache diary and used it to compare recall reliability of headache frequency and disability among pediatric migraine patients and their parents. Several studies have used daily headache diaries to assess recall of migraine disability (14,17–19). It is imperative in such studies that the diary accurately measures daily headaches and associated disability over the given study period. Stone and colleagues evaluated diary compliance among adult pain patients using a paper diary that electronically tracked opening and closing of the diary binder (20). Their subjects reported diary compliance rates of 90.5%, but had a very low actual compliance rate of 10.9% based on the diary tracking feature. Seventy-five percent of subjects had at least one day on which the diary was not opened, but they had reported diary compliance for that day. Our ability to monitor daily diary entries in real time allowed us to measure compliance directly and to ensure that missed entries were completed in a reasonable period of time through email and telephone contact. Recall studies that use a headache diary but do not incorporate some form of date verification risk a loss of accuracy as entries placed distant from the measured event may be completed in error as a consequence of inadequate recall. In our study, 68.7% of diary entries were submitted during the 24-hour calendar day, and less than 11% of entries exceeded 72 hours’ delay (Table 1). Furthermore, our diary design prevented the submission of incomplete diary entries. Families were contacted by telephone to update any missing information when late entries were completed by email or through the diary comment section. Our frequent email and telephone contact likely enhanced overall diary compliance.
Both patient and parent recall reliability improved when comparing 30-day recall of PedMIDAS elements to 90-day recall. We consider our study to be exploratory because the 30-day recall period, the separate measures for parent and child, and our Internet headache diary have not been validated. The notion that participants would have a better memory of events over the past 30 days compared to events over the past 90 days may seem self-evident. Interestingly, eight parents and three patients reported higher confidence ratings for the 90-day period. However, their higher confidence for the longer recall period was not reflected in the magnitudes of their respective recall errors. It is possible that active participation with the diary artificially improved accuracy on the PedMIDAS questionnaire and that this effect would be stronger over the 30-day period than over the 90-day period. Our diary design prevented the review of prior diary entries, a feature meant to hamper the “study” of past headaches. Also, we did not administer an initial PedMIDAS prior to starting the diary because we thought it could contribute to this effect. Had active diary participation been the sole cause of recall improvement, we would not have expected similar improvements in parental scores. Patients with <1 headache and >15 headaches monthly were excluded from the study because we wanted to prevent undue bias of disability recall with very low and very high relative headache frequencies.
The ideal recall period for a measure of migraine disability optimizes recall accuracy while providing an adequate time frame to capture infrequent migraine attacks. There are clinical situations for which a three-month scale would be preferable, particularly when patients have migraines that occur less than once monthly. The three-month PedMIDAS provides the clinician with a reasonable tool to assess trends in migraine disability over clinical follow-up periods that often exceed one month. However, clinical trials that rely on accurate patient self-report of migraine frequency and disability at different stages of intervention might benefit from an instrument with a shorter recall period. When therapeutic effects are monitored over periods greater than one month or headache frequencies are expected to be lower than once monthly, an instrument with 30-day recall could be administered repeatedly at monthly intervals. We encourage further study of the validity of shorter recall periods for migraine disability assessment in children and adolescents.
Patients had better recall of headache disability and headache frequency than their parents for every study measure. When assessing retrospective headache frequency and headache-related disability, patient-parent agreement should not be assumed. In a large adolescent sample, Nakamura and colleagues showed that parent-child agreement over the mere presence or absence of prior headaches can be quite low (21). Our data underscore the recommendations of others that parental report of headache and related disability should not be viewed as a substitute for the child’s report (21,22). It is possible that our study design contributed to the disparities between patient and parent recall if diary participation by the patient led to less parental involvement in day-to-day headaches. The original validation of the PedMIDAS combined patient and parent input. Based on our results, we recommend that children old enough to complete the PedMIDAS on their own be encouraged to do so.
Lastly, we explored the impact of patient and parent confidence and patient age on recall error. We hypothesized that greater confidence and greater age would each have positive effects on patient recall accuracy, while parent recall would improve with higher confidence and lower patient age as parents would be more involved in the daily management of headaches when children are younger. However, our results did not support our hypotheses. Neither confidence nor age statistically affected parent recall. Trends and differences in patient recall were demonstrated for different confidence and age datasets, but these did not occur consistently. Andrasik and colleagues concluded that adolescent patient age did not predict recall accuracy (23), while van den Brink and colleagues found that older children had less accurate recall than younger children among subjects ranging in age from 9 to 16 years (17). Based on our data, confidence levels and patient age (10–18 years) are poor predictors of recall accuracy.
There are several limitations to our study. As described above, active participation with the diary may have led to improved recall. We attempted to limit this effect by preventing families from reviewing any diary entries after submission. We also remained vague about the intent of the study as a measure of recall until completion of the full 90 days. We intended that these approaches would limit the “study” of diary headaches in anticipation of a PedMIDAS quiz. The small decrease in headache frequency at 30 days compared to 90 days probably resulted from headache treatment in some patients and may have contributed to the relative improvements in 30-day recall. Subject selection also represents a study limitation. We did not choose patients randomly. While it is possible that our highly motivated and very compliant subjects had better recall than the general population, we do not believe that they would have had poorer recall. We acknowledge that this sort of subject selection may affect the generalizability of our results. Unfortunately, our study design would not offer an adequate measure of recall among poorly compliant patients. We did not distinguish between recall during the school year and the summer holiday. It is possible that recall differs between these periods. We did not specifically differentiate between migraine disability and disability due to tension headaches, although all subjects met migraine or probable migraine criteria. Lastly, we used the Internet diary as our measure of headache frequency and daily scoring of PedMIDAS elements. Our diary has not been validated against any other daily measures. Some patients missed consecutive diary entries (see distribution, Table 1), and lapses in recall during those relatively short periods may have decreased diary accuracy. We accept this as a potential study limitation.
Conclusion
In this study of recall reliability among children and adolescents with migraine or probable migraine and their parents, we found that recall of PedMIDAS elements and headache frequency at 30 days is superior to recall at 90 days. The optimal recall period for a measure of migraine disability would maximize recall accuracy while providing a time frame that adequately captures infrequent migraine attacks. We encourage further study of the validity of shorter recall periods for migraine disability assessment in children and adolescents. Patients completing our study had better recall than their parents in all study measures. Accordingly, parent report of migraine disability should not be used as a replacement for patient report in a similarly aged population. We recommend that children old enough to complete the PedMIDAS on their own be encouraged to do so. Self-reported levels of recall confidence and patient age serve as poor predictors of recall accuracy. Lastly, our daily headache diary had several features (Internet access, real-time monitoring of diary compliance, date and time stamping of all entries, and prevention of incomplete entry submission) that bolstered its use as a daily measure of headache frequency and disability. These features should be considered in clinical trials that use daily diaries entries to assess clinical outcomes.
Clinical implications
Recall of Pediatric Migraine Disability Assessment (PedMIDAS) elements and headache frequency at 30 days is superior to recall at 90 days. Patients aged 10–18 have better recall of migraine frequency and disability than their parents. Self-reported levels of recall confidence and patient age serve as poor predictors of recall accuracy.
Footnotes
Funding
This work was supported by a Nationwide Children’s Hospital Intramural Grant (Grant #278311; GL Heyer).
Conflicts of interest
None declared.
Author contributions
Study concept and design: G. Heyer.
Subject recruitment: G. Heyer, S. Aylward, J. Lee.
Data collection: G. Heyer, S. Perkins, S. Rose.
Statistical analysis: G. Heyer.
Drafting and revision of manuscript: G. Heyer, S. Perkins, S. Rose, S. Aylward, J. Lee.
Acknowledgments
We would like to thank Aggie LeGros, RN, for her coordination of this study and her numerous email and telephone reminders to families. We would also like to thank Dr Kathi Kemper for helpful recommendations regarding manuscript preparation.