Abstract
Aim
The aim of this article was to develop and apply an Internet-based headache diary (i-diary) for adolescents and compare it with a paper-diary (p-diary) regarding adherence, user acceptability and recorded headache activity.
Methods
In a cross-sectional school-based study, a representative sample of 488 adolescents aged 12–18 years were randomly allocated by cluster sampling to record for three weeks in i-diaries or p-diaries their headache intensity, disability, and use of acute medication.
Results
A significantly (p = 0.008) higher proportion of adolescents in the i-diary group used the diary at least once during the 21-day period (86% vs 76% for the p-diary). However, the p-diary group completed a significantly (p < 0.001) higher number of diary days (20.8 vs 15.0 days for the i-diary). The response rate for the i-diary-group was largely evenly distributed over the study period; conversely, approximately two-thirds of the adolescents using the p-diary responded on all 21 days, whereas one-fourth did not respond at all. The two diary types were rated as equal in easiness to remember (p = 0.25), but the i-diaries were more bothersome to use (p = 0.029).
Conclusion
Although p-diary users completed a higher proportion of diary days, i-diaries provided more reliable and credible estimates of headache parameters because of better real-time assessment.
Background
Headache is one of the most common self-reported health complaints among school-aged children and adolescents (1–5). It is associated with higher levels of school absence, anxiety and depression, lower quality of life, comorbid pain conditions and other somatic complaints (6–8).
During the last decade, there has been a substantial increase in the number of epidemiological surveys and clinical studies estimating headache parameters in adolescents by different assessment methods (9,10). The data collection technique is one of the key determinants of the quality and validity of data. In epidemiological surveys, retrospective information in self-reports has been gathered primarily by self-administered questionnaires, whereas interviews have been used less often (9). Paper diaries (p-diaries) have been used to estimate the frequency, intensity and temporal changes of headaches among children and adolescents in prospective recordings, in particular in treatment outcome studies (10,11).
Recently, there has been a growing interest and effort to document pain as near in time as possible to the individual’s actual experience using real-time data capture methods (11–13). Advances in information and communication technology have resulted in the development of several electronic diaries (e-diaries), both for handheld devices specially designed for studies and for the participants’ own computers or mobile phones using the Internet (i-diary) (12,14). However, to date, only a few studies exist that have evaluated the validity of standard recordings in p-diaries compared with e-diaries of chronic pain including headaches among children and adolescents.
A recent review (11) reported five studies in which the assessment of headaches among children, adolescents or adults was conducted using e-diaries, and all of the studies were performed over the past 15 years (15–19). While none of the five e-diaries showed evidence of acceptable reliability and validity, these diaries showed evidence of feasibility. In addition, 16 clinical trials were presented where e-diaries were used for data collection, and most of these studies were based on small clinical samples. Only two of the five studies and one of the clinical trials included adolescents (16,18,20). In addition, three clinical trials on headache included children and adolescents and used only e-diaries without comparison to standard p-diary recordings (21–23).
Because of the lack of comparative studies evaluating the validity and feasibility of e-diaries and i-diaries vs p-diaries for the recording of headaches among adolescents in the general population, the purposes of this study were: (1) to develop an i-diary with a reminder function to record headaches in adolescents and to compare this method with p-diaries in regard to user adherence, (2) to investigate potential differences between the diary types in regard to four headache-related measures, and (3) to compare the perceived acceptability of the diaries by their users.
Methods
Sample selection and recruitment procedure
From a representative sample of students, stratified by school location, age and sex, from all schools in the county of South-Trøndelag in Norway, 899 eligible students were invited to participate in the study from three grade cohorts (8th grade: age 12–13 years; 10th grade: age 14–15 years and 12th grade age 16–18 years) in equal proportions (see participant flow in Figure 1).
Flow of participants in the study.
Of this invited sample, 488 adolescents aged 12–18 years in 13 schools participated in the study and were asked to report prospectively on headache occurrence at the end of the day for a three-week period (276 girls and 212 boys). To obtain a sample corresponding to the distribution of the population in the county, the schools were initially drawn with the following probabilities: 59% from the urban region, 28% from the inland-mountain region and 13% from coastal municipalities. Data from schools in the inland/mountain and coastal regions were combined and labeled “rural.” Details of the recruitment procedures are provided elsewhere (24).
The participants were asked to record their daily headaches in one of two diary types (see details below). To avoid comparisons between students assigned to different groups, and to reduce teacher burden to administer and collect the p-diaries, the sampling unit was the school. Initially, a total of 13 schools were included; however, seven schools declined to participate in the study and were replaced by new schools. Thus, seven schools with 208 students were randomized to i-diary recordings, and six schools with 280 students were randomized to the p-diary format. Before the students began their diary entries, they were interviewed (24) so that we could compare participants with and without a history of headaches.
Data collection period
The students were asked to record their headaches prospectively for a three-week period, which has been recommended as an optimal length of recording for research purposes (25–27).
Diary characteristics
In both diary formats, participants were asked to respond to the following four questions every evening: (I) “How intense was your worst headache today?” The results were rated on a numerical rating scale (NRS) ranging from 0 to 10, where 0 = “no pain” and 10 = “worst imaginable/unbearable pain,” (II) “How intense was your headache on average during the day?” to be rated on a similar NRS, (III) “How have you functioned in your daily activities today?” The results were rated on a scale of 0–3, where 0 = “No difficulties,” 1 = “Minor difficulties,” 2 = “Medium difficulties” and 3 = “Major difficulties,” and (IV) “Did you take any acute medication because of headache today?” (No/Yes). Having a headache was defined as a score of ≥1 on an 11-point NRS (0–10 NRS). Moderate/severe headache intensity was defined as a score of ≥5 on the NRS according to a recommendation by Hirschfeld and Zernikow (28).
An Internet-based program was developed for i-diary recordings in collaboration with the Faculty of Informatics and e-Learning, AITeL, at the University College of Sør-Trøndelag. The participants received a username and password to log on to the server via a private computer, smartphone or a tablet device. When logged on, they first arrived at a page with instructions on how to proceed and respond to the four questions outlined above. Daily reminders were sent out through a short text message service (SMS) to those students who had not logged on before 9 p.m., and retrospective headache recordings were not possible after midnight. In the i-diary, all data were entered directly into a password-protected database. Web-based diaries enabled the use of a private personal computer (PC), mobile phone (for SMS only), smartphone or tablet to perform recordings, without the need for other devices.
The p-diaries were distributed at the beginning of each week by the teacher and collected one week later by the teacher and the first author (ABK). Diary recordings took place in the participants’ homes.
All participants received training on how to fill out the diaries in face-to-face sessions with the first author (ABK). Those who registered in i-diaries could also contact the first author by email if they had any problems or questions regarding recordings in the diary.
User evaluation of diaries
User evaluation for all adolescents and differences in ratings by diary type.
Statistics and data
Descriptive data were presented as means and standard deviations (SD) and as medians with interquartile ranges (IQR) for ordinal variables. Cluster effects of school sampling were examined and found negligible. Differences between groups for ordinal variables were analyzed with Mann-Whitney and Kruskal-Wallis tests. Pearson Chi-square tests were used to analyze associations between dichotomous variables, and Friedman’s test of variance by ranks was used to analyze the completeness of responses to the four items across the three-week period. Conditional logistic regression analyses with student identification (ID) as a matching variable were used to analyze the associations between responses and day of the week. A p value of 0.05 or less indicated statistical significance in two-tailed tests. To avoid biased estimates in comparison of the proportion of days of diary use due to technical problems, the number of total eligible days and eligible days for the i-diary group were adjusted taking into account the number of days when the server was inoperative. The response proportion was defined as the number of completed questionnaires divided by the number of eligible sample units. The data completeness was defined as the proportion of the number of missing items in the diary.
IBM SPSS Statistics for Windows version 21 and R version 2.13.1 were used for statistical analyses.
Ethics
The Norwegian Ethical Committee for Medical Research approved the research protocol, and informed consent was documented in accordance with their requirements.
Results
Description of participants
The flow of participants presented in Figure 1 shows that 488 (54%) of the 899 invited adolescents consented to record their complaints in a diary. Of these, 280 were allocated to the p-diary and 208 were allocated to the i-diary format.
Proportion of adolescents using the diaries
The proportion of participants who returned p-diaries or recorded at least one entry in the i-diaries during the 21-day period was significantly (p = 0.008) higher in the i-diary group (87%) than in the p-diary group (76%) (Figure 2). However, this difference was observed only in urban (n = 26, 87% vs n = 123, 71%), but not in rural (n = 153, 86% vs n = 91, 86%), schools.
Distribution of participants and number of days with diary recordings.
Response proportion of days
Response proportion (total number of recording days) and comparisons on diary type, sex, grade and school location for all allocated students.
Comparisons: aDays with response by sex for both diaries and by diary type. bDays with response by grade for both diaries and by diary type. cDays with response by school location for both diaries, and by diary type. dDays with response by having headache or being headache free for both diaries and by diary type. Response is defined as having filled out at least one of four diary items during a particular day. eHaving headache was defined as a score of ≥1 on a 0–10 numerical rating scale (NRS).
Figure 2 depicts a detailed comparison of the distribution of participants with regard to the number of days of diary use. While the response rate in the i-diary-group was evenly distributed over most of the 21-day period, most adolescents in the p-diary group responded either on all 21 days (68.9%) or not at all (23.6%).
Among all the 488 adolescents, p-diaries were completed on an average of 15.9 ± 8.9 days (median 21) compared with an average of 12.9 ± 6.9 days (median 15) for i-diaries during the three-week period (see Figure 2). The mean number of days of diary use among the 393 adolescents who recorded at least once during the three-week period was 20.8 ± 0.9 (median 21) for p-diaries and 15.0 ± 4.8 (median 16) for i-diaries.
Completeness of responses to the different diary items during days of diary use
Completeness of response to the different diary items by diary type.
Completeness is defined as the number of days items were answered in relation to eligible days.
Pain maximum = Worst pain today rated on a numerical rating scale (NRS) 0–10 scale.
Pain mean = Mean pain intensity today rated on a NRS 0–10 scale.
Disability = Performance of today’s activities (none, minor, medium, major).
Medication = Use of acute headache medication today (no/yes).
Comparison between completeness of response to pain maximum, pain mean, disability and medication for both diary types.
Comparison between completeness of response for pain maximum, pain mean, disability and medication for the paper diary type.
Comparison between completeness of response for pain maximum, pain mean, disability and medication for the Internet diary type.
Diary use (for at least one day) for each of the three weeks by diary type
The proportion of adolescents using the diaries decreased significantly (p < 0.001) across the three weeks in both diary groups pooled with the greatest decline observed in the i-diary group (p < 0.001) (Figure 3), in which there was a significantly (p < 0.001) higher response proportion during the first week compared with the following two weeks (67.4% for week 1, 60.7% for week 2 and 57.6% for week 3 in the i-diary group and 76.3% for week 1, 75.6% for week 2 and 75.5% for week 3 in the p-diary group) (Figure 3).
Diary use (on at least one day) for each of the three weeks by diary type.
Days of response (having replied to at least one question) and response proportion (eligible days of all allocated students) by diary type and weekday.
= comparison of different days of the week for both diary types.
= comparison of different days of the week for the paper diaries only.
= comparison of different days of the week for the Internet diaries only.
Differences in headache-related measures between diary types
Days with any headache, headache with moderate/severe intensity, headache with moderate/severe disability and headache medication in relation to sex, school grade and school location by diary type.
Having headache is defined as ≥1 on an 11-point numeric rating scale (NRS) (0–10). bModerate/severe headache intensity is defined as ≥5 on the NRS. cHeadache with moderate/severe disability is defined as ≥1 on the NRS for intensity and ≥ moderate disability.
Perceived acceptability of diaries
A total of 374 adolescents (77% of all 488 allocated participants) completed the diary acceptability ratings (209 from the p-diary group and 165 from the i-diary group; 75% and 79%, respectively). The majority of participants in both groups found it either “easy” or “quite hard” to remember to fill out the diaries without any significant difference between diary types (p = 0.25). However, a significant difference (p = 0.029) was found between the groups regarding whether diaries were bothersome to fill out. Although the majority of adolescents in both diary groups did not find them at all bothersome, a “big” or “very big bother” was reported more than twice as often by the i-diary users than p-diary users (Table 1). The majority of participants in both groups liked the diary appearance, and there was no significant difference (p = 0.25) between the two formats in that regard (Table 1).
Technical problems
The server was down for two days, so 61 adolescents were not able to perform their i-diary recordings these days (0.6% of the eligible days). Furthermore, a hurricane on the coast of the region occurred, disconnecting all electricity, Internet and mobile phone connections for one day, and 24 students were not able to record their data that day (0.2% of the eligible days). Another five adolescents had written on the diary evaluation form that they had no Web access or mobile coverage some days because they had been at a cabin or on mountain hiking tours on some of the weekends while the study took place. One girl had forgotten the password.
Discussion
To the best of our knowledge, this is the first general population-based study to develop, apply and compare an Internet-based e-diary, and it is the first study to us a paper diary for recording headaches in adolescents in regard to adherence, levels of headache activity and perceived acceptability. Although quite good adherence was found for both diary types, a pronounced difference in the pattern of adherence was observed. A significantly higher proportion of adolescents allocated to the i-diary format used the diary at least once during the 21-day period. However, among the users, adolescents in the p-diary group completed a significantly higher proportion of diary days than those in the i-diary group (20.8 vs 15.0 days, respectively). While the response rate from adolescents in the i-diary group, who were urged to register each evening, was mostly evenly distributed across the 21-day period, two-thirds of the adolescents in the p-diary group, who had the opportunity to back-fill data for the last week, responded on all 21 days, and a quarter did not use the diary at all.
Similar findings, that there were twice as many complete non-users in the p-diary group (24%) than in the i-diary group (14%), have also been described in a previous study examining children and adolescents with chronic pain, but shows an even greater difference than the present study. All the children using e-diaries completed at least one or more diary recordings, whereas 40% of the p-diary group did not return any diary records at all (18). However, concerning the proportion of eligible days when the diary was used, that study reported a higher proportion of registrations in the e-diary group than in the p-diary group, which is in contrast to the findings of the present study (18).
One possible reason why the i-diary group in the present study reported a lower proportion of days than the p-diary group could be that they did not have the opportunity to back-fill data. The finding that adolescents who used p-diaries in the present study generally recorded either all or no days indicates that they might have back-filled responses, which has also been described for p-diaries in earlier clinical and epidemiological studies (12,16,18,29). Some statements from adolescents in the evaluation form of the present study also confirmed that back-filling of data occurred. For example, one girl stated that “I used to forget it and therefore I wrote several days at a time,” and another girl said, “I had trouble remembering, so I did [record] it every Sunday.” Another study in which electronic recordings were used showed that real-time adherence in p-diary registrations was only 11%, indicating a high level of faked adherence (12).
Another reason for a lower proportion of recorded days in the i-diary group could be that the i-diaries were significantly more often regarded as bothersome to fill out. Some participants in the i-diary group in the present study experienced technical problems involving temporarily limited access to the Internet. This may have resulted in some of the youths finding the i-diaries difficult to use.
A third reason for the lower proportion of registration days in the i-diary group in the present study could be that some participants gradually lost interest, resulting in a decline in headache recordings from week to week (67.4%, 60.7% to 57.6%). Such a decline was not observed in the p-diary group, possibly because the participants could back-fill data at the end of each week.
The decline in diary use during the three-week period might also partly explain the lower proportion of daily registrations in the i-diary group in the present study compared with the e-diary groups in the aforementioned clinical studies, in which recordings lasted only one week (16,18). Another explanation for the lower proportion of registered days in i-diaries in the present study compared with participants in the clinical studies (16,18) could be that the latter included children/adolescents seeking help for headaches and who were therefore more motivated to use a headache diary than those in a general population-based sample.
In regard to differences in registered headache activity, the only indication of non-equivalence between the two diary types was a higher proportion of headache days with moderate or severe disability recorded in p-diaries than in i-diaries. One explanation for this could be that the participants in the i-diary group were so bothered by the headaches on such days that they could not bear to sit down at the computer and make their registrations before the deadline at midnight. Another explanation for this difference might be related to a recall bias connected to after-registrations in p-diaries with data lumped together for several days at a time. In two epidemiological studies, data from prospective diary recordings were compared with retrospective data based on recall in standardized questionnaires showing a recall bias, where adolescents and adults tended to report greater headache-related disability in retrospect (13,27). Given the assessment of headache in real time (on the same day) in i-diaries and the lack of optional retrospect recordings, the validity of the obtained information is strengthened, thus providing more credible data than p-diaries.
A limitation of the present study was the ability for adolescents to back-fill data in the p-diary group. The rationale for this was pragmatic, i.e. we wanted to refrain from putting further burden on the teachers by asking for daily reminders to all students using p-diaries. Also, we wanted to examine how the two diary assessment formats worked in real life and as p-diaries were used in previous studies (27,29).
For some participants in the i-diary group, technical difficulties with the Internet prevented them from conducting registrations on certain days. The major strengths of the study are the comparably large sample sizes of representative adolescents from the general population and the use of a longer period of time for diary registrations than in most previous clinical studies, The random sample of school children with none, infrequent, frequent or daily headaches enables generalization of the results for the general population, which is a unique strength of this study.
Conclusion
The study revealed a pronounced difference in the pattern of adherence between the two diary formats. While there were fewer non-users in the i-diary group, more days of diary recordings were completed in the p-diary group, possibly because of back-filled responses. Although the study showed that an Internet-based headache diary with a reminder procedure was useable for prospective recordings of headaches occurring on the same day, considerable improvement potential was identified. Although i-diaries ensure better real-time assessment than p-diaries, further research on how to improve adherence to i-diaries is recommended.
Clinical implications:
Complete non-users were more common for paper-based (p-diaries) than Internet-based (i-diaries). The completeness of responses among users was higher in p-diaries than in i-diaries. Participants in the p-diary group reported back-filling, which was not possible with i-diaries. I-diaries ensure better real-time assessment than p-diaries.
Footnotes
Funding
This research received no specific grant from any funding agency in the public, commercial, or non-profit sectors.
Conflict of interest
None declared.
Acknowledgments
The authors express their gratitude to the participating students, their directors and all staff of the participating schools for their cooperation and contributions to this study. The authors also want to thank Dean Per Borgersen and students Per Stian Hoff, Malene Dae and Ida Rosèn at the Faculty of Informatics and e-Learning at the University College of Sør-Trøndelag, Norway, for their excellent cooperation on the development and maintenance of the procedures and program for the i-diary.
Further data from the underlying research material can be obtained upon request to the corresponding author.
Author participation is as follows: study concept and design: B. Larsson, M. Linde; acquisition of data: AB. Krogh; statistical analysis: Ø. Salvesen; drafting of the article: AB. Krogh, M. Linde, B. Larsson, Ø. Salvesen; critical revision of the manuscript for important intellectual content: M. Linde, B. Larsson; final approval of the version to be submitted: AB. Krogh, M. Linde, B. Larsson, Ø. Salvesen.