Abstract
Background
International guidelines recommend diaries in migraine trials for prospective collection of headache symptoms. Studies in other patient populations suggest higher adherence with electronic diaries instead of pen-and-paper. This study examines the feasibility of a text message-based (texting) diary for children and adolescents with headache.
Methods
This is a secondary analysis of data from a study validating a pediatric scale of treatment expectancy. We developed a Health Insurance Portability and Accountability Act-compliant texting diary collecting headache characteristics, medication use, and disability with 3–5 core daily questions for 4 or 12 weeks depending on headache treatment. Adherence was incentivized.
Results
93 participants consented to the expectancy study. Five participants opted for a paper diary for follow-up. 88 participants chose the texting diary with 28 4-week and 60 12-week participants. Five participants did not complete the enrollment visit. Of those remaining 83, 89% of 4-week and 93% of 12-week participants responded on at least 80% of days. On average, participants fully completed 88% (4-week cohort) and 90% (12-week) of diary entries.
Conclusions
Text messages are a promising method for collecting patient-reported data. Adherence was similar to that reported for paper diaries in other pediatric migraine trials, but time-stamped entries ensure real-time data collection.
Introduction
Migraine affects 9.1% of children and adolescents (1), causing substantial disability (2) impaired school performance (3) and reduced quality of life (4). As new therapeutics are tested, the Guidelines of the International Headache Society for controlled trials of preventive treatment of migraine in adults and children recommend prospective collection of a headache diary that captures migraine symptoms (5,6). Historically, headache diaries have been done on paper, but studies in other patient populations have described equivalent to increased diary completion rate and participant retention with electronic diaries (7–9). One review of adherence between headache diary modalities suggested increased adherence to electronic diaries, though technical problems were a clear barrier (10). Another review concluded that electronic diaries allow for a better clinical assessment of headache diagnosis and burden (11).
There are drawbacks to the existing electronic diaries, which are web-based or smartphone application-based. These diaries often require participants to install an additional app (which may be limited to a particular platform) or to carry an additional device, compromising feasibility and usability (12,13). In addition, these systems do not prompt the patient to enter diary information, whereas a system of regular reminders has been shown to increase participant adherence in some studies (14–16). Some headache studies have found high adherence to diaries collected via research electronic data capture (REDCap), which can send a link via email or text to an interactive web-based survey (17–20). However, this still requires participants to click on a link sent via text message and use a mobile browser to complete a survey which–depending on the type of phone–may make surveys appear less visually intuitive and make diary completion more tedious.
Communication via text messages, also known as texting, may serve as a promising venue for diary collection, and could address these concerns. In May 2018, Pew Research Center reported that 95% of 13- to 17-year-olds in the United States have access to a smart phone, with over 90% saturation across socio-economic groups (21). Texting permits interactive, adaptive responses in real time, and can utilize the participants’ own devices. Concerns of using texting to transmit medical information include encryption, privacy, and compliance with the Health Insurance Portability and Accountability Act (HIPAA) This study aims to determine feasibility of a HIPAA-compliant algorithmic texting diary to collect daily measurements of headache frequency and symptoms. We hypothesized that the texting-based diary would have a completion rate at least as high as the completion rates for prior paper headache diaries.
Methods
Setting and participants
This is a secondary analysis of data collected during follow-up from a validation study of a pediatric adaptation of the Stanford Expectations of Treatment Scale, a measure of a patient’s expectations of clinical outcome prior to the start of treatment (22). The study was conducted at the Children’s Hospital of Philadelphia, a large academic children’s hospital in the north-eastern United States, with approval from the hospital’s institutional review board. We recruited children and adolescents (ages 7–17) who presented to the institution’s emergency department or outpatient neurology clinic seeking acute or preventative treatment for headache between November 2018 and December 2020. As our diary platform was only available in English and the expectation scale was only available in English or Spanish, we excluded patients who were not proficient in English or parents who were not proficient in either English or Spanish. We also excluded patients where there was a high level of concern for secondary headache. We obtained informed consent from the parents or guardians along with participant assent.
Study design
We asked patients to complete the headache diary for 4 weeks or 12 weeks depending on whether they received an acute treatment for headache exacerbation (4-week diary) or they started a new preventive treatment (12-week diary). Participants reported the occurrence of any headache and associated characteristics daily via yes-no responses to text messages sent by an artificial intelligence-powered and HIPAA-compliant program called CareBot created by partner Patient.ly. The CareBot used artificial intelligence driven text analysis to respond to most interactions (Figure 1) such as asking about presence of migraine if the participant reported having a headache or asking about use of medications if they did not report headache. When the participant gave a response that deviated from the pre-written script, the CareBot alerted staff to provide a customized response.
Conversion tree between CareBot and participant for a typical diary entry. Blue boxes represent prompts given by CareBot. Green and red ovals represent potential responses given by participant. Text provided courtesy of Children's Hospital of Philadelphia. ©2021 The Children’s Hospital of Philadelphia. All rights reserved.
In addition to completing the daily 3–5 question diary, participants were asked to complete a more detailed assessment of their headache characteristics on randomly selected 20% of days with a self-reported headache. These were yes-no questions. If the answers indicated a change from the initial questionnaire, such as new aura symptoms, a REDCap survey link was sent via text message to gather additional data (18). On a weekly basis, participants rated their level of improvement via the Patient Global Impression of Change (PGIC) scale, and the severity of any side effects of their headache treatment via text message. Participants also filled out a weekly feedback survey, collected via REDCap, to identify which questions confused participants and why. Every 4 weeks, we sent a REDCap survey link to participants where they described their level of pain-related disability according to using the PROMIS Pain Interference Scale (23), once for 4-week participants and three times for 12-week participants during their follow-up period. On their final day of the study, participants completed the PedMIDAS scale2 to report headache-related disability also collected via REDCap.
Participants specified their preferred time to be messaged between 7:00PM and 12:00AM local time, with a 24-hour window to respond. The CareBot also sent reminder texts both one and two hours after sending an unanswered prompt. Any correspondence sent by the CareBot past 4:00 AM the following morning changed prompts from “today” to “yesterday” to help reduce confusion about the entry in question. To incentivize adherence, participants received $20 per 4 weeks if they responded on at least 23 of 28 (82%) of days, meaning at least partial completion of 80% of diary entries. When text-diary participants missed entries for two or more consecutive days, the CareBot alerted staff who then contacted participants to encourage participation.
We offered a pen-and-paper diary to participants who declined to use the texting-based headache diary; the incentive was the same. If participants chose the paper diary, they received a package during their enrollment visit, containing one survey for each day of the follow up period. The package also included an envelope with prepaid postage for the participant to return the completed package at the end of the follow up period. If we did not receive the paper diary within two weeks after the scheduled completion of the follow up period, research staff contacted the family via email to remind them to return the package.
Outcome measure and other variables
Our primary outcome to demonstrate feasibility was the proportion of diary entries completed for both 4-week and 12-week groups. A fully completed entry requires that the participants either responded to all prompting text messages sent by our CareBot or completed all available fields in a provided REDCap survey. Our diary entries employed branching logic such that a participant’s responses determined later prompts. We also examined rates of partial completion, in which a participant responded to at least one field in the diary. This schema of full or partial completion was applied to our additional scales (PGIC/Adverse Events, PROMIS Pain Interference, and PedMIDAS).
Statistical analysis
Participant demographics were characterized with descriptive statistics (proportions, median, etc). Proportion of participants who responded on a given day or week were reported as percentages.
Results
Feasibility: Core headache diary (3–5 daily questions)
Ninety-three children and adolescents consented to participate in the primary expectancy study (median age = 15.9 years; IQR = 13.8–17.0 years; Figure 2); five participants who consented to the study later did not complete the enrollment visit and were not included in this analysis. Another five participants elected to complete a 4-week pen-and-paper diary (median age = 12.3 years; IQR = 11.8–13.7 years). All five paper-diary participants failed to return their paper diaries despite repeated attempts by coordinators to make contact and were lost to follow-up. The remaining 83 participants elected to complete the electronic diary: 27 participants in the 4-week arm and 56 participants in the 12-week arm (Table 1). In most cases, participants had direct access to a cellular device with texting capabilities and could respond directly to our prompts. Seven 12-week participants did not have direct access to a suitable device and agreed to complete the diary using the cellular device owned by the parent who initially consented to the study. Participants were predominantly female, adolescent, and non-Hispanic White, consistent with the demographics of the patients who typically present to the institution’s outpatient neurology clinics.
Flow diagram of the progress through the phases of our study across both 4-week and 12-week arms including recruitment, enrollment, follow-up, and data analysis.
Demographic characteristics by length of follow-up period.
1number (%); 2Median (Interquartile range)yo: years old.
An additional four participants, two in the 4-week arm and two in the 12-week arm, completed some diary entries before being lost to follow-up between Days 11–71. The remaining 79 participants submitted entries throughout their respective follow-up periods. Excluding participants who did not complete the enrollment visit, 24/27 participants in the 4-week arm and 52/56 in the 12-week arm provided texting data for at least 80% of available days. In total, we collected 5111 partially completed and 4846 fully completed diary entries.
Early in the study, there was a programming error with CareBot resulting in the loss of 62 person-days of diary entries from the 12-week arm during which the CareBot failed to prompt participants. Despite this loss of data and a few participants being lost to follow-up, adherence to the texting headache diary remained high with an average 88% of diary entries being fully completed per subject in the 4-week arm, 90% in the 12-week arm. Mean adherence for at least partially completed entries increased to 91% and 95% for both arms (Figure 3).
Estimated mean rates of diary entry completion, both partially and fully, for each subject over time for 4-week (a) and 12-week (b) participants. Translucent ribbons represent the standard error for the estimated mean.
Five participants reported being confused by the questions: three reporting confusion from the surveys conducted via REDCap directly and two reporting confusion from both text message and REDCap survey questions. However, most participants provided either neutral or positive feedback.
Feasibility of text-based diary: Additional questionnaires
Participants also had high adherence rates for answering additional questions via text message and REDCap-administered surveys for both 4-week and 12-week participants (Figure 4). Overall, adherence across multiple subsequent measures of headache features, disability, and adverse events was comparable to that for the core diary.
Mean rates of full completion for non-diary scales for each subject over time for 4-week (a) and 12-week (b) participants. The single star-shaped point represents the completion rate for the PedMIDAS Scale which was completed at the end of follow-up. The first 3 scales—Adverse Event Assessment, Detailed Headache Questions, and Patient Global Impression of Change—were measured weekly via text message while the PROMIS Pain Interference scale was measured every 4 weeks via REDCap, and the PedMIDAS scale at the end of follow-up via REDCap.
Discussion
In this study, we demonstrated feasibility of a texting-based headache diary and related patient-reported outcomes completed by children and adolescents for up to 12 weeks, with over 80% adherence in reporting outcomes and additional scales. We found high adherence to our texting-based diary regimen throughout both 4-week and 12-week follow-up periods. In addition, participants had similarly high levels of adherence when completing additional questions on headache characteristics, adverse events, and disability given less frequently. These data support the use of text messages as a worthwhile platform for collecting pain diary information in real time from children or adolescents who have access to a cell phone with mobile data.
In comparing our results to other published studies, the overall adherence rates were similar to those reported for the paper diary used in the CHAMP trial (24) and for those reported in headache studies using REDCap-based diaries (17,19,20,25). The addition of confirmed time entries may be considered to enrich data validity compared to paper diaries. One study of Norwegian adolescents found increased adherence for paper headache diaries compared to a web-based platform (26). However, adolescents assigned to complete paper diaries tended to complete either all or none of the diary entries, which led the authors to speculate that participants were backfilling prior entries (26). Indeed, another study found that adults completing chronic pain diaries appeared to have high levels of adherence to the paper diary, but electronic timestamps from when the diary was opened suggested participants were largely backfilling (27). While backfilling has not been proven to alter the accuracy of the information, our diary platform’s use of electronic timestamps ensures that participants are recording diary entries close to the event of interest, and still maintains high rates of completion.
Our diary also addressed many of the flaws of the available electronic diaries. The text messages were not specific to a certain type of mobile device nor software and were compatible with a wide range of devices as a result. Unlike most publicly available headache- and pain-related electronic platforms (28), this diary was developed by clinicians. Our diary platform is similar to previous REDCap-based diaries in that they both use branching logic to address a variety of potential answers and can automatically remind participants to complete still-pending diary entries. However, our diary platform also alerts staff to participants who fail to complete surveys over consecutive days, allowing for fewer steps in monitoring an ongoing study. The language was developmentally appropriate and understood by the majority of participants. However, the texting platform could be further improved by adding an option to visualize trends in data for both patient and clinician (13,28). While integration with the electronic health record is feasible for this platform, we have not yet explored that for the headache diary.
It is also important to consider the limitations of this texting-based diary. For one, none of the participants who chose paper diaries returned them and it is unknown if they were completed at all. Therefore, we were unable to compare adherence between both diary platforms. Since we did not receive any paper diaries back, we did not attempt to have anyone complete both paper and texting diary, so we cannot assess the reliability or validity of responses from our texting-diary participants in comparison to responses on the more traditional paper diary. We could not assess whether it was the paper diary itself, or some confounding factor that influenced both the choice of a paper diary and loss to follow-up. It is possible that participants who elected to use the pen-and-paper diary did not have access to a cell phone with available texting. Though the majority of adolescents in the United States have access to a cell phone (21), this is not universal and there may be limitations in access to mobile data or excessive fees for text messaging. It is possible that sole use of a texting-based diary within a study could exclude data collection from some children and adolescents, particularly those from lower-income backgrounds. One possible solution is to reimburse data service fees associated with text messages for participants with cell phones but limited access to data.
Additionally, even though clinical trials for patients with chronic pain conditions often require use of a pain diary, there has been some concern that diaries—both paper and electronic—can lead to poorer recovery outcomes. Three months after a neck or lower back injury, adults who completed a 4-week paper diary were less likely to report a full recovery compared to adults without a pain diary (29,30). Though we are not aware of any data on negative effects of keeping a headache diary, anecdotal discussions with other headache clinicians have echoed the concern that diaries can be detrimental by increasing the focus on symptoms. That concern may lead some eligible patients to decline use of a diary for fear of exacerbation.
In addition, specific features of a headache diary may render it overwhelming, such as entries that ask for substantial detail every day or reminders to complete an entry that come too frequently or do not mesh well with a participant’s schedule (12,31,32). Either could lead to data entry fatigue and a decrease in participant adherence and satisfaction. We believe that most of our participants did not experience such issues as mean adherence remained above 75% throughout follow-up for both 4-week and 12-week groups. The brevity of questions in this diary may have enhanced adherence but does not enable us to determine whether every self-reported migraine-day met full criteria for migraine. In the future we will use the additional characteristics collected on a subset of days to examine the concordance between self-reported migraine and migraine features. Participants in this study were offered an incentive for adherence, which limits generalizability in non-incentivized circumstances. Another possible limitation to note is that being contacted by research staff could also be an overwhelming experience for some participants (12) and requires active oversight from research staff, which may not be feasible for larger studies or clinical use.
In conclusion, text messages appear to be a relatively untapped, but promising resource for diary data collection. One clear advantage is that this modality allows researchers to collect real-time data from participants on a longitudinal scale. Further, it taps into a widely used communication method for adolescents making timely responses more likely. To improve the system, we are actively working on the automatic transfer of data into REDCap with plans to share our platform for use by other headache researchers (18,33). The texting platform could be made even more useful in clinical care by adding an option to visualize trends in data for both patient and clinician, and direct integration with the electronic health record (13,28). Given the consistency and quality of the data collected, we plan to continue using this platform for data collection in future projects.
Clinical implications
Children and adolescents with headache had high rates of adherence to a text message-based diary. Use of an electronic platform for a headache diary is a feasible way to ensure real-time data collection during follow-up periods as long as 12 weeks.
Footnotes
Declaration of conflicting interests
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The institution of DJ Kellier receives research support from the NIH NINDS (3U24NS115691-01S2). JT Farrar has received personal compensation for serving as a consultant for Vertex Pharma and for serving as an associate editor for PharmacoEpidemiology and Drug Safety. The institution of JT Farrar has received research support from the FDA and the NIH NINDS. AD Hershey or his institution have received compensation for serving as a consultant for AbbVie, Amgen, Biohaven, Eli Lilly, Lundbeck, Supernus, Teva, Theranica and Upsher-Smith. His institution has also received research support from Amgen, Biohaven, Eli Lilly, Theranica, Upsher-Smith, and the NIH NINDS/NICHDS. CL Szperka or her institution have received compensation for serving as a consultant for Teva, Lundbeck, and Impel. She has received personal compensation for serving on a scientific advisory or data safety monitoring board for Eli Lilly and Upsher-Smith. She has also received research support from the FDA and the NIH NINDS (K23NS102521). CL Szperka has received compensation for serving as an editor from Elsevier.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the NIH National Institute of Neurological Disorders and Stroke (1K23NS102521-01A1 and 3U24NS115691-01S2).
