Abstract
Many veterans returning from service in Afghanistan or Iraq suffer from post-traumatic stress disorder or mild traumatic brain injury. Treating these conditions can be challenging because of high rates of relapse and associated memory impairments. We report on a pilot study that assessed the utility of mobile health (mHealth) technologies, including personal digital assistant-based ecological momentary assessment and two-way interactive text (SMS) messaging, for providing treatment feedback to clinicians, encouraging and motivating veterans throughout treatment, and monitoring participants for relapse after treatment discharge. The results of the pilot suggest that mHealth technologies are feasible adjuncts to traditional mental treatment in the veteran population. Additional work is needed to establish the degree of clinical and economic value.
Background
More than 1.6 million men and women have been deployed from the USA in support of Operation Iraqi Freedom (OIF) and Operation Enduring Freedom (OEF) since 2001. 1 Because of improvements in body armor, many soldiers survive attacks but are left with the ‘signature’ wounds of these conflicts: mild traumatic brain injury (mTBI) and/or post-traumatic stress disorder (PTSD). 2 The reported prevalence of these conditions in this population is significant: recent estimates suggest that from 5% to 20% of Service members are likely to have PTSD based on self-reported symptoms 3 and up to 20% of Service members report symptoms consistent with mTBI. 4 Owing to stigma and concerns about the effects of seeking mental health care on one’s military career, it is likely that the true prevalence of these conditions is even higher.
Both the US Military Health System and the US Department of Veterans Affairs have seen substantial increases in funding over the last several years in support of large-scale expansions in mental health and substance abuse treatment capacity.5–7 Despite the expansion in system capacity, a number of challenges remain in providing effective treatment. First, the nature of these two conditions may complicate effective treatment planning. Both PTSD and mTBI are associated with memory impairments that may reduce a patient’s ability to provide retrospective feedback on progress between outpatient visits.8–10 Second, the risk of PTSD relapse remains high. One recent study showed 26% of veterans with PTSD relapsed over the course of a 12-week trial; 11 another showed about 20% relapsing over a one-year follow-up period. 12
A specialized psychological measurement tool developed in the mid-1990s—ecological momentary assessment (EMA)—holds the potential to help address both of these challenges. EMA is a technique which samples attitudes, behaviors, and experiences in real time in subjects’ natural environments. EMA data have been used for studies of a wide variety of topics, including coping, 13 smoking behavior, 14 and eating disorders, 15 and have been shown to produce reliable data, improve treatment compliance, participant motivation, treatment outcomes, and system capacity. 16
Whilst EMA data collection has often been performed using paper and pencil diaries, the widespread availability of personal digital assistants (PDAs)—and, more recently, the cell phone—has now opened up a new route for minimally-obtrusive EMA data collection using devices most people are either familiar with or already own. Mobile health (mHealth) applications using these devices are rapidly increasing in popularity as healthcare delivery tools in a wide variety of settings, including for chronic disease self-management,17–19 weight loss, and other healthy behaviors,20,21 as well as in support of AIDS care in Africa. 21 Despite the popularity of mHealth tools, there is still relatively little evidence on the feasibility and uptake of these tools in the populations where they have been deployed. As adoption of, and engagement with, new technologies are critical precursors to mHealth tools becoming effective complements to traditional means of care delivery, this article contributes to that body of knowledge by reporting on the design of and lessons learned from a pilot study focused on assessing the feasibility of mHealth-based EMA as an adjunct to treatment and follow-up care in a population of veterans receiving or recently discharged from residential treatment for mTBI and/or PTSD.
Methods
The pilot sought to examine the feasibility of mobile-technology-assisted data collection in both the active phase of treatment, whilst veterans were in residence, but also during the follow-up phase of treatment after they had been discharged to their homes and regular mental health providers.
Setting
The pilot was conducted in a small population of veterans receiving care for PTSD and mTBI at The Pathway Home—a residential treatment facility in northern California. The Pathway Home offers a residential recovery program specifically created for veterans, featuring 45 inpatient beds and an average treatment stay of between 4 and 5 months. The pilot was conducted over a 12-month period across two distinct cohorts (one early in the study, another later in the study) of veterans who had been admitted for treatment. The protocol for the pilot was reviewed and approved by an institutional review board.
Participants
Study participants were recruited from the population of veterans and service members receiving residential treatment. The pilot included veterans suffering from PTSD and/or mTBI—in some cases coupled with psychiatric comorbidities, including substance use disorder (SUD), and major depressive disorder (MDD). A total of 27 individuals who were either in active treatment or were admitted during the study period consented to participate in the pilot. Eighteen individuals were part of an initial cohort. Owing to admissions and discharges over the study period, we also recruited a second cohort of nine individuals later in the project.
Protocol
The protocol was designed to use technology in two distinct ways. During the active treatment phase, EMA data were collected on a daily basis from participants via PDAs. After discharge in the follow-up phase of treatment, patients continued to participate through interactive text messaging technology (SMS) as they received and responded to check-in messages on their personal cell phones.
Active phase
We developed a questionnaire to support daily EMA data collection in conjunction with clinical staff and an informal focus group of recently-discharged patients. The group was used to pre-test and further refine question formats and timing. A subset of 32 questions (shown in Appendix A) was adapted from one of several standardized instruments including the Symptom Checklist-6, 22 the BriefCOPE, 23 and Beck Depression Inventory-II 24 accompanied by questions assessing level of pain, hassles, and uplifts (as potential indicators of stress triggers), as well as a series of questions designed to gather feedback from participants about the previous day’s activities. The questions were developed in an electronic survey tool designed to support data collection via PDAs. Each participant was issued a PDA to keep with him/her during waking hours. The EMA data collection was initially planned to occur once daily at a random times during this window and was initiated by an alarm notification on the device. When the participant responded to the alarm, the PDA prompted with the question ‘Do you have time now to complete the survey?’.
EMA data were downloaded from the PDAs several times a week by a research assistant who then compiled the responses into a database. Using these data, simple graphs of responses to EMA questions (see example in Figure 1), focusing on an additive index of negative emotions (items 2–8 from the EMA instrument recoded so that higher values indicated negative emotions), were produced and shared with clinical staff in order to provide a richer understanding of day-to-day progress toward recovery. These data were intended to be shared regularly with participants by the treatment team for incorporation into therapy sessions and the treatment planning process.
Example negative emotions graph
Follow-up phase
Beginning late in residential treatment and continuing for up to three months post-discharge, SMS messaging provided by LifeWIRE Corporation was integrated into treatment to continue to collect information on and support the veteran’s progress toward recovery. All SMS messages included the name of the residential treatment facility to remind participants that the communication was study-related. Also, as part of the follow-up phase of interaction, participants identified one or two ‘buddies’ to provide social support as a part of the SMS messaging interactions. All participants and buddies consented to participation in the messaging activity.
SMS messaging was used in two ways during the follow-up phase. First, messaging was used to provide motivational reminders to participants. Together with the treatment team, each veteran crafted a number of reminder messages that could be sent on a daily basis in the post-discharge period with the goal of providing individualized motivational support. Table 1 displays a subset of messages and their delivery schedules for a sample of participants; unless otherwise requested by the participant, the messages were sent either at 09:00 h or 17:00h local time. The motivational messages selected by participants varied widely, from a generic message sent every Friday encouraging weekend relaxation, to very specific reminders that reminded them to think of the individuals who are counting on their recovery, such as ‘Remember you are leaving a legacy for your child’.
Example motivational messages selected by patient and treatment team
SMS messaging was also used in a second way to conduct ‘check-ins’ in order to support the participant’s recovery and provide a periodic assessment of the participant’s level of distress. The protocol for the check-in messages is summarized in Table 2. The assessment of functioning was done with a pain scale analogue that asked the participant ‘How are you doing overall?’. Participants were asked to respond via SMS message with a single numeric response where the number 1 indicated they were doing ‘great’ and the number 5 indicated they were doing ‘lousy’. Clinical staff and participants together set a threshold (e.g. anything above a response of ‘3’) that would trigger a call to action message back to the participants, well as notification to clinical staff and the participant’s buddy system that follow-up was warranted. The technology employed in the study was able to receive and parse the return message and trigger notification to clinical staff and the patient’s self-identified ‘buddies’ if the pre-set threshold was exceeded. Notification to clinical staff was also provided when a patient failed to respond to two or more SMS message check-ins.
Follow-up messaging protocol content
Note: [response] indicates a field which can fill the patient’s response to the question and [full name] indicates a placeholder in which the patient’s full name can be inserted.
Results
Table 3 summarizes the characteristics of the two cohorts of patients. The sample was exclusively male, predominantly young and generally exhibited a high level of psychiatric symptoms. The SCL-6 score in our population (3.5 across both cohorts) was substantially higher than in a reference population (2.2) of patients with PTSD. 22
Participant characteristics
The first round of EMA data collection yielded only 17 completed questionnaires in 75 attempts for a response rate of 23% (Figure 2). Informal feedback from patients and clinical staff indicated that the low response rate was due to the perceived intrusiveness of randomized EMA delivery. After the process was redesigned to incorporate EMA data collection into a daily group therapy session, the response rate jumped dramatically to 89% (763 completions in 854 attempts; Figure 2).
Percent of data collection attempts completed follow-up phase: Random and scheduled times
Patient adherence to the SMS messaging protocol used in the follow-up phase of treatment was good. Participants participated in messaging for an average of 72 days during the follow-up phase of treatment (Table 4). Over the study period, an average of 16.5 motivational and other system messages were sent to participants along with an average of 9.2 check-in prompts every 30 days. Participants responded to check-in messages an average of 8.2 times over the same interval. Across both phases, 92% of participants responded to at least one SMS message in the first month of use (Figure 3). In months two and three the rate of at least one response was 95% and 91%, respectively. The volume of outgoing messaging was roughly one message every 3–4 days. Messages from participants coming back into the system were somewhat less frequent—about once a week on average. Given that some of the outgoing messages were motivational and required no response, the rate of incoming messages shows a reasonable rate of participation.
Follow-up phase SMS messaging participation (n 27)
Participant engagement with SMS messaging at 30 days, 60 days and 90 days (n = 27) Note: Engagement defined as responding to at least one SMS interaction in each 30-day period
Participants generally found the messaging program useful. Five veterans in the second cohort were surveyed with a short, forced-choice and open-ended survey instrument. Sixty percent found the overall messaging experience to be helpful, 80% reported the affirmation messages and the buddy notification of distress to be helpful and all participants found the notification of clinical staff when a participant reported distress through the system to be helpful. Open-ended responses to the satisfaction questionnaire included one patient who reported that ‘I have found it to be very useful and cannot think of anything that needs to be fixed’. Another patient suggested that the message wording could be more effective if it were altered to ask about how the patient is feeling at the moment the message is received. One patient whose participation was minimal reported that ‘I don’t [use] text messages’.
Discussion
Because this was a small sample pilot project designed to assess the feasibility of using technology as an adjunct to treatment, no statistical analyses of outcomes data were conducted. The pilot did, however, produce a number of valuable findings that promise to inform future trials of this technology, which bear relevance to future mHealth applications.
First, our results strongly echo accumulated wisdom that for mobile technology to be an effective adjunct to treatment it must provide only minimal disruption to daily routines.25,26 Our initial protocol called for EMA to be delivered at random times during waking hours, which proved disruptive to participants’ schedules and led to a very low response rate. After the data collection was made part of the daily routine, response rates improved dramatically.
Second, our experience in this pilot confirms the need to account for users’ current and prior mobile technology experience and preferences in designing mHealth interventions. 27 Two devices were used initially in this protocol: non-networked PDAs were used to collect EMA data during the active phase of treatment and patients’ personal cell phones were used for messaging during the follow-up phase of treatment. Participants proved to be highly resistant to carrying two electronic devices on their person. In the first cohort that participated in EMA data collection, patients routinely left their assigned PDA in the housing unit during the day, which contributed to the low EMA response rate. The PDAs, despite being of the most recent generation, were viewed as being clunky and out of date compared with the smart phones ubiquitous among patients, echoing the experiences of other mHealth innovators, which suggest that innovative devices may be more likely to be adopted. 28 No similar challenges to adoption rates were observed for the follow-up phase interactions which occurred through the personal cell phones which participants already routinely carried and were accustomed to having on their person. Subsequent efforts to augment behavioral health treatment with EMA would benefit from the use of a single device. A smart phone which features both internet and SMS/MMS capability would permit both EMA data collection, as well as interaction via SMS message in the follow-up phase.
Third, our pilot experiences emphasize the importance of timely, easy-to-read data presentations to clinicians. Whilst the SMS data were transmitted to clinicians in real time, the EMA data collected by PDAs called for data downloads from the PDAs several times a week and weekly data uploads to a secure data center. After allowing 1–2 days for processing, data reports were returned to clinical staff between 10 and 14 days after collection. Feedback from clinical staff revealed that whilst the data reporting was valuable for historical and analytical purposes, it was often not timely enough for the clinician to react to stress or crisis situations. As a result, these data were not shared as often with clients as part of the treatment process as had been intended. Echoing the findings of other mHealth work, delivering data in a timely fashion, as well presenting data in a clinician-friendly format, has the strongest potential to produce changes in clinical decision-making. 29
Fourth, and finally, our experiences emphasize the importance of providing regular feedback to participants. Particularly in the active treatment phase of the study where we were collecting EMA data on a daily basis, participants’ casual remarks to program staff indicated that they didn’t believe the data they were providing were doing any good because they could not see any effect on their treatment. On a few occasions participants temporarily ceased participating in EMA data collection after researchers and clinicians failed to respond to a stress or crisis event recorded in EMA data. Participants invested a significant amount of time and emotion providing EMA data. Their frustration underscores the importance of providing regular positive feedback to participants and making clear to participants how the data they provide are useful. This is reinforced by the positive response of the participants to the SMS message interactions when they were involved in creating their interactions, they received a follow-up response to their replies, and, depending on their response, a buddy and care manager were notified.
In short, the results of the pilot suggest that the mobile ecological momentary assessment and SMS messaging tools are feasible adjuncts to traditional mental treatment in the veteran population. Additional work is needed to establish the degree of clinical and economic value.