Abstract
There has been growing interest in utilizing mobile phone applications (apps) to enhance traditional psychotherapy. Previous research has suggested that apps may facilitate patients’ completion of cognitive behavioral therapy for insomnia (CBT-I) tasks and potentially increase adherence. This randomized clinical trial pilot study (n = 18) sought to examine the feasibility, acceptability, and potential impact on adherence and sleep outcomes related to CBT-I Coach use. All participants were engaged in CBT-I, with one group receiving the app as a supplement and one non-app group. We found that patients consistently used the app as intended, particularly the sleep diary and reminder functions. They reported that it was highly acceptable to use. Importantly, the app did not compromise or undermine benefits of cognitive behavioral therapy for insomnia and patients in both groups had significantly improved sleep outcomes following treatment.
Chronic insomnia affects more than 1 in 10 people. 1 Rates are even higher among veterans using Veterans Health Administration (VHA) services. 2 Cognitive behavioral therapy for insomnia (CBT-I) is an effective treatment for insomnia3–7 with superior long-term efficacy 8 and fewer risks compared to hypnotic medications. 9 As such it is widely recommended as a first-line treatment for insomnia. 1
The VHA has an ongoing national dissemination initiative to train its licensed mental health care providers (i.e. non-sleep specialists) to deliver CBT-I. 10 Patient outcomes from this training program are comparable to those of published research trials.11,12 However, patient non-adherence to treatment recommendations (e.g. going to bed only when sleepy and limiting naps) may be undermining the benefit for some patients.
Mobile applications (apps) utilized with smartphones have the potential to improve traditional psychotherapy by enhancing access to psychoeducation and psychotherapy skills (e.g. relaxation techniques), facilitating monitoring of symptoms and outcomes, and assisting with relapse prevention.13,14 Emerging literature is beginning to accumulate showing the promise of apps for depression, stress, psychosis, eating disorders, and substance use15–19 and integrating mental health apps into treatment within a VA setting shows initial promise. 20 Although there is increasing availability of consumer sleep technologies for sleep improvement, many of the existing apps focus on monitoring sleep and providing sleep education, rather than assisting with active therapy.21–23
The VHA National Center for Post-Traumatic Stress Disorder (PTSD), in partnership with Stanford University School of Medicine and the Department of Defense’s National Center for Telehealth and Technology, built CBT-I Coach, a patient app designed to be used as an adjunct to face-to-face CBT-I. It was released for both iOS 24 and Android 25 mobile devices in 2013. It was specifically designed to help facilitate patients’ completion of CBT-I tasks, potentially increasing patient adherence to the protocol. 26 The design and content of the app has been described in detail in previous publications.26,27 Briefly, the app provides education about sleep processes, developing positive sleep routines, and improving sleep environments. Key features of the app include a sleep diary to record daily sleep variables (see Figure 1), ability to update a sleep prescription (recommended bedtime and wake time) in consultation with CBT-I providers, tools and guided exercises for quieting the mind (see Figure 1), education about sleep and sleep-health behaviors, reminder functions and alarms to help change sleep habits (e.g. reminders for when to stop caffeine intake, start wind-down time, and alarms for prescribed bedtime and wake time). See Figure 1 for a screenshot of the home page of the app.
Example screenshots from CBT-I Coach.
In a recent survey, VA CBT-I clinicians indicated that they believe this app may improve care and increase adherence, 26 and an initial feasibility study investigating the use of the app among patients engaged in a cannabis cessation attempt indicated daily engagement with the app. 27 However, it is uncertain if integrating CBT-I Coach into the existing VA CBT-I protocol is feasible in terms of whether patients will consistently use it as intended and if they will find it acceptable to use while engaged in this therapy. It is also unknown if CBT-I Coach will actually improve patient adherence to the therapy and, importantly, whether using the app will compromise or undermine benefits of CBT-I, a well-established evidence-based treatment, on sleep-related outcomes. Therefore, the current study sought to examine the feasibility and acceptability of CBT-I Coach, as well as explore the potential impact of this device on adherence and sleep outcomes. The first-author, in conjunction with the developers of the app (second, sixth, and seventh authors) utilized the existing app in a sample of patients receiving CBT-I in a clinical setting. We hypothesized that participants randomized to CBT-I with the app and without the app would report significant improvements in sleep, but that the app group would have higher adherence than the non-app group.
Methods
Participants
This study was approved by the Veterans Affairs Institutional Review Board. Forty-one consecutive referrals for CBT-I at a Midwestern VA Medical Center were pre-screened following CBT-I intake assessments by project clinicians, with further screening via phone by the project coordinator. Inclusion criteria included commitment to begin CBT-I, ownership of a smartphone, and willingness to use CBT-I Coach. Exclusion criteria included moderate or greater suicidal or homicidal ideation, significant alcohol or drug use, and active psychotic symptoms. Patients not meeting these criteria where referred elsewhere or underwent CBT-I outside of the study.
Figure 2 presents the flow of participants through the study from March 2014 to November 2014. Twenty-three potential participants were excluded, most commonly for not owning a smartphone (n = 9) and not imminently planning to begin CBT-I (n = 8). Two participants declined to participate due to lack of interest. After obtaining informed consent, 18 participants were randomized equally to either CBT-I plus CBT-I Coach (app group) or CBT-I without the app (non-app group) by the project coordinator, utilizing a computerized random number generator. Table 1 presents baseline characteristics. The average age was 48.50 years (standard deviation (SD) = 14.93), and participants were primarily White males with Android smartphones. There were no significant differences between the two conditions on baseline insomnia symptom levels or demographics, with the exception of marital status (p < .05), where those in the app group were more likely to be married compared to the non-app group (55.56% vs 11.11%). On average, participants reported clinical levels of insomnia. Three participants dropped out of the non-app group (two lost interest in treatment and dropped out after three sessions, one moved and dropped out after two sessions); none dropped out of the app group. There were no significant differences between retained participants and those who dropped out on baseline insomnia symptom levels or demographics.
Flow of participants through the study.
Baseline characteristics.
SD: standard deviation.
Based on n = 8 in non-coach due to missing data.
Significant χ2 difference at p ⩽ .05. Insomnia Severity Index ⩾ 15 indicates clinical levels of insomnia.
Procedure
Participants completed semi-structured interviews at baseline and following treatment, as well as self-report measures prior to each CBT-I treatment session. Participants received a US$25 gift card for completing the baseline assessment and a US$50 gift card for the final assessment. Pre-treatment interviews were identical for both app and non-app groups and were used to obtain demographic information and baseline experience with apps, including questions about phone usage and comfort, and app usage. Following the initial interview, participants assigned to the app group were shown how to download and use the app. The app participants used the app in conjunction with the standard CBT-I procedures, including using it to complete sleep diaries. Non-app group participants completed CBT-I according to standard procedures without CBT-I Coach.
In order to explore the impact of the app on adherence to treatment recommendations and sleep outcomes, participants in both groups reported the number of days they completed homework, amount of time spent on homework each week, and completed an insomnia questionnaire before each therapy session. Therapists completed a measure of patient adherence to treatment recommendations at the end of each session for all participants. At post-treatment, app group participants completed semi-structured interviews focused on their use and engagement with the app, including their impressions of CBT-I Coach, which elements they used, barriers to app usage, perceived value of the app, and potential enhancements of the app. The non-app group participated in a semi-structured interview during which they were shown the app and asked for their thoughts on integrating the app into CBT-I and suggestions for enhancements.
Treatment and therapists
Participants in both conditions completed CBT-I with one of two VA clinical psychologists who completed the VHA CBT-I training initiative.10,11 Treatment consisted of weekly 1-h individual therapy sessions based on the CBT-I manual developed by VA. 10 The protocol consists of five treatment sessions, with patients attending fewer or more sessions if clinically indicated. The average number of treatment sessions completed in this study were four (66.67% of patients), with almost a third of patients completing five sessions (27.78% of patients). The basic components of this protocol include: (1) sleep restriction, which involves limiting time in bed to consolidate sleep; (2) stimulus control, which involves restricting the bed/bedroom to sleep; and (3) cognitive restructuring, which addresses maladaptive thoughts and beliefs about sleep.
Measures
Insomnia Severity Index
The Insomnia Severity Index (ISI) 28 is a 7-item self-report measure designed to provide a global measure of difficulties sleeping at night and daytime impairment and was completed by participants during each session. Respondents rate statements using 4-point scales from 0 to 4 (response options differ by item) and items are summed to provide a total score. Scores of 0–7 indicate no clinical insomnia, 8–14 indicate subthreshold insomnia, 15–21 indicate moderate insomnia, and 22–28 indicate severe insomnia. This instrument has adequate psychometric properties, including internal reliability (coefficient alphas ranging from .76 to .78) and concurrent validity with sleep diaries and polysomnography. 28
Adherence scale
Adherence to CBT-I recommendations was measured using the Patient Adherence Form that was created for the VA CBT-I Training Program. 10 Starting at the end of session two, therapists rated the extent to which participants followed six specific recommendations (e.g. adhering to recommended bedtime and wake time, limiting naps, and scheduling worry time) on a scale from 1 (no adherence) to 6 (complete adherence) or not applicable (NA) if the recommendations had not yet been introduced during the therapy. Scores were averaged to create a total adherence score that has shown good psychometric properties. 12
Statistical analyses
Descriptive statistics were calculated for variables related to feasibility and acceptability from the semi-structured interviews. Independent samples t-tests were calculated to examine variables potentially related to treatment impact, including patient adherence and homework completion. Treatment outcomes analyses were conducted using intent-to-treat for ISI scores. These analyses included all available data from 17 participants; one participant was excluded due to errors in coding outcome data. Hierarchical Linear Modeling (HLM) was conducted using SAS (PROC mixed) to determine mean values at each time point and test the effect of treatment and condition on ISI scores during five treatment sessions. Fixed effects were specified for time and condition, whereas random effects accounted for the nested nature of the data with repeated measures over time within individuals. Effect sizes were calculated using Cohen’s d, which represents the standardized differences between means.
Results
Feasibility and acceptability of integrating CBT-I Coach into therapy
Table 2 presents descriptive statistics of smartphone and mobile app variables for all study participants. The majority of participants indicated having their phone with them all of the time (n = 14, 77.8%) and owning a smartphone for at least 5 years (n = 13, 72.2%). Most participants indicated using their smartphone for calls (n = 16, 88.9%) and apps (n = 15, 83.33%) at least daily and most indicated being very comfortable using their smartphone generally (n = 16, 88. 9%) and apps specifically (n = 14, 77.8%). Very few participants had used apps for mental health or sleep (n = 2, 11.1%) and none had used CBT-I Coach previously. Demographic variables, including age, gender, marital status, ethnicity, and education, were not significantly related to degree of comfort with apps or frequency of app use.
Pre-treatment interview: feasibility and acceptability of mobile applications (n = 18).
Table 3 summarizes findings from the post-treatment semi-structured interview with the app group participants. The most commonly used element of the app was the sleep diary, followed by the educational materials, relaxation exercises, and reminders. All participants reported that the sleep diary was a helpful component (n = 9, 100%; e.g. “the sleep efficiency numbers made me feel better about my sleep and decreased my anxiety about sleep. I liked being able to push enter and get the results”), followed by reminders (n = 2, 22.2%; e.g. “Reminders kept me on track”). All participants indicated that they would recommend the app to family or friends. In general, feedback from participants in the app group was positive and focused on the personalized feedback provided by the app, particularly as it related to the sleep diary information (e.g. “I like the app because it gives you info on what you are doing compared to what you think you are doing”). The non-app participants also provided feedback about CBT-I Coach. One participant indicated that it would increase compliance: “I think it would be very helpful because no one loses their phone, but I lost my sleep log.” Another stated, “I think it’s a good idea, it’s a lot easier to do homework because you always have your phone with you.”
Post-treatment interview: feasibility and acceptability of CBT-I Coach with participants in the app condition.
CBT-I: cognitive behavioral therapy for insomnia.
Impact of CBT-I Coach on CBT-I adherence and outcomes
There were no statistically significant differences between the app and non-app group on average time spent on homework (d = .66 in favor of non-app group), number of days completing homework (d = .11 in favor of non-app group), and days completing sleep diaries (d = .36 in favor of app group). The effect size for the group difference on adherence scores was large (d = .76), favoring the app group, but it was not statistically significant (t(16) = 1.53, p = .15).
HLM intent to treat analysis estimates for time was significant for ISI scores, F(4, 42) = 8.84, p < .001, d = 1.84 (see Figure 3). The main effect for treatment condition (d = .21) and the interaction of treatment time by condition (d = .73) were statistically non-significant. Among participants who completed treatment, four (26.7%) had no insomnia, eight (53.3%) reported subthreshold insomnia, one (6.7%) reported moderate insomnia, and two (13.3%) reported severe insomnia according to ISI scores.
Hierarchical linear model predicted Insomnia Severity Index (ISI) scores at each therapy session (n = 17). Error bars represent standard error above and below predicted values.
Discussion
This is the first study to report the feasibility and acceptability of integrating an app with CBT-I. Previous research suggests that providers see the app as potentially improving care and increasing adherence 26 and this study suggests that integrating CBT-I Coach with individual CBT-I is highly feasible and acceptable to patients. Overall, the app was favorably received by all participants in the app group and participants in the non-app group responded favorably when they were introduced to the app at the end of therapy. The qualitative data suggest that patients were using the app as it was intended (particularly the sleep log and reminder functions) and the app improved accessibility to therapy materials.
This study is also the first to report on the impact of CBT-I Coach on process variables, including homework completion, adherence, and accessibility to therapy components. As hypothesized, CBT-I remains an effective treatment after integrating the app. Use of the app did not appear to erode or dilute the basic elements of the therapy. There was also some indication that app use related to better adherence to therapeutic recommendations. Although this finding was non-significant, the large effect size produced suggests that members of the app group are perceived as being more adherent by therapists. The study was clearly underpowered, but this potential effect represents a promising area for future research.
Although this study represents an important first step in investigating the role of mobile interventions in the treatment of insomnia, there are several limitations that should be noted. The trial was non-blinded, which may have biased adherence ratings by therapists (e.g. higher adherence scores for patients in app group). The research context may have increased homework compliance and sleep diary completion by patients in both groups (e.g. higher homework completion across all patients due to weekly monitoring, resulting in non-significant group differences). In addition, the app did not provide time-date stamped data, and so we could not confirm if participants completed the sleep diary within an hour or two of awakening as is recommended. Moreover, we were not able to collect objective information on the amount of time spent using the app, as this capacity is not built into the app.
Integrating mobile health (mHealth) into behavioral sleep treatments represents a promising area for future research, particularly within the context of stepped care models. Demand for sleep treatment often exceeds the availability of trained providers, which validates and necessitates the use of stepped care models. These models are often conceptualized in the shape of a pyramid, with the base representing low intensity, low resource treatment modalities (e.g. self-help) as an entry point, with treatments growing progressively more resource heavy and intensive as one moves to higher levels.29,30 Apps based on CBT-I principles may represent an entry level step for some patients and pilot trials are needed with stand-alone CBT-I apps to determine treatment efficacy. In addition to potentially serving as a stand-alone treatment method, behavioral sleep treatment apps provide an efficient way to collect information (e.g. baseline sleep diary data) and deliver basic sleep education to patients while they are waiting for care. Finally, the app could potentially be utilized as a screening tool to help providers determine what level of care is necessary for an individual patient, in accordance with baseline symptom levels and initial improvement in sleep following independent app utilization; additional studies with apps are needed to determine how they can contribute to personalized treatment by matching patients with appropriate levels of care.
Footnotes
Acknowledgements
The views expressed in this article are those of the authors and do not reflect the official policy or position of the US Department of Veterans Affairs.
Funding
This material is the result of work supported with resources and the use of facilities at the Minneapolis VA Health Care System and the VA Palo Alto Health Care System.