Abstract
The impact of computer-based cognitive-behavioral self-help therapy programs is limited by high attrition. This study explored reactions to computer-based cognitive-behavioral self-help therapy use among individuals not completing a full treatment course. Individuals receiving outpatient substance use disorder treatment at a Veterans Health Administration clinic who enrolled in a study implementing a computer-based cognitive-behavioral self-help therapy for insomnia, but subsequently dropped out prior to completion, were interviewed. Reactions to use and reasons for attrition were explored through thematic analysis of interviews. Among barriers to use, themes of competing demands, personal attributes, the computer-based format of computer-based cognitive-behavioral self-help therapies, and negative experiences with the specific program used were identified. Among facilitators of use, themes of personal support, the computer-based cognitive-behavioral self-help therapy format, and personal attributes were identified. Recommendations for future implementation efforts to include additional person-to-person contact during computer-based cognitive-behavioral self-help therapy participation were made. These themes may be employed to develop strategies for computer-based cognitive-behavioral self-help therapy implementation in order to maximize program engagement and completion.
Keywords
Introduction
Computer-based cognitive-behavioral self-help therapies (cCBTs) are self-guided programs delivered over a computer or the Internet and focused on improving knowledge, awareness, or behavior change. 1 Participants use these therapies at a pace and in a setting of their choosing and can be provided varying levels of guidance or support during participation. Numerous evidence-based mental health–oriented cCBTs have shown efficacy for the treatment of multiple disorders including depression, anxiety, substance use, and insomnia, the disorder targeted in this study.2–4 The potential benefits of cCBTs include convenience, reduced travel barriers, provision of treatment for people who cannot come to a clinic during business hours, and reduced stigma associated with mental healthcare.5,6 The programs may also increase self-care and health-system productivity.7–9
A major issue with regard to the effectiveness and successful dissemination of these treatment programs is attrition, defined as non-completion of the course of treatment for a given program. 10 In large trials and reviews of cCBT use, attrition can commonly range up to 50 percent, and even 99 percent in programs where personal support is not provided.11,12 In a recent pragmatic feasibility trial of a cCBT for insomnia implemented among Veterans in outpatient care, 57 percent did not complete the program. 13 In comparison, attrition from large trials of face-to-face cognitive-behavioral therapy (CBT) for insomnia ranges from 10 to 20 percent.14–16 Attrition is one of a number of factors which determine the effectiveness of such programs and is likely vital as studies have shown that program completion is associated with improved clinical outcomes. 17
Prior research on reasons for attrition from cCBTs has involved analysis of data from clinical trials either in the form of structured follow-up surveys or the qualitative analysis of participant interviews. This literature is associated primarily with programs for the treatment of depression or anxiety; few studies have evaluated cCBT engagement or attrition for individuals in substance use or insomnia treatment. There is evidence that the most important facilitator of engagement and completion is the provision of support or guidance to individuals as they participate. 11 Support has been provided in the form of face-to-face, telephone, or messaging interactions and is thought to encourage engagement and completion, as well as improve outcomes.18,19 Secondary factors associated with attrition can be loosely grouped into “patient-” and “treatment”-related categories as Johansson et al. 20 have discussed in one of the only analyses of interviews specifically with individuals who dropped out of cCBT treatment. Patient-associated factors include low computer literacy, competing demands on participant time or energy, symptom improvement prior to completion, and the potential negative consequences of improvement.11,20–24 Treatment-associated factors include difficulty with Internet/computer access, problems with specific program content (e.g. too boring or too demanding), and technology fatigue.11,20,21
Limiting attrition will be important to improving cCBT effectiveness and dissemination in outpatient care. Therefore, a better understanding of the reasons for attrition is needed in order to support program engagement and completion. This study was undertaken to validate findings from previous qualitative and quantitative studies on attrition from cCBTs in a unique population: Veterans in outpatient Veteran Health Administration (VHA) substance abuse care. The study employed semi-structured interviews to explore personal reactions to using a cCBT for insomnia.
Methods
Study design
The feasibility of implementing a cCBT for insomnia in a VHA outpatient clinic for substance use treatment was tested in an open trial with 51 participants and has been described previously. 13 The cCBT, RESTORE™, is a six-session, Internet-based, self-administered program consisting of therapeutic components common to other face-to-face and self-help CBTs for insomnia. The RESTORE program has shown efficacy in a randomized and controlled trial. 25 In our prior study, RESTORE was implemented using a strategy consisting of face-to-face provider and patient education sessions, on-site Internet access, and clinician telephone support. 13 The program and implementation strategy were usability tested in two individuals prior to the start of the trial. Participants who chose to stop using the program prior to completing all sessions were asked to participate in a one-on-one interview with research personnel during their follow-up contact session.
Sample
Participants were English speaking with chronic insomnia and no evidence of sleep apnea, restless leg syndrome, recent psychiatric instability, or concurrent treatment with buprenorphine, methadone, or CBT for insomnia. Of 29 individuals who did not complete the program, 12 (41%) agreed to an interview. Participants provided informed consent in accordance with protocols approved by the institutional review boards of the VA Connecticut Healthcare System and Yale University School of Medicine.
Data gathering
The principal investigator (E.D.A.H.) conducted and audio recorded interviews in a quiet private area of the clinic. Interviews were semi-structured using an interview guide that introduced the purpose of the study and allowed for exploration of topics, such as reasons for attrition and recommendations for future efforts. Interviews lasted between 30 min and 1 h.
Data analysis
Interviews were transcribed verbatim to obtain a final set of data for analysis. Thematic Analysis, a foundational approach in the analysis of in-depth interviews, was used to analyze interview content and included a five-step process used by the first four authors: (1) familiarization with transcribed data; (2) generation of initial codes (short grammatical phases labeling data segments from interviewee statements); (3) collating codes into potential themes; (4) reviewing, discussing, and modifying themes in relation to coded extracts; and (5) developing a typology of and defining themes. 26 To facilitate in-depth analysis and decrease travel burden, the first four authors divided into two teams to perform an initial analysis and coding of interview content (phases 1, 2, 3 above) by randomly dividing the interviews into two groups. Phases 4 and 5 used a group process involving all authors to develop consensus. Discrepancies were resolved through discussion with final adjudication by the first author. Codes within themes were then reviewed using the entire data set to identify examples and counter-factuals.
Results
On average, the sample of Veterans interviewed consisted of middle-aged males with a high school education. An equal proportion reported being White or another race/ethnicity, and most reported at least one psychiatric disorder. The sample completing interviews did not differ statistically from those not interviewed by socio-demographic characteristics or reported psychiatric diagnoses (Table 1).
Socio-demographic characteristics of individuals interviewed among those not completing a cCBT for insomnia.
cCBT: computer-based cognitive-behavioral self-help therapy; SD: standard deviation.
Between-group comparisons were tested with chi-square for categorical variables and analysis of variance for continuous variables.
After the analysis of interviews, personal reactions to using a cCBT for insomnia were organized into three groups: barriers to completion, facilitators of engagement, and participant suggestions for future implementation efforts.
Barriers to completion
Barriers to completion were noted in all interviews, with four identified sub-themes: competing demands from other activities, characteristics of individual participants, aspects of the computer-based format, and negative experiences with the specific program.
Competing demands
Competing demands were the most commonly cited barriers, found in 11 interviews (92%). Interviewees described spending significant time and effort on the performance of or motivation to perform activities other than those related to the cCBT. A prominent competing demand (n = 7, 58%) was difficulty prioritizing the program among activities related to other health problems. The high number of medical and mental health appointments as well as time spent attending to other health difficulties were described as direct competitors with cCBT completion (the direct effect of psychiatric and medical symptomatology is discussed in a later section). One Veteran said, “I got these appointments that they [the VHA] are bombarding me with … so there’s a lot of stuff I gotta do …” Many participants (n = 6, 50%) said their simultaneous enrollment in college courses interfered with cCBT program completion, even though some said they had thought, prior to engagement, that the cCBT would dovetail nicely with coursework: “I started school right at the same time and way overloaded myself; took on way more than I could handle first time back in school in fourteen years.” A meaningful portion (25%) were also struggling to meet basic needs such as paying bills and obtaining stable housing and talked about the negative impact of these demands on program completion: “I mean, you gotta make sure you have what you need in order to even begin to think about something else.”
Attributes of the individual
Participants talked about personal traits, characteristics, or symptoms that contributed to difficulty completing the program. In nine interviews (75%), cognitive symptoms including difficulty concentrating, forgetfulness, and distractibility, as well as anxiety, were noted. Anxiety symptoms were described either as having no antecedent cause or as associated with other factors related to discontinuation, such as competing demands from other activities: “I won’t be able to concentrate and actually take in what I’m doing. I get easily distracted and I just can’t feel comfortable enough to get the full benefit.” In three interviews, medical disorders, such as acutely elevated blood sugar or chronic pain, interfered with concentration and the ability to attend to the program. “I could sit down, but I have to get up. I have to get up so that my legs actually don’t freeze up on me.” In addition, several participant statements revealed a general lack of receptivity to new treatment options or ideas related to their care: “But me, I’m not as sharp as I used to be. I’m not as receptive as I used to be for learning new things.” Alternatively, several non-completers said, in effect, “I’ll try anything.” perhaps suggesting that they agreed to participate in the program without an adequate understanding of the commitment or work needed for completion. A similar attitude of “I’ll try anything for the good of Veterans” was described by other individuals and is discussed below as a facilitator to engagement.
Aspects of the computer-based format
Participants identified features of the computer-based format of cCBTs and their lack of familiarity with computer technology as barriers to program completion. Eight participants (67%) mentioned stress related to a lack of experience with, or ability to use, computers or the Internet as reasons for non-completion: “… I didn’t know how to turn it on. That’s one of my things that puts me in a tense or anxious situation. So I try to eliminate that.” Related to the stress associated with low technology literacy were additional negative feelings toward computers such as fear or distrust associated with prior experience with them, noted by seven participants (58%): “I hated it. They [Army] make you sit at the computer for three hours at a time and do stuff, and do some stupid thing on line that doesn’t even matter.” Seven participants (58%) said that limited access to a computer with an Internet connection in a safe, private, space was a barrier to engagement and completion. Several in this group noted that they had recently lost a previously stable source of Internet or computer resources:
… at the time I started the study my personal computer crapped the bed and that left me to either have to come to the VA or go to a library or public facility or something like that … I could have put more effort into going to the library but like I just don’t feel comfortable in a public place to do stuff like that.
Negative experiences with the program
This theme related to characteristics of the specific program used, RESTORE, separate from the computer-based format. Most participants cited one main negative experience with the program, while also echoing other common complaints. The four most common complaints describing negative experiences are shown in Table 2.
Negative experiences with the specific program.
Facilitators of engagement
Almost all individuals who did not complete the program and were interviewed, n = 11 (92%), also noted themes that facilitated engagement in the program, such as clinician support, attributes of the individual, and characteristics of the computer-based format.
Staff support
Part of the strategy used to implement the program involved a clinician educating participants during an initial session, supplemented by weekly follow-up phone contact. Eight participants (67%) said this support was helpful, by encouraging them to engage, providing technical help, or simply knowing help was available if needed: “I was probably getting a little lost in my own situation. By you checking up on me, it kind of brought my attention back to me thinking more, getting more personalized with this study.” In addition, two participants spoke of helpful support from family members.
Attributes of the individual
Seven participants (58%) mentioned a variety of personal characteristics that facilitated enrollment in the program. Two characteristics stood out—the willingness of participants to try anything they believed might help improve their symptoms and a sense of duty or public service: “My thought at the beginning of the program was maybe it will be beneficial to myself as well as others. If I can do it, I’ll do it … Basically, I’ll try anything for the benefit of future veterans.” This sense of duty and public service differed from the “I’ll try anything” attitude described as a barrier, because it focused on service to others and was described as a reason why individuals signed up to participate in the research project.
Aspects of the computer-based format
Seven individuals (58%) noted specific aspects of the computer-based format that facilitated engagement, such as the privacy it provided, the convenience of 24/7 access to treatment, and the opportunity to proceed at one’s own pace: “You could have more time to think about stuff before you put it in there. Because you’re doing it on your own instead of sitting in front of someone getting asked questions.”
Suggestions for future implementation efforts
At the end of each interview, Veterans were asked “Do you have any suggestions on how the use of this program [cCBT] could be improved?” Six individuals (50%) indicated a desire for increased personal interactions in the form of face-to-face support, stating that these interactions would increase motivation and provide a format for trouble shooting problems and getting questions answered. Several participants also suggested the addition of a face-to-face peer support group of individuals using the program as a way to facilitate this support:
It [a support group with other participants] maybe a “meet and greet.” You know, to feed off of each other … I wouldn’t want to do the group all the time … even once a month …. You might think of something while talking to them, “Yeah, that’s how I felt,” and bounce ideas off each other.
Discussion
Although there is strong evidence for the efficacy of many cCBTs for common mental health conditions, attrition is one of the factors that limits program effectiveness in clinical practice. 12 To develop strategies for implementing cCBTs that support program completion and limit attrition, a better understanding of the reasons for attrition from cCBTs must be developed, especially among Veterans in outpatient care where this work has yet to be completed. This study explored personal reactions to using a cCBT for insomnia among Veterans in substance abuse treatment. The most important barriers to program completion were patient factors such as competing demands for participants’ time, attention, and energy, followed by personal attributes such as anxiety about technology and low technology literacy. Participants also identified face-to-face support from clinicians and other Veterans as facilitators of engagement and recommendations for future implementation efforts.
The scant data available on reasons for attrition from cCBTs come primarily from participant assessments during controlled clinical trials and interviews with individuals who have completed their respective programs.21,22,24,27 A recent similar qualitative study in individuals who did not complete a program for generalized anxiety by Johansson et al. 20 helpfully divided reasons for non-adherence into patient- and treatment-related factors. Patient factors may be the most influential as they appear to be most widely shared among studies, the most important of which are competing demands from other activities and low computer literacy in some participants. Competing demands (alternatively described as “life factors,” 20 feeling “overwhelmed,” 24 and “motivation” in the face of other demands), 11 especially with participants’ other medical care and educational activities as identified here, are some of the most important barriers in this as well as other studies. 21 However, treatment factors such as characteristics of the specific program, especially its ability to hold the participants attention, and aspects of the computer-based format, such as the need for reliable Internet or computer access, are also important factors in attrition revealed in a number of other studies.11,20
Other barriers identified here have not been noted in previous studies such as physical illness. The theme of physical illness is closely associated with the competing demand of medical care and may have been more pronounced in this study because VA patients are generally older and have more medical comorbidities than members of the general population. 28 However, it is notable that while technology access and literacy were cited as barriers and are thought to be common among the elderly, the barrier of competing demands appeared to outweigh these considerations.
Another key finding of this study was a group of themes facilitating engagement in this group of participants who actually did not complete the program. This finding suggests that the changes in the strategy used to implement cCBTs may encourage increased engagement and completion. Many of the facilitators and suggestions revealed here have been emphasized in other studies, such as the relative freedom from scheduling and travel barriers associated with face-to-face therapy as well as the anonymity cCBTs provide.21,24
Themes identified in this study will be helpful in developing a comprehensive strategy to support the implementation of cCBTs in VHA outpatient care by reducing identified barriers and strengthening facilitators. As observed in a recent large-scale effectiveness trial of cCBTs for depression in the United Kingdom, a lack of engagement and completion is associated with a lack of program effectiveness. 12 Primary among participant’s suggestions for future implementation was the need for users to have direct contact with individuals while participating in the program.11,19,20 This study has led our group to define a cCBT implementation strategy, which includes training a clinical intermediary, other than the referring provider, who will provide face-to-face and other modalities of support and education to patients throughout their engagement with the program. We also recommend provider and staff facilitation and education to encourage referral, as well as stepped-care for those requiring additional treatment. 29
There are several characteristics of this study that make it unique within the cCBT attrition literature. As discussed above, this is one of the only studies utilizing interviews with individuals who dropped out of a cCBT program. 20 Although the literature on cCBT programs for the treatment of insomnia is growing, this is the only work in this group which specifically addresses attrition through the analysis of participant interviews. Likewise, this is the only cCBT study in the substance abuse treatment setting to address attrition and insomnia treatment. Prior work in this area has primarily focused on alcohol and tobacco use.30,31 Findings from these studies in conjunction with this study and its unique sample suggest that barriers and facilitators of cCBT use may not be disorder specific but hinge more on factors related to the cCBT platform and an individual’s response to the platform.
Primary limitations of this study concern the sample: individuals in VHA outpatient substance use treatment who did not complete a cCBT for insomnia. The group interviewed consists of Veterans, who are characteristically older with more medical and psychiatric comorbidities compared to the general population and receive care within a unique, integrated, and nationwide healthcare system. Their experience may differ from those who are non-VHA service users, or receive treatment in non-specialty care settings. Moreover, only 41 percent of participants not completing the cCBT volunteered to be interviewed, potentially introducing a selection bias, as their views may be different than those not interviewed. In addition, the process of interviews, which were completed by the researcher conducting the trial, may have affected responses and the topics discussed. Therefore, the results cannot be regarded as representative for all individuals who receive computer-based self-help treatment, especially those who completed the program. However, the immediate goal of this work was to identify an initial and general set of barriers, facilitators, and user suggestions on which to base the content of an implementation strategy for cCBTs in VHA outpatient care. Therefore, the selection of this sample was appropriate in this context. An additional limitation is that successful implementation of cCBTs hinge on barriers and facilitators identified by providers and administrators, who also play key roles in such efforts, and institutional factors that may either support or compromise the successful implementation of cCBT programs. 32 Efforts are currently underway to uncover such factors.
Limiting attrition from cCBTs will be a key factor in realizing their effectiveness and successful dissemination in outpatient care. This study uncovered barriers to cCBT completion such as competing demands on Veteran time and limited technology literacy, as well as facilitators of cCBT engagement and suggestions for future implementation, such as face-to-face support from clinicians and other Veterans. This information will be used to develop robust implementation strategies to foster increased engagement in and completion of these programs with the goal of increasing access to evidence-based mental healthcare.
Footnotes
Acknowledgements
The RESTORE™ program was provided through a contract and data use agreement with Cobalt Therapeutics, LLC, for whose support and encouragement we are grateful.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research outlined here was supported by the Department of Veterans Affairs, Veterans Health Affairs, VISN 1 Career Development Award (grant number V1CDA12012-17) to E.D.A.H., MD, and the VA New England Mental Illness Research, Education, and Clinical Center and the VA Connecticut Healthcare System. The funding sources had no role in the design, analysis, or interpretation of data or in the preparation of the report or decision to publish.