Computer-based training for cognitive behavioral therapy (CBT4CBT): A mixed methods investigation

Abstract

Computer-Based Training for Cognitive Behavioral Therapy (CBT4CBT) is an online intervention for individuals with substance use disorder (SUD).

Objective

The aim of this study was: (1) to investigate changes in CBT related skills, quality of life, and SUD severity in adults completing a CBT4CBT intervention, and (2) to explore the participant experience of CBT4CBT in a tertiary hospital specializing in mental health and substance use health in Canada.

Methods

Participants included 51 adults seeking treatment for SUD. Participants received access to CBT4CBT over 8 weeks. Measures assessing CBT skills and associated constructs (distress tolerance, change assessment, drug-taking confidence, and change strategies inventory), substance use outcomes, and quality of life were collected at baseline and post-treatment. A qualitative interview was conducted with 26 participants post-treatment. Statistical analysis was conducted using a series of linear mixed effects models examining changes from week 0 to week 8 across all measures.

Results

Significant effects of time were found for SUD symptoms, change strategies, distress tolerance, and quality of life. Qualitative analysis found that participants reported the skills modules on managing triggers and dealing with cravings to be the most helpful. Further, participants found the convenience and relatability of the program scenarios most beneficial. Participants suggested that a live connection to a researcher or a practitioner and some technology enhancements would improve the program.

Conclusion

The results suggest that CBT4CBT is linked with a reduction in SUD symptoms and an increase in CBT skills, including change strategies and distress tolerance, as well as improved quality of life.

Keywords

Substance use digital intervention mixed methods treatment cognitive behavioral therapy

Effective treatment for substance use disorders (SUD) is critical given the high prevalence and devastating impacts of SUD in Canada.¹ Approximately 1 in 5 Canadians meet criteria for diagnosis of SUD during their lifetime.² Evidence supports the use of cognitive behavioral therapy (CBT), as a monotherapy or in combination with pharmacological interventions, as treatment of a range of SUDs.³ CBT for SUD targets cognitive, affective, and environmental risks for substance use, and includes training in self-control skills to help clients achieve substance use goals.⁴ Consistent with the overall CBT model, CBT for SUD involves targeting maladaptive behavioral and cognitive patterns by developing coping, problem-solving, and emotion management skills. Increases in these skills have demonstrated efficacy in helping individuals reduce substance use by supporting their ability to understand their urges and triggers and learn how to cope adaptively.⁵

Despite the high prevalence of SUD, data indicate that only one in eight individuals receive specialized services such as CBT.⁶ Significant barriers to accessing quality treatment include affordability, accessibility, societal stigma, and attitudes of healthcare workers.^7–10 Lastly, many care providers often do not have the training or supervision to provide treatments for SUD such as CBT, particularly with fidelity to the protocols evaluated in randomized controlled trials (RCTs).¹¹ Delivering CBT digitally may provide invaluable opportunities for evidence-based care for a broader group of individuals, including those with mobility issues, who live in remote locations, and who may not seek treatment due to other mental health concerns or stigma.¹² Findings from Marsch and colleagues (2014) suggest digital interventions can be effective in creating positive outcomes for individuals with SUD, including developing key skills to manage SUD.^13,14 In response to this clinical need, Carroll and colleagues developed Computer-Based Training for Cognitive Behavioral Therapy (CBT4CBT) for adults seeking treatment for SUD. CBT4CBT is an online program that teaches individuals key CBT skills related to substance use. The program was based on an established CBT protocol for treating cocaine use disorder by the National Institute of Drug Abuse.¹⁵

Accumulated evidence supports the efficacy of CBT4CBT as both an adjunct and stand-alone intervention. Carroll and colleagues randomized 77 adults with substance dependence to receive either outpatient treatment as usual (TAU) or TAU plus CBT4CBT. Results revealed that individuals in the CBT4CBT group were more engaged in treatment, had higher module completion, and exhibited greater abstinence from substances compared to the TAU group; benefits were maintained at a 6-month follow-up.^16,17 Since this seminal trial, RCTs have supported CBT4CBT as an efficacious adjunct treatment for individuals with specific SUDs such as cocaine and alcohol.^18,19 Evidence has further supported CBT4CBT as an efficacious adjunct and stand-alone treatment for individuals who use a variety of substances including alcohol, cannabis, opioids, and cocaine.^19,20 Three different versions of CBT4CBT have now been developed, including a version with a general focus on all SUDs,¹⁶ another with a more specific focus on alcohol use disorder,¹⁹ and an adaptation for Spanish-speaking populations.²¹

Applied research has increasingly focused on skill building and broader changes that occur during CBT4CBT, to further inform its implementation in clinical settings and care pathways. CBT4CBT teaches seven strategies including: recognizing and changing patterns of use, coping with craving and urges to use, challenging and changing negative thoughts, improving decision making skills, learning how to say “no” effectively, and improving problem-solving skills.²² In an early investigation, Kiluk and colleagues examined how CBT4CBT is linked to improvements in coping skills using a role-playing scenario.²³ Participants with SUD were asked to role play in a scenario that simulated a high risk for using drugs or alcohol immediately following 8 weeks of CBT4CBT. Results indicated that participants who received CBT4CBT demonstrated higher quality coping strategies than those who received TAU, as assessed by independent raters. This effect was not replicated in a subsequent investigation of cocaine use disorder, however.²⁴ More recently, Roos et al., evaluated changes in coping skills in adults with SUD who were randomized to receive in-person CBT, CBT4CBT plus brief monitoring, or TAU.²⁵ Results suggested that long-term (but not short-term) increases in change strategies may be a mechanism of change in CBT4CBT. Overall, skills acquisition shows promise as a mechanism of behavior change in those receiving CBT4CBT, but further investigation is needed to understand the nature of these effects.¹¹

The present study aimed to assess change in a range of clinical outcomes relevant to skills acquisition in an ecologically valid investigation, that is, within a heterogeneous group of treatment-seeking individuals with SUD in Canada. A recent mixed-methods investigation supported the therapeutic benefits of CBT4CBT delivered as a stand-alone treatment in a Canadian tertiary care setting.²⁶ The current study extended this research in several important ways. First, this is the first qualitative analysis of participant experience, and provides a rich understanding of the benefits and barriers of this treatment. Qualitative analysis provides a much-needed lived experience voice to the existing empirical foundation for CBT4CBT, which to date has prioritized rigorous quantitative support. Second, the current study included novel clinical outcomes, including an important functional outcomes such as quality of life. Improvements in functional outcomes may have greater ecological validity, and relevance to patients, families, and other key knowledge users, and have evinced greater prediction of long-term outcomes than consumption-based outcomes such as abstinence.^27–29 Third, potential mechanistic factors related to skills acquisition and treatment outcomes (distress tolerance, coping abilities, confidence) were included. These outcomes were identified to be of transdiagnostic relevance and potential clinical utility within complex patient populations, while also providing initial evaluation and insights into the mechanisms of change in CBT4CBT.^30,31 Fourth and finally, this study was conducted with a newly updated version of CBT4CBT, which retains all previous intervention components but in a new user interface, which has yet to be included in a trial. Taken together, these study features strongly support the capacity of the present study to advance the empirical support for CBT4CBT.

The first research question sought to evaluate change in SUD severity, distress tolerance, coping abilities, confidence in the ability to not take substances, and quality of life over 8 weeks of CBT4CBT as an adjunct to standard care. Both quantitative and qualitative approaches were utilized to improve understanding of CBT4CBT in this clinical context. Our hypotheses for the quantitative analysis were as follows:

H1a

We predicted that 8 weeks of CBT4CBT will be associated with a reduction in severity of SUD symptoms as a primary outcome.

H1b

We predicted an increase in distress tolerance, coping abilities, confidence, and quality of life as secondary outcomes as indicated by participant self-report responses at week 0 (pre-treatment) and week 8 (post-treatment).

The qualitative analysis sought to understand two research questions. First, which aspects of the CBT4CBT program best support skills building and therapeutic benefit, and second, which aspects may create barriers to success. Due to the exploratory nature of these analyses, no specific hypotheses are stated.

Methods

Procedure

The current study is a single-arm pragmatic clinical trial examining CBT4CBT delivered as an adjunct to standard care in adults seeking support for SUD. A multifaceted recruitment strategy was utilized, including clinician referral, self-referral in response to study advertisements, or a hospital registry including those who had interest in being contacted for future research. Prospective participants were provided an overview of the study, and if interested, provided oral and written informed consent and completed an eligibility assessment. Eligibility criteria included: being 18 years old or over; fluent in English; willing to comply with study requirements; meet diagnostic criteria for current SUD assessed via the Structured Clinical Interview for the Diagnostic and Statistical Manual for Psychiatric Disorder (DSM-5; SCID-5); using substances in the last 30 days; and seeking treatment for substance use. Exclusion criteria included: inability to read at least at a sixth-grade level; untreated or unstable severe psychiatric or medical disorder that would impede participation; monosubstance use of nicotine; and acute suicidal or homicidal ideation requiring immediate intervention. Eligible participants completed an initial assessment of demographic and clinical characteristics over a videoconferencing platform, and then received access to CBT4CBT, including a user access code for use when visiting https://cbt4cbt.com/. Study outcomes were collected at Week 0 (baseline), and at Week 8 after acute treatment. All necessary permissions were acquired for the tools and questionnaires used in this study (if applicable). As a smaller N was required for qualitative analyses, qualitative interviews were conducted at Week 8 according to participant availability. Importantly, no statistically significant differences in demographic or clinical variables are present between the individuals that completed Week 8 versus those that did not, and those that completed the qualitative interviews and those that did not (all P-values were > .05).

Participants

A total of 51 adults recruited from the Centre for Addiction and Mental Health (CAMH), a tertiary care setting in Toronto, Canada, participated in the present study. Data was collected from January to April 2021. Participants’ age ranged from 22 to 72 years (M = 40.6, SD = 12.1). Participants were majority men (66.7%), white (56.9%), university educated (49%) and employed full time (62.7%). Additional participant demographics are available in Table 1. Participants met DSM-5 diagnostic criteria according to the SCID-5 for SUD, including alcohol use disorder (n = 43; 9.3% mild, 14% moderate, 76.7% severe), cannabis use disorder (n = 17; 13.3% mild, 26.7% moderate, 60% severe), and other substance use disorders (stimulant, opioid, and other substances combined for confidentiality; n = 11; 27.3% moderate, 72.7% severe). Participants also met DSM-5 diagnostic criteria for the following co-occurring disorders: generalized anxiety disorder (n = 16), social anxiety (n = 10), current major depressive episode (n = 15), and persistent depressive disorder (n = 21). Other diagnoses included bulimia nervosa, obsessive-compulsive disorder, and specific phobias (n=<5% per diagnosis).

Table 1.

Participant demographics.

Characteristic	n	%
Sex
Female	16	31.4
Male	35	68.6
Gender
Woman	15	29.4
Man	34	66.7
Other (do not identify as man or woman)	1	2.0
Sexuality
Heterosexual	43	84.3
Homosexual	4	7.8
Bisexual	4	7.8
Marital status
Never married	24	47.1
Separated	2	3.9
Married	12	23.5
Divorced	8	15.7
Domestic partnership	5	9.8
Children^a	20	39.2
Highest educational level
High school/some college	16	31.4
College/Associates degree	10	19.6
University/Post graduate degree	25	49.0
Employment
Working	32	62.7
Temporary leave	3	5.9
Disabled	4	7.8
Unemployed	6	11.8
Student	3	5.9
Retired	2	3.9
Ethnicity
Latin American or Hispanic	4	7.8
South or Southeast Asian	3	5.9
White	29	56.9
Other ethnic group	16	31.5

Note. Additional options for gender and ethnicity were provided in the questionnaires; however, only those selected by participants were reported in the table. Any value less than 3 is combined into an “other” group.

Reflects the number and percentage of participants answering “yes” to this question.

CBT4CBT intervention

The current version of CBT4CBT (https://cbt4cbt.com/) was implemented in this trial; to maximize ecological validity, participants were free to choose the general or alcohol specific version of the program. For both versions, CBT4CBT included seven modules, each focusing on a skill to support reduction or abstinence from drugs or alcohol. Modules consist of a variety of learning methods, such as videos (varying in length) and interactive exercises, including specific examples of situations individuals may experience in their journey of managing SUD. The two versions differ primarily in the examples provided in the content. The general version was chosen by 16 participants, and includes broad examples that can be applicable for a range of substances, whereas the alcohol version was chosen by 34 participants and includes examples addressing challenges unique to alcohol use disorder, alcohol-specific scenarios and strategies, and alcohol specific psychoeducation. In line with previous studies, the study team recommended that participants complete one module per week over the 8-week treatment phase; however, participants had the flexibility to complete the modules at their own convenience, including revisiting modules that may be a specific area of interest or concern.²⁶ Participants completed an average of 4.48 modules (SD = 2.85; range = 0–7).

Measures

Distress Tolerance Scale (DTS) : The DTS is a 10-item self-report questionnaire assessing the capacity to experience and withstand negative psychological states.^32,33 Participants respond using a 5-point Likert scale (1 = strongly agree, 5 = strongly disagree). The total score is calculated by reverse coding all 10 items, then calculating the sum of all items. Empirical evidence supports the psychometric properties of the DTS, including internal validity and concurrent validity (α=.70).³⁴ Internal reliability for the full-scale is very high in the present study (Week 0 α=.93; Week 8 α=.93).³⁵

Change Assessment Inventory (CAI; URICA–University of Rhode Island Change Assessment Questionnaire)³⁶: The URICA is a 32-item self-report questionnaire assessing an individual's perspective on approaching problems in their life. Participants respond using a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). A total “change readiness” score is calculated by adding the contemplation, maintenance, and action subscales together and subtracting the precontemplation scale. This measure has demonstrated construct validity in a large sample of patients with drug and alcohol use disorders.³⁷ The URICA demonstrates good reliability in the present study (Week 0 α=.75; Week 8 α=.84).

Severity of Dependence Scale (SDS)³⁸: The SDS is a 5-item self-report questionnaire assessing the severity of SUD symptoms, with a focus on psychological features such as impaired control and preoccupation with drug use.³⁹ Participants respond on a 4-point scale (0 = never/almost never, 1 = sometimes, 2 = often, 3 = always/almost always). The SDS is calculated using the sum score of the five items. The SDS has demonstrated excellent validity with high specificity and sensitivity.⁴⁰ The SDS demonstrated high internal reliability in the present study (Week 0 α=.79; Week 8 α=.84).

Drug-Taking Confidence Questionnaire (DTCQ)⁴¹: The DTCQ is a 50-item questionnaire assessing self-efficacy in the context of substance use. Participants are asked to rate how confident they are, on a scale of 0–100, in their ability to resist substance use in different high-risk situations, where 0 indicates the participant has no confidence at all about their ability to resist the urge to use drugs in the given situation and 100 indicates they are 100% confident right now they could resist the urge to use drugs in the given situation.⁴¹ Items were averaged to create a full-scale total score. Furthermore, extensive analyses have been conducted demonstrating the convergent and discriminant validity of the measure in both clinical and research settings.⁴² The DTCQ total demonstrated high reliability in the present study (Week 0 α=.97; Week 8 α=.98).⁴¹

World Health Organization Quality of Life Assessment (WHOQOL-BREF)⁴³ is a 26-item questionnaire assessing four domains related to quality of life: physical, psychological, social, and environmental health.⁴⁴ Participants respond on a 5-point scale from 1 to 5 (1= “not at all,” “very poor,” and “very dissatisfied,” 5= “completely,” “very good,” and “very satisfied”). Each domain score is calculated as a sum of its items score. The WHOQOL-BREF subscales have demonstrated high internal reliability and various forms of validity, and are widely used in health research (α=.70 to .77).⁴⁴ The WHOQOL-BREF subscales demonstrated high reliability in the present study (DOM1 Physical: Week 0 α=.84; Week 8 α=.80; DOM2 Psychological: Week 0 α=.78; Week 8 α=. 81; DOM3 Social: Week 0 α=.77; Week 8 α= .74; DOM4 Environmental: Week 0 α=.86; Week 8 α=.78).

Change Strategies Inventory (CSI) is a modified version of the Coping Strategies Scales (CSS).^45,46 Carroll and colleagues modified this existing measurement tool to quantify change in the skills taught during the CBT4CBT program. The CSI includes 17 items rated using 5-point Likert scale. The 17 items are a single scale with the total score calculated by using the sum of each item. A confirmatory factor analysis demonstrated a good fit with the 6 topics covered in the CBT4CBT modules, as well as internal reliability.⁴⁵ Furthermore, Sugarman and colleagues have supported the reliability and validity of this measure in samples of individuals with substance use disorder.⁴⁵ The CSI demonstrated high internal reliability in the present study (Week 0 α=.87; Week 8 α=.81).

Statistical/data analysis

A series of linear mixed effects models (LMM) were conducted to evaluate the change in nine variables over 8 weeks of CBT4CBT. Each of the nine variables was analyzed separately and added fixed effects of time (Week 0 and Week 8). LMM was used for this analysis to permit the modeling of participant-specific variability via random effects, to permit the inclusion of covariates, and to permit the use of all available data. LMM was selected for several key reasons. First, LMM is ideal for a repeated measures structure, such as the two time points used in the present study.⁴⁷ Second, traditional methods, including independent t-tests or ANOVAs, assume independence of observations, which is violated in repeated measures data.⁴⁸ Third, the LMM appropriately models within-subject correlation while accounting for individual differences.⁴⁹ Lastly, LMM uses a flexible approach to handling missing data using the assumption of missing at random, as opposed to a repeated-measures ANOVA which uses listwise deletion.⁵⁰ The LMM approach to missing data allows more data to be retained and included in the analysis.

Assumptions were tested for the LMM. First, normality of residuals was met with Shapiro-Wilk test demonstrating P > .05 for all measures and subscales. Homoscedasticity was tested with a visual inspection of scatter plots demonstrating the relationship between residual and predicted value outputs for each measure. All scatterplots demonstrated random cloud shapes indicating the assumption of homoscedasticity is met. The assumption of linearity is not applicable in this model as the fixed effect of time is categorical, with two time points. We examined the assumption of independence of residuals by specifying the repeated measure structure in the model and comparing to alternative covariance structures (e.g., compound symmetry and AR(1)). Although less critical with only two time points, this assumption was evaluated, and model comparisons indicated that the assumption was adequately met, with no evidence of residual autocorrelation.

Semi-structured interviews were conducted, including questions regarding health, well-being, use of technology to support health, what aspects of the program participants felt most benefited them, and what barriers they may have experienced. Participant interviews were audio-recorded and transcribed. Qualitative interview data was analyzed following the qualitative content analysis method, which focuses on latent meaning rather than exact wording.⁵¹ Context and language can vary greatly from participant to participant, which makes the use of qualitative content analysis highly effective for this study.⁵² Using this method allowed for the inclusion of every reference or inference of what participants found either beneficial or unhelpful in the program. Based on our research questions, we developed an initial coding frame for the first qualitative research question with the main categories of CBT skills, convenience, and indirect benefits. After reviewing all interview transcripts and coding values inferred or directly noted from the interviews, sub-categories were added to each of these categories until all interviews had been analyzed and all pertinent data had a sub-category to which it was allocated. The qualitative interviews were reviewed and coded by two independent researchers who were instructed to review the transcripts using the qualitative content analysis method.⁵¹

Results

Quantitative analyses

Descriptive statistics for each measure are displayed in Table 2. As shown in Table 3, there was a significant small effect of time for SUD severity (b_time = −0.79, P = .04, d = −.25).³⁹ There was also a significant medium effect in the use of change strategies (b_time = 6.08, P = <.001, d = .56), indicating enhanced application of CBT-related coping skills over time. Distress tolerance scores increased with a small yet significant effect from baseline (b_time = 2.41, P = .03, d = .31), also suggesting improved capacity to manage negative emotions without substance use.³⁴ Significant improvements, with small to medium effect sizes, were observed across all domains of quality of life (Physical b_time = 0.75, P = .01, d = .40; Psychological b_time = 0.78, P = .02, d = .40; Social b_time = 1.06, P = .01, d = .29; Environmental b_time = 1.19, P=<.001, d = .56). There was no significant effect of time for change assessment (b_time = –0.39, P = .05, d = –38) and drug-taking confidence (b_time = 6.40, P = .38, d = .31).⁴¹ Across measures, the marginal R² values ranged from .00 to .40, indicating that fixed effects alone accounted for a small to moderate proportion of variance in most outcomes, with the exception of the environmental domain of the quality of life measure, where fixed effects explained 40% of the variance, suggesting a large effect. Conditional R² values ranged from .59 to .82, demonstrating that the full models explained a considerable proportion of variance in all measures. The significant effects found in this study were generally small effect sizes regarding only fixed effects, however large effect sizes when considering the full model. Figures 1 and 2 visually present the standardized mean changes in each outcome variable from week 0 to week 8.

Figure 1.

Standardized mean change in functional outcome measures Week 0 to Week 8.

Figure 2.

Standardized mean change in secondary outcome measures Week 0 to Week 8.

Table 2.

Measure descriptives.

		n	M	SD
Substance dependence scale	0	50	9.92	2.75
	8	36	9.22	2.95
Change assessment	0	50	9.10	1.30
	8	36	8.69	0.55
Change strategies	0	49	32.33	10.32
	8	36	38.36	11.84
Distress tolerance	0	49	15.42	10.67
	8	36	18.78	10.80
Drug taking confidence	0	49	138.3	50.75
	8	36	147.39	59.76
WHOQOL—physical	0	49	11.17	2.14
	8	36	12.24	2.20
WHOQOL—psychological	0	49	11.75	1.98
	8	36	12.62	2.37
WHOQOL—social	0	49	12.00	4.02
	8	36	13.15	3.77
WHOQOL—environmental	0	49	14.50	3.05
	8	36	16.04	2.38

Table 3.

Linear mixed effects model.

Variable	b _time	SE	P	CI
CAI	−.39	.20	.05	−.79, .01
DTS	2.41	1.09	.03	−.19, 4.62
SDS	−0.79	.37	.04	−1.54, −.03
CSI	6.08	1.42	<.001	3.20, 8.95
DTCQ	6.40	7.19	.38	−8.14, 20.95
WHOQOL—physical	.75	.27	.01	.21, 1.29
WHOQOL—psychological	.78	.32	.02	.13, 1.45
WHOQOL—social	1.06	.40	.01	.23, 1.88
WHOQOL—environmental	1.19	.32	<.001	.54, 1.84

Note. B_time represents the change in variable scores from Week 0 to Week 8. CAI: Change Assessment Inventory; DTS: Distress Tolerance Scale; SDS: Substance Dependence Scale; CSI: Change Strategies Inventory; DTCQ: Drug Taking Confidence Questionnaire; WHOQOL: WHO Quality of Life.

Qualitative analyses

CBT skills

Of the CBT4CBT modules, 19 of the 26 participants (65.51%) reported the most helpful modules in teaching skills were managing triggers and cravings, titled “Recognize the Triggers” and “Deal with Cravings,” respectively. The “Recognize the Triggers” module was reported by 13 participants to be especially helpful, with participants noting specifically that this module encouraged introspection to understand their specific triggers for use. Being able to recognize triggers, participants noted, is an important skill that allows them to predict and prepare for situations that may challenge their ability to reach substance use goals. One participant stated, “for the triggers, it helps me identify what situations I get myself into that always lead to use, so that was helpful.” Another participant noted,

There's no question. You can tell my drinking has decreased. It gave me a lot to think about, I think. Each module, I know has its own different topic and yeah, it gave me a lot to think about… There was one session with a girl who had a boyfriend who was using, and she was struggling to stop. And as I related to her to try and avoid triggers, I also saw myself in the dude who was pressuring her to use.

The cravings module was also highlighted by participants. Participants found that implementing the suggested strategies to deal with cravings helped them reduce or eliminate substance use. Participants mentioned various CBT skills including coping strategies, boundary setting, and self-reflections that were helpful in their use of the program. The structure of the program in how to approach and use all these skills was also noted as realistically applicable help.

Convenience

The convenience of online asynchronous training was noted by 7 interviewees as a benefit of the program. Participants indicated that the flexible timing for completing modules and the ability to complete the modules at home (removing any commuting time) was beneficial. Autonomy over the program was considered a main benefit aligned with convenience as participants had the autonomy to complete the program as it suited them including re-watching modules, going at their own pace, and taking breaks when needed. A participant said,

Personally, I like to be by myself, and I like to just be able to rewind, go over it again and watch it over, and learn at my own pace. I don’t feel rushed. Like I can even stop and take a break and then resume it, so I like that aspect of it.

Relatability

The relatability of scenarios faced by actors in the modules was mentioned by 14 participants as being the most beneficial aspect of the program. Participants indicated that being able to relate to the scenarios portrayed in the modules made them feel less alone, more normal, and reduced self-stigma. A participant noted, “There was three different types of people that you wouldn’t expect to be alcoholics, you know, so it really normalized it for me.” Another participant stated, “I liked the validation like, of normalizing the troubles that I have, um, which was kind of nice in a weird way.” Participants also indicated that their confidence and self-esteem increased, such that they felt more confidence to reduce substance use and/or talk to family and friends about their use.

Barriers of CBT4CBT

Connection

Increase of live connection was suggested by 7 participants, including live interaction via a live chat, group meetings with other participants, and one-on-one meetings with a professional. Interviewees indicated that it was difficult to maintain motivation without the accountability enforced by a live meeting. Participants expressed desire to discuss their learnings with someone after completing the modules to further grasp the skills. A participant stated,

The online component was delivered great. I think the gap is just having someone there to kind of go through what you just learned, and I think that just comes onto like, maybe just education or just basic education in terms of how people process information… I think maybe a group context would be better, because I did have questions immediately after that I’ve probably forgot by now… But it would have been great to talk it through with someone, and kind of explore certain answers.

Relatability

In contrast, some participants found that the scenarios were not relatable, specifically for the safe sex module. A participant mentioned “the last [module]… I found it did really like… it wasn’t relevant to me necessarily.” Participants indicated that the lack of relatability to this scenario turned them off from the program as being unhelpful and unrealistic. This may have been particularly memorable as the safe sex module was the final module participants completed, therefore likely most top of mind.

Technology

Issues with technology were also noted as barriers to the program. Interviewees indicated the user interface of the program warranted improvement. Furthermore, participants had difficulty with navigation, video speed, and quality of the videos. A participant noted that the program was “just a little bit less easy to use than I feel like it could have been”.

Discussion

Consistent with previous studies of CBT4CBT, this study demonstrated that the severity of SUD decreased from week 0 to week 8, with scores suggesting a modest but clinically meaningful reduction in the severity of SUD. As such, the current investigation extends previous research to support the feasibility and clinical outcomes of adults with access to CBT4CBT in real-world treatment settings.⁵³ Secondary outcomes, including coping strategies, distress tolerance, and quality of life, also improved during acute treatment, and further extended prior research, which has not included this full range of indicators.⁵⁴ Notably, participants did not demonstrate a difference in their perspectives on changing substance use from week 0 to week 8. As the sample included individuals enrolled in a substance use treatment program, it may be that participants largely felt they were ready to make changes, potentially creating a ceiling effect. Similarly, individuals did not have a significant change regarding the confidence in their abilities to not take drugs. This may be attributable to several features, from the short duration of follow-up to fear of relapsing, and requires further study.⁵⁵ Coping and distress tolerance demonstrated a significant change from week 0 to week 8, indicating that participants improved in key CBT skills. This result is consistent with other CBT4CBT efficacy studies providing evidence to suggest that CBT4CBT is administering the anticipated skills to support individuals with their substance use goals.³⁰ Lastly, participants reported a significant increase in quality of life from week 0 to week 8. This important overall outcome is frequently described as an ultimate goal for individuals who experience functional impairment due to SUD.⁵⁶

The qualitative analysis provided significant depth of insight to participant perspectives on the CBT4CBT program. Qualitative insights are particularly important when the statistical significance of a result would indicate there was no meaningful change over time, yet qualitatively participants note the impact of how they felt during the experience–a meaningful aspect to consider when evaluating treatment efficacy.⁵⁷ When discussing the benefits or highlights of the program, many participants noted the relatability of the characters and scenarios. Participants felt their challenges with substance use were accurately portrayed, providing a sense of credibility to the program and increasing confidence and encouragement related to their own SUD journeys. The relatability of the characters and scenarios also provided participants with a sense of normalcy, such that these occurrences are so common as to be included in a video module means they are not alone in what they are experiencing. Relatedly, participants noted subjective increases in self-esteem, confidence, and a reduction in self-stigma.

These qualitative results, however, were not reflected by quantitative improvements of self-efficacy. Despite this, the CBT skills acquired during the current study were reported as highly beneficial by the participants. The two most beneficial modules noted by participants included the “triggers” and “cravings” modules. These were practical modules with tangible actions and outcomes. This may indicate that participants are most interested in actionable steps with demonstrable progress as a key element of their SUD treatment journey.

The convenience of the program was noted as a key benefit for participants. The ability to participate in modules remotely with flexibility in scheduling provides the opportunity for many individuals to access care that may not be able otherwise. The increased convenience of online asynchronous modules comes at the cost of decreased human interaction and potential technological difficulties, which were both noted by participants as barriers to the program. Furthermore, participants described how accountability and motivation are often higher when they are responsible to someone other than themselves. In line with this, a systematic literature review recently indicated that guided or synchronous online interventions were superior to unguided interventions for internet-based mental health care.⁵⁸ Therefore, web-based interventions may benefit from some synchronous contact component to increase patient–clinician interaction.

The additional insight provided by the qualitative analysis provides context not otherwise understood by the quantitative results. While quantitative measures are interpreted by the lens of the literature and the researcher, qualitative results are direct insights from participants. Overall, the use of mixed methods in this study was critical to understanding participants true perspectives on the program, allowing for rich and meaningful feedback for improvement. As noted, this study is the first qualitative analysis of CBT4CBT and provides key input from those with lived experience participating in the CBT4CBT program. Furthermore, the inclusion of functional outcomes and potential mechanisms of change provides information about ecologically valid impacts of the program, as well as the transdiagnostic risk and resilience factors that may underlie changes over time in those accessing this treatment. Indeed, functional outcomes often align more closely with participant treatment goals than substance use frequency or SUD severity, further establishing their importance as outcomes in substance use research. Finally, this trial is the first to employ the current version of CBT4CBT, which allowed for gaining important participant insight on the usability and aesthetic updates to the program.

Strengths and limitations

There are a number of key strengths that support the present study in being a valuable contribution to the literature on SUD treatment. This study has high ecological validity, as eligibility criteria were inclusive and CBT4CBT was administered in addition to standard care, which is the common recommendation for digital interventions in tertiary care settings. Participants exhibited diversity in demographic and clinical features, further increasing the generalizability of results. Validated measures covered a broad range of clinical features related to skills acquisition and therapeutic outcomes, and included both self-report and interviews. Lastly, the complexity of the sample provides strong evidence for the use of this treatment for individuals with complex clinical presentations. Limitations are important to consider as part of the present study as well. First, the study did not include a control group or randomization, which precludes the ability to speak to efficacy. Second, although best practices were applied to conduct the qualitative analysis using a thematic coding scheme, qualitative analysis is inherently subjective, especially when conducting a content analysis to interpret latent meaning. This was mitigated by using multiple researchers to conduct the qualitative analysis and create a consensus on the results. Furthermore, a constructivity and interpretivist paradigm formed the foundation of this work, in which interactive, context-dependent interpretation provides a valuable, deep engagement with data.^59–62 Third, as a pragmatic study of CBT4CBT as an augmentation to standard care, we did not control or track the CBT4CBT version received or compliance with the recommended pace of module completion, as well as other concurrent pharmacological or psychological treatment. Lastly, the data collection occurring during early stages of the COVID-19 pandemic in the region, resulting in impacts on participant retention and power. The attrition at Week 8 was therefore a limitation to the study; however, as noted, there were no statistically significant differences in participants who did and did not complete Week 8 for both demographic and clinical outcomes. Further, qualitative analysis reached saturation, suggesting that sufficient Week 8 interviews were available to inform regarding the participant experience of CBT4CBT and their acquisition of skills.

Future directions

While this study demonstrated significant improvements in key areas, several areas warrant further investigation. Future research should explore the long-term effects of CBT4CBT by incorporating greater sample sizes and extended follow-up periods beyond the immediate post-treatment phase. Given the qualitative findings indicating the importance of human interaction for accountability and motivation, future iterations of CBT4CBT could incorporate synchronous or guided elements. Studies assessing the impact of integrating periodic therapist check-ins or peer support components within the program could provide further insights into optimizing digital substance use treatments. Moreover, technological barriers were noted by participants, highlighting the need for future research to evaluate the accessibility and usability of CBT4CBT across diverse populations, particularly among individuals with limited digital literacy.

Lastly, further exploration of individual differences, such as baseline levels of motivation or prior treatment experiences, may also help identify subgroups of participants who may benefit most from specific program elements.

Conclusions

In summary, these results support positive outcomes after 8 weeks of CBT4CBT treatment regarding substance use severity and other psychological outcomes. This study contributes to the growing body of literature supporting the efficacy of CBT4CBT in real-world clinical settings. Participants exhibited significant improvements in key functional outcomes, including distress tolerance, coping strategies, and quality of life, highlighting the potential impact of CBT-based digital interventions beyond symptom reduction. Qualitative findings emphasized the importance of relatability, practicality, and accountability in treatment engagement. The mixed-methods approach was instrumental in capturing both the statistical and experiential aspects of treatment, underscoring the importance of incorporating patient perspectives into intervention development and evaluation.

Footnotes

Acknowledgements

We would like to thank Dr Kathleen Carroll, who was Co-Principal Investigator of this seed grant and contributed substantially to project design; her inspiration and guidance cannot be overstated. We would also like to thank all participants for sharing their insights and experiences with us.

ORCID iDs

Danielle Downie

Bernard Le Foll

Ethical considerations

The study was approved by the CAMH Research Ethics Board. All procedures performed in studies involving human participants were in accordance with the ethical standards of Canadian national and international guidelines, as well as provincial legislation and hospital policies, and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The study received ethical approval from the CAMH REB (REB #: 073/2018) on July 24, 2018.

Consent to participate

This study was conducted according to the GCP ethical standards for the treatment of human subjects. Identifying information was separated from the dataset upon analysis. Participants were informed that their participation was voluntary, that they could leave the study at any time, and that their data would be treated anonymously. Oral and written informed consent was obtained and documented by a research team member.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was funded by the CAMH Foundation Discovery Fund. Dr. Bernard Le Foll has obtained funding from Indivior for a clinical trial sponsored by Indivior. Dr. Le Foll has in-kind donations of placebo edibles from Indivia. Dr. Le Foll has obtained industry funding from Canopy Growth Corporation (through research grants handled by the Centre for Addiction and Mental Health and the University of Toronto). He has participated in a session of a National Advisory Board Meeting ( Emerging Trends BUP-XR) for Indivior Canada and is part of the Steering Board for a clinical trial for Indivior. He has been a consultant for Shinogi and ThirdBridge. He got travel support to attend an event by Bioprojet. He is supported by CAMH, Waypoint Centre for Mental Health Care, a clinician-scientist award from the Department of Family and Community Medicine of the University of Toronto and a Chair in Addiction Psychiatry from the Department of Psychiatry of University of Toronto.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

The data that support the findings of this study are available from the corresponding author, LCQ, upon reasonable request.

References

Kranzler

Soyka

. Diagnosis and pharmacotherapy of alcohol use disorder: a review. Jama 2018; 320: 815–824.

Government of Canada SC. Health at a glance. Mental and substance use disorders in Canadahttps://www150.statcan.gc.ca/n1/pub/82-624-x/2013001/article/11855-eng.htm (2015, accessed 4 October 2024).

McHugh

Hearon

Otto

. Cognitive behavioral therapy for substance use disorders. Psychiatr Clin 2010; 33: 511–525.

Ray

Meredith

Kiluk

, et al. Combined pharmacotherapy and cognitive behavioral therapy for adults with alcohol or substance use disorders: a systematic review and meta-analysis. JAMA Netw Open 2020; 3: e208279.

van Emmerik-van Oortmerssen

Vedel

Kramer

, et al. Integrated cognitive behavioral therapy for ADHD in adult substance use disorder patients: results of a randomized clinical trial. Drug Alcohol Depend 2019; 197: 28–36.

Shams

Wong

Nikoo

, et al. Understanding eHealth cognitive behavioral therapy targeting substance use: realist review. J Med Internet Res 2021; 23: e20557.

Martino

Ball

Nich

, et al. Community program therapist adherence and competence in motivational enhancement therapy. Drug Alcohol Depend 2008; 96: 37–48.

Marchand

Beaumont

Westfall

, et al. Conceptualizing patient-centered care for substance use disorder treatment: findings from a systematic scoping review. Subst Abuse Treat Prev Policy 2019; 14: 1–5.

Muthulingam

Bia

Madden

, et al. Using nominal group technique to identify barriers, facilitators, and preferences among patients seeking treatment for opioid use disorder: a needs assessment for decision making support. J Subst Abuse Treat 2019; 100: 18–28.

10.

Hewell

Vasquez

Rivkin

ID.

Systemic and individual factors in the buprenorphine treatment-seeking process: a qualitative study. Subst Abuse Treat Prev Policy. 2017;12:1–10.

11.

Kiluk

. Computerized cognitive behavioral therapy for substance use disorders: a summary of the evidence and potential mechanisms of behavior change. Perspect Behav Sci 2019; 42: 465–478.

12.

Marsch

Carroll

Kiluk

. Technology-based interventions for the treatment and recovery management of substance use disorders: a JSAT special issue. J Subst Abuse Treat 2014; 46: 1–4.

13.

Bergman

Kelly

. Online digital recovery support services: an overview of the science and their potential to help individuals with substance use disorder during COVID-19 and beyond. J Subst Abuse Treat 2021; 120: 108152.

14.

Levy

Deister

Fantegrossi

, et al. Virtual care in an outpatient subspecialty substance use disorder treatment program. J Addict Med 2022; 16: e112–e117.

15.

Carroll

A cognitive-behavioral approach: Treating cocaine addiction. US Department of Health and Human Services, National Institutes of Health, National Institute on Drug Abuse; 1998.

16.

Carroll

Ball

Martino

, et al. Computer-assisted delivery of cognitive-behavioral therapy for addiction: a randomized trial of CBT4CBT. Am J Psychiatry 2008; 165: 881–888.

17.

Carroll

Ball

Martino

, et al. Enduring effects of a computer-assisted training program for cognitive behavioral therapy: a 6-month follow-up of CBT4CBT. Drug Alcohol Depend 2009; 100: 178–181.

18.

Carroll

Kiluk

Nich

, et al. Computer-assisted delivery of cognitive-behavioral therapy: efficacy and durability of CBT4CBT among cocaine-dependent individuals maintained on methadone. Am J Psychiatry 2014; 171: 436–444.

19.

Kiluk

Devore

Buck

, et al. Randomized trial of computerized cognitive behavioral therapy for alcohol use disorders: efficacy as a virtual stand-alone and treatment add-on compared with standard outpatient treatment. Alcoholism: Clin Exp Res 2016; 40: 1991–2000.

20.

Kiluk

Nich

Buck

, et al. Randomized clinical trial of computerized and clinician-delivered CBT in comparison with standard outpatient treatment for substance use disorders: primary within-treatment and follow-up outcomes. Am J Psychiatry 2018; 175: 853–863.

21.

Paris

Silva

Jamarillo

, et al. Randomized clinical trial of a culturally adapted computer-based cognitive behavioral therapy for spanish speakers with substance use disorders: primary outcomes. Am J Public Health 2018; 108: 1535–1542.

22.

Carroll

. (CBT4CBT) computer-based training for cognitive behavioral therapy. Comput Based Train Cognitive Behav Therapy (CBT4CBT) 2018. https://cbt4cbt.com/

23.

Kiluk

Nich

Babuscio

, et al. Quality versus quantity: acquisition of coping skills following computerized cognitive–behavioral therapy for substance use disorders. Addiction 2010; 105: 2120–2127.

24.

Kiluk

DeVito

Buck

, et al. Effect of computerized cognitive behavioral therapy on acquisition of coping skills among cocaine-dependent individuals enrolled in methadone maintenance. J Subst Abuse Treat 2017; 82: 87–92.

25.

Roos

Carroll

Nich

, et al. Short-and long-term changes in substance-related coping as mediators of in-person and computerized CBT for alcohol and drug use disorders. Drug Alcohol Depend 2020; 212: 108044.

26.

Patel

Corman

de Oliveira

, et al. Computer-Based training for cognitive behavioural therapy for substance use disorder: a randomized controlled trial including quantitative and qualitative health and economic outcomes: formation informatisée pour la thérapie cognitivo-comportementale pour les troubles liés à l'usage de substances: un essai randomisé contrôlé y compris les résultats quantitatifs et qualitatifs en matière de santé et d’économie. Canadian J Psychiatry 2024; 70: 9–20.

27.

Davis

Bryan

, et al. Importance of measuring quality of life in aging and chronic disease research. J Aging Health 2020; 32: 105–115.

28.

Cisler

Zweben

. Development of a composite measure for assessing alcohol treatment outcome: operationalization and validation. Alcoholism: Clin Exp Res 1999; 23: 263–271.

29.

Witkiewitz

. Success” following alcohol treatment: moving beyond abstinence. Alcoholism: Clin Exp Res 2013; 1: E9–E13.

30.

Boumparis

Schaub

. Recent advances in digital health interventions for substance use disorders. Curr Opin Psychiatry 2022; 35: 246–251.

31.

Maisto

Moskal

Firkey

, et al. From alcohol and other drug treatment mediator to mechanism to implementation: a systematic review and the cases of self-efficacy, social support, and craving. Alcohol: Clin Exp Res 2024; 48: 1677–1692.

32.

Brown

Lejuez

Kahler

, et al. Distress tolerance and early smoking lapse. Clin Psychol Rev 2005; 25: 713–733.

33.

Brown

Burton

Abbott

. The relationship between distress tolerance and symptoms of depression: validation of the distress tolerance scale (DTS) and short-form (DTS-SF). J Clin Psychol 2022; 78: 2609–2630.

34.

Simons

Gaher

. The distress tolerance scale: development and validation of a self-report measure. Motiv Emot 2005; 29: 83–102.

35.

Leyro

Bernstein

Vujanovic

, et al. Distress tolerance scale: a confirmatory factor analysis among daily cigarette smokers. J Psychopathol Behav Assess 2011; 33: 47–57.

36.

Dozois

Westra

Collins

, et al. Stages of change in anxiety: psychometric properties of the university of Rhode Island change assessment (URICA) scale. Behav Res Ther 2004; 42: 711–729.

37.

Field

Adinoff

Harris

, et al. Construct, concurrent and predictive validity of the URICA: data from two multi-site clinical trials. Drug Alcohol Depend 2009; 101: 115–123.

38.

Gossop

Griffiths

Powis

, et al. Severity of dependence and route of administration of heroin, cocaine and amphetamines. Br J Addict 1992; 87: 1527–1536.

39.

Gossop

Darke

Griffiths

, et al. The severity of dependence scale (SDS): psychometric properties of the SDS in English and Australian samples of heroin, cocaine and amphetamine users. Addiction 1995; 90: 607–614.

40.

Cuevas

Sanz

Fuente

, et al. The severity of dependence scale (SDS) as screening test for benzodiazepine dependence: SDS validation study. Addiction 2000; 95: 245–250.

41.

Sklar

Turner

. A brief measure for the assessment of coping self-efficacy among alcohol and other drug users. Addiction 1999; 94: 723–729.

42.

Sklar

Annis

Turner

. Development and validation of the drug-taking confidence questionnaire: a measure of coping self-efficacy. Addict Behav 1997; 22: 655–670.

43.

Whoqol Group. Development of the world health organization WHOQOL-BREF quality of life assessment. Psychol Med 1998; 28: 551–558.

44.

Skevington

Lotfy

O'Connell

. The world health organization's WHOQOL-BREF quality of life assessment: psychometric properties and results of the international field trial. A report from the WHOQOL group. Qual Life Res 2004; 13: 299–310.

45.

Sugarman

Nich

Carroll

. Coping strategy use following computerized cognitive-behavioral therapy for substance use disorders. Psychol Addict Behav 2010; 24: 689.

46.

Litt

Kadden

Cooney

, et al. Coping skills and treatment outcomes in cognitive-behavioral and interactional group therapy for alcoholism. J Consult Clin Psychol 2003; 71: 118.

47.

Meteyard

Davies

. Best practice guidance for linear mixed-effects models in psychological science. J Mem Lang 2020; 112: 104092.

48.

Hallgren

Witkiewitz

. Missing data in alcohol clinical trials: a comparison of methods. Alcoholism: Clin Exp Res 2013; 37: 2152–2160.

49.

Singer

Willett

. Applied longitudinal data analysis: modeling change and event occurrence. Oxford, England: Oxford University Press, 2003.

50.

Snijders

TAB

Bosker

. Multilevel analysis: an introduction to basic and advanced multilevel modeling. 2nd ed. Los Angeles: Sage, 2012.

51.

Schreier

Qualitative content analysis. London: Sage Publications Ltd, 2012.

52.

Elo

Kyngäs

. The qualitative content analysis process. J Adv Nurs 2008; 62: 107–115.

53.

Tetrault

Holt

Cavallo

, et al. Computerized cognitive behavioral therapy for substance use disorders in a specialized primary care practice: a randomized feasibility trial to address the RT component of SBIRT. J Addict Med 2020; 14: e303–e309.

54.

Kelpin

Parlier-Ahmad

Jallo

, et al. A pilot randomized trial of CBT4CBT for women in residential treatment for substance use disorders. J Subst Abuse Treat 2022; 132: 108622.

55.

Kelly

Myers

Brown

. A multivariate process model of adolescent 12-step attendance and substance use outcome following inpatient treatment. Psychol Addict Behav 2000; 14: 376.

56.

Laudet

. The case for considering quality of life in addiction research and clinical practice. Addict Sci Clin Pract 2011; 6: 44.

57.

Dawadi

Shrestha

Giri

. Mixed-methods research: a discussion on its types, challenges, and criticisms. J Pract Stud Educ 2021; 2: 25–36.

58.

Baumeister

Reichler

Munzinger

, et al. The impact of guidance on internet-based mental health interventions—A systematic review. Internet Interv 2014; 1: 205–215.

59.

Braun

Clarke

. Conceptual and design thinking for thematic analysis. Qual Psychol 2022; 9: 3.

60.

Braun

Clarke

. Reflecting on reflexive thematic analysis. Qual Res Sport Exerc Health 2019; 11: 589–597.

61.

Nowell

Norris

White

, et al. Thematic analysis: striving to meet the trustworthiness criteria. Int J Qual Methods 2017; 16: 1609406917733847.

62.

Saldaña

Coding techniques for quantitative and mixed data. In The routledge reviewer’s guide to mixed methods analysis. 1st ed. New York: Taylor & Francis Group, 2021, pp.151–160.