Sage Journals: Discover world-class research

Abstract

French

Objectives

Heavy alcohol and drug use is reported by a substantial number of Canadians; yet, only a minority of those experiencing substance use difficulties access specialized services. Computer-Based Training for Cognitive Behavioural Therapy (CBT4CBT) offers a low-cost method to deliver accessible and high-quality CBT for substance use difficulties. To date, CBT4CBT has primarily been evaluated in terms of quantitative outcomes within substance use disorder (SUD) samples in the United States. A comparison between CBT4CBT versus standard care for SUDs in a Canadian sample is critical to evaluate its potential for health services in Canada. We conducted a randomized controlled trial of CBT4CBT versus standard care for SUD.

Methods

Adults seeking outpatient treatment for SUD (N = 50) were randomly assigned to receive either CBT4CBT or treatment-as-usual (TAU) for 8 weeks. Measures of substance use and associated harms and quality of life were completed before and after treatment and at 6-month follow-up. Qualitative interviews were administered after treatment and at follow-up, and healthcare utilization and costs were extracted for the entire study period.

Results

Participants exhibited improvements on the primary outcome as well as several secondary outcomes; however, there were no differences between groups. A cost-effectiveness analysis found lower healthcare costs in CBT4CBT versus TAU in a subsample analysis, but more days of substance use in CBT4CBT. Qualitative analyses highlighted the benefits and challenges of CBT4CBT.

Discussion

Findings supported an overall improvement in clinical outcomes. Further investigation is warranted to identify opportunities for implementation of CBT4CBT in tertiary care settings.

Trial Registration: https://clinicaltrials.gov/ct2/show/NCT03767907

Plain Language Summary Title

Evaluating a digital intervention targeting substance use difficulties

Plain Language Summary

Why was the study done?

Heavy alcohol and drug use is frequent in the Canadian population, although very few people have access to treatment. The digital intervention, Computer-Based Training for Cognitive Behavioural Therapy (CBT4CBT), may provide a low-cost, high-quality, and easily accessible method of treatment for substance use difficulties. Limited research on this digital intervention has been conducted in Canadian populations, and few studies thus far have evaluated participants’ subjective experience using the intervention, along with the cost on the Canadian healthcare system.

What did the researchers do?

The research team recruited participants and provided access to either CBT4CBT or to standard care at a mental health hospital for 8 weeks. Participants were asked questions about their substance use and related consequences, quality of life, and thoughts on the treatment they received. Information regarding healthcare use and the cost to the healthcare system was also gathered.

What did the researchers find?

Participants in both groups improved with regards to their substance use, some related consequences, and psychological quality of life. Participants provided insight on the benefits and challenges of both types of treatment. It was also found that the CBT4CBT intervention was less costly.

What do these findings mean?

These findings support that adults receiving CBT4CBT and standard care both improved to a similar degree in this sample. Participant feedback may inform future studies of how best to implement this intervention in clinical studies. Future studies with larger samples are needed to further examine whether CBT4CBT can increase access to supports and be beneficial in the Canadian healthcare system.

Keywords

addictions substance use disorders cognitive behavioural therapy e-mental health clinical trials

Introduction

Heavy alcohol and drug use is reported by a substantial number of Canadians, and accounts for 40 billion dollars to Canadian society each year.¹ However, only a minority of those experiencing substance use difficulties access specialized services,² highlighting the need for scalable treatment alternatives. Cognitive behavioural therapy (CBT) is empirically supported for the treatment of substance use disorders (SUDs); yet, this treatment is rarely implemented with high fidelity in clinical practice.³ The components of CBT are adaptable, however, and have exhibited promise in reducing substance use.⁴

Computer-Based Training for Cognitive Behavioural Therapy (CBT4CBT) is an online treatment program⁵ that leverages the potential of this digital format, providing psychoeducation and skills building using an interactive form of delivery. CBT4CBT initially demonstrated efficacy as an adjunct to standard care in adults seeking addiction treatment^5,6; effects were then replicated in an investigation of cocaine misuse.⁷ Kiluk and colleagues⁸ reported the first investigation of CBT4CBT as a stand-alone treatment in adults with alcohol use disorder (AUD), where CBT4CBT was comparable to standard care or standard care plus CBT4CBT. In a subsequent investigation,⁹ CBT4CBT as a stand-alone treatment was superior to standard care in adults with a range of SUDs, but not different from clinician-delivered CBT. In this study, CBT4CBT was associated with better retention compared to standard care or in-person CBT.

The current investigation aims to extend this research to include qualitative outcomes and the first cost-effectiveness analysis of this innovative program in Canada. Economic investigations are required to support the implementation of novel treatment approaches into health services.¹⁰ We compared the efficacy and cost-effectiveness of a stand-alone implementation of CBT4CBT to treatment-as-usual (TAU). The primary hypothesis was that CBT4CBT will be associated with greater decreases in substance use frequency compared to TAU. Secondary hypotheses were that CBT4CBT will be associated with (a) greater improvements in secondary outcomes such as SUD symptom severity and quality of life, and (b) lower healthcare costs and thus, cost-effectiveness, compared to TAU. Hypotheses were not specified for qualitative analyses, which aimed to elucidate the patient experience of this program in this clinical context.

Methods

Participants

The study included 50 participants, ranging in age from 22 to 73 years (M_age= 39.98, SD = 11.42). Participants were registered with outpatient services at a psychiatric hospital in Ontario, Canada. Participants were included if they met the following criteria: ≥ 18 years of age; fluency in English; understanding and willingness to comply with study requirements; meet DSM-5 criteria for current SUD according to the Structured Clinical Interview for DSM-5 (SCID-5);¹¹ have used substances in past 30 days according to the Timeline Follow-Back (TLFB);¹² seeking treatment for substance use. Exclusion criteria: untreated or unstable severe psychiatric or medical disorder that would impede participation; acute suicidal or homicidal ideation requiring immediate intervention; unable to read at a sixth-grade level. The institutional Research Ethics Board approved study procedures and all participants provided written informed consent.

Procedure

We conducted a parallel design randomized control trial with a 1:1 allocation ratio (ClinicalTrials.gov identifier: NCT03767907. See https://clinicaltrials.gov/ct2/show/NCT03767907; Figure 1). Participants were recruited from January 2019 to August 2020 following standard registration procedures. Prospective participants completed an initial telephone interview to assess basic demographic and clinical characteristics, and then a more in-depth in-person assessment to obtain fulsome demographic and health information. Notably, study assessments were switched to remote during the COVID-19 pandemic. Eligible participants were enrolled and randomized after the baseline “Week 0” assessment. Participants completed a post-treatment “Week 8” assessment including quantitative and qualitative measures. The same measures were administered at a 6-month follow-up (“Month 6”). Healthcare utilization and costs were extracted from the health record from baseline to Month 6. Randomization was stratified according to sex and diagnosis (AUD only vs. other) in randomly generated block sizes, without replacement. Randomization was conducted by an independent biostatistical consultant (sequences generated using R Studio), with allocation concealment ensured using sequentially numbered, opaque, sealed envelopes. Outcomes assessors blind to group assignment rated interview measures.

Figure 1.

Consort diagram.

Interventions

CBT4CBT: CBT4CBT consisted of seven online modules, including interactive videos presenting characters portrayed by professional actors struggling with real-life situations.⁵ These characters experience a common risky situation or problem and then, after the skill is presented and used, negotiate that situation without substance use. The program is user-friendly, requires minimal computer experience or reading, and includes interactive exercises to teach and model effective use of skills and strategies.

TAU: The active control intervention consisted of group therapy, supported by individual therapy as needed. Group therapy protocols ranged from 8 to 12 sessions. Models of therapy were primarily cognitive behavioural, with some groups including motivational enhancement, dialectical behavioural therapy concepts and mindfulness. Participants attended M = 3.58 (SD = 3.29) groups during the 8 weeks; no individual sessions were requested. All participants, regardless of their randomized condition, received psychiatric and emergency services as needed.

Measures

All quantitative measures were administered at Weeks 0, 8, and Month 6. The primary outcome was the number of days of use of the primary substance over the past 30 days, as measured by the TLFB.¹² Secondary outcomes were as follows: severity of the primary SUD, measured using the SCID-5 module E;¹¹ psychiatric functioning using the Psychiatric Composite score of the Addiction Severity Index (ASI);^13,14 severity of drug dependence from the Severity of Dependence Scale (SDS)¹⁵ (α=0.77); Global Severity Index (GSI) from the Brief Symptom Inventory (BSI)^14,16 (α=0.97); and all 4 domains of the World Health Organization Quality of Life Assessment (WHOQOL-BREF)^17,18 (physical α=0.82; psychological α=0.84; social relationships α=0.69, environment α=0.88).

Analyses

Efficacy Analysis: To evaluate efficacy, a generalized linear mixed model was conducted where the number of days the primary substance was used in the past 30 days was the level 1 primary outcome nested within participant. A Poisson distribution with a log link function was used. Time (pre-treatment, post-treatment) and group (CBT4CBT, TAU) were entered as fixed effects and individual subjects were specified as random effects.

To evaluate efficacy for secondary outcomes, mixed effect models were conducted, with each secondary outcome evaluated in separate models. Time was entered as a within-subject factor with two levels (pre-treatment, post-treatment), and group as a between-subject factor with two levels (CBT4CBT, TAU); individual subjects were specified as random effects. Mixed effect models handle missing values using maximum likelihood estimation.¹⁹ We followed the intention-to-treat principle and analyzed all randomized participants. Models were run with and without covariates, including age, sex, primary diagnosis (AUD only vs. other), and the COVID-19 pandemic (acute treatment outcome assessed before vs. after onset). All models were also conducted with the inclusion of the 6-month time point to evaluate similarity of effects over time.

An a priori power calculation indicated that the sample size required to locate a small-to-medium effect size based on Kiluk et al.,⁹ a power of 80% (for main effects or interactions), and an alpha level of 0.05 is 54 participants or 27 per group.

Economic Evaluation: A cost-effectiveness analysis was undertaken for the primary outcome – the number of days used of the primary substance – in line with Canadian Agency for Drugs and Technologies in Health (CADTH) guidelines²⁰ and Consolidated Health Economic Evaluation Reporting Standards (CHEERS).²¹ We adopted the perspective of the publicly funded healthcare payer.

CBT4CBT cost included the time per participant spent to show how to use the program ($10 for 15 min based on an hourly wage rate of $35.73) and the program fee ($140 per participant). Data on healthcare utilization and time spent by staff per service were extracted from the electronic health record. Staffing costs included physician visits and other OHIP-funded visits to allied health professionals. Three costing categories were identified: counselling, medical and administrative services. Total service cost was estimated by multiplying the hourly compensation for the service by the time spent providing the service.^22,23 (see Supplemental Table S1 for more details). Although the cost-effectiveness analysis solely included the costs of services received at the study site, participants were also asked at each visit regarding their expenditures on healthcare outside of this site, using the Program and Client Costs Substance Abuse Treatment.^24,25

Costs and outcomes were compared at follow-up, as mean values by arm with standard errors. An incremental cost-effectiveness ratio (ICER), defined as the mean difference in cost between groups divided by the mean difference in effect (i.e., the extra cost required to gain an additional unit of health), was calculated for the cost-effectiveness analysis. Of the participants randomized to treatment, 33 had the required follow-up outcome assessment (i.e., TLFB) to complete the cost-effectiveness analyses.

Qualitative Analysis: The qualitative component of the study was organized as a qualitative description design.²⁶ Semi-structured interviews were deployed with a focus on (1) what was and was not working for participants based on how they gave meaning to their experiences, and (2) “health work”²⁷ or practices surrounding health, including what individuals do to take care of themselves, and how they access and utilize health information and services.²⁸ All interviews were audio-taped and transcribed verbatim. Here, we contrast what did and did not work well for participants with a focus on CBT4CBT. The sample consisted of 21 participants at week 8 (CBT4CBT n = 11, TAU n = 10; 25 min on average) and 17 at month 6 (CBT4CBT n = 9, TAU n = 8; 12 min on average). Analyses were conducted on participants for whom COVID-19 did not impact acute treatment, until saturation was met.

Thematic analysis of the transcripts was conducted to offer a summative description based on how participants gave meaning to their experiences. Data analysis focused on identifying, analyzing and reporting patterns²⁹ and included a process that involved both open and focused coding.³⁰

Results

Participant Descriptives

The majority of participants (N = 50) were heterosexual (81.6%), identified as a man (69.4%), and were never married at the time of study (53.1%). The majority of the sample identified as White European and/or White North American (67.3%; see Table 1). Overall, 76% of individuals reported alcohol as their primary substance, 10% reported cannabis, 10% reported cocaine, 4% reported another substance as their primary substance of concern. 42.9% of the sample was diagnosed with AUD only, whereas 57.1% was diagnosed with either AUD plus another SUD, or another SUD without AUD. Co-occurring conditions were present in 58% (n = 29), including: persistent depressive disorder (36%, n = 18), major depressive disorder (30%, n = 15), generalized anxiety disorder (20%, n = 10), social anxiety disorder, panic disorder, agoraphobia or specific phobia (24%, n = 12), post-traumatic stress disorder (10%, n = 5), and other (6%, n = 3). 36% (n = 18) met criteria for more than one co-occurring diagnosis. Baseline demographic characteristics between groups were not significant.

Table 1.

Participant Descriptives.

Variable	CBT4CBT(N = 25)	TAU(N = 25)
	M (SD) / N (%)	M (SD) / N (%)
Age	40.83 (12.45)	38.64 (10.30)
Sex
Male	18 (72.0%)	18 (72.0%)
Female	7 (28.0%)	7 (28.0%)
Gender
Man	17 (68.0%)	18 (72.0%)
Woman	6 (24.0%)	7 (28.0%)
Missing	2 (8.0%)	0 (0%)
Sexual orientation
Heterosexual/straight	19 (82.6%)	22 (91.7%)
Other	4 (17.4%)	2 (8.0%)
Missing	2 (8.0%)	1 (4.0%)
Ethnicity
White	17 (68.0%)	16 (64.0%)
Other	10 (40.0%)	9 (36.0%)
Marital status
Never married	13 (52.0%)	14 (56.0%)
Married	8 (32.0%)	7 (28.0%)
Other	4 (16.0%)	4 (16.0%)
Employment status
Employed	15 (60.0%)	16 (64.0%)
Unemployed	6 (24.0%)	6 (24.0%)
Other	4 (16.0%)	3 (12.0%)
Level of education
High school or some college	5 (20.0%)	13 (52.0%)
Bachelor's degree	10 (40.0%)	8 (32.0%)
Advanced degree	10 (40.0%)	4 (16.0%)
Household income
$0 - $49,999	7 (28.0%)	5 (20.0%)
$50,000–$99,999	5 (20.0%)	8 (32.0%)
$100,000+	11 (44.0%)	10 (40.0%)
Number of days substance used (past 30 days)	22.96 (7.94)	18.04 (11.11)
Diagnosis
AUD only	13 (52.0%)	9 (36%)
Other (AUD + SUD comorbidity, other SUD)	12 (48.0%)	64 (66.7%)

Note. M = mean; SD = standard deviation; AUD = alcohol use disorder; SUD = substance use disorder. Cell sizes < 4 were combined into “other” categories to protect privacy. Other sexual orientation included gay, lesbian, homosexual, and bisexual. Other ethnicity included Indigenous, Arab, Black, Asian, and Latin American. Other marital status included separated, divorced, and domestic partnership. Other employment status included on disability, temporary leave, and retired.

Efficacy Analyses

Primary Outcome: Generalized linear mixed effects models revealed a main effect of time (F = 6.90, P = 0.01), such that both groups improved from Week 0 (M = 16.70) to 8 (M = 14.49). There was also a main effect of group (F = 4.76, P = 0.03), such as that individuals in the CBT4CBT group had higher overall number of days used of the primary substance (M = 19.70) compared to the TAU group (M = 12.29).

Secondary Outcomes: Mixed effects models revealed main effects of time for multiple study outcomes, such that scores improved from Week 0 to 8 on SCID-5 Module E, BSI GSI, and WHOQOL-Bref Psychological domain scores (see Table 2a). There were no significant interactions between time and study group for any outcomes.

Table 2.

Efficacy Analyses.

	With covariates			Without covariates
	Time	Group	Time *group	Time	Group	Time *group
	F (P)	F (P)	F (P)	F (P)	F (P)	F (P)
A. Generalized Linear Model & Mixed Effects Models (Week 0 and Week 8)
TLFB	6.90 (0.01)	4.76 (0.03)	0.89 (0.35)	6.20 (0.02)	4.57 (0.04)	1.17 (0.28)
SCID-5	21.82 (<0.001)	0.002 (0.96)	0.00 (0.98)	24.59 (<0.001)	0.13 (0.72)	0.01 (0.94)
SDS	0.03 (0.86)	0.05 (0.82)	0.11 (0.75)	0.01 (0.94)	0.04 (0.84)	0.40 (0.53)
BSI – GSI	7.29 (0.01)	0.01 (0.92)	1.09 (0.30)	4.50 (0.04)	0.00 (0.99)	0.42 (0.52)
WHOQOL
Physical	0.00 (0.99)	0.002 (0.96)	0.17 (0.68)	0.05 (0.82)	0.09 (0.77)	0.26 (0.61)
Psychological	10.69 (0.002)	0.21 (0.65)	0.16 (0.69)	9.71 (0.004)	0.46 (0.50)	0.13 (0.72)
Social	2.47 (0.13)	0.05 (0.83)	1.82 (0.19)	1.95 (0.17)	0.09 (0.77)	1.95 (0.17)
Environment	1.37 (0.25)	0.04 (0.84)	0.12 (0.74)	1.02 (0.32)	0.11 (0.74)	0.17 (0.74)
ASI – Psychiatric	0.50 (0.49)	0.03 (0.87)	0.36 (0.56)	0.40 (0.53)	0.27 (0.61)	0.16 (0.69)
B. Generalized Linear Model & Mixed Effects Models (Week 0, Week 8, Month 6)
TLFB	4.72 (0.01)	6.20 (0.01)	2.23 (0.11)	4.35 (0.02)	5.98 (0.02)	2.50 (0.09)
SCID-5	28.11 (<0.001)	0.30 (0.59)	0.56 (0.58)	29.40 (<0.001)	0.95 (0.34)	0.90 (0.41)
SDS	0.59 (0.56)	0.30 (0.58)	0.46 (0.64)	0.27 (0.77)	0.00 (0.99)	0.32 (0.73)
BSI – GSI	6.88 (0.002)	0.27 (0.61)	1.55 (0.22)	6.14 (0.002)	0.09 (0.77)	0.85 (0.43)
WHOQOL
Physical	0.38 (0.68)	0.30 (0.59)	3.83 (0.03)	0.70 (0.50)	0.55 (0.46)	3.07 (0.053)
Psychological	7.12 (0.002)	0.77 (0.39)	1.01 (0.37)	6.32 (0.003)	1.19 (0.28)	1.16 (0.32)
Social	1.84 (0.17)	0.13 (0.72)	3.17 (0.049)	1.44 (0.24)	0.04 (0.83)	2.69 (0.08)
Environment	1.04 (0.36)	0.004 (0.95)	1.19 (0.31)	1.01 (0.37)	0.00 (0.95)	0.89 (0.41)
ASI – Psychiatric	0.39 (0.68)	1.38 (0.25)	0.44 (0.65)	0.35 (0.71)	3.19 (0.09)	0.87 (0.42)

Note. TLFB = Timeline Followback; SCID-5 = Structured Clinical Interview for DSM-5; SDS = Severity of Dependence Scale; GSI = Global Severity Index of the Brief Symptom Inventory, WHOQOL-Bref = World Health Organization Quality of Life Assessment, ASI = Addiction Severity Index. Statistically significant results highlighted in boldface.

Analyses incorporating the 6-month follow-up assessment demonstrated the same pattern of effects (see Table 2b). Two time×group effects were located in these analyses for two domains of the WHOQOL; however, as these effects were not replicated across analyses with and without covariates, they are not interpreted substantively. See Supplemental Table S2 for marginal means for each outcome.

Cost-Effectiveness Analyses

Average and incremental healthcare resource utilization are reported in Table 3a for both groups. Compared to TAU, CBT4CBT had fewer counselling sessions on average per participant. The categories in percentage of total costs were medical care (CBT4CBT: 27%; TAU: 30%), administrative tasks (CBT4CBT: 7%; TAU: 7%) and counselling (CBT4CBT: 54%; TAU: 65%).

Table 3.

Average and Incremental Healthcare Utilization, Healthcare Costs and Health Outcomes.

	CBT4CBT (n = 15)	TAU (n = 18)	(CBT4CBT) – (TAU)
	Mean (SE)	Mean (SE)	Mean (SE)
A. Healthcare utilization
Counselling sessions	5.8 (3.2)	16.1 (2.8)	−10.3 (4.2)*
Administrative tasks (number)	5.9 (1.8)	9.6 (1.5)	−3.7 (2.3)
Medical care (number)	5.9 (1.9)	6.7 (1.1)	−0.8 (2.2)
B. Healthcare costs (2022 CAD)
Counselling sessions	685.7 (497.3)	1457.0 (249.6)	−771.3 (529.1)
Administrative tasks	83.3 (28.7)	147.9 (24.8)	−64.7 (37.7)
Medical care	346.8 (113.7)	632.8 (103.4)	−286.1 (153.7)
CBT4CBT intervention	149.8 (0.3)	0 (0)	149.8 (0.3)*
Total	1,266.1 (544.9)	2,237.8 (304.7)	−971.7 (598.2)
C. Health outcomes
Mean effect (substance use days)	22.7 (2.5)	13.3 (3.0)	9.3 (4.0)*

Note. CBT4CBT = computer-based training for cognitive-behavioural therapy; TAU = treatment as usual; SE = standard error; CAD = Canadian dollars.

Health provider average hourly compensation was retrieved from the Ontario Health Insurance Plan fee schedule.²² Administrative staff average hourly compensation came from CAMH People & Experiences department. All costs were adjusted to 2022 Canadian dollars amount using the Consumer Price Index Health and Personal Care from Statistics Canada.³¹

*P-value < 0.05.

Table 3b and Table 3c presents the two groups’ average and incremental costs and outcome differences. Although the average incremental total cost was not statistically significant, it indicated a lower average treatment cost of $972 per participant. Within this subsample analysis, CBT4CBT was associated with more substance use days compared to TAU. The ICER for the primary outcome was $104.2/substance use days, suggesting that CBT4CBT was less costly but was associated with more substance use days. Sensitivity analyses controlling for age, sex and baseline values of substance use days showed similar results. Notably, only one participant reported limited treatment expenditures during the study period.

Qualitative Analyses

CBT4CBT: Themes identified during the qualitative analysis are reviewed below, with illustrative quotes in Table 4. At Week 8, improved subjective well-being was reported by many participants as a benefit of CBT4CBT, although this was often described as small in size. As 109 summarized, “health-wise I feel a little better.” Some participants reported substantial changes in substance use, describing how they are “drinking less…I feel like my drinking has, um, reduced quite a bit. And that was my goal, to kind of get that more under control” (41).

Table 4.

Qualitative Analysis: Themes and Illustrative Quotes.

Theme	Assessment	Quote
CBT4CBT
Positive outcomes	Week 8	“experienced more positive periods of time alongside that, that program; like when I do the modules for it” (119)“do feel different and I do feel better” (128)“[roommate was] surprised…oh, the beer's still in there’…like I’ve lived with him for a year and like four months.’ He's like, ‘I’ve never seen that from you…It's been sitting there for days.’ I’m like, ‘yeah, I know.’” (109)
	Month 6	“Well, it's been six months and I’ve cut my drinking in half, which is what I wanted. I wanted to have more control over my drinking, um, so yeah…a year ago, I don’t think I would have thought I could have done this” (41)“specifically helped keep my mind towards that, kind of, healthy living” (41)
Awareness/Insight	Week 8	“found that it did help with like recognizing certain triggers. And then, but, like really identify them, and knowing that I’m not the only one who has those issues” (132)“recognizing that it was a trigger or…craving” (128)“bit of a window on yourself” (113)
	Month 6	“understand my issue” (109)“thinking about triggers” (11)
Tools/strategies	Week 8	“think through some of the strategies…deep breathing” (128)“Since I watched the module yesterday afternoon and then I had [an event to go to] in the evening, and in the evening, um, uh, and, you know, I wanted to…everybody was drinking because it's kind of how these things are done…So I, I allowed myself to have the one drink and then I, I kind of just, um, I found just the wording to stop from having more after that… and people just seemed to accept that, I mean, without giving further details on what I'm going through…But just having the right wording on why, uh, that I won't, but, uh, just hang out and have a club soda for the rest of the night, that sort of thing.” (41)
	Month 6	“some valuable skills” which resulted in their ability to “avoid situations,” and ultimately “reduc[ed] desire to drink” (171).
Reappraisal	Week 8	“think more positively” (117)“the little films and the questions are reflective of other people. And, it's kind of a…well, it's nice to see that you’re not as bad off as the next [Chuckles] guy.”
	Month 6	“Um, yeah, it just gets me in the right mindset…gets me thinking about how I can make little changes to things I do throughout the day and every like that. I found that if I used it at the start of the day it would stay with me. I’d be able to keep things in mind throughout the day when I'm making decisions or feeling emotional about things.” (41)“look differently at alcohol…And it's helped me to think differently to…[I] don’t want to drink…[it] helped me to change my way… [It] help me to stop, uh, and help me to like understand” (117).
Increased self-confidence	Week 8	“more in control” of substance use (128)“it's given me a little more confidence…[in] recovery” (132)“I was doing something. I was doing something towards fixing a problem” (119)
Flexible delivery	Week 8	“you don’t have to talk to another person if you don’t want to, and you can kind of just think through the stuff on your own terms and that's beneficial.” (128)“Like I’m shy when I’m talking to people. It's not really help me. But, when I was with this program it means I’m just me in that program. It's nobody judge me and nobody like…I don’t have to share extra things. And, it's very clear that explained what is right and what is wrong; what you have to do about this.” (117)“more [on] your own time” (109)
	Month 6	“great delivery model” (171)
Lack of accountability	Week 8	“because you don’t have, like I imagine you don’t have that feeling of obligation of keeping an appointment” (128)
Simplistic format/content	Week 8	“just common sense…[not geared towards a] more sophisticated audience”“little repetitive”lacking a “modern” or “cutting edge” feel
TAU
Positive outcomes	Week 8	“able to start smiling again…[I] feel more relaxed and not so tense” (138).“changed my relationship with my daughter… it's also improved my relationship with my wife.” (22)
	Month 6	“well [the program] pretty much stopped me from doing coke on a weekly basis, so that's good. I mean, that's the biggest thing that's come out of it. Uh, not completely free of it, but definitely a humungous decrease…that's, that's the biggest thing and the most, you know, concrete example of the benefit” (87).“I think it's improved my relationship with my family. It's definitely improved my relationship with my daughter…and it's improved my relationship with my dad.” (22)
Insight	Week 8	It made me realize that if I kept this type of lifestyle going on that like this sounds bad. But, I could end up like some of those people in group that are homeless or that are, like, in bizarro-world…It made me realize that I can get help. And, it helped me to not be embarrassed” (31)
	Month 6	“new perspective on things” (131)
Skill development	Week 8	“mindfulness…taking time to be aware of your surroundings” (118)“being mindful of how much you’re actually drinking and paying attention to that…” (138)“listening to other people and their experiences…what they did to get rid of this habit or to reduce…the cravings and stuff like that…were very helpful” (145)
	Month 6	to “control my whole habit…you know, cut down and everything…talking about it does help, you know, and other things like, uh, practicing what other people kind of ideas like, you know, what people try, other people try and then you just try to implement those ones, but yeah, they were very helpful in the end.” (22)“the idea of challenging automatic thoughts… recognizing triggers and the idea of, uh, urge surfing, is another one that we talked about” (22).
Increased hope	Week 8	“new perspective on things” (131)“you can be fairly addicted to something and it's not, it's not the end of the world. You can treat it, and you can, uh, come out of it.” (87)“…it's been good for me to realize that it's more common than I thought. And that like there are more normal like, um, high functioning people like me who suffer from the same situation that I do” (107).
Community	Week 8	“you feel less alone” (22)“in the same boat” (87)“best part of the group is the community…open and accessible, and welcoming, and like pressure-free, and honest place where you can go and talk about things. And, you don’t have to say a word” (107)“The most things that motivate me is that the person who is actually running the group has gone through this, uh, scenario himself…how everyone in the group are fighting a battle, the same battle, I would say, and that made me think that I’m not the only one.” (155)
	Month 6	“feeling less alone” (107), “not alone” (118), or “you’re not the only one out there” (170)
Safe space	Week 8	“during the week where…it was a place where there's no drinking, so that was always, that was a positive” (138)
	Month 6	“necessarily feel safe enough to, uh, really express myself, you know, like emotionally speaking…I felt like, uh, I felt like I, I had to hold back, you know…it was easy to hide sometimes.” (118)
Format benefits	Week 8	“timeframe was good. Not too much, not too little” (136)
Content too general / not tailored	Week 8	“good refresher, in terms of that sort of information about, you know, just dealing with it and how to, how to manage it with your addiction…It was good to go over…[But] I’m not sure how many things I learned that I didn’t, kind of, already know.” (87)“I think that's the future of group therapy is having group therapy and then taking a little bit individually on your own. And then, almost like Legos, you’re just building a wall up all the time. And, the more pieces that you have the better it's going to be.” (31)
Triggering / upsetting	Week 8	“triggering. Like, guys would come in smelling like weed, especially the first couple of weeks, so that was triggering” (22)“Um, like the one time when we did the mindfulness, the first time, I left feeling great. But then, a lot of the time or most of the times, I left feeling worse off than when I went in.” (118)“hopeless…like I’m not going to change” (118)
Self-responsibilizing	Month 6	“my own self [got in the way]. [Chuckles]…I have my own way of self-sabotaging and staying in my head and not being disciplined or, enough to continue with it kind of thing” (118)“not [being] great at applying [the tools learned]” (107)

Participants who received CBT4CBT reported “small but positive” (128) and larger benefits from the intervention. Benefits included increased awareness and insight regarding triggers and cravings, and learning tools or strategies to meet substance use goals such as deep breathing and refusal skills. For example, CBT4CBT allowed many participants to recognize triggers and “choose something else to do,” facilitating healthier choices, and reducing harm associated with substance use – “like notice why I was going to drink, and remind me that I have an option to do something else” (141).

Participants reported that CBT4CBT videos supported skill development and cognitive reappraisal, helped them feel more confident, and know they are not alone in their struggles. The program's flexible format provided greater clarity: “I try one more CBT group actually with people; like with 10 people sitting. And uh, and like two workers … for me it doesn’t work. It's a lot of opinions. A lot of different, uh, experiences. And, it's a big mix for me in my head” (117). This format was also helpful for those with concerns about judgment from others.

Indeed, some described the program as a “brilliant delivery model … I think the convenience of delivery, the privacy of delivery to certain people. Like, I mean that's a big deal for me” (171). Others described that once they got used to the online format, “[it] actually did help…I felt it, it worked pretty good. It didn’t like fully stop me from drinking. But, it gave me ideas and ways to cope, and ways to look at things from a perspective that I didn’t really look at” (109). Furthermore, while some participants were “firm believer[s] in that….one-on-one…would work even better,” the online program met many participants where they were at and was a good “fit” for many (109).

While some participants described the online format as very positive due to how simple it was to use and its ability to reduce barriers, some participants described program format as a “blessing and a curse” (128) as they found it challenging to keep themselves accountable. Importantly, some participants were critical of the program for being “pretty simplistic” (113), a “little repetitive” (128), and thus less impactful – “I’ll say this again for the record, that like I think that it would be impactful for a lot of people. Uh, for myself I don’t think it's impacted me” (132).

Many of the benefits at Week 8 were still present at the Month 6 mark, such as tools to support better choices: “But, I think the value is that it took it to the front of mind, um, and reminded me of things that I maybe just wouldn’t have been thinking of … it helped me think” (119). Other participants described the maintenance of insight and skill development at Month 6, which resulted in behavioural change and decreased craving. Overall, many participants described that CBT4CBT “sticks with me” (132), ranging from descriptions of a profound effect – “…on the whole it was really useful for me” (125) – to a “change [in] my attitude towards mental health” (132) to, at the very least, being a “step” in the right direction – “a great starter” (109). For those who did not find the program useful at Month 6, a positive outcome was realizing what does work for them. [For a description of the qualitative results for TAU, please see Supplemental Results: TAU Qualitative Description.]

Discussion

The present study extends the literature by examining the effects of CBT4CBT versus TAU in a sample of diverse treatment-seeking individuals with SUDs. We examined not only quantitative outcomes of substance use and health, but also adopted a qualitative and economic evaluation lens to understand the impacts of CBT4CBT in a Canadian context. Findings regarding the number of days an individual's primary substance was used showed improvement for both groups; contrary to hypotheses, however, there were no differences in improvement across study groups. This contrasts with previous research examining CBT4CBT as a stand-alone compared to individual CBT and TAU, in which both CBT4CBT and individual CBT were associated with greater reduction in the frequency of substance use than TAU.⁹ Notably, a main group effect was also exhibited for the primary outcome, suggesting that individuals from TAU had fewer days used compared to CBT4CBT. No other group effects were observed for SUD or mental health outcomes, suggesting that this difference was specifically related to use rather than harms or consequences related to use. Consistent with hypotheses, CBT4CBT was associated with lower healthcare use related to counselling in a subsample analysis. The economic evaluation suggested that CBT4CBT was associated with lower healthcare costs; however, higher substance use days in this group suggested that this cost reduction could be linked to reduced outcomes.

There were also no significant interactions of group and time observed in our secondary outcomes. Participants in both CBT4CBT and TAU exhibited improvements in the number of SUD symptoms endorsed, global severity of psychological symptoms, and psychological quality of life from pre-treatment to post-treatment. These results also differ from other studies of CBT4CBT as a stand-alone treatment versus standard care,^8,9 which may suggest that differences across treatment conditions might vary across geographical regions and patient populations. However, evaluating other outcomes would be valuable.^32,33

Exploring the impact of CBT4CBT from a qualitative perspective provides important insights into this digital intervention's potential. This qualitative analysis complements what the efficacy analysis and economics evaluation reported and brings to the fore what might have been left in abeyance in the quantitative analysis, mainly, how participants construct their social world³⁴ in relation to their participation in this study. As Hills argues, “Qualitative methods provide the means by which to explore and explain people's experiences in ways that cannot be captured by quantitative methods or reported by statistical data.”³⁵ Future research may extend this important analysis; for example, a discourse analysis may provide insights into participants’ language use and contribute to refining the CBT4CBT intervention.

Limitations: The recommended economic evaluation by CADTH is a cost-utility analysis; given the lack of utilities for the WHOQoL-BREF, undertaking a cost-utility analysis was not possible. Furthermore, the subsample used to undertake the cost-effectiveness analysis was small; this and overall group differences may have implications for the interpretation and generalizability of the findings. Additionally, the current sample size was less than anticipated due to the COVID-19 pandemic, which also impacted power and generalizability of efficacy analyses. Notably, qualitative research does not aim to be generalizable per se; rather, the aim is to shed light on participant's lived-experiences. With that said, qualitative findings are typically transferable to other contexts.³⁶ Finally, attrition over the follow-up was high, which may have biased estimates. The heterogeneity and complexity of the sample, and more accurate effect size estimates and power calculations, are critical considerations for future research.

Conclusions

CBT4CBT has demonstrated promise in improving access to treatment for those struggling with substance use. Although this investigation found that participants receiving CBT4CBT improved over acute treatment and longer-term follow-up on the primary and some secondary outcomes, they did not differ from those who received standard care. These results cannot be interpreted as support of equivalence between these treatments, however, as target accruals were not met due to the COVID-19 pandemic. Yet, the economic analysis supported that participants who received CBT4CBT had fewer healthcare costs than those who received TAU, and qualitative analyses identified areas of strength and growth for this digital health resource. Data from this investigation can inform continued evaluation and implementation of this program, incorporating unique and key information, including local estimates of clinical outcomes and costs and patient perspectives of treatment format and content. The current investigation is thus a crucial step to realizing this innovation in clinical care to innovation in clinical practice.

Supplemental Material

sj-docx-1-cpa-10.1177_07067437241255100 - Supplemental material for Computer-Based Training for Cognitive Behavioural Therapy for Substance Use Disorder: A Randomized Controlled Trial Including Quantitative and Qualitative Health and Economic Outcomes: Formation informatisée pour la thérapie cognitivo-comportementale pour les troubles liés à l'usage de substances : un essai randomisé contrôlé y compris les résultats quantitatifs et qualitatifs en matière de santé et d’économie

Supplemental material, sj-docx-1-cpa-10.1177_07067437241255100 for Computer-Based Training for Cognitive Behavioural Therapy for Substance Use Disorder: A Randomized Controlled Trial Including Quantitative and Qualitative Health and Economic Outcomes: Formation informatisée pour la thérapie cognitivo-comportementale pour les troubles liés à l'usage de substances : un essai randomisé contrôlé y compris les résultats quantitatifs et qualitatifs en matière de santé et d’économie by Alina Patel, Michael Corman, Claire de Oliveira, Joyce Mason, Nibene Somé, Danielle Downie, Esha Jain, Michelle Patterson, John A. Cunningham, Tony P. George, Bernard Le Foll and Lena C. Quilty in The Canadian Journal of Psychiatry

Supplemental Material

sj-doc-2-cpa-10.1177_07067437241255100 - Supplemental material for Computer-Based Training for Cognitive Behavioural Therapy for Substance Use Disorder: A Randomized Controlled Trial Including Quantitative and Qualitative Health and Economic Outcomes: Formation informatisée pour la thérapie cognitivo-comportementale pour les troubles liés à l'usage de substances : un essai randomisé contrôlé y compris les résultats quantitatifs et qualitatifs en matière de santé et d’économie

Supplemental material, sj-doc-2-cpa-10.1177_07067437241255100 for Computer-Based Training for Cognitive Behavioural Therapy for Substance Use Disorder: A Randomized Controlled Trial Including Quantitative and Qualitative Health and Economic Outcomes: Formation informatisée pour la thérapie cognitivo-comportementale pour les troubles liés à l'usage de substances : un essai randomisé contrôlé y compris les résultats quantitatifs et qualitatifs en matière de santé et d’économie by Alina Patel, Michael Corman, Claire de Oliveira, Joyce Mason, Nibene Somé, Danielle Downie, Esha Jain, Michelle Patterson, John A. Cunningham, Tony P. George, Bernard Le Foll and Lena C. Quilty in The Canadian Journal of Psychiatry

Footnotes

Acknowledgments

We would like to thank Dr. Kathleen Carroll, who was Co-Principal Investigator of this seed grant and contributed substantially to project design; her inspiration and guidance cannot be overstated. We would also like to thank all participants for sharing their insights and experiences with us.

Abbreviations

Data Access

Data is available at .

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: MC, CdO, MP, JC, TG, BL, KMC, LCQ were awarded a Centre for Addiction and Mental Health (CAMH) Foundation Discovery Fund Seed Grant for this project. BL is supported by CAMH, Waypoint Centre for Mental Health Care, a clinician-scientist award from the Department of Family and Community Medicine of the University of Toronto, and an Addiction Psychiatry Chair from the Department of Psychiatry of the University of Toronto. TPG is Co-Principal Editor of Neuropsychopharmacology (NPP), the Official Journal of the American College of Neuropsychopharmacology (ACNP). All other authors disclose no relevant conflict of interest.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded by the CAMH Foundation Discovery Fund. This funding agency was not involved in study design, analyses, results or conclusions.

ORCID iDs

Nibene Somé

John A. Cunningham

Tony P. George

Lena C. Quilty

Supplemental Material

Supplemental material for this article is available online.

References

Rehm

Gnam

Popova

, et al. The costs of alcohol, illegal drugs, and tobacco in Canada, 2002. J Stud Alcohol Drugs. 2007;68:886-895.

Substance Abuse and Mental Health Services Administration. Results from the 2010 National Survey on Drug Use and Health: Summary of national findings, NSDUH Series H-41, HHS Publication No. (SMA) 11-4658. Rockville, MD: Author; 2011.

Carroll

. Lost in translation? Moving contingency management and cognitive behavioral therapy into clinical practice. Ann N Y Acad Sci. 2014;1327:94-111.

Boumparis

Karyotaki

Schaub

Cuijpers

Riper

. Internet interventions for adult illicit substance users: a meta-analysis. Addiction. 2017;112:1521-1532.

Carroll

Ball

Martino

, et al. Computer-assisted delivery of cognitive-behavioral therapy for addiction: a randomized trial of CBT4CBT. Am J Psychiatry. 2008;165:881-888.

Carroll

Ball

Martino

Nich

Babuscio

Rounsaville

. Enduring effects of a computer-assisted training program for cognitive behavioral therapy: a 6-month follow-up of CBT4CBT. Drug Alcohol Depend. 2009;100:178-181.

Carroll

Kiluk

Nich

, et al. Computer-assisted delivery of cognitive-behavioral therapy: efficacy and durability of CBT4CBT among cocaine-dependent individuals maintained on methadone. Am J Psychiatry. 2014;171:436-444.

Kiluk

Devore

Buck

, et al. Randomized trial of computerized cognitive behavioral therapy for alcohol use disorders: efficacy as a virtual stand-alone and treatment add-on compared with standard outpatient treatment. Alcohol Clin Exp Res. 2016;40:1991-2000.

Kiluk

Nich

Buck

, et al. Randomized clinical trial of computerized and clinician-delivered CBT in comparison with standard outpatient treatment for substance use disorders: primary within-treatment and follow-up outcomes. Am J Psychiatry. 2018;175:853-863.

10.

Drummond

Coyle

. The role of pilot studies in the economic evaluation of health technologies. Int J Technol Assess Health Care. 1998;14:405-418.

11.

First

Williams

JBW

Karg

Spitzer

. Structured clinical interview for DSM-5—Research Version (SCID-5 for DSM-5, Research Version; SCID-5-RV). Arlington, VA: American Psychiatric Association; 2015.

12.

Sobell

. Timeline follow-back: a technique for assessing self-reported alcohol consumption. In: Measuring alcohol consumption: psychosocial and biochemical methods. Totowa, NJ: Humana Press; 1992:41-72.

13.

McLellan

Kushner

Metzger

, et al. The fifth edition of the Addiction Severity Index. J Subst Abuse Treat. 1992;9:199-213.

14.

Kiluk

Roos

Aslan

, et al. Detecting change in psychiatric functioning in clinical trials for cocaine use disorder: sensitivity of the Addiction Severity Index and Brief Symptom Inventory. Drug Alcohol Depend. 2021;228:109070.

15.

Gossop

Darke

Griffiths

, et al. The Severity of Dependence Scale (SDS): psychometric properties of the SDS in English and Australian samples of heroin, cocaine and amphetamine users. Addiction. 1995;90:607-614.

16.

Derogatis

. Brief Symptom Inventory: Administration and procedures manual - I. Baltimore: Clinical Psychometric Research, Inc; 1982.

17.

Group

. Development of the World Health Organization WHOQOL-BREF quality of life assessment. Psychol Med. 1998;28:551-558.

18.

Skevington

Lofty

O’Connell

. The World Health Organization’s WHOQOL-BREF quality of life assessment: psychometric properties and results of the international field trial. A report from the WHOQOL group. Qual Life Res. 2004;13:299-310.

19.

Hallgren

Witkiewitz

. Missing data in alcohol clinical trials: a comparison of methods. Alcohol Clin Exp Res. 2013;37:2152-2160.

20.

CADTH. Guidelines for the economic evaluation of health technologies: Canada. 4th ed. Ottawa: Author; 2017.

21.

Husereau

Drummond

Augustovski

, et al. Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations. Int J Technol Assess Health Care. 2022;38:e13.

22.

Ministry of Health, Schedule of Benefits: Physician Services Under the Health Insurance Act. 2022.

23.

Canadian Institute for Health Information. Hospital spending: Focus on the emergency department. Ottawa, ON: CIHI; 2020.

24.

Jofre-Bonet

Sindelar

Petrakis

, et al. Cost effectiveness of disulfiram: treating cocaine use in methadone-maintained patients. J Subst Abuse Treat. 2004;26(3):225-232.

25.

Olmstead

Sindelar

Easton

Carroll

. The cost-effectiveness of four treatments for marijuana dependence. Addiction. 2007;102(9):1443-1453.

26.

Sandelowski

. Whatever happened to qualitative description? Res Nurs Health. 2000;23:334-340.

27.

Mykhalovskiy

McCoy

Bresalier

. Compliance/adherence, HIV, and the critique of medical power. Soc Theory Health. 2004 Nov 1;2:315-340.

28.

Nichols

Fridman

Ramadan

Ford Jones

Mistry

. Investigating the social organization of family health work: an institutional ethnography. Crit Public Health. 2016 Oct 19;26(5):554-565.

29.

Scharp

Sanders

. What is a theme? Teaching thematic analysis in qualitative communication research methods. Communication Teacher. 2019 Apr 3;33(2):117-121.

30.

Taylor

Bogdan

DeVault

. Introduction to qualitative research methods: A guidebook and resource. London: John Wiley & Sons; 2015 Oct 19.

31.

Statistics Canada. Consumer Price Index, annual average, not seasonally adjusted. 15 December 2023]; Available from: https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1810000501

32.

Kiluk

Nich

Babuscio

Carroll

. Quality versus quantity: acquisition of coping skills following computerized cognitive-behavioral therapy for substance use disorders. Addiction. 2010 Dec;105(12):2120-2127.

33.

Sugarman

Nich

Carroll

. Coping strategy use following computerized cognitive-behavioral therapy for substance use disorders. Psychol Addict Behav. 2010 Dec;24(4):689.

34.

Carter

Ritchie

Sainsbury

. Doing good qualitative research in public health: not as easy as it looks. N S W Public Health Bull. 2009 Sep 7;20(8):105-111.

35.

Hills

. Human science research in public health: the contribution and assessment of a qualitative approach. Can J Public Health. 2000 Nov 1;91(6):I4.

36.

Corman

. The positives of caregiving: mothers’ experiences caregiving for a child with autism. Fam Soc. 2009 Oct;90(4):439-445.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.21 MB

0.02 MB