Canadian Studies on the Effectiveness of Community Treatment Orders: An Updated Systematic Review of Quantitative Data: Études canadiennes sur l’efficacité des ordonnances de traitement en milieu communautaire : mise à jour d’un examen systématique des données quantitatives

Abstract

Objectives

Community treatment orders (CTOs) for people with severe mental illnesses are used across most of Canada. Our previous systematic review of 10 years ago found that the evidence-base was limited to small studies, with only one including controls. This review updates the evidence using studies conducted in Canada over the last decade.

Methods

A systematic search of PubMed/Medline, Embase, CINAHL, and PsycINFO for any Canadian study of outcomes following CTO placement from March 2015 to January 2025.

Results

We identified four articles from three studies. Adding these studies to the previous search gave a total of nine articles from seven studies. None could be included in a meta-analysis. There were reductions in readmission rates and bed-days following CTO placement, while psychiatric symptom, outpatient attendance, treatment adherence participation in psychiatric services and housing all improved. In one study, perceived coercion was no greater in the CTO cases than the controls and being on an order preferable to being in hospital. However, many of the studies were small and only two included controls, of which solely one adjusted for potential confounders using either matching or adjusted analyses. The certainty of evidence was therefore rated as very low.

Conclusions

The evidence-base for the use of CTOs in Canada remains limited. This research gap contrasts with other countries that have conducted large studies using randomized or matched controls and adjusted analyses. There is a need for larger studies with more standardized reporting methods to allow for the pooling of results.

Protocol Registration Number

Prospectively registered with PROSPERO registration number CRD42024615480

Introduction

This review updates our systematic review (SR) of 2016 on the impacts of community treatment orders (CTOs) in Canada. ¹ The 2016 SR identified five papers from four quantitative studies plus three qualitative evaluations. It found only limited evidence from largely uncontrolled studies that CTOs reduced inpatient service use while improving outpatient attendance, participation in the community, and housing. However, conclusions were limited because many of the studies were small (median n per study = 46), and only one included a control group. Regression to the mean may have therefore contributed to some of the observed reductions in admissions and bed-days. It was also not possible to meta-analyze the data because of the wide range of methodologies and outcomes. In this update, we limited our search to quantitative studies to ensure a uniform and standardized assessment of both quality and certainty of evidence across studies.

Method

Search Strategy

We registered the protocol for this SR with PROSPERO (CRD42024615480) and followed guidelines from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses. The following databases were searched from March 2015 (the date of our last search) to the latest available (January 2025) for papers in English or French (Canada's two official languages): PubMed/Medline, Embase, CINAHL and PsycINFO. ² Table 1 shows the PubMed/Medline search terms. Ethical approval was not required as all data had previously been published.

Table 1.

Search Terms for PubMed.

Database

Search terms

PubMeda

((Canad* OR “British Columbia” OR Alberta OR Saskatchewan OR Manitoba OR Ontario OR Quebec OR “Newfoundland and Labrador” OR Newfoundland OR “Prince Edward Island” OR “Nova Scotia” OR “New Brunswick” OR “Northwest Territories” OR “Yukon” OR Nunavut) AND ((“Psychiatry” AND “Commitment of Mentally Ill"[Mesh]) OR “Forensic Psychiatry"[Mesh] OR “Mandatory Programs"[Mesh] OR “community treatment order*” OR “involuntary outpatient treatment” OR “involuntary outpatient commitment” OR “extended leave” OR “compulsory community treatment” OR “treatment authority” OR “supervised discharg*” OR “conditional release” OR “mandatory outpatient treatment” OR (“extended release” NOT “Pharmaceutical Preparations”[mesh]) OR (“extended outpatient” AND “civil commitment”))) NOT (“COVID*” OR “pandemic” OR “vaccin*”)

^aThe same search terms were used for Embase, PsycINFO, and CINAHL with the absence of MESH terms.

Two authors independently screened records/abstracts and extracted data, consulting a third reviewer if consensus could not be achieved. The reference lists of selected retrieved papers were also screened.

Inclusion and Exclusion Criteria

We included any of the following study designs conducted in Canada that reported on quantitative outcomes following CTO placement: randomized controlled trials, cohort, case–control cross-sectional studies and before-and-after studies. We excluded studies of drug or alcohol use.

Outcomes of CTO Placement

Primary outcomes were hospital admissions, bed-days and community contacts in the 12 months following CTO placement. The following were secondary outcomes: psychiatric symptoms as measured by a standardized psychiatric instrument; concordance with psychiatric treatment; employment; contacts with the criminal justice system; perceived coercion as measured by surveys; housing; and quality of life.

Study Quality

All the papers were independently assessed for quality using the Joanna Briggs Institute (JBI) tool for quasi-experimental studies as they were cross-sectional or before-and-after designs (with or without controls). ³ The JBI tool assesses the timing of any cause and effect, the presence and comparability of controls, and whether outcomes were measured in a reliable and comparable way.

Certainty of Evidence

The overall certainty of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework. ⁴ This ranges from high for randomised controlled trials (RCTs) and low for observational studies. Ratings can be altered up or down based on study quality, inconsistency, indirectness, imprecision and publication bias.

Results

We found 905 citations of interest in the updated search. There were 114 duplicates leaving a total of 791 articles. Of these, 62 full-text papers were potentially relevant and assessed for eligibility (Figure 1). Four articles from three studies met inclusion criteria.^5‐8 Reasons for exclusion were that articles were conference abstracts or editorials, or that they did not contain primary data, relevant outcomes or suitable controls (Figure 1). Adding these studies to the previous search meant that there were nine articles from seven studies in total (Figure 1 and Table 2).^5‐13 All the studies came from either Ontario or Quebec.

Figure 1.

PRISMA flow diagram.

Table 2.

Included Studies.

First author		Year	Design	N	Mean age	Outcomes
O’Brien		2005	UBA	25 60% male	42	↓admissions and bed-days up to 12-month follow-up ↑ contact with community services ↑ supportive housing arrangements
O’Brien		2009	UBA	84 60% male	45	↑ contact with community services in 80% of the sample ↑ supportive housing arrangements
Same study	Frank	2005	UBA	42 52% male	48.4	↑ time to readmission particularly in the period immediately after the CTO
	Nakhost	2012	UBA	72 52% male	46.3
Hunt		2007	CBA	224 CTO cases 92 controls 45% male	51% aged between 31 and 40	Unadjusted analyses only. Although the two groups were not significantly different in terms of socio-demographic or clinical variables, CTO cases had significantly greater health service use prior to placement ↓admissions at 0- to 12-month follow-up in CTO cases ↓ bed-days at 6- to 12-month follow-up in CTO cases
Levya		2018	UBA	37/38 FEP cases 29% male	22.6	↓ ER visits and bed-days/month in acute units up to 24-month follow-up No difference in bed-days/month in short stay units up to 24 months ↑ case manager contacts and cognitive therapy sessions but also weight Improvements in SAPS, SANS, GAF and SOFAS up to 24-month follow-up
Nakhosta		2018 2019	Cross-sectional	69 CTO cases 69 matched controls 62% male	38.4 39.4	↑ perceived coercion in CTO cases on unadjusted but not adjusted analyses Coercion in whole sample had positive correlation with past experience and negative with sense of procedural justice on adjusted analyses Both groups thought that a CTO could improve mental health CTO cases thought a CTO was better than being in hospital and that they should be able to contest their CTO CTO cases significantly more optimistic about the potential positive impact of CTOs on other service users’ QoL and mental health
Franka		2020	UBA	53/77 66% male	46.0	↓ admissions/year ↓ bed-days in the admission preceding and after a CTO (subsample of n = 26) ↑ mental health service contacts per year and LAI adherence The latter, in turn ∼ admissions ↓

Note. UBA = uncontrolled before-and-after study; CBA = controlled before-and-after study; MHS = mental health services; Qol = quality of life; SAPS = Scale for the Assessment of Positive Symptoms; SANS = Scale for the Assessment of Negative Symptoms; GAF = Global Assessment of Functioning Scale; SOFAS = Social and Occupational Functioning Assessment Scale; FEP = first episode psychosis; LAI = long-acting antipsychotic injections.

^aStudies from updated search/data.

As in our previous SR, the overwhelming majority of studies were small, uncontrolled before-and-after designs. Only two studies included controls and, of these, only one adjusted for potential confounders using either matching or multivariate analyses. It was not possible to meta-analyze the data, or derive common effect sizes, given the wide range of methodologies and outcomes, as well as the way in which they were reported. Attrition was also a problem in several studies, with low numbers for some of the reported outcomes.^8,11,12 For instance, data were missing from between 22% and 66% of the sample in one study depending on the comparison, leading to very small cell sizes. ¹¹ In another study, 28 out of 90 participants (31%) could not be included because of missing data at various time points. ¹² In a third, only 53 out of 77 (69%) individuals were successfully followed up, and data on bed-days were only available in a subsample of 26 individuals. ⁸

Effect of CTOs on Inpatient and Emergency Department Use

Inpatient use was reported using either admissions or bed-days, although there was considerable variation in how these were presented. This included the length of follow-up (Table 2). In all of the four included studies, admissions, bed-days, and the time to readmission were reduced following CTO placement compared to before (Table 2). In the one study with controls, the reductions were greater for CTO cases than for the controls in unadjusted analyses. ¹³ Although the two groups were not significantly different in terms of socio-demographic or clinical variables, CTO cases had had significantly greater health service use prior to placement (Table 2).

One of the studies divided inpatient units into acute and short-stay facilities with there only being a difference following a CTO for admissions to acute units. ⁵ This study also reported on emergency department visits, reporting a significant reduction following CTO placement (Table 2).

Effect of CTOs on Outpatient/Community Outcomes

This was reported in four studies (Table 2). Outcomes included outpatient visits, contacts with mental health services or a case manager, attendance at behavioural therapy sessions and adherence with long-acting injectable antipsychotics (LAIs). In every case, CTO placement was associated with significantly greater involvement in before-and-after analyses. In the study of LAI use, greater adherence was, in turn, associated with reduced admissions. ⁸ None of the above studies compared outcomes with controls.

Secondary Outcomes

One study reported statistically significant improvements in a range of psychiatric symptoms and psychosocial outcomes with up to 24 months of follow-up (Table 2). ⁵ Two papers from a single study compared perceived coercion and attitudes to CTOs in people on orders with those of voluntary controls.^6,7 In the first, people on CTOs reported significantly greater levels of perceived coercion in unadjusted analyses. ⁷ However, on adjusted analyses including socio-demographic factors, diagnoses and insight, individuals’ perception of coercion was directly linked to their past probation experiences and inversely related to their sense of procedural justice in their treatment regardless of CTO status. ⁷ In the second paper both CTO cases and the controls thought that a CTO could improve mental health. CTO cases thought being on an order was better than admission but also that they should be able to contest their CTO. ⁶ In general, they were also significantly more optimistic about the potential positive impact of CTOs on other service users’ quality of life and mental health (Table 2).

Certainty of Evidence

As all of the included studies were observational, the relevant outcomes were initially rated as being of low certainty. This was downgraded to being of very low certainty because of study quality and low numbers in some papers.

Discussion

This is an update of our first SR of outcomes following CTOs in Canada. ² Our search yielded only four additional papers from three studies with a grand total of nine papers from seven quantitative studies. As in our earlier review, CTOs reduced inpatient service use while improving outpatient attendance, participation in the community, and housing. In addition, there were benefits in terms of psychiatric symptoms and treatment adherence. In one study, perceived coercion was no greater in the CTO cases than the controls and being on an order preferable to being in hospital. However, as before, all the included studies were small and limited to a single population. Only two studies included controls and, of these, only one adjusted for potential confounders using either matching or multivariate analyses. As a result, the certainty of evidence was rated as very low. Outcomes were also largely restricted to health service use such as hospital readmissions and length of stay with less information on other outcomes such as quality of life or satisfaction.

There are several limitations. All the included studies were observational, and many used administrative health data. These may be subject to recording bias and lack information on social disability. We were unable to investigate any differences by psychiatric diagnoses. Most of the studies used before-and-after designs that are possibly subject to regression to the mean and an over-estimation of treatment effect Where present, controls may have differed from CTO cases in ways for which it was not possible to match or adjust, such as insight. We have also only demonstrated significant associations not causality. Finally, we were unable to combine results in meta-analyses.

Although there are conflicting views on the appropriateness of randomized controlled trials in this area, there is near universal agreement of the need for large controlled observational studies with adjusted analysis. It is therefore surprising such studies have not been carried out in Canada given the availability of good-quality province-level data.