Abstract
Objective:
There is debate about the effectiveness of community treatment orders in the management of people with a severe mental illness. While some case–control studies suggest community treatment orders reduce hospital readmissions, three randomised controlled trials find no effects. These randomised controlled trials measure outcomes over a longer period than the community treatment order duration and assess the combined effectiveness of community treatment orders both during and after the intervention. This study examines the effectiveness of community treatment orders in a large population-based sample, restricting observation to the period under a community treatment order.
Methods:
All persons (n = 5548) receiving a community treatment order in New South Wales, Australia, over the period 2004–2009 were identified. Controls were matched using a propensity score based on demographic, clinical and prior care variables. A baseline period equal to each case’s duration of treatment was constructed. Treatment effects were compared using zero-inflated negative binomial regression, adjusting for demographics, clinical characteristics and pre-community treatment order care.
Results:
Compared to matched controls, people on community treatment orders were less likely to be readmitted (odds ratio = 0.90, 95% confidence interval = [0.84, 0.97]) and had a significantly longer time to their first readmission (incidence rate ratio = 1.47, 95% confidence interval = [1.36, 1.58]), fewer hospital admissions (incidence rate ratio = 0.90, 95% confidence interval = [0.84, 0.96]) and more days of community care (incidence rate ratio = 1.55, 95% confidence interval = [1.51, 1.59]). Increased community care and delayed first admission were found for all durations of community treatment order care. Reduced odds of readmission were limited to people with 6 months or less of community treatment order care, and reduced number of admissions and days in hospital to people with prolonged (>24 months) community treatment order care.
Conclusion:
In this large population-based study, community treatment orders increase community care and delay rehospitalisation while they are in operation. Some negative findings in this field may reflect the use of observation periods longer than the period of active intervention.
Introduction
Community treatment orders (CTOs) or outpatient commitment (Churchill et al., 2007) provide a legal mechanism ‘designed to benefit persons with serious mental illness who need ongoing psychiatric care and support to prevent relapse, hospital readmission, homelessness, or incarceration but have difficulty following through with community-based treatment’ (Swartz and Swanson, 2004). They are widely used in Australia and internationally despite limited evidence regarding their effectiveness for achieving these goals (Kisely et al., 2017; Maughan et al., 2014; Swartz and Swanson, 2004). Some (Hunt et al., 2007; Segal and Burgess, 2006b, 2006c; Swartz et al., 2010c), though not all (Geller et al., 1998; Kisely et al., 2004; Segal and Burgess, 2006a; Zanni and Stavis, 2007), case–control studies have found that CTOs are effective in reducing hospital readmission; however, three randomised controlled trials (RCTs; Burns et al., 2013; Steadman et al., 2001; Swartz et al., 1999) have found no (Burns et al., 2013; Steadman et al., 2001) or only limited (Swartz et al., 1999) beneficial effect for CTOs. This has prompted debate about their effectiveness and the ethical justification for their use (Churchill et al., 2007; Hiday, 2003; Kisely et al., 2007; Maughan et al., 2014) and calls for further evidence, particularly from large, naturalistic studies (Johnson, 2013; Swartz and Swanson, 2015).
All three RCTs had observation periods longer than the duration of the CTO and different effects upon the level of community care. Steadman et al. (2001) examined the effect of CTOs in New York on rehospitalisation, quality of life, criminality and treatment adherence over 12 months. Overall, 35% of their sample had orders of 6 months or less, and overall, no differences were seen between the groups in the level of community care or on any clinical outcome. The North Carolina study (Swartz et al., 1999) examined service use over 12 months – initial CTOs were typically of 30–60 days duration, and 64% of cases had orders of less than 180 days. A significant effect was seen in those individuals given an order of greater than 180 days duration with a reduction in the number of admissions, the number of hospital days and a higher number outpatient service events while under a CTO. The Oxford Community Treatment Order Evaluation Trial (OCTET) study in England (Burns et al., 2013) followed participants randomised to a CTO at discharge from a hospital admission over 12 months with a median CTO duration over this period of 183 days. No differences were seen between the groups in clinical outcomes or level of community contact. In OCTET’s 3-year follow-up (Burns et al., 2015), the CTO group had an average of 488 days on a CTO (within the 1095 days follow-up period) as against 393 days on a CTO in the ‘non-CTO’ randomised arm. Over this longer period, there was a non-significant trend for the CTO arm to have fewer admissions. In a later analysis of their data over the full 3 years, being on a CTO was associated with more frequent contact and better continuity of care but an increased rate of admission suggesting that more unwell patients were being targeted with increased care (Puntis et al., 2017).
Therefore, it is possible that these RCTs conflate two distinct questions: (1) Are CTOs effective while they are in place? and (2) Do CTOs have a sustained effect on service use after their conclusion? In seeking to assess whether CTOs have a sustained effect, it is first necessary to determine whether they are effective while in place. Another important factor in this debate is the difficulty in comparing the use of CTOs across different legal frameworks (Dawson, 2006) and mental health systems (Churchill et al., 2007). Given the continued debate surrounding this question and possible changes to mental health legislation in many states in Australia, further work in the Australian context was seen as useful.
This study specifically focuses on changes in care while a person is on a CTO. We use a case-control methodology to provide a large, population-based observational study in a jurisdiction (New South Wales [NSW], Australia) where CTO use is widespread and well established. We hypothesise that compared to matched controls, people on CTOs will have greater community care and reduced hospital admissions while under a CTO.
Method
Setting
The study included all state-operated (‘public’) specialised mental health services in NSW, Australia: population 7.5 million. These services provide all involuntary inpatient and community care for individuals with psychoses and other serious mental illnesses. A CTO application can be initiated by a medical officer during an involuntary mental health admission and then approved by a mental health inquiry overseen by the NSW Mental Health Review Tribunal (MHRT). CTOs mandate attendance at a community mental health centre for treatment and medication. Non-adherence can trigger a procedure allowing forcible return to hospital for assessment, medication or possible readmission. CTOs can be granted for a maximum of 12 months (increased from 6 months in 2007), and the mental health service can apply for renewal of a CTO prior to its expiry.
Sample
We examined people subject to a CTO made by the MHRT between 1 January 2004 and 30 December 2009. Subjects were identified from MHRT’s routine administrative data collection. Linkage between MHRT and NSW Health data was undertaken using person and NSW Health service identifiers recorded within the MHRT administrative dataset. Health service utilisation was obtained using a unique statewide person identifier. Specialist mental health inpatient and community care occurring between 1 July 2001 and 30 July 2012 were recorded. Cases were excluded from matching unless a valid CTO duration or diagnosis was recorded. Cases whose index CTO was made during admission to a forensic psychiatric hospital were excluded (n = 22). The study was approved by the NSW Population Health and Service Research Ethics Committee. The study is reported according to Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines (Von Elm et al., 2007).
Controls
All admissions of people aged over 18 years to a mental health unit between 1 January 2004 and 30 December 2009 were identified as potential controls. A fixed baseline period of 30 months was used for matching cases and controls. For each admission, data were extracted on age, sex, service location, primary and comorbid diagnoses, and the number of hospital admissions and contacts with community mental health services.
A propensity score was constructed for case–control matching (Austin, 2011; Austin et al., 2007). Binary logistic regression was used to calculate the likelihood that an admission to a NSW mental health unit would result in initiation of a CTO. Variables included age, sex, country of birth, indigenous status, type of psychosis diagnosis (grouped as schizophrenia, affective psychosis, other psychoses or no psychosis), presence of a comorbid substance use disorder diagnosis, presence of a comorbid personality disorder diagnosis, number of hospital admissions and the number of community treatment days in the prior 30 months. The performance of the propensity score as a predictor of persons receiving the intervention was tested by receiver operating characteristic (ROC) curve analysis.
Matching of controls
We selected two controls for each case, matched on propensity score for receipt of a CTO. Propensity scores were matched to a maximum range (calliper width) of 0.2 standard deviations (SDs) of the logit of the propensity score (Austin, 2011). Each control individual was matched with only one case. Cases where fewer than two controls could be identified were excluded.
Establishing variable observation period for each case
Because the duration of CTO care varies between individuals, the study used a mirrored design, defining an observation period for each subject according to their CTO duration and comparing it to service use for a matching period prior to their first CTO. For controls, the observation period (pre and post) was defined by that of their matching CTO subject.
The length of each CTO was defined by the date of expiry of the order, as set by the MHRT at the time of making the order; early cessation of CTO due to admission or clinical decision was not recorded. Successive or renewed CTOs for an individual were joined to create a single episode of continuous care on a CTO. Therefore, while hospital admission may have terminated an individual CTO, where a further CTO was obtained, it was possible to calculate the number and length of hospital episodes that occurred during ongoing CTO care. The observation period for each subject began with the date of initiation of the person’s first (index) CTO and ended with the expiry date of the person’s last CTO. If there was a gap of more than 30 days from expiry of a CTO to initiation of a new CTO, this was treated as marking a new episode of CTO care. We only included the first episode of continuous CTO care per person.
Clinical and service use variables
The primary outcome measures were as follows: (1) the number of community treatment days, defined as a day on which at least one mental health service contact occurred, (2) the likelihood of any mental health readmission, (3) the number of admissions of more than 1-day duration to a mental health unit, (4) days to first readmission and (5) total mental health inpatient days.
We obtained demographic details (age, gender, country of birth), primary diagnoses and the presence of primary or comorbid substance or personality disorders from the information entered into NSW Health data after the admission ending in the index CTO. Primary diagnoses were grouped into categories of schizophrenia & schizoaffective disorder (F20, F24), affective psychoses (F30, F31 and specific codes indicating psychosis within F32, F33), other psychoses (other codes in F20–F29, plus specific codes for substance-induced or organic psychosis within F00–F19) or no psychosis (all other codes). We calculated pre- and post-CTO care (hospital admissions, days in hospital, community treatment days) from the date of discharge from the index admission.
Statistical analysis
We used Stata SE v13 for the statistical analyses. CTO and control groups were considered suitably matched if the Standardised Mean Difference (SMD) for each variable was less than 0.1 (Austin, 2011). For continuous variables, incidence rate ratios were compared using zero-inflated negative binomial regression, first using univariate analysis then adjusting for the pre-CTO baseline rate of the variable being examined. Binary outcome variables were assessed using binary logistic regression. The effect of duration of CTO care on findings was examined by subgroup analysis, splitting the group by the total duration of the first continuous episode of CTO care.
Results
The MHRT made 17,506 CTOs on 6567 individuals during the study period. Of these, 630 (9.6%) could not be linked to NSW health service data using the identifiers recorded. A further 166 cases (2.5%) were excluded prior to matching because they had no recorded mental health service contacts in the study period (n = 107), no recorded diagnosis (n = 25) or no CTO duration (n = 12). The regression-derived propensity score performed well in predicting the presence of a CTO (area under the curve = 0.86). Of 5771 cases available for matching, two propensity score matched controls could be identified for 5548 (96%). Of the 223 cases lost, no control cases could be identified for 207 and only one control for 16 cases.
People receiving CTOs at discharge from hospital were typically males, with a mean age of 38.5 years, a primary diagnosis of schizophrenia or schizoaffective disorder, and high rates of comorbid substance misuse or personality disorder (see Table 1). The distribution of duration for continuous CTO care was skewed; the mean duration of continuous CTO care was 369 days (SD = 313 days), but most individuals (54.2%) had a continuous duration of CTO care of 6 months or less. A small group (10.1%) continued on a CTO for greater than 24 months. For the 30-month pre-CTO period used in matching, case and control groups were acceptably matched (SMD < 0.1) on most variables, but CTO cases had more days in hospital.
Characteristics of study group (N = 5548) and matched controls (N = 11,096).
CTO: community treatment order; SMD: standardised mean difference; SD: standard deviation.
Duration of pre-CTO baseline varies: mirrors the duration of CTO for each case and their matched controls.
Compared to matched controls, people on CTOs had a reduced likelihood of readmission, a longer time to readmission, fewer total hospital admissions and an increased rate of community care (Table 2). These effects persisted after adjusting for between-group differences in pre-CTO baseline. CTO cases and controls did not differ significantly in total days in hospital during CTO care. The number needed to treat for an additional beneficial outcome (NNTB) of a reduction of one less hospital admission was 10 (95% CI = [6, 20]), and to result in one additional person moving into the ‘no readmissions’ group was 48 (95% CI = [28, 167]).
Inpatient and community mental health care during treatment on a community treatment order (CTO), compared to controls matched on pre-CTO person, illness and care characteristics.
OR: odds ratio; IRR: incidence rate ratio; CI: confidence interval; SD: standard deviation.
For hospital readmission, treatment effect is OR, calculated using binary logistic regression. For other variables, treatment effect is IRR using zero-inflated negative binomial regression.
p < 0.005; and ***p < 0.0005.
We then examined if the duration of the CTO had an effect upon outcomes. Across all durations of CTO care (Table 3), there was a significantly longer time to first readmission and increased community care days. A shorter (≤6 months) duration of CTO care was additionally associated with a significantly reduced likelihood of hospital readmission, while a longer duration (>24 months) of CTO care was additionally associated with a significant reduction in the number of hospital admissions and the number of hospital days.
Service use and treatment effects by duration of CTO.
CTO: community treatment order; OR: odds ratio; IRR: incidence rate ratio; SD: standard deviation; CI: confidence interval.
p < 0.05 and ***p < 0.0005.
Discussion
This study was motivated by the continuing controversy surrounding the use of CTOs (Johnson, 2013; Swartz and Swanson, 2015). This population-based, naturalistic case–control study focused only on the period during which a CTO was in place, and followed individuals for a variable period of up to 2.5 years. The large scale of the study, covering services for a population of 7.5 million people, allowed for close matching on a wide range of variables.
We found that compared to their pre-CTO baseline and to matched controls, people placed on a CTO had a reduced likelihood of readmission, a longer time to readmission, fewer total hospital admissions and an increased frequency of community care. These effects were similar in direction and degree for shorter and longer CTOs. This supports the hypothesis that, while present, CTOs are effective in increasing the level of community care and delaying or reducing hospital readmission. Despite the reduction in readmission, there was minimal effect upon the number of days in hospital, with a CTO needing to be in force for more than 24 months to achieve a significant reduction in bed days. This is disappointing as it undermines one of the hopes that CTOs would result in shorter hospital stays.
It is possible that the negative findings of RCTs of CTOs partly reflect type II error arising from a shared factor in all three research designs; all examine outcomes over a period significantly longer than the duration of the intervention. Therefore, they test the combined effectiveness of CTOs both during and after the period of intervention. For an intervention to be justified, it should first demonstrate effectiveness while being applied. However, many effective interventions in mental health care do not have a sustained or illness-modifying effect which persists after their cessation. For example, antipsychotic medications are effective in early psychosis, but 12-month relapse rates on discontinuation may exceed 70% (Zipursky et al., 2014). An RCT which used 6 months of antipsychotic medication but a 12-month follow-up period may well find the medication to be ineffective.
Our findings are consistent with several previous studies which assessed individuals only when they were on a CTO. In a continuation of the New York assisted outpatient treatment programme, Swartz et al. (2010) found a reduction in admissions, improved receipt of psychotropic medication and improved engagement with services while participants (n = 3609) were on a CTO. A large Australian register-based study (Segal and Burgess) found that extended CTOs were associated with increases in community treatment but that these were not sustained after their termination.
We note that a duration of CTO of greater than 6 months has also been associated with improved outcomes in other studies (Hunt et al., 2007; Swartz et al., 1999, 2010; Van Dorn et al., 2006). This may reflect clinical factors; for some individuals, an extended period may be required to stabilise illness and build relationships with a clinical team. However, it is also possible that it represents methodological factors, since in those studies subjects were under a CTO for a greater proportion of the observation period.
The primary aim of CTOs, as implemented in the NSW Mental Health Act, is to increase the amount of community care and enforce adherence to medication. We found a significant increase in the delivery of community care for those on a CTO. Increased community care has been associated with reduced need for admission in most (Kisely et al., 2006, 2013; Segal and Burgess, 2006b; Swartz et al., 1999, 2010) but not all (Puntis et al., 2017) studies. On the other hand, if a CTO does not produce increased community care, it is unlikely to result in a reduced risk of hospitalisation. This increase in community care was not seen in two of the RCTs (Burns et al., 2013; Steadman et al., 2001) that failed to find an effect for CTOs. A later analysis of the 3-year outcomes for the OCTET study found that being on a CTO improved continuity of care with ‘fewer days between community contacts … and fewer 60 day breaks in contact’ (Puntis et al., 2017). Increased contact and CTOs appeared to be a marker of the severity of illness as less frequent contact was associated with a reduced rate of admission and fewer days in hospital (Puntis et al., 2016).
Being placed on a CTO did not radically change outcome as measured by readmission, although the benefit in our study of being placed upon a CTO appeared to be considerably more than that estimated in a meta-analysis of the three RCTs which calculated an NNTB of 142 (Kisely et al., 2017). The number of CTOs needed to prevent one readmission in this study was estimated at 10 (95% CI = [6, 20]). This benefit must be weighed against concerns that CTOs alienate people and make continued care less likely after their end. There is a growing understanding among mental health professionals, consumers and carers that CTOs place a responsibility upon community mental health services to provide care and that this must more than balance the coercion, loss of dignity and over reliance on medication often associated with CTOs (Corring et al., 2017; Light et al., 2017) independent of evidence of their effectiveness. Without adequate care, the considerable curtailment of the rights of individuals placed upon them by a CTO cannot be justified.
Our study has a number of limitations. The linkage used service identifiers recorded in administrative datasets, and overall the study was unable to link 15% of individuals given a CTO. Matching of cases and controls was imperfect on one important item, the number of days in hospital prior to the CTO, because that variable did not strongly predict the likelihood of a CTO and therefore had little influence on the propensity score used in matching. This may have biased the study against finding a treatment effect for the CTO group as a greater duration of inpatient stay has been suggested to predict poor outcome (Harrison et al., 1996). During the study period, some CTOs could be initiated through a separate legal process that did not form part of the MHRT database used in the study. Those CTOs were not captured in our data until and unless the person was presented to the MHRT for renewal of the CTO. Therefore, some controls may have been subject to a CTO, and in some subjects, we may not have the true beginning of their CTO or their pre-CTO baseline may include some period on a CTO. The main effect of these issues would be to obscure differences between cases and controls, and reduce the likelihood of demonstrating a treatment effect. In using routinely collected administrative data, we are unable to measure several potentially important clinical or care variables which may have differed between cases and controls, including levels of risk, adherence to treatment, insight or clinical relationships.
Legislation surrounding CTOs can vary greatly from jurisdiction to jurisdiction and the conclusion made from this study may not be generalisable to other legal systems.
It is likely that people with longer CTOs differ systematically from people with shorter CTOs. However, we are unable to determine the individual clinical reasoning behind decisions to seek extension of a CTO. For some individuals, continued readmission is likely to trigger increased attempts to enforce treatment and lead to a longer period on a CTO. However, a different clinician might conclude that CTOs were not effective for this individual and thus they should be stopped. Conversely, a reduction in admissions might be seen as a sign of success of the use of a CTO in the management plan and hence motivate the clinician to request a continuation of the order.
This large population-based case-controlled study has found that CTOs increase the delivery of community care, delay the time to readmission and decrease the number of admissions while they are in force. This must be seen in the broader context of mixed findings from other jurisdictions regarding their effectiveness. Future work should explore the determinants of this finding, the duration of the CTO, the quality and frequency of care provided in the community, to help maximise the benefit accruing to the individual placed upon it. These findings reflect the legal framework for CTOs in NSW, Australia, and, most probably, the availability of community care; however, we think in light of these findings calls for the abandonment of CTOs (Burns et al., 2015) may be premature.
Footnotes
Acknowledgements
This project used de-identified and linked individual data from NSW Health and NSW Mental Health Review Tribunal data collections. Linkage of these collections, including a waiver of participant consent, was approved by the NSW Population and Health Services Research Ethics Committee. The research dataset is not available to other researchers.
Declaration of Conflicting Interests
A.H. has received consultancy fees from Janssen Australia and Lundbeck Australia. He has been on an advisory board for Sumitomo Dainippon Pharma. He has received payments for educational sessions run for Janssen Australia and the Lundbeck Institute. Other authors report no disclosures.
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.
