Abstract
Keywords
Early intervention usually refers both to the timeliness of services (early detection and treatment), and to the quality of services (provision of optimal treatment after the initiation of care) [1–5]. While the precise characteristics and efficacy of optimal treatment are yet to be established [6], there are now guidelines for services wishing to adopt ‘best practice’ in their treatment of young people with early psychosis. To what extent have mental health services been successful in adopting guideline concordant care?
Services that have implemented optimal treatment practices for early psychosis have reported improved client outcomes. For example, improvements in negative symptoms and quality of life were observed by the Early Psychosis Prevention and Intervention Centre (EPPIC) for young people who received a package combining intensive case management, psychosocial interventions, family support and education, and low-dose medication strategies [5], [7], [8]. Other comprehensive intervention programs have also reported reductions in illness severity associated with biopsychosocial treatments combined with early detection [9] and intensive inpatient and community intervention [10], [11]. A review of empirical evidence concluded that individuals experiencing a first episode of psychosis respond to lower doses of antipsychosis medication, that relapse is reduced by the continuation of medication for at least 12 months, that a low dose atypical produces fewer extrapyramidal sideeffects than conventional medications and that benzodiazepines may be a useful adjunctive therapy [12]. There is, as yet, insufficient research into the efficacy of specific psychosocial treatment components. However, some favourable outcomes have been reported for group programs [13] family interventions [14] and psychological interventions [15], [16].
The Australian clinical guidelines for early psychosis (ACG-EP) were produced in 1998 by the National Early Psychosis Project (NEPP). These provided six general strategies to guide early psychosis interventions: (i) engaging with and developing a supportive therapeutic alliance with the patient; (ii) using a balanced biopsychosocial approach; (iii) using a low dose antipsychosis medication regimen; (iv) developing an awareness of the phases of illness; (v) ensuring continuity of care; and (vi) involving family, carers and friends in the therapeutic process [17].
The critical challenge for mental health services is to change the practices of clinicians, making evidencebased best practices available to all people for whom they are suited [18], [19]. The aim of this study was to assess how effective an Area Mental Health Service was in introducing optimal treatment strategies for young clients presenting with first-episode psychosis. It was hypothesized that interventions provided after service development strategies were implemented would be more consistent with the ACG-EP than those provided previously. This study is one of three components of a larger project that also evaluates early psychosis intervention outcomes [20] and changes in staff attitudes [21].
Method
Setting
Northern Sydney Health Area Mental Health Services (NSHMHS) (population 750 000) is comprised of four sub-area mental health services offering integrated 24-hour crisis intervention and assertive community care, acute inpatient care, community care for adolescents, residential and rehabilitation services.
Sample
The sample included all individuals aged 15–26 years, residing in the Northern Sydney catchment area who presented to any team for first treatment of a psychotic illness during two 6-month periods. The first period, February to July 1997, was prior to the implementation of any service development strategies and the second period, April to September 1999, was after implementation.
The computer databases of all treatment teams were used to identify eligible individuals. Individuals who were known to have received prior treatment for psychosis were excluded. Fifty-eight individuals during the first period and 74 during the second were found to meet criteria for inclusion in the audit. Eleven files from the first sampling period and four files from the second were not available for audit, leaving 47 and 70 clients in the sample, respectively. The expected incidence is 60 in a 6-month period, based upon an estimated 9/10000 population affected per annum in the age range 15–24 years [22]. The greater incidence in the second period may indicate that after training, our early psychosis teams were identifying individuals who had previously been inaccessible or not accepted into treatment. Most young people (66%) received treatment from two or more clinical teams. In the first period, no clients received treatment from a specialized early psychosis team, while during the second period, 51% of clients (n = 36) received at least some of their treatment from a specialized early psychosis team.
Procedure
Service development strategies
Strategies to encourage widespread dissemination and implementation of evidence-based guidelines began in 1997 with the formation of the Early Psychosis Prevention and Intervention Network for Young People (EPPINY) and a series of staff training workshops. These were attended by interested clinicians from all four sub-areas and all service types. Topics included optimal prescribing practices, adolescent development, family interventions, group work, cognitive behaviour therapy and the ACG-EP. Staff surveys conducted in 1998 and 1999 provided a means for staff attitudes and concerns to be monitored and addressed throughout the change process [21].
During 1998–1999, three of the four sub-areas restructured their services to create specialized community early psychosis teams, each servicing its own geographical area and providing assertive community case management for young people (18–26 years) from first presentation for up to 18 months. The fourth sub-area chose to have a dedicated early psychosis co-ordinator working within the general mental health team. The early psychosis services offer optimal medication strategies, social, vocational and psychoeducational group programs, psychosocial interventions and family support and information evenings. They aim to intervene as early as possible from the time of initial contact and to provide flexible and appropriate intervention in a youth-friendly environment.
Evaluation
Researchers were employed to audit files independently and retrospectively so as to minimize any impact of this process on clinical services and to maximize consistency and objectivity. All inpatient and community mental health records were audited for the first 12 months of treatment for each individual. Ten files (9% of n = 117) were independently audited by a second trained rater for the purpose of determining inter-rater reliability.
Measures
The audit instrument is a 49-item checklist of demographic information, service use indices and clinical indicator criteria. The research team, in consultation with clinicians and management, developed treatment recommendations for early intervention based upon the ACG-EP, research literature and our emerging local clinical practice guidelines [23], [24]. Key aspects of intervention that have not yet been addressed by our service, such as early detection, were not included. Clinical indicators were derived for each treatment recommendation and then culled to exclude those that were unlikely to be adequately documented or readily located in medical records. Treatment recommendations and indicators are presented in Table 1. Piloting of the instrument was conducted independently by two auditors to assess feasibility, completion time and reliability. Indicators were modified, definitions were improved and the tool restructured with an aim to maximize validity and reliability. The final instrument assessed 27 indicators of service provision.
Clinical indicators and treatment recommendations with Australian Clinical Guidelines for Early Psychosis (ACG-EP) strategies from which they were derived
Data analysis
Changes in service provision were analyzed by comparing the proportion of medical records meeting criteria for each indicator at Time 1 and Time 2. Comparisons were also made at Time 2 between clients who received some treatment from an early psychosis team and those seen exclusively by other services. Chi-square analyses were used throughout.
Results
Sample characteristics
The mean age of subjects on entry to the service was 20.2 years (SD = 2.88, range = 15–25). Sixty-four percent were male. The diagnoses documented at the time of prescribing the first antipsychosis medication were: first-episode psychosis (44%); schizophrenia (16%); drug-induced psychosis (13%); bipolar disorder (8%); psychotic depression (5%); and brief reactive psychosis (2%). The audit did not record a diagnosis for 14% of clients.
Reliability assessment
Inter-rater reliability of the audit indicators ranged from Kappa [25]: κ = 0.2 (unacceptable) to 1.0 (perfect agreement). Three indicators with a κ-score less than 0.6 (medication dose, clinician contact with the GP, documentation of the date of illness) were considered unreliable and were excluded from the analysis. Thus, only indicators with at least ‘good’ reliability are reported [26].
Comparison of service provision across the two sampling periods
Demographic variables did not differ significantly between the two samples. Where there was no change over time, percentages for the whole sample are presented. The proportion of subjects meeting recommended treatment criteria at each time-point and significance levels are displayed in Table 2.
Proportion of clients receiving indicator concordant treatment
Treatment recommendations
(1) Provide accessible services within a least restrictive, non-traumatic setting
Sixty-three percent of all subjects were initially assessed in a community setting, 90% of these within 48 hours as recommended by the ACG-EP. Thirty-three percent received no inpatient treatment.
The audit recorded the incidence of ‘traumatic’ events, such as selfharm, occurring during the first hospital admission, as well as events that have the potential to be traumatic, such as involuntary admission. Eighty-five percent of individuals who were admitted experienced one of these events. The average number of events was almost three per person with the maximum being nine. The most commonly experienced ‘traumas’ were involuntary admission (70% of those who were admitted), intramuscular/IV medication (38%); police involvement with admission (27%); and severe medication side-effects (27%). Less common ‘traumatic’ events included seclusion (11%), self-harm (5%) and harm to others (4%).
(2) Comprehensive biopsychosocial approach to assessment and treatment
The proportion of subjects receiving psychoeducation increased significantly from 36% at Time 1 to 56% at Time 2. There was a significant increase in the number of young people invited to attend a group program, from 17% to 37% and in those who attended from 9% to 27%. Evidence of any documented relapse prevention planning was suboptimal despite a significant increase from 11% at Time 1 to 26% at Time 2. Physical assessments were conducted for most subjects (62%) but rarely included an assessment of weight (15%). In both cases, most occurred in an inpatient setting. Assessment of substance use was routinely documented.
(3) Provision of antipsychosis medication to minimize side-effects
The proportion of subjects receiving an atypical antipsychosis medication within the first 3 weeks of treatment increased significantly from 62% to 87%. There was also a significant increase (17% to 46%) in the use of benzodiazepines as the only medication used ‘as required’. Prescribing of typical antipsychosis medications ‘as required’ reduced from 40% to 31%. Comparison of inpatient and community data indicated that this change in practice occurred in the community setting while inpatient prescribing of medication ‘as required’ remained unchanged. The increase in ‘as required’ benzodiazepines in the community setting occurred in the context of an overall reduction in ‘as required’ medication from 68% to 50%. Depot medication was rarely prescribed.
Documentation by a doctor of a thorough assessment for abnormal movements using a method such as AIMS or DISCUS was found in less than 5% of cases.
(4) Provision of assessment and treatment that is appropriate to illness phase
Our indicators examined two aspects of phase-specific assessment: assessment of illness onset and assessment of recovery and treatment resistance. Some documentation regarding onset of positive symptoms (dates or descriptions of early signs) was found in almost every case (97%). For almost 50% of individuals, auditors were able to locate at least one page, 3–6 months into treatment, documenting a comprehensive reappraisal of persisting and comorbid symptoms conducted by a doctor.
(5) Community case manager involved in all phases of care
For those individuals whose first contact was the inpatient unit, there was a significant increase in the proportion who received a visit from their community case manager during the first week of admission (26% to 63%). For those individuals admitted after having been first treated in the community, 58% received an inpatient visit from their case manager. There was a significant increase in the recording of the name of the young person's GP, from 40% to 66%.
(6) Provision of family support and opportunity to be involved in assessment and treatment
A meeting between at least one family member and members of the clinical team for the purposes of assessment, psychoeducation or support, in addition to the initial contact or referral occurred within five days for 50% of families and within the first month of treatment for 75%. Overall, 44% of families were documented to have received psychoeducation in a group or individual setting. The number of families invited to attend family evenings increased significantly, from 6% to 43% while the proportion who attended also increased significantly from 2% to 13%.
Comparison of service provision between early psychosis and non-early psychosis teams at Time 2
Clients and families who attended early psychosis teams were significantly more likely to receive psychoeducation than clients who (only) attended other services (Client psychoeducation, p = 0.006; Client attended group, p = 0.001; Family psychoeducation, p = 0.005; Family attended group, p = 0.002). Clients who attended early psychosis teams were significantly less likely to receive benzodiazepines as their only ‘as required’ medication (p = 0.021) and were significantly more likely to have an admission to an inpatient unit (p = 0.017). On all other indicators there were no significant differences between the service provided by early psychosis teams and non-early psychosis teams.
Discussion
This study measured the extent to which a ‘real world’ mental health service has been successful in implementing optimal treatment for early psychosis as defined by the ACG-EP. Results indicate significant improvements in service provision over the 18-month period for 10 of the 24 indicators. No indicator showed significant adverse change.
Indicators showing significant improvement represented four of the six treatment recommendations, including family involvement, case manager involvement and change in medication regimen and the psychological aspects of a biopsychosocial approach. The proportion of young people receiving psychosocial interventions (psychoeducation, groups, relapse prevention strategies) following a staff education program and service restructuring increased significantly. Visits by case managers to inpatients, attendance of families at group programs and the identification of clients' GPs also increased. Significant changes in medication strategy included a preference for atypical antipsychosis agents and the use of benzodiazepines as adjunctive treatment.
Several indicators did not improve significantly over the time period audited but were clearly routine practice. Access to services was timely (within 48 hours) and occurred mostly in a community setting. Only a small proportion of young people received depot medication. Family meetings and the assessment of substance use occurred routinely while our admission figures (66%) were similar to those published for a recent EPPIC sample [27].
Several aspects of service provision clearly require further attention. Physical assessments including the assessment of weight are not yet routine practice, especially in the community setting. A formal assessment of involuntary movements was not frequently documented. This area of examination should be addressed as good practice, even though involuntary movements are less problematic for this group of first-episode patients taking atypical antipsychosis medications. Re-assessment of symptoms at 3–6 months and family psychoeducation are also identified for further improvement. Despite significant change, rates of adherence were considered sub-optimal for individual psychoeducation, relapse prevention and participation in group programs. In the inpatient setting, experiences which may cause distress or psychological ‘trauma’ remained common. Such experiences have been found to be associated with posttraumatic stress disorder [28] and should be minimized. The most common potentially traumatic event at both time-periods was involuntary admission (70% of those who were admitted). Further consideration is needed with regard to the definition of the optimal rate of involuntary admission in a ‘least restrictive’ service, where only individuals who are extremely unwell are admitted to hospital care. Our rate of involuntary admission was comparable with that published for a sample of early psychosis patients treated by a generic Australian mental health service (84%) [27].
On three of the indicators demonstrating significant change (client psychoeducation, client-attended group and family groups), the effect appears to have been largely carried by early psychosis teams. For example, only three of the 34 clients who did not attend an early psychosis team participated in a group program, compared with 21 of the 47 clients who had received some treatment from an early psychosis team. Surprisingly, clients of early psychosis teams were more likely to have experienced an admission to an inpatient unit during their first year of treatment. Further examination of the data revealed that, in most cases, this admission occurred prior to any contact with the early psychosis team. Clients of an early psychosis team were also more likely to receive medication other than benzodiazepines ‘as required’; however, no typical antipsychotic was prescribed ‘as required’ by an early psychosis team.
The findings from this project along with observations made during the audit process have been used to inform ongoing service developments. It was observed, for example, that one team which had introduced a standard format for early warning-sign identification and prevention planning had utilized this in every case. In some cases it was identified that services did not have the necessary equipment (e.g. scales) to provide guideline concordant care. Strategies to promote optimal practices have been considered and have subsequently included the development of local clinical guidelines accompanied by further staff training [23], [24].
Methodological considerations
The audit method used in this study was objective, cost effective and amenable to retrospective study. Three key indicators were excluded from the present study due to poor inter-rater reliability. It was unfortunate that the audit was unable to reliably measure medication dose, case manager contacts with GPs and duration of untreated psychosis. The latter was redefined after the data were analyzed with a less precise but more reliable indicator, requiring only some evidence that the clinician had recorded either the first signs of illness or the date these occurred. Although thresholds were clearly defined for each indicator, these were not equivalent, thus performance across indicators should be compared with caution. In some cases the criteria were quite stringent, for example the assessment of involuntary movements required the documented use of a formal instrument such as DISCUS. In other cases, thresholds were less precise, for example the redefined measure of illness onset required only some description of onset such as a date or symptom.
For all indicators, a service was considered to be provided only if it was documented in the client's medical record and was also able to be reliably located by auditors. This may have led to an increase in a type II error rate, with a low rate of performance on an indicator reflecting documentation failure rather than a lack of service provision. A subsequent study, incorporating additional assessment methods, such as clinician interviews and review of other records [B. Moss, unpublished data] is attempting to address this issue and has indeed found this to be the case, with logged attendance at groups often not being documented in the medical record.
Conclusion
This comprehensive audit of medical records before and after staff training and service restructuring has provided evidence for significant improvements in some important services provided to young people with first-episode psychosis, suggesting that we have made substantial progress in disseminating evidence-based guidelines in a real-world setting. They also indicate that further improvements are needed to achieve optimal levels of service provision and documentation in relation to the ACG-EP guidelines.
Footnotes
Acknowledgements
We thank the management and clinicians of Northern Sydney Health, medical records staff, Moy Dibdin and Vivienne Miller. This research was supported by a Northern Sydney Health Area Research Grant and is part of a larger study funded by the Commonwealth Department of Health and Aged Care. The data collection and preparation of papers was partially funded by Janssen-Cilag. Louise Nash received a New South Wales Institute of Psychiatry Fellowship in 1997.
Aspects of this paper were presented at The Mental Health Services Conference (TheMHS) of Australia and New Zealand, Sydney, August 2002 and the 3rd International Conference on Early Psychosis, Copenhagen, September 2002.