Abstract
Australian mental health care providers have sought uniform collection of health-care data; largely stimulated by the National Mental Health Policy [1], which recommended regular outcome review and national measurement standards. This initiative mirrors the exercise in the UK that led to development of the Health of the Nation Outcome Scales (HoNOS) [2]. Advantages of routine outcome assessment include the determination of illness severity, measurement of treatment effectiveness, comparisons between services and assistance in planning the service provision. Such measures may also predict length of stay and treatment responsiveness. To assist instrument selection a report to the Australian National Mental Health Information Strategy Committee reviewed potential instruments [3]. The committee sought brief and easily administered, psychometrically acceptable instruments that measured change. Three clinician-rated instruments, including the HoNOS, and three self-rated instruments, including the Medical Outcomes Short Form (SF-36) [4], met the criteria. The authors suggested that all six instruments be trialed in Australia.
Private mental health care providers made recommendations that were more specific. The Strategic Planning Group for Private Psychiatric Services (SPGPPS) [5] selected the 12-item clinician-rated HoNOS because it was comprehensive, clinically relevant and quick to administer [1, 6]. They selected the SF-36 [7, 8] (or the 14 mental health items alone [5]) as a patient self-report measure that was not disease- or treatment-specific; measured both the distress and disability associated with mental disorders; and was easy to administer, freely available and widely used in Australia. Given these recommendations, it is important to evaluate how these tests fare in sensitivity to change and in their predictive abilities.
At least three large Australian field trials have used the HoNOS in psychiatric settings, but only Boot, Hall and Andrews[9] concurrently employed the SF-36. They reported outcome assessments of 18 acute-care public, private and general psychiatric hospitals. Patients were assigned to a diagnosis-related group, administered the HoNOS and asked to complete the SF-36. However, low completion rates of the SF-36 necessitated its exclusion from analyses. Improved outcomes were noted in all institutions, although variation in the level of change was noted between settings. Diagnosis-related group, hospital group and older age predicted length of stay and scores on five HoNOS subscales moderately increased this prediction. Boot et al. therefore recommended routine use of the HoNOS.
More recently, Trauer et al. [10] examined the usefulness of the HoNOS in five public psychiatric services, encompassing hospital and community care. Most items had reasonable test–retest reliability and the usefulness of the HoNOS was supported by a strong positive correlation with prior and current service utilisation. Finally, there was no significant change in consecutive HoNOS scores for patients tested twice within the 3-month study period, although admissions and discharges to hospital were associated with increased and decreased scores respectively. The authors suggested this lack of sensitivity to detect change in patients within the community might be related to the short interval between assessments, a lack of change in patients or the failure of the instrument itself.
Thus, the endorsement (albeit with reservations) of the HoNOS by Boot et al. [9] contrasts with the study by Trauer et al. [10]. However, Trauer et al. note that their sample did not include private psychiatric patients. Therefore, a study by Goldney, Fisher and Walmsley [11] is of interest because it reported admission and discharge data on consecutive patients, over a 3-month period, from six Australian private psychiatric hospitals. Total HoNOS scores (and most subscales) had reduced by discharge, but there was no association between the total score and length of stay. However, there were some relationships at the level of the individual subscales. There was a weak negative association between length of stay and overactive/aggressive behaviour, problem drinking, and hallucinations and delusions subscales. There was also a weak positive association between length of stay and depressed mood and problems with activities of daily living. The authors contended that the lack of association with length of stay obviated a role for HoNOS in guiding resource utilisation.
Similar trials in psychiatric settings have not been carried out using the SF-36, although the instrument discriminates well between depressed and non-depressed groups [12]. It has been used extensively in Australia as a quality of life indicator across various settings including medical and surgical inpatients [13, 14], the well aged [15] and substance abusers [16].
Since few studies have used both the HoNOS and SF-36 [9, 17] the current study reports HoNOS and SF-36 data routinely collected in a private psychiatric facility with the aim of exploring some psychometric properties of the HoNOS and SF-36 not undertaken in the studies cited above. The aims were to assess (i) patient disability levels in comparison with other Australian studies; (ii) treatment efficacy in comparison with other Australian studies; and (iii) the ability of the HoNOS and SF-36 to predict length of stay and changes in mood from admission to discharge.
Method
Subjects
The potential sample comprised consecutive inpatients and outpatients for whom questionnaire data were available both at admission and discharge treated at the Perth Clinic over a 2-year period. Each patient was diagnosed according to DSM-IV [18] criteria by their treating psychiatrist, who provided ongoing management. Of the potential sample (n = 1763), 77% were treated as inpatients and to ensure comparability with other studies, analysis was restricted to these patients alone (n = 1360), of whom 1152 provided questionnaires upon admission and 754 upon discharge. The mean age of the sample was 42.5 years (SD = 14.2), 69.1% were female, and each subject spent an average of 8.6 (SD = 8.4) days in hospital. The final sample comprised 754 patients. The primary diagnoses, clustered using the Australian National Diagnostic Related Groups, indicated that most (66.9%) were suffering from mood disorders (mostly major depression), 12.5% had personality disorders, and 7.8% had anxiety disorders. Psychotic, eating and substance-abuse disorders each accounted for approximately 3% of diagnoses.
Description of programmes
In addition to standard psychiatric assessment and ongoing psychiatric care, all patients were involved in group therapy streams depending on their functional ability. The Acute Admission Stream involves crisis care. It uses interpersonal and cognitive therapies to provide strategies for interpersonal coping, life transitions, conflict and management of anxiety and depression. The Prescriptive Care Stream addresses re-motivation and skills training. It is based on treating moderate-to-high impairment of coping and involves psychoeducational and prescriptive activities. The Structured Care Severe Stream focuses on acute disorders and psychosis. This is based on assessment, engagement in therapy and re-motivation and uses a highly structured programme. The Interpersonal Psychotherapy Stream is offered during the acute phase for patients whose predominant problems are within the relationship and interpersonal domain. The final stream involves an intensive cognitive–behavioural therapy (CBT) closed group conducted over a period of 10 working days for a total of 60 h. The programme includes psychoeducation, cognitive therapy (with self-monitoring), behavioural assignments to challenge thoughts and beliefs, goal setting, assertion and self-esteem training, stress management, information on a healthy lifestyle, relaxation training, and a supporters’ session.
Design and measures
Data were collected from all patients upon admission and discharge. The HoNOS [2] was rated by hospital staff. At the same time, staff rated the patient's Global Assessment of Function (GAF) [18]. Staff completed them at admission and discharge. All staff had regular HoNOS training throughout the study period.
Patients completed five self-report inventories. First, patients completed the SF-36 [7], which is a 36-item self-report index of physical and psychological symptoms. The Beck Depression Inventory (BDI) [19] is a 21-item self-report scale designed to measure the level of depression among clinical and non-clinical populations and is widely used in research on depression. The Spielberger State Anxiety Questionnaire (STAI) [20] is a self-report inventory that measures state and trait anxiety, however, only state anxiety was measured for the purposes of the study. The state scale is an indicator of transitory anxiety and has been widely used in clinical and research settings. The Locus of Control Scale (LOC) [21] assesses an individual's sense of control over themselves and their lives. A low score indicates an internal sense of control. The final measure was the Rosenberg Self Esteem Scale (RSES) [22], a measure of general self-concept consisting of 10 items using a four-point Likert-type response format. A high score reflects a high level of self-esteem. (Since the SF-36 and RSES questionnaires were introduced during the course of the study, the effective sample size for these is smaller than the remaining measures.)
Results
Table 1 shows substantial patient improvement from admission to discharge, with a 1.45 standard deviation improvement in the GAF and changes of a similar magnitude in the emotional indices. Measures of more enduring constructs that were not the focus of treatment (e.g. LOC and RSES) improved to a smaller degree. Comparisons of the HoNOS and the SF-36 revealed that the HoNOS was more sensitive to treatment change, with effect sizes ranging from 0.28–1.56, whereas the SF-36 ranged between 0.01 and 0.53.
Summary of treatment effects
To assess the comparability of the change observed in the present sample with those obtained from other Australian psychiatric hospitals, a total HoNOS score was calculated and compared with other published studies. As shown in Table 2, the ratings of the present sample are comparable to the ratings at admission to other Australian private psychiatric hospitals. Likewise, the degree of improvement in ratings from admission to discharge is comparable to the changes in ratings of patients of other private hospitals.
Comparison of treatment change with other Australian hospitals
The HoNOS is a clinician-coded assessment of a variety of symptom clusters that cover the diverse array of possible psychiatric presentations. Consequently, it is not surprising that the internal consistency of the scale was only moderately high (α = 0.59). The construct validity is demonstrated in part by the correlation between the HoNOS ratings of depression and the self-reported BDI scores (r = 0.26, p < 0.001). In passing it should be noted that the size of the correlation between BDI and HoNOS would be anticipated to be low because the HoNOS score is based on a Likert-type rating of a single item and therefore the restricted range would attenuate any correlation.
The HoNOS and SF-36 subscales at admission were used to predict length of hospital stay, change in BDI scores from admission to discharge and change in STAI scores. In predicting length of stay, the SF-36 subscales reflecting their physical function at admission (β = −0.16) and that they were ‘more healthy than a year ago’ (β = 0.17) predicted 7% (R = 0.25) of the variance. In predicting changes in depression, the SF-36 subscales of mental health (β = 0.16) and vitality (β = 0.14) combined with the self-harm subscale of the HoNOS (β = −0.23) predicted 15% (R = 0.39) of the variance of change in BDI scores. In predicting changes in anxiety, the mental health (β = 0.27) and the physical role function subscales of the SF-36 (β = −0.14) combined with the self-harm subscale of the HoNOS (β = −0.14) predicted 12% (R = 0.35) of the variance of change in STAI scores.
Discussion
The HoNOS data showed that the patients were comparable in initial severity to those in other Australian hospitals and that outcomes were similar to, albeit somewhat larger than, those reported previously. Thus, there is nothing to suggest limitations in generalizability of the results to other Australian psychiatric hospitals. Interpreting any similarities and differences across sites highlights one of the weaknesses of the HoNOS. That is, it is possible that the differences are due to real differences in the patients, differences among raters or a combination of these. The impossibility of disambiguating these effects highlights the wisdom of collecting both clinician-rated and patient-rated measures and as both sets of measures become available it will be easier to compare data sets.
Considering the relative utility of the two measures, the HoNOS was more sensitive to treatment change, but this may reflect the fact that the SF-36 used was the 4-week version and so some patients may have been reporting on status that overlapped with admission. In terms of predictive capabilities, both the SF-36 and the HoNOS subscales were able to predict length of stay and symptom change. Although neither of the indices explained much of the variance, they were relatively better able to predict symptom change.
In summary, the HoNOS and the SF-36 provided reliable and valid measures of aspects of patient function. The version of the SF-36 used in the present study was not sensitive to change and was a poor predictor of outcome. The HoNOS was sensitive to change, but like the SF-36 was a poor predictor of outcome. Thus, while the HoNOS and SF-36 can provide useful descriptive data for comparison purposes, from the present data neither seems particularly useful in guiding clinical decisions about the length of stay or expected outcomes.
Footnotes
Acknowledgements
The assistance and support provided by Gavin Andrews, David Castle, Stephanie Jones and Moira Munro in preparation of this manuscript is much appreciated.