Abstract
The measurement of outcome has become an important topic for mental health services recently [1] and a leading instrument in this regard is the Health of the Nation Outcome Scales (HoNOS) [2–3]. The HoNOS has been quite widely used in this country following a comprehensive review of outcome measures [4] and it was included in a range of measures in a large national casemix study [5].
In recent years in Australia, there have been several reports of the use the HoNOS in assessing the impact of acute psychiatric hospitalisation. The method is simple; the scale is completed at admission and again at discharge and the numerical difference represents the change in the patient's health status. This change in health status may be regarded as an operational definition of outcome, and it is customary to attribute it to the effects of the hospitalisation event.
This approach can shed light in several important areas. First, by dividing the change in health status by the number of days that the hospitalisation lasted, an index of efficiency, or ‘benefit per day’, can be derived. Second, the progress of groups of patients can be compared according to characteristics of interest. The typical one is diagnosis. Thus, the appropriateness and effectiveness of hospitalisation for different kinds of problems can be addressed. Third, comparisons can be made across different kinds of wards and hospitals. The typical comparison is between public and private facilities, with their differing costs and purchasers.
Goldney et al. [6] administered the HoNOS at admission and discharge to 150 inpatients at an acute private psychiatric hospital. They noted significant declines (improvements) in score in patients with affective, anxiety, and obsessive-compulsive disorders, but not in those with schizophrenia, other acute psychoses, and alcohol and substance abuse problems. Boot et al. [7] reported admission and discharge HoNOS data collected in a number of public and private psychiatric facilities. They computed mean outcome per 10 days of stay for different facilities and for different diagnostic groups. They found that the public facilities were able to achieve approximately the same improvement as the private facilities, but in a shorter length of stay. Goldney et al. [8] extended their earlier pilot to a study of six private psychiatric hospitals. They found significant improvements in all hospitals and for all diagnoses. They found low and nonsignificant correlations between admission HoNOS scores and length of stay.
In 1996, the Mental Health Branch of the Victorian Department of Human Services conducted a field trial of the HoNOS as a potential State-wide outcome measure [9]. Five of Victoria's 22 area mental health services participated in the trial, of which one, the Geelong service, has continued to use the HoNOS on a routine basis. This study presents a comparison of the Geelong inpatient HoNOS data with the results obtained by Boot et al. [7] and Goldney et al. [6–8].
Method
In Geelong, the public mental health services are provided by an ‘integrated team’ approach [10] whereby a single multidisciplinary team of mental health workers is responsible for the care and management of patients both in hospital and in the community. Over a 15-week period, from April to August 1997, staff assessed all inpatients within a few days of admission and again within a few days of discharge using version 4 of the HoNOS [2]. Health of the Nation Outcome Scales ratings, and the dates on which they were made, were recorded.
In the study period, 95 patients were involved in a total of 107 admission episodes. Forty-six had a principal psychiatric diagnosis of schizophrenia or other psychosis, and 38 were diagnosed with a major affective disorder. The Victorian State-wide psychiatric database records diagnoses according to ICD-9 [11]. Given the very small numbers of cases with other conditions, the present analyses will be confined to the two diagnostic categories of schizophrenia and major affective disorder, which have been shown to comprise the great majority of patients within the public psychiatric hospital system [7]. The patients had a mean age of 39.6 years and 62 of them (58%) were male.
Boot et al. [7] and Goldney et al. [6] used version 3 of HoNOS, which comprised 11 items rated on a five-point scale, followed by a rating of functional disability rated between 0 and 100. Goldney et al. [8] and the Geelong service used version 4, in which all 12 items are rated on a five-point scale. In order to render total scores comparable across studies, all total scores were computed from only those 11 items that are common to both versions.
Methods
The present analysis will follow that of Boot et al. [7], in which outcome, or effect size was computed as the admission total score minus the discharge total score, divided by the admission standard deviation [7, table 2; 12, p.20]. Mean outcome per 10 days of stay was computed as the outcome divided by the length of stay in days, multiplied by 10. The relevant statistics from the Boot et al. and Goldney et al. studies were taken direct from their articles. Additional figures from the Goldney et al. studies were required for some of the calculations [Fisher L: personal communication].
Schizophrenia
Inpatient Health of the Nation Outcome Scales (HoNOS) results for patients with schizophrenia
Table 2 compares the HoNOS item scores of Geelong patients with schizophrenia with those reported in Goldney et al. [8].
Mean Health of the Nation Outcome Scales (HoNOS) item scores at admission and discharge for patients with schizophrenia in the Geelong service (n=51) and reported in Goldney et al. [8] (n=97)
Major affective disorder
Inpatient Health of the Nation Outcome Scales (HoNOS) results for patients with major affective disorders
Mean Health of the Nation Outcome Scales (HoNOS) item scores at admission and discharge for patients with affective disorders in the Geelong service (n=42) and reported in Goldney et al. [8] (n=517)
Using the same Bonferroni adjustment of significance levels as was applied in Table 2, it may be seen that at admission, patients with affective disorders in the public service were rated significantly higher on items 2 (self-harm), 3 (drug and alcohol abuse), 6 (hallucinations and delusions) but significantly lower on item 7 (depression). At discharge, significant differences, albeit at a much lower level, were still present on items 3, 6 and 7.
Discussion
We have compared data collected in a public psychiatric service with previously published results from other Australian public and private institutions. A fairly consistent pattern was found. For the two diagnostic groups studied, admission severities tended to be higher in the public settings, but lengths of stay tended to be shorter. Thus, while most facilities were able to achieve substantial symptom reduction in their patients, an index of efficiency, the mean symptom reduction per 10 days of stay, favoured the public facilities.
Differences between public and private services appeared to focus on certain items of the HoNOS. The most consistent and most significant difference was on item 2, self-harm. One reason for this difference may well be associated with legal status, for which we did not have available data. Serious risk of self-harm is a criterion for involuntary admission. In Victoria, the mental health legislation [13] requires that only designated hospitals may accept involuntary patients, and only public hospitals are so designated. Similar arrangements exist in other states. Thus, public psychiatric units will have a mixture of voluntary and involuntary patients, while private units will have exclusively or predominantly voluntary patients.
Systematic differences in either overall severity or in length of stay, or both, according to legal status, could go some way to accounting for the public-private differences observed. It is reasonable to expect that involuntary patients will have more severe problems than their voluntary counterparts [14]. Unpublished data from the Victorian field trial of the HoNOS in the public sector showed that the mean total score (computed on just the 11 items common to HoNOS versions 3 and 4) was 13.2 for 64 voluntary inpatients and 14.6 for 172 involuntary inpatients. The difference was greater for patients with schizophrenic diagnoses (12.3 vs 14.6) than for those with affective disorders (13.7 vs 14.4).
It is not clear whether systematic differences in length of stay exist between voluntary and involuntary patients. Studying general and state hospitals in the United States, LaWall [15] and Okin [16] found involuntary patients having shorter lengths of stay, but Michalon and Richman [17] found the opposite. Probable greater severity and possible shorter lengths of stay of involuntary patients would combine to give an impression of greater efficiency in public inpatient units. This effect would be magnified if the greatest reductions in severity occurred early in the admission episode.
Apart from the difference on the self-harm item, the only other significant difference for patients with schizophrenia was on item 5 (Physical problems), with patients in the public service being rated higher (i.e. worse). It is not clear why this should be. The Victorian HoNOS field trial [9] showed this item to associated with age, so any differences in the ages of patients in the two services might account for the finding. The mean ages of the public patients in Geelong and Boot et al. [7] were 39.6 and 40 years, respectively, while the mean ages of the private patients in Boot et al. [7] and Goldney et al. [6–8] were 46, 46 and 43.7 years, respectively. Thus, despite lower age, the public patients scored higher on the physical problems item of the HoNOS.
Among the patients with affective disorders, the public patients were rated higher than the private patients at both admission and discharge on item 6 (hallucinations and delusions) but lower on item 7 (depression). The former finding could arise if individuals with private insurance who develop overtly psychotic symptoms are admitted to or transferred to public facilities, where, as mentioned, they may be detained involuntarily. The reason for significantly higher rating of the private patients with affective disorders on the depression item is not immediately apparent to us.
The relationship between HoNOS scores and length of stay needs to acknowledge that certain items, notably those reflecting behavioural problems, are more amenable to rapid improvement during acute hospitalisation than certain others, like the impairment and social items. Since behavioural problems, like self-harm, are more likely to lead to involuntary admission, and this in turn is associated with public treatment, it can be expected that the public facilities will be more ‘efficient’. Studies which examine the rate of resolution of admission problems could shed useful light on this possibility.
It has not been the purpose of this paper to draw an invidious comparison between the public and private psychiatric sectors. While certain of the findings may superficially appear to favour the public sector (with which the authors are more strongly associated), the more important implication is that comparisons of agencies or sectors need to take into account external explanatory factors, especially when these are beyond the control of the facilities concerned. Where risk of self-harm leads to involuntary detention, statutory requirements represent factors that are not wholly under the control of the public or private facilities. Other potential differentiators between the public and private sectors, not examined in this study, are contact time between staff and patient, financial security and the availability of community supports.
Our findings have more general implications for outcome measurement. First, this study, and those from which it builds, has relied exclusively on the recorded principal psychiatric diagnosis. Many patients, however, have multiple psychiatric and/or medical diagnoses. Studies have shown that level of functioning is inversely related to the number of conditions that patients have [18], and that patients with both psychiatric and medical conditions are more consuming of resources [19]. Increased severity or complexity may not be consistently associated with increased service use; however, Russo et al. [20] found that patients with longer length of stay tended to have no substance abuse diagnosis. Future work would benefit from attention to medical and psychiatric comorbidities.
Second, studies using routinely recorded information, such as diagnosis and length of stay, are comparatively easy to conduct. Other factors, such as social variables which are more difficult to assess, may have a significant explanatory contribution to make. For example, Russo et al. [20] found that frequency of family visits was the strongest functional quality-of-life predictor, relating to positive outcomes, and that patients' quality of life before psychiatric inpatient treatment predicted treatment outcomes independently of psychiatric status, demographic characteristics, and level-of-care variables. The moral may be that we need to look where we expect to find, rather than where the light is best.
Finally, outcome always needs to be explicitly defined [1]. The term tends to have different connotations for providers and purchasers. The former are inclined to think that the severity methodology is not sufficiently clinically meaningful and is of limited focus, while the latter are typically concerned with financial outcomes and the need for evidence on which to make informed purchasing decisions [21, p.350]. Part of the problem arises from the providers holding a predominantly individual focus, while the purchasers hold a group focus, but both will rely in different ways on the same set of outcome data. It is widely agreed, however, that: ‘Most existing severity measures were intended to be used across groups of patients and are not well suited to making inferences about individual cases’ [21, p.202]. Thus, it is important that simple, single, outcome measures, unadjusted for uncontrollable external factors, not be interpreted too liberally.