Abstract
The estimation of need for health care on a population level has long been discussed in terms of its implications for resource allocation and service planning [1]. More recently, discussions have focused on assessments of the individual needs of patients and their role in service delivery and evaluation [2,3]. When applied routinely in mental health service settings, accurate needs assessments have the potential to inform the choice of treatments for individuals.
Individual need has been a difficult concept to define and consequently measures for assessing need have differed in their focus and design and importantly in whether clients or clinicians make the ratings [4–6]. The focus of the measures is potentially important as there is evidence to suggest that clinicians and clients differ in their perceptions of need and that staff assessments alone may not be sufficient for determining need for care [2,7–9].
The Camberwell Assessment of Need (CAN) is one measure of need which has been shown to be reliable and valid, takes into account both clinician and client views and is suitable for both research and routine use [10]. Although widely used in the United Kingdom, this instrument has not been formally trialled in Australia. There are considerable differences in the Australian and UK systems of social welfare and health care [11]. There are also important differences in the organisation of mental health service delivery and in the training of clinical staff. Clinicians' assessments of need for individual clients will be influenced by their role within the wider health system and their professional training [12]. In addition, the relationship between measures of need and other widely used measures of disability, such as the Health of the Nation Outcome Scales (HoNOS) [13], has not been clearly identified. If the CAN is to be considered as an assessment tool, then its applicability in an Australian setting, and its relationship to currently used outcome measures, should be examined.
The aims of this study, therefore, were to trial the CAN in a community mental health centre in Australia to compare clinician and client ratings of need and to compare ratings of need with ratings of disability using the HoNOS.
Method
Study setting
The current study was carried out within Southeastern Sydney Area Health Service, a large urban service providing health care to approximately 700 000 people in the inner city and south-eastern suburbs of Sydney. Two mental health services within this area participated in the trial. Both services provide comprehensive inpatient and community mental health care to defined catchment area populations and both services have access to a 24-h psychiatric crisis team based in the community. The area encompasses a wide range of socioeconomic groups and includes the inner city of Sydney, where there are high rates of homelessness and social deprivation [11,14].
Procedure
Fifteen case managers (each having seven to 32 clients, median = 9) who were already participating in a trial of routine outcome measures were asked to participate. Case managers came from a range of teams within the service, and therefore clients with varying levels of disability were included in the study. Case managers were asked to complete the HoNOS (fourth draft) [13] and the CAN (short version) [10]. Case managers were trained in the use of the measures. Clients were excluded from the study if they were considered by their case managers to be too unwell or did not have sufficient English to complete the interview. Clients who met inclusion criteria were interviewed on the CAN by researchers, either face-to-face or over the telephone. Case managers were blind to the results of these interviews.
Sample
There were 230 clients who were considered for inclusion in the study. Of these, 57 (25%) clients were excluded because their case managers considered them to be too acutely ill, and four (2%) because of insufficient English. Of the 169 clients who met the inclusion criteria, 123 (73%) were assessed by their case managers on the HoNOS and the CAN. Forty-six clients were not assessed because case managers were too busy or on extended leave. Of the 123 clients assessed by case managers, 78 (63%) were also interviewed by the researchers on the CAN. Fifty-five clients were not interviewed because they were unable to be contacted or refused interview. These data therefore gave a sample of 123 completed HoNOS and CAN pairs, and 78 completed clinician and client CAN pairs.
Statistical analysis
The CAN assesses 22 areas of functioning and provides both clinician and client ratings of need. It rates the presence or absence of a need in each of these areas and assesses whether the needs are met or unmet. The instrument can be scored in a number of ways to obtain either total scores for met and unmet need or scores on each of five subscales. For the purposes of this trial, the following scores are reported: for each of the 22 functioning areas, for both clinician and client versions, all levels of need are reported (i.e. no need, met need or unmet need). This is to allow comparison with other samples. The kappa coefficient was used to examine level of agreement [15]. Kappa was not calculated across all levels of need as small numbers in each cell render the statistic unstable [16]. Instead, ratings of ‘not known’ were combined with ‘no need’ on the basis that if need is subjective then not knowing that one has a need is equivalent to not having one.
Total scores on the HoNOS were compared with total scores on the CAN (total number of needs, total number of met needs and total number of unmet needs) for both the client and clinician versions. This was to allow comparison of total levels of need with total levels of disability but also to allow comparison of disability with met versus unmet need. Pearson product moment correlation coefficients were used for these comparisons.
Results
Characteristics of sample
Fifty-three percent of the sample (n = 78) were male and their mean age was 44 years (SD = 14). Eighty-one percent had a clinical diagnosis of schizophrenia, 14% bipolar disorder and 5% depression. There were no significant differences between those with complete CAN data and those without (n = 78 vs n = 152) in terms of age (t = 0.84, df = 179, p = 0.40), sex (χ2 = 0.71, df = 2, p = 0.70), or diagnosis (χ2 = 0.43, df = 3, p = 0.23). The mean score for the sample on the HoNOS was 10.1 (SD = 7.0), indicating a moderately disabled group.
Staff versus client ratings of need
Staff and client ratings of need are presented in Table 1. Staff identified a mean number of 7.3 needs per client (SD = 5.0) compared with 6.0 (SD = 2.4) identified by clients. This difference was significant (t = 2.44, df = 77, p < 0.05, two-tailed test). The clinician and client ratings on each of the 22 areas of need assessed are shown in Figs 1–3. The pattern across each of the 22 areas is similar for clinicians and clients and ratings are most similar for identification of unmet needs. In general, there were more ratings of no need in the 22 areas assessed by both clinician and clients than there were ratings of either met or unmet needs. The kappa coefficients for agreement between clinicians and clients in identification of need (no or unknown need, met need or unmet need) are shown in Table 2. The mean level of agreement for ratings is 0.18. The kappa coefficients ranged from poor to moderate (0–0.45). High levels of discrepancy between clinicians and client ratings were in the areas of education, welfare and sexual expression.
Clinician and client ratings (%) of need in each of 22 areas on the MINI-Camberwell Assessment of Need
Clinician (____) and client (_____) ratings: no need
Clinician (____) and client (—–) ratings: met need
Clinician (____) and client (______) ratings: unmet need
Comparisons of ratings of need and disability
Clinician ratings of total number of needs on the CAN and ratings of disability on the HoNOS were highly correlated (Pearson's r = 0.82, p < 0.001). When the CAN scores were split into the number of met and unmet needs, unmet needs were still highly correlated with total HoNOS scores (Pearson's r = 0.80, p < 0.001), but ratings of met need were only moderately correlated (Pearson's r = 0.52, p < 0.001). In contrast, while ratings of met need made by clients showed a significant relationship with the clinicians' ratings of disability, the correlation was weaker (Pearson's r = 0.23, p < 0.01). Further, ratings of unmet need made by clients were not significantly correlated with HoNOS scores (Pearson's r = 0.22, p = ns). The strongest correlation was between client ratings of total number of needs and total score on the HoNOS (Pearson's r = 0.37, p < 0.01).
Agreement between clinician and client ratings of need in each of 22 areas on the MINI-Camberwell Assessment of Need
Discussion
Clinicians identified a mean of 1.3 more needs (out of a possible 22) than clients. In general, comparisons between clinician and client identification of need showed poor to fair agreement. Poor to moderate agreement was also found between ratings of need made by clients and ratings of disability made by clinicians on the HoNOS. In contrast, clinician ratings of overall level of client need were highly correlated with their ratings of disability. When examined more closely clinician ratings of unmet need were more highly correlated with disability than ratings of met need.
The disparity between clinician and client ratings of need observed in our sample is consistent with previous work in other countries [2,7–9]. Large discrepancies were found in the areas such as education, welfare, intimate relationships and sexual expression, while higher levels of agreement were found in areas such as accommodation, drugs and money. Consistent with the results of Slade et al. [2], there may be less discrepancy in the areas for which there is a defined service response. These areas may be those for which there have already been discussion between the clinician and the client about the extent of problems and of potential management strategies. For example, in this study ensuring appropriate accommodation for clients is a clearly defined role of clinicians, whereas there is a less defined service response for needs in the area of sexual expression or intimate relationships. Many clinicians had never discussed this area of potential need with clients and consequently, were not able to identify whether or not these needs were present. Similarly, 40% of clients could not identify whether company of others was an area of need. For many of them, this may be another area about which they have had little discussion with case managers. Use of a comprehensive, structured measure such as the CAN may identify areas of need not previously considered.
Another example in this study was that while it is known that substance use in this population leads to significant consequences in terms of disability, it is rarely assessed [17]. The CAN assessment showed that 23% of clients were rated as having a need in the area of alcohol. There are effective interventions for alcohol [17] and the clinical utility of the CAN is in identifying where such needs exist. Not all identified needs will be meetable within a given service, however, awareness may lead to appropriate referral, and ultimately to the need being met. Comprehensive and reliable assessment is an essential component of good clinical care.
The discrepancies observed in this and other studies may be due to differences in interpretation of the concepts underlying ratings of need and disability. The assessment of these constructs is complex in the context of ongoing care. For example, an individual may experience disability that is alleviated by help that they receive. A client who is reminded daily about self care, may as a result maintain an adequate level of care for themselves, a met need. However, if such support were not available, the client may experience significant disability in this area, an unmet need. The client has an underlying need in the area of self care but it is masked by the level of support they are receiving. In this case, a disability measure would not reflect the level of support needed to alleviate the disability and maintain it at the level reported. The clinicians in the present study rated areas of met need as areas of low disability, reflected in the only moderate agreement between met need scores on the CAN and total scores on the HoNOS. This was in contrast to the strong agreement found between clinician ratings of unmet need and disability.
Previous studies noting discrepancies between client and staff ratings of need have postulated several explanations [2,7]. One such explanation is that clinicians rate needs in the same way as they rate a client's disability, whereas clients may be more likely to include the social consequences, thereby rating their own handicaps [2]. In this paper, we were able to directly compare clinician ratings of need with clinician ratings of disability. It appears that clinicians rate unmet need in the same way as they rate disability, but that met needs are not consistently rated as areas of disability. These results highlight the importance of understanding the way in which measures are applied to individuals. It also points to the potential to examine changes in levels of need alongside changes in disability as service provision varies.
In practice, the accurate assessment of need is rarely attempted in routine settings. This was a naturalistic study, the clinicians were faced with ongoing staff shortages, and no additional resources were available for the application of the measures. Further, the complex nature of the patients' problems and the demanding nature of the inner city area added to the challenge of standard measurement. The completion rates for the measures while low, were similar to those found in comparable research studies in the UK [2]. Of eligible clients in this study, 37% were unable to be interviewed by the researcher. None of the clients knew the researcher before the study began. In routine use of a measure such as the CAN, a higher completion rate might be expected when clients are interviewed by case managers with whom they have an ongoing clinical relationship.
In summary, this study provides support for the application of structured needs assessments such as the CAN in the context of an Australian mental health service. In conjunction with other studies it points to the importance of obtaining both clinician and client perspectives when assessing need for care. Furthermore, it highlights the importance of both the role of the rater and the context in which they are making the ratings when applying need and disability assessments in clinical practice.
Footnotes
Acknowledgements
The authors wish to thank Jill Gallagher for her help collecting data for this evaluation, and the staff and clients of St Vincent's Mental Health Service and Eastern Suburbs Mental Health Service, Sydney, for their participation.