Abstract
The assessment of disability ought to include the views of clinicians as well as an understanding of the subjective experiences of patients, particularly in the field of rehabilitation in psychiatry [1]. Assessments will ideally use frameworks and instruments developed for health care generally, or for mental illness specifically, depending on the purpose.
People with long-term psychotic disorders often have significant disability, particularly at the social level, as a result of restrictions in their social participation. While considerable heterogeneity in long-term outcome is reported in follow-up studies of patients with psychotic disorders, social disability appears to be a persistent phenomenon. Wiersma et al., from the six European centres participating in the WHO International Study of Schizophrenia, found a high proportion of incidence cohort patients with significant social disability at 5-year follow-up [2]. In the recent Australian National Survey of Mental Health and Wellbeing almost half (47%) of the patients living with psychotic disorders were rated as seriously impaired in their ability to function in social and occupational domains and 59% were found to have evidence of impairment in their social functioning over the previous year [3].
Clinician measures of disability have been developed for use in health care and rehabilitation generally as guides to management and as outcome measures. The recent International Classification of Impairments, Disabilities and Handicaps-2 (ICIDH-2), now known as the ICF (International Classification of Functioning and Disability), was developed by the World Health Organization (WHO) as a major conceptual and practical work [4]. We reported that this framework was potentially useful in the clinical assessment of patients with psychotic disorders [5]. Other clinician measures of disability include the Life Skills Profile (LSP) and the Health of the Nation Outcome Scale (HoNOS) which are widely used as measures of outcome in Australian mental health services [6], [7].
The validity of the self-reported experience of patients with long-term psychotic disorders has been questioned because of the presumed influence of psychotic symptoms. However, recent work indicates that assessment of quality of life can be feasible and meaningful in this group and there is a risk of marginalizing patients with mental illness if the validity of their perspectives is discounted [8]. Other self-report instruments are used as outcome measures in Australian mental health services, including the Mental Health Inventory (MHI), the Short Form 36 (SF-36) health survey and the Behaviour and Symptom Identification Scale (BASIS-32), all of which have been rated favourably by patients in terms of their utility [9]. These measures are designed to refer specifically to people with mental illnesses, rather than as generic instruments.
The development of the WHODAS II
The original WHO Psychiatric Disability Schedule (WHODAS) was developed to assess social functioning in patients with psychiatric disorders specifically, using the ICIDH model as its framework. The WHODAS is a semistructured instrument comprising items related to patients' overall behaviour, social role performance, social functioning in the ward and occupational settings, as well as items related to functioning in the home environment [10]. Several cross-cultural studies, including the WHO Collaborative Study on the Assessment and Reduction of Psychiatric Disability, have demonstrated its validity and reliability as a tool in the specific measurement of psychiatric disability [11]. Sections of this instrument were used in the Australian National Survey of Mental Health and Wellbeing [3].
The WHO Disability Assessment Schedule II (WHODAS II) is a new instrument distinct from the original WHODAS. It is not designed to apply specifically to patients with psychiatric disorders. It is a self-report generic questionnaire, whereas the WHODAS is a clinician-rated instrument [12]. The WHODAS II is based on the ICIDH-2 framework and can be used to assess disability in individuals irrespective of diagnosis. However, while the ICIDH-2 was developed as a framework to record clinicians' overall impressions of a patient's level of disability, the WHODAS II rates the nature of disability directly from the patient's responses [12]. This instrument can be considered to be complementary to the WHO Quality of Life instrument (WHOQOL-BREF) which assesses individuals' level of satisfaction or difficulty with several aspects of their lives and functioning in the context of the society and culture in which they live [13].
Description of WHODAS II domains
The 36-item WHODAS II contains 32 questions covering six domains of assessment. These are based on the four ICIDH-2 dimensions: impairments in body functions and structure; activity limitations; participation restrictions; and environmental factors [12]. The WHODAS II domains include: understanding and communicating; getting around; self-care; getting along with people; life activities; and participation in society.
Patients are asked to rate how much difficulty they experienced in the last 30 days because of their health condition in performing various activities. A 5-point rating scale is used; the patient may rate the level of difficulty experienced as none, mild, moderate, severe or extreme.
Method
This study is based on the WHODAS II Reliability and Validity, and the WHODAS II Sensitivity to Change 1999 WHO Field Trial Protocols [12]. We aimed to: assess the feasibility of the WHODAS II instrument as a selfreport measure of the level of disability in patients with long-term psychotic disorders; assess the reliability of the WHODAS II using a test-retest analysis; compare the WHODAS II assessments with clinician-rated measures of functioning, including the ICIDH-2, LSP and HoNOS, and compare the WHODAS II assessments with those from the WHOQOL-BREF.
Setting
The study was performed at two sites of St. Vincent's Mental Health Service: the Continuing Care Unit (CCU), a 20-bed residential rehabilitation unit and the Mobile Support and Treatment Service (MSTS) at the Clarendon Clinic community mental health service, which provides intensive case-management service.
St. Vincent's Mental Health Service Melbourne is a ‘mainstreamed’ state-funded area mental health service established in September 1995 to provide specialist mental health services to adults aged 16–64 years residing in the local government areas of Yarra and Boroondara (a population of approximately 216 000). St. Vincent's Mental Health Service is managed by St. Vincent's Health and is affiliated with the University of Melbourne Department of Psychiatry. It operates from four sites: a 50-bed inpatient unit, the CCU and two community mental health services, each with approximately 400 registered patients.
Participants
Our study was part of an international multicentre field trial coordinated by WHO, in which the minimum number of 20 patients was stipulated. Furthermore, for our own purposes, our investigation was a pilot study to determine the potential usefulness of the WHODAS II instrument for future research in this group of patients. For these reasons we chose to study the application of the five instruments in a small group of 20 patients. We assessed patients who were selected on the basis of having a long-term psychotic disorder and who required intensive rehabilitation, either residential or non-residential. From a group of 27 potentially eligible patients approached for the study and who were reassessed after 6 months, four refused and three were unable to persist with the interviews. Ten patients from each of the CCU and MSTS participated.
Materials
For each patient the first author (PC) consulted with treating clinicians to complete:
While the LSP-16 is used in routine practice in mental health services in Victoria, one of its limitations is the exclusion of the Bizarre subscale, included in the longer LSP-39 and in the LSP-20 to assess psychotic symptoms [6].
Each patient completed two questionnaires, the WHODAS II and WHOQOL-BREF, during interviews with the first author and the WHODAS II interview was repeated by the second author (JC). Both investigators assisted the completion of these interviewer-administered questionnaires by asking the patient to answer the various questions and then recording their responses. The 36-item intervieweradministered version of the WHODAS II was chosen in preference to the 12-item version, the latter being recommended for brief assessments of overall functioning [12]. The Australian version of the WHOQOL-BREF has 26 questions about perception of various aspects of quality of life [14].
Demographic information as well as psychiatric and medical diagnoses were recorded for each patient.
Design
For each patient the initial assessment using all five instruments was completed by PC. Within 5 days the WHODAS II interview was repeated by JC for the purpose of test-retest analysis. Approval for the study was granted by the St. Vincent's Hospital Human Research and Ethics Committee.
Data analysis
An assessment of the feasibility of the WHODAS II was based on our experience of administering the questionnaires and the reactions of the patients when completing the questionnaires. We assessed testretest reliability of the WHODAS II question items using raw agreement and weighted kappas based on squared weights as measures of agreement. Weighted kappas allow differential credit to be given according to the degree of agreement; that is, the larger the discrepancy the lower the weighted kappa. Ninety-five percent confidence intervals were determined for weighted kappas. The statistical package Stata-Corp developed by Stat Statistical Software was used for these analyses [15]. SPSS V. 12.0.1 software package for Windows was also used to determine a paired t-test analysis between the mean test and retest item scores for each code [16]. A qualitative comparison of the WHODAS II and WHOQOL-BREF ratings with the clinician measures of functioning and disability was made.
Results
Demographics
All patients had a primary diagnosis of long-term schizophrenia except one patient with bipolar affective disorder. There were seven men and 13 women with a mean of 11 years of education. Most patients were unmarried, were receiving a disability support pension and had a mean age of 42 years. This profile is typical for patients with long-term psychotic disorders receiving either residential or non-residential rehabilitation [3].
Feasibility
A number of factors were identified which limited the feasibility of the WHODAS II in the patient group studied.
Patients appeared hard-pressed to distinguish difficulties attributable to their health condition from other difficulties. For example, many patients reported difficulties in mobility, although they had no apparent physical disability. The distinction became more complicated when patients were asked to report difficulties due specifically to health conditions in the other more subjective areas, such as getting along with people, life activities and participation in society.
Test-retest reliability
The test-retest analysis for WHODAS II items at the baseline assessment is shown in Table 1. Overall there was a fair level of agreement. It can be seen that the 95% confidence intervals for the weighted kappas varied markedly. The interpretation of these ratings is limited by the small number of participants. Using paired t-test analyses, there were four codes for which patients gave a statistically significant higher rating at the time of the first interview (p < 0.05). These codes were: D1.2 Memory; D6.4 Time spent on health; D6.7 Impact of health condition on family; and D6.8 Relaxation activities. Specifically, the means of the rating scores given at the time of the initial interview by PC for these four codes were higher than the means of the scores given at the time of the retest interview by JC.
Test-retest analysis for WHO Disability Asessment Schedule II items
Comparing WHODAS II with other instruments
ICIDH-2
The response to each question in the WHODAS II is potentially affected by all of the dimensions described in the ICIDH-2. That is, all of the ICIDH-2 dimensions of impairments with body structure and function, activity limitations, participation restrictions and environmental factors, influence each of the activities in the WHODAS II domains. For example, a question in Domain 5 (Life activities) of the WHODAS II [12] asks the patient to state how much difficulty they experienced due to their health condition in taking care of household responsibilities in the previous 30 days. This is potentially influenced by impairments in body function such as attention and higher level cognitive functions. It is also affected by limitations in the activity of performing a task such as getting daily necessities, as well as participation restrictions including caring for home and possessions and environmental factors such as the availability of items for personal consumption.
Regarding the second ICIDH-2 dimension of activity limitations, corresponding to Domain 3 (Self-care) of the WHODAS II, many patients with significant negative symptoms did not acknowledge difficulties with self-care. However, most patients, including those with clinical evidence of self-neglect, reported difficulties in interpersonal activities, detailed in Domain 4 of the WHODAS II.
Domain 6 of the WHODAS II and the ICIDH-2 dimensions of participation restrictions and environmental factors can be compared more directly. Many patients with significant social disability as rated using the ICIDH-2, reported difficulty in joining in community activities, even those who minimized other difficulties such as self-care and maintaining their household.
LSP-20
As with the ICIDH-2 assessment, problems with social withdrawal and self-care noted in the LSP-20 were under-reported in the WHODAS II. The total LSP-20 score ranged from 32 to 53 with a mean of 40.6, indicating a high level of overall impairment of functioning [6].
HoNOS
Similar comments apply to the comparison with the HoNOS. The total HoNOS score ranged from 5 to 19 with a mean score of 12.5, also indicating an overall high level of impairment in functioning [7]. Patients identified as having problems with daily living, which in the HoNOS includes problems with basic self-care as well as complex skills such as budgeting and other household activities, under-reported these difficulties in the WHODAS II.
WHOQOL-BREF
The four domains incorporated in the brief version of the WHOQOL include physical health, psychological health, social relationships and environment [13]. The questionnaire allowed patients to rate their level of satisfaction or difficulties with various aspects of their lives and generated additional relevant information.
Discussion
Our study illustrates the challenge of assessing disability in patients with psychotic disorders. We found the WHODAS II to be useful, although there were limitations in applying this questionnaire in this group of patients.
Many patients, when asked to consider difficulties due to health conditions, reported that they were healthy and denied ‘emotional or mental problems’ as described in the WHODAS II. Hence, a lack of insight into their illness appeared to make it difficult for patients to report on difficulties as framed in this questionnaire. Comparison between the WHODAS II scores and the ICIDH-2 dimension of impairments with body structure and function ratings is thus affected because many patients, despite the presence of psychotic symptoms, did not acknowledge that they suffered from an illness. Hence, their ratings for all of the activities contained in the WHODAS II interview could be interpreted as ‘difficulties experienced’ rather than difficulties specifically due to health conditions.
It required effort on the part of the researchers to keep patients focused on answering the questions. For example, there was a tendency for patients to rate their difficulties as either none or extreme, perhaps reflecting a ‘black and white’ cognitive schema.
However, the finding of fair test-retest reliability for the WHODAS II items gives support to the notion that patients' perspectives should not be discounted. As stated, the mean scores for the codes D1.2 Memory, D6.4 Time spent on health, D6.7 Impact of health condition on family and D6.8 Relaxation activities, were consistently higher at the first interview as compared with the retest interview. It can be interpreted that patients on reflection gave a lower rating of impairment for these particular codes when asked again by a second rater. For the other 28 variables, test-retest differences were due to the responses moving in either direction.
Many patients reported difficulty in joining in community activities, even those who minimized other difficulties such as self-care and maintaining their household. Most patients reported problems because of barriers in their environment; the WHODAS II questionnaire does not allow patients to report which barriers were experienced as significant, although many patients had limited family or other social support with the exception of health care professionals.
As noted earlier, a considerable limitation of our study is the small number of participants, so that detailed comparisons of clinician and self-report measures could not be made. In addition, all participants in the study were patients with long-term psychotic disorders. Hence, we cannot comment on the applicability of the WHODAS II to other groups of patients with mental illnesses. However, a number of conclusions are drawn.
First, the ‘generic’ assessment of disability is applicable to people with long-term psychotic disorders. Hence, the integration of psychiatric care and rehabilitation with ‘mainstream’ services may be facilitated through better mutual understanding.
Second, the manner in which assessment of disability is made has meaning for the patient. As clinicians, there is an inclination to document areas of weakness or deficits the patient has, so that these can be identified for treatment. WHODAS II questions are framed in terms of asking patients to rate their level of difficulty experienced with various tasks. Reframing questions to ask patients about their ability rather than difficulty experienced may make the interview more acceptable, as it would characterize patients' strengths rather than weaknesses. This was highlighted by a patient who commented that the interview should be about what she ‘could do’ rather than what she ‘couldn't do’. This approach may also be useful when assessing patients with little insight into their illness, who may be unable or reluctant to acknowledge difficulties experienced.
Third, incorporating the viewpoint of patients into the assessment of disability adds additional useful information. As with the clinician assessments of disability, including the ICIDH-2, the WHODAS II does not directly ask patients regarding their personal experience and does not enquire regarding the patients' perception of the meaning of their illnesses. We found that the quality of life assessment, using the WHOQOL-BREF, was complementary to the WHODAS II interview, the crucial difference being that the WHOQOL-BREF includes an assessment of satisfaction experienced by the patient in addition to an appraisal of difficulty experienced in various specific tasks. Question 6 in the WHOQOL-BREF Australian version was of particular interest: ‘To what extent do you feel your life to be meaningful?’ [14]. This goes beyond the task of the WHODAS II of assessing to what extent the patients are disabled. Patients' responses varied; some significantly disabled patients reported that they felt their life was meaningful to a great extent, while some patients with apparently less severe disability reported that they felt their life was not at all meaningful. Hence, it is clear that satisfaction and perception of quality of life are not solely dependent on the level of disability. The experience of using the WHOQOL-BREF in a previous study of similar patients demonstrated that this instrument was feasible and provided meaningful assessments [8]. As discussed, there is a need to consider both clinician and self-report measures of disability and quality of life. Self-report measures can be considered among the group of relevant outcome measures in assessing clinical or service-level interventions. There is a need to include measurements of disability by clinicians and the reported experience of patients in order to define needs more broadly rather than solely focusing on clinical areas of interest. The next step involves the redefinition of outcomes, including measures of disability and quality of life according to priorities identified by patients themselves.
Footnotes
Acknowledgements
Prem Chopra was supported by a grant from the Lilly Fellowship. We thank Tom Trauer and Tony Pinzone for assistance with data analysis.