Abstract
In the last two decades, Australian mental health services have implemented interventions that are recovery orientated [1]. In this context ‘recovery’ has been described as ‘a deeply personal, unique process of changing ones attitudes, values, feelings, goals, skills, and/or roles …. it is a way of living a satisfying, hopeful, and contributing life even with limitations caused by illness’ [2]. To enable the use of interventions that are effective in fostering recovery, it is necessary to have a psychometrically sound measure of recovery. However, there are few published measures that specifically target recovery-oriented outcomes. One of the first measures is the Recovery Assessment Scale (RAS) [3], but to date there are limited data regarding its psychometric characteristics and no Australian data are available. The present study aims to assess the validity of the RAS in a sample of individuals with severe and persistent psychiatric disability. The term ‘recovery’ is used in multiple ways within the literature, depending on whether one takes a medical, rehabilitative, psychological or sociological definition [4]. In this paper, recovery is viewed in more psychological terms, consistent with patient views of recovery as a process. This is in contrast to a rehabilitation definition, which often implies disability and skill deficit, or a medical approach, which equates recovery with symptom reduction or cure.
A number of scales have been developed that attempt to measure components of the subjective experience of recovery [5]. The construction of these scales has included strong reference to the first-hand accounts of people who have been through recovery. Patient empowerment has led to increasing emphasis on the importance of mental health professionals understanding the recovery experience from the patient's point of view. Previously, measures of recovery from severe mental illness have been dominated by the ‘return to non-diseased state’ perspective that is often characterized by the presence or absence of psychotic symptoms. Liberman et al. view this conceptualization as an incomplete gauge of a person's recovery and propose several criteria to measure a ‘recovery outcome’ including employment [6].
Difficulty has arisen in attempts to measure recovery due to the uniqueness of each individual's journey [7]. There are many measures that tap into individual components that are theorized to be part of the recovery concept, but there are few that aim to capture multiple components specifically related to recovery. Examples of empirical recovery scales developed so far include the Crisis Hostel Healing Scale, the RAS and the Mental Health Recovery Measure. The Crisis Hostel Healing Scale has indicated poor construct validity [8]. The Mental Health Recovery Measure (MHRM) was developed by Young and Ensing [9]. Following interviews and focus groups with 18 consumers, they outlined a three-phase model of recovery consisting of six aspects: phase I, overcoming stuckness; phase II, discovering and fostering self-empowerment, learning and self-redefinition, and return to basic functioning; and phase III, striving to attain overall well-being and striving to reach new potentials. The MHRM is a 41-item scale that comprises six subscales assessing these six aspects of recovery. Results of psychometric testing are contained in the Compendium, and include coefficient alpha for the total scale (α = 0.91) and for the subscales (α = 0.55–0.83). There are, however, limited data on the MHRM, and it was used as a reference point to better understand the RAS.
The RAS is one of the few recovery-oriented scales with any published psychometric information and thus was the focus of the present study. This scale was developed by Giffort et al.[3]. They used participatory action research and narrative analysis from four people with severe mental illness. Thirty-nine items were generated for a recovery scale. These items were reviewed by another group of 12 patients, resulting in the RAS that currently consists of 41 items. Corrigan et al. found positive relationships with scores on the RAS and the self-esteem (r = 0.55), self-orientation (r = −.071) subscales of the Empowerment scale [cited in 10], Quality of Life scores (r = 0.62) and social support (r = 0.48) [10].
Corrigan et al. conducted a factor analysis of the RAS and found five factors comprising 24 items [11]. Factors identified were described as (i) personal confidence and hope; (ii) willingness to ask for help; (iii) goal and success orientation; (iv) reliance on others; and (v) symptom coping [11]. These findings are similar to other components of recovery reported in previous conceptual studies [9, 12, 13]. For a scale to gain credibility for use across the mental health system it must demonstrate stability as a measure with sound psychometric properties [14]. Although the study by Corrigan et al. had a sample of >1750 people, it provided only limited concurrent validity data and did not compare the RAS with other recovery measures [11]. In addition, there is a need to extend comparisons of recovery-oriented measures such as the RAS with some of the Australian routine outcome assessment measures (e.g. Health of the Nation Outcome Scales, HoNOS) to clarify the extent that it appears to be capturing unique domains.
The present study aimed to explore the dimensionality of the RAS and compare the obtained structure against that found by Corrigan et al. [11]. An initial investigation of convergent validity was tested by correlating the RAS and its subscales with other measures of recovery. Finally, divergent validity would be demonstrated if the RAS and its subscales had lower correlations with more traditional symptom-oriented or functional measures of outcome. Such relationships would also help further clarify the extent that the RAS is capturing distinct or novel outcome data compared to some of the Australian routine outcome measures.
Method
Data were obtained from a baseline assessment of part of a larger research project, the Australian Integrated Mental Health Initiative (AIMhi) project. The AIMhi project is part of an National Health and Medical Research Council health partnership grant set up to evaluate the effectiveness of the Collaborative Recovery Training Program [15].
Participants
The AIMhi project includes participants selected from Area Health Services in Queensland, NSW and Victoria and three non-government organizations. The sample procedure used was one of convenience, with case managers being asked to identify people with psychosis and high support needs. Therefore inclusion criteria were (i) age ≥18; (ii) diagnosis of a psychotic disorder of ≥6 months duration; and (iii) ‘high support needs’ (≥5 needs) as assessed by the Camberwell Assessment of Need Short Appraisal Schedule [16]. Patients with dementia, moderate or severe mental retardation or brain injury were excluded from the study whereas patients with comorbid substance misuse or personality disorders were included. Informed consent was obtained from all participants. Diagnosis was based on clinical judgment based on information available from medical records. In addition, a diagnostic criterion checklist based on DSM-IV was also included. Clinicians were asked to complete the checklist for each patient participating in the study.
Archival data from 168 patient-participants (98 men and 58 women) who completed baseline measures were used in the present analysis. This constituted 83% of the total sample who had agreed to participate in the AIMhi study at the time data were extracted. Ages ranged from 19 to 68 years (mean = 38.98 years, SD = 12.1 years). Of the participants 84% were single, 9% were married or in a de-facto relationship and 7% were divorced or widowed. Most had been diagnosed with mental illness for at least 5 years (84%), with 12% receiving a diagnosis between 1 and 4 years prior and only four people indicating they had been diagnosed for <1 year.
Measures
In addition to the RAS, the current study used the Kessler-10 (K-10), the Mental Health Recovery Measure [17], Stages of Recovery measure and the clinician completed HoNOS. The K-10 and HoNOS were included in the present study because they were used as part of routine outcome assessment in the services, with assessment conducted on a 3 monthly basis. The AIMhi project is an effectiveness oriented field study in which integration with existing systems was essential.
The 24-item RAS is derived from the original 41-item scale. Patients respond to each item using a five-point Likert scale (1, strongly disagree; 5, strongly agree). Sample items include, ‘I have goals in life I want to reach’ and ‘I can handle what happens in my life’. Previous studies have found the RAS to have good internal consistency (α = 0.93) and validity [10].
The K-10 is a 10-item self-report measure of psychological distress. The survey asks participants how often in the past 4 weeks they experienced psychological distress such as feelings of hopelessness, depression and nervousness. Participants answer on a 5-point Likert scale (1, none of the time; 5, all the time).
The MHRM was also completed by patient-participants. This is a self-report, 41-item measure of elements of recovery based on the conceptual model of recovery by Young and Ensing [9]. Participants respond on a five-point Likert scale (0, strongly disagree; 4, strongly agree). Sample items include ‘I want to take care of myself for my own good’ and ‘Every day is a new opportunity for learning’. The MHRM has shown good internal consistency (α = 0.93) and good convergent validity with the Community Living Skills Scale (r = 0.57) and the Empowerment Questionnaire (r = 0.67) [18].
The Stages of Recovery Measure (SRM) is a patient-completed measure that was designed as a concise measure of the five stages of recovery as described by Andresen et al. [19]. The scale includes five brief statements, each representing a stage of recovery. Participants respond to each statement with reference to which best describes their current experience of recovery, in the past, currently or never. For, example the statement corresponding to stage 3 is, ‘I'm starting to learn how I can overcome the illness. I've decided to start getting on with my life’.
The HoNOS is a 12-item scale with four subscales of behaviour, functioning, symptoms, social and a global score. It has shown satisfactory reliability, validity and sensitivity to change. The HoNOS is widely used as an outcome measure for severe mental illness [20].
Procedure
Clinicians completed the HoNOS measure after patients had provided consent to participate in the study. The procedure for the majority of patients (approx. 90%) involved research assistants (RAs) arranging a time to meet at the location that they were receiving services (e.g. community mental health team, rehabilitation unit, supported accommodation). Measures were administered, and RAs provided support by reading items if assistance was needed. Those remaining participants (10%) who were able to complete the measures independently returned them by mail via case managers.
Results
The 24 items from Corrigan et al. [11] were included in an exploratory factor analysis (EFA). Corrigan et al. used a two-step process to determine a reliable factor structure for the RAS. First, they conducted an EFA on a randomly selected subset of the data. The factor structure obtained in this step was then cross-validated using confirmatory factor analysis (CFA) on the second half of their data.
To obtain the most robust and stable factor structure for the 24 items from Corrigan et al. multiple extraction methods (principal axis factoring, maximum likelihood with oblimin and promax rotation and principal component) were used. If factors and loadings were similar across extraction methods then this was deemed to provide evidence for factor structure stability. The specific rotation methods were used because the factors were assumed to be correlated. In addition, promax rotation is computationally inexpensive. If a factor structure is stable then there will be little variation in the structure across extraction methods. The reliability of each factor was then determined including item-total correlation analysis. Reliability analysis was also carried out for the factors obtained of Corrigan et al. [11] and comparison between the reliability results of the present study and that of Corrigan et al. [11] were determined.
The items loading on the five factors are highly consistent with those found by Corrigan et al.[11] with the exception of items 5 and 22. The results of the multiple extraction methods of Exploratory factor analysis (EFA) indicate the factor structure to be relatively stable. Results of the five factors using the promax rotation on a principle axis factor extraction are reported in Table 1. The total variance explained by the factors was 51.66%.
Item factor loadings
CFA-1 confirmatory factor analysis of present study; CFA-2, confirmatory factor analysis of Corrigan et al. [11].
Reliability analyses were performed on each of the five factors using Cronbach alpha and compared with reliability analysis of the factors by Corrigan et al.[11]. All reliability coefficients were >0.7, suggesting good consistency of the items in each factor. The corrected item-total correlations ranged from 0.39 to 0.83. This finding indicates that all items are discriminating well and there is consistency within the total test. This analysis found a slightly higher alpha level for ‘reliance on others’ in the present study (0.66–0.73).
A CFA (reported as CFA-1 in Table 1) was also done to determine the fit of these factors and to establish the stability of the factor structure obtained in the EFA. It is not common practice to conduct both an EFA and CFA on the same dataset. In this instance the initial use of EFA was to identify a suitable measurement model using an Australian sample, and identify poor item loadings. The use of confirmatory techniques was to examine the latent factors once poor items have been removed. In addition, given that the factor structure of the RAS been researched, it was deemed appropriate to examine the extent to which that structure is robust using an Australian sample.
The factor loadings for items 5 and 22 are not reported in the CFA-2 column of Table 1 because this column refers to previous results of Corrigan et al.[11] in which the items loaded onto different factors (‘willingness to ask for help’ and ‘personal confidence and hope’ respectively). Results of the CFA indicate an acceptable fit of the model to the data. The minimum discrepancy between the model with the five factors and the structure inherent in the data was χ2=353.15. Similarly, by being <2 [21], the normed chi-squared parameter indicated an adequate fit at 1.68. The Tucker–Lewis coefficient, incremental fit index and comparative fit index indices (0.90, 0.91 and 0.87 respectively), also indicated that the model showed a reasonable fit to the data. For these two indices, values range from 0 (no fit) to 1 (perfect fit) as a gauge of improvement in model fit as compared with an independence model [21]. Another indicator that there is a satisfactory fit of the model to the data is the root mean square of approximation (RMSEA = 0.06).
Overall the RAS correlated positively with the subscales of the SRM and MHRM and negatively or non-significantly with the HoNOS and K-10 measures. Significant positive correlations with the factors of the RAS and the stages of the SRM ranged from r = 0.21 to 0.49, p < 0.01 (Table 2). The factor ‘not dominated by symptoms’ was the only RAS factor not to correlate with any of the SRM subscales. All factors of the RAS positively correlated with the subscales of the MHRM (Table 3). Significant correlations ranged from r = 0.20 to 0.68 (p < 0.01). These results indicate the RAS to have good convergent validity with other recovery-oriented scales.
Correlations of RAS subscales with the SRM subscales
SRM, Stages of Recovery Measure; RAS, Recovery Assessment Scale.
∗Correlation is significant at the 0.05 level (two-tailed); ∗∗correlation is significant at the 0.01 level (two-tailed). Columns 1–2 are subscale scores based on Corrigan et al. scale items. Columns 6–7 subscale scores based on current study findings.
Correlations of RAS subscales with the MHRM subscales
MHRM, Mental Health Recovery Measure; RAS, Recovery Assessment Scale.
∗Correlation is significant at the 0.05 level (two-tailed); ∗∗correlation is significant at the 0.01 level (two-tailed). Columns 1–2 are subscale scores based on Corrigan et al. scale items. Columns 6–7 subscale scores based on current study findings.
The intercorrelations among the RAS subscales and the HoNOS and K-10 subscles are presented in Table 4. With regards to the HoNOS, the present study's revised ‘personal confidence and hope’ factor correlated with the symptoms subscale (r = − 0.22, p<0.01) as well as the HoNOS behaviour subscale (r = − 0.23, p < 0.05). The only other factor that showed a significant correlation with the HoNOS was ‘not dominated by symptoms’, which was negatively correlated (r = − 0.19, p < 0.05) with the symptoms subscale. Factors ‘personal confidence and hope’, ‘goal and success orientation’ and ‘not dominated by symptoms’ were negatively correlated with K-10 scores (r = − 0.43, r = − 0.26, r = − 0.21, p < 0.01). Both the HoNOS and K-10 are scales evaluating external attributes of recovery such as functional behaviours or presence/absence of symptoms. The negative correlations suggest that several factors may be measuring a distinct aspect of psychological recovery that is not simply symptoms or functional domains. These results support the divergent validity of the RAS. This is a positive result because it indicates that the RAS appears to be assessing an aspect of recovery separate from these external measures.
Correlations of RAS subscales with HoNOS and K-10 subscales
HoNOS, Health of the Nation Outcome Scales; K-10, Kessler-10; RAS, Recovery Assessment Scale.
∗Correlation is significant at the 0.05 level (two-tailed); ∗∗correlation is significant at the 0.01 level (two-tailed). Columns 1–2 are subscale scores based on Corrigan et al. scale items. Columns 6–7 subscale scores based on current study findings.
Discussion
The results of the present study support the theoretical adequacy of the RAS as a measure of components of recovery. The following discussion addresses possible expansions of the analysis for future research, limitations of the study and the overall implications of this study for psychology.
There was a strong correlation between the two factors ‘personal confidence and hope’ and ‘goal success and orientation’. This finding is to be expected considering both factors contain similar elements relating to hope or positive future-orientation. This is also evidence that hope is essential and broadly interconnected to recovery. For the RAS to develop as a more succinct scale of recovery, future analyses may benefit from focusing on the possible reduction of similarity between these two factors.
The different exploratory rotation methods presented two of the items (items 20 and 25) as inconsistent in the factor structure. Two of the extractions resulted in these items loading onto ‘personal confidence and hope’. In both the analyses the factor loadings for these two items were low to moderate: factor loadings (principal axis factoring extraction method rotating by promax with Kaiser normalization) were 0.32 for item 20 and 0.32 for item 25; and for items 20 and 25 they were (principal component analysis rotating by promax with Kaiser normalization) 0.44 and 0.58, respectively. The items did not reach the cut-off of <0.30 [20] for the other two extractions, leaving them without a place in the five factors. The content of these items (item 20 reads ‘I have an idea of who I want to become’ and item 25 reads ‘I continue to have new interests’) seems more pertinent to ‘goal success and orientation’ rather than ‘personal confidence and hope’.
The findings of the Corrigan et al. factor analysis placed these two items in factor 1 [11]. All items contained in ‘personal confidence and hope’ relate to a concise concept except items 20 and 25. This as well as the overlap of ‘personal confidence and hope’ and ‘goal and success orientation’ in items referring to ‘hope’ necessitate further analysis of the two factors. Preliminary reliability analysis of ‘personal confidence and hope’, excluding these items, indicates only slight reduction of the alpha (0.84). Future research could further assess the possible exclusion of these two items from the RAS because a scale shorter in length would be more appropriate for this population.
Item 22, ‘something good will eventually happen’, is another item that subsequent research into the psychometric properties of the RAS could focus on. Although this item is stable in ‘reliance on others’, theoretically it does not seem to fit into the conceptualization of this factor. It would be expected that this item would fit into ‘personal confidence and hope’ because it alludes to aspects of hope, or ‘goal success and orientation’ in that it is future orientated. Future analysis of the RAS could also focus on the possible reduction of the scale from this 24-item version to a 21-item scale.
Limitations
A limitation of the current study may be that a large proportion of respondents reported their marital status to be single. The confirmation in the current study of the Corrigan et al.[11] factor that relates to social support ‘reliance on others’ may not be a true representation of a component of recovery, but instead be attributed to the marital status of the participants. This factor should not be dismissed altogether because social support has been referred to recurrently in the recovery literature.
The self-report nature of the scale may also be a limitation of the study. For a person in the initial stages of recovery, completing a questionnaire could prove a difficult task. The present analysis may be restricted because the RAS may require an adequate level of recovery to elicit responses. This is supported by the fact that the majority of participants had been diagnosed >5 years prior to the baseline assessment. The factor structure found in the current study may only be applicable to consumers at a later stage of recovery.
The inverse relationship between the RAS and the HoNOS and K-10 along with the confirmation of the concurrent validity of the five RAS factors with other recovery subscales, indicate that the scale is measuring recovery as subjective psychological processes, rather than symptoms or behavioural excesses or deficits. With more refinement, the RAS could be used as a tool to help clinicians tailor interventions in more consumer-relevant directions. The factors of the RAS may be useful as subscales to enable clinicians to examine which particular aspects of recovery an individual needs to focus on. It is noted that although this scale has displayed good reliability and validity, the divergence of the items from those on the MHRM indicates that it may not measure all aspects or the full range of recovery components. Similarly, not all elements of recovery as described in the introduction appear to be captured by the factors confirmed in the RAS.
Although assessment of all features of recovery cannot be managed by the RAS at this stage, the scale is brief enough to be supplemented with other outcome measures. Mental health organizations are continually implementing routine clinical outcome evaluations of their services to promote better quality services [22]. The RAS potentially offers a measure that has high relevance for mental health consumers and at the same time provides relatively unique information for clinicians.
Conclusion
Although the RAS is still in its initial stages of development the findings in the present study support its reliability and validity as a measure of recovery. As with most measures the RAS has strengths (e.g. brevity) and weaknesses (e.g. comprehensiveness). The present study has provided data regarding the relationship of the RAS with other routinely completed outcome measures. The utility of the measure in clinical practice and what it can add to our understanding of recovery at both an individual and service level remain to be determined.
Footnotes
Acknowledgements
This research was supported by the Australian Integrated Mental Health Initiative (AIMhi) which was partly funded by the National Health & Medical Research Council (NHMRC, 219327) Health Partnership Grant Scheme and participating partner organizations. Contributing organizations to the High Support Stream of this project, in alphabetical order include Aftercare; Illawarra Health, Mental Health Service; La Trobe Valley Health; NEAMI; Prince Charles Hospital Health, District Mental Health Service; Psychiatric Rehabilitation Association; Richmond Fellowship Queensland and Richmond Fellowship NSW; University of Wollongong; Wentworth Area Health Service.