Abstract
Work has been well known as a potential contributor and an important element of rehabilitation for severe mental illness [1,2]. In Hong Kong, Tsang and Chen found that returning to work was regarded as a significant factor of recovery by health-care professionals [3]. According to the statistics reported from the General Household Survey [4], Hong Kong has 86 600 persons with mental illness/mood disorder with a prevalence rate of 1.3%. Only 20.3% of them (17 580 people), however, were actively employed or actively participated in economic activities. The traditional vocational rehabilitation (TVR) service adopts a stepwise approach that offers pre-vocational training at the beginning of the service. Participants are paid less than the minimum wage, and rates for competitive employment are usually <20% [5,6]. Due to the obvious limitations of TVR, supported employment service has been a popular model of vocational rehabilitation in recent years [7,8].
Individual placement and support (IPS) is currently regarded as the most evidence-based supported employment approach for people with severe mental illness. Its success rate averaged >50% employment during a 6 month period of service [9,10]. The job retention, however, of IPS participants is not satisfactory. Because people with severe mental illness demonstrate serious deficits in interpersonal behaviour and social functioning that may explain their premature job terminations [11], we developed an Integrated Supported Employment (ISE) programme, which combines IPS with work-related social skills training (WSST), to augment the vocational outcomes of participants. The most distinctive feature of the ISE protocol is that a manualized WSST module [12] is given to the participants before the process of obtaining employment. After they have succeeded in securing a job, ongoing support will then be given to help them improve their social relationship in their workplace. Tsang et al. reported that ISE participants had significantly higher employment rates (78.8% vs 53.6%) and longer job tenure (23.84 vs 12.34 weeks) after 15 months of services than those receiving IPS only [10].
Other than vocational outcomes, limited studies have been devoted to showing the effects of supported employment on such non-vocational outcomes as self-esteem, control in psychiatric symptoms, quality of life, and psychosocial well-being [1,13]. Bond et al. examined the cumulative effects of work on different non-vocational outcomes for 18 months and found that competitive employment led to better symptom control and self-esteem [14]. Long-term non-vocational outcome, however, remains unknown. The aim of the present study was therefore to fill this knowledge gap.
Most studies on supported employment had follow-up data up to 24 months [15,16]. Only two studies in the USA to our knowledge had information beyond 24 months [17,18]. To date, there has not been any study on the long-term vocational and non-vocational outcomes of supported employment in a non-US culture.
We had already reported the satisfactory vocational outcomes of the ISE programme up to 15 months of follow up [10]. To our knowledge, no other study has attempted to explore the long-term effectiveness of this supported employment protocol. In the present study we extended our understanding on the impact of this protocol by examining its long-term outcomes up to 3 years in Hong Kong. The aims and objectives of the study were (i) to study and compare the vocational outcomes and non-vocational outcomes of IPS, ISE programme, and TVR up to 15 months follow up; (ii) to study vocational outcomes of IPS and ISE programme over a period of 3 years in terms of job satisfaction, job tenure, job stress coping, and job mobility; (iii) to study non-vocational outcomes of IPS and ISE programme over a period of 3 years in terms of symptom control, relapse rate, self-esteem, and subjective personal well-being; and (iv) to compare the long-term effects of ISE programme and IPS in terms of vocational and non-vocational outcomes.
The main hypothesis was that the group receiving the ISE programme would have better vocational and non-vocational outcomes in terms of job seeking, maintenance, and adjustment than the other two groups receiving IPS and TVR.
Method
Study design
The research study was a randomized controlled trial comparing three different types of vocational services. The subjects were randomly assigned to three different vocational services, ISE, IPS, and TVR.
Participants
The subjects were recruited from two community mental health service units and three day hospitals. The service units provided a range of services by a multidisciplinary team consisting of psychiatrists, rehabilitation managers, case workers, and social workers. The recruitment was based on the following selection criteria: (i) severe mental illness (operationally defined as schizophrenia, schizoaffective disorder, bipolar disorder, recurrent major depression, or borderline personality disorder); (ii) at least 2 years of major role dysfunction; (iii) medium–high functioning and free from serious role dysfunction for the past 3 months; (iv) unemployment; (v) willing and cognitively competent to give informed consent; (vi) completion of primary education; and (vii) desire to work. The participants were excluded if they had memory impairment, learning disorder, and neurological or medical illness that would preclude their working or participation in assessment. The diagnosis of the participants was based on documented medical records and was made by certified psychiatrists in Hong Kong following DSM-IV criteria. The Ethics Committee of the Hong Kong Polytechnic University reviewed and approved this study. Individual written informed consent was obtained after description and explanation of the study. In addition, patient anonymity was preserved.
Comparison of the demographic data of the ISE, IPS and TVR participants is summarized in Table 1.
Subject characteristics
IPS, individual placement and support; ISE, integrated supported employment; TVR, traditional vocational rehabilitation.
Interventions
Traditional vocational rehabilitation
TVR participants received comprehensive vocational assessments and pre-vocational training conducted in the vocational rehabilitation centres. Vocational assessments included work samples, vocational interest exploration, and situational vocational assessments. After the establishment of participants’ baseline work performance, pre-vocational training on entry-level job tasks were provided in order to help them develop specific job skills and work habits. The participants were placed in a sheltered environment in various work groups such as clerical training, computer training, and cleaning training. The aim of the workshop-based training was to promote the participants to sheltered workshop or competitive employment.
IPS service protocol
IPS represents a synthesis and standardization of eight key principles of support employment [19]: (i) a single-minded focus on competitive employment (in the present study the employment specialist adhered to this principle in helping the participants obtain open employment); (ii) eligibility for services based solely on client choice, with no exclusion on the basis of work readiness, substance use problems, lack of motivation, treatment non-compliance and so on; (iii) rapid job search upon programme admission using the ‘place then train’ approach; (iv) attention to client preferences in the job search, rather than dependence on job availability; (v) close integration between the employment services and the mental health treatment team; (vi) ongoing, individualized support and job training after clients obtain employment; (vii) systematic benefits counselling; and (viii) consultation with employer or job supervisor including advocacy accommodations.
ISE service protocol
The ISE participants joined the ISE programme, which combined IPS and WSST. The main difference with IPS is that it was enhanced by 10-session WSST. The social skill training was provided to ISE participants before obtaining employment. A problem-solving approach was used to help participants handle interpersonal conflicts throughout the follow-up period.
Figure 1 illustrates the similarities and differences of the IPS and ISE protocols compared in the present study:
Protocol of IPS and ISE. IPS, individual placement and support; ISE, integrated supported employment; WSST, work-related social skills training.
Outcomes
The outcome measures of the study included the vocational aspects (i.e. employment rate, job tenure, salary, and number of job terminations) and non-vocational aspects. The following psychometrically valid instruments were used.
Medical history, work history, and demographic data
This is a specially designed demographic data sheet [20] and was pilot-tested for psychiatric patients. Medical history referred to previous admission to medical hospitals and work history referred to whether the participants had been employed for ≥3 months.
Primary outcomes
Employment Outcome Checklist. This assesses employment outcome of participants including number of jobs obtained, number of hours per week worked, and salary received from each job that was obtained [20].
Chinese Job Termination Checklist. This was used to collect information regarding job termination [21].
Secondary outcomes
21-item Chinese Job Stress Coping Scale. This was used to assess stress coping strategies of participants in the workplace.
Personal Well-being Index. This is an 11-item scale ranging from ‘0’ (extremely dissatisfied) to ‘10’ (extremely satisfied) and was used to assess subjective well-being [22].
Chinese General Self-efficacy Scale. This is a 10-item scale tapping how confident the respondents were regarding their abilities to deal with novel or demanding situations [23].
Data collection
Written consent from each participant was obtained before data collection began. Counting from the commencement of the 3 month initial service provision, the programme lasted for 39 months for the ISE and IPS groups. The TVR group received traditional vocational service for 15 months. Assessments was conducted before and after the completion of the 3 month service, and follow up was done at 7 months, 11 months, 15 months, 21 months, 27 months, 33 months and 39 months using the aforementioned instruments by an independent, trained, and blind assessor who was a registered occupational therapist. Data for the TVR group, however, was available only up to 15 months. We did not collect further follow-up data among the TVR group due to the limited improvement in vocational and/or non-vocational outcomes. The differences between the TVR group and the two supported employment protocols were determined using follow-up data up to 15 months.
Training of service team and quality control
Three registered occupational therapists were employed as employment specialists to implement either the ISE or IPS protocol. Social workers were also involved in the ISE and IPS groups, and they referred cases to the study. Prior training for the employment specialists was provided by Professor Gary Bond, Dr Robert Drake, Ms Debbie Becker, and the first author. The 15-item supported employment fidelity scale was adopted to ensure the quality of our service [24,25]. The first author met with the employment specialists at 1, 4, 9, 15, 21, 27 and 33 months of the study and used the scale to check the adherence of the two protocols to the principles of supported employment. The score of IPS ranged from 66 to 68 out of 75 (88–91%) and ISE ranged from 65 to 68 out of 75 (87–91%). Both protocols demonstrated good fidelity to supported employment implementation. For the TVR group, services were provided by the staff members of service centres providing sheltered vocational training in the community. Employment specialists met with the participants mainly in their workplace or restaurants close to their workplace. If face-to-face meetings could not be arranged, discussion took place via phone calls.
Data analysis
Intent-to-treat analyses of the employment rates were conducted on the entire randomized sample (n = 189). Because the aim was to compare the long-term outcomes of IPS and ISE, only these two groups were followed up at 21, 27, 33 and 39 months. The demographic variables were compared using χ2 statistics or one-way ANOVA among the participants of the three groups. Independent t-test was used to compare the number of contacts for the ISE and IPS groups by the employment specialists. Comparisons of the overall job nature, and the programme attrition rates and the cumulative employment rates among the three groups were done using χ2 statistics. Post-hoc test for employment rates was done using exact logistic regression. We defined success in competitive employment as having continuously worked in the job for ≥2 months for at least 20 h per week. Repeated-measures ANOVA with post-hoc analysis was used to determine whether there was significant difference in job tenure and non-vocational outcomes among the groups. Job tenure was regarded as the longest duration in number of weeks that participants worked in the same job during the study.
In terms of salary and number of job terminations, comparisons were done only between the ISE and IPS groups because only a few participants in the TVR groups were employed. As job termination was recorded separately in each follow up period, missing data of job termination was prominent when the participants were unemployed or did not have job termination during the follow-up assessments. The missing data of salary was prominent when the participants were unemployed during the follow-up assessments. Thus, χ2 statistics was used to determine whether IPS or ISE participants experienced more work-place interpersonal difficulties that resulted in job termination.
Results
Between 2003 and 2006 we recruited 189 participants: 58 were randomly assigned to the ISE group, 65 to the IPS group, and 66 to the TVR group. Figure 2 summarizes the randomized trial process.
Trial procedure. IPS, individual placement and support; ISE, integrated supported employment; TVR, traditional vocational rehabilitation.
Comparison of intensity of the two interventions
From the beginning of the service to 15 months and from 16 months to 39 months of the study, no significant differences were found in telephone (t = −1.34, p = 0.19) or face-to-face (t = −0.89, p = 0.38) contacts between the ISE and IPS groups.
Programme attrition rates
The ISE and IPS participants were followed up at 7, 11, 15, 21, 27, 33 and 39 months after they had joined the programme (seven intervals), whereas the TVR participants were followed up only at 7, 11 and 15 months (three intervals). Forty-four ISE participants (75.9%) and 41 IPS participants (63.1%) completed the follow-up assessment at 39 months. Fifty-four TVR participants (81.8%) completed the 15 month follow-up assessment. The programme attrition rates among the three groups were not significantly different.
Employment rates
During the 39 month follow up, 82.8% of ISE participants (n = 48) and 61.5% of IPS participants (n = 40) successfully obtained competitive employment. Only 6.1% of TVR participants (n = 4) were able to obtain competitive employment before 15 month follow up. Significant differences in employment rates were found between the three groups at the 7 month, 11 month and 15 month follow ups. Table 2 summarizes the employment rates of the ISE, IPS and TVR participants throughout the study period.
Employment rate vs group and follow-up interval
IPS, individual placement and support; ISE, integrated supported employment; TVR, traditional vocational rehabilitation. ∗p < 0.05; ∗∗∗p < 0.007 (Bonferroni adjustment: p = 0.05/7)
Job characteristics
Twenty-seven ISE participants (57.4%), 21 IPS participants (55.3%) and three TVR participants (100%) worked full time, whereas 20 ISE participants (42.6%) and 17 IPS participants (44.7%) worked part time. A few missing data for the ISE (n = 1), IPS (n = 2) and TVR (n = 1) groups were noted during long-term follow up. No significant difference in job nature among the three groups was found (χ2 = 2.294, df = 1, p = 0.318). The three groups did not differ in type of jobs. Most of the participants worked at entry-level jobs such as office assistant, cleansing worker, security guard, shops assistant, and delivery worker.
Job tenure
The ISE participants worked longer in a job than the IPS (p < 0.001 to p = 0.008) and TVR (p < 0.001) participants at the 7 month, 11 month and 15 month follow ups. Significant difference in job tenure was found among the three groups (F = 9.526, df = 4,316, p < 0.001).
As one the main objective of the present study, the long-term follow-up data from 21 months to 39 months for the ISE and IPS groups were compared with their baseline data. Overall significance (F = 3.394, df = 4,100, p = 0.012) on time × group interaction between the ISE and IPS groups was found. Post-hoc comparison indicated that the job tenure of the ISE group was significantly longer than the IPS group at 21 month follow up (p < 0.011). Although no significant difference was noted between the two groups for the remaining follow-up intervals, the trend for the ISE participants to have longer job tenure than the IPS participants at 27, 33 and 39 month follow up was still observed.
Salary
The findings suggested that the two groups did not differ in salary at any of the follow-up intervals.
Job termination
The average number of unwanted job terminations for IPS participants was significantly higher than for the ISE participants at 7 months (t = −0.239, df = 38.108, p = 0.031), 11 months (t = −3.107, df = 32.669, p = 0.004) and 15 months (t = −3.737, df = 37.444, p = 0.001). Although there was no significant difference at 21 month, 27 month, 33 month, or 39 month follow ups, the number of job terminations for the IPS groups was still more prominent. Only 8.7% of ISE participants had faced interpersonal problems in quitting their jobs, compared with 24.2% of IPS participants. χ2 tests showed that IPS participants experienced more workplace interpersonal difficulties in resulting job termination (χ2 = 11.654, df = 1, p = 0.001).
Non-vocational outcomes: job stress coping, personal well-being and general self-efficacy
Regarding the average scores of the Chinese Job Stress Coping Scale (CJSC), the Personal Well-being Index (PWI), and the Chinese General Self-efficacy Scale (CGSS) among the ISE, IPS and TVR groups at 7 month, 11 month and 15 month follow ups, ISE participants (p < 0.001) and IPS participants (p = 0.004) obtained higher CGSS average score than the TVR participants at 15 month follow up. No significant difference, however, was found in the average scores of CJSC, PWI and CGSS between the ISE and IPS groups for the long-term follow-up intervals. Table 3 summarizes the relevant information regarding the average scores of CJSC, PWI and CGSS.
Average scores of PWI, CJSC and CGSS vs group and follow-up interval
CGSS, Chinese General Self-efficacy Scale; IPS, individual placement and support; ISE, integrated supported employment; CJSC, Chinese Job Stress Coping Scale; PWI, Personal Well-Being Index; TVR, traditional vocational rehabilitation. Effect sizes (partial η2 index with observed power) of the average scores of CJSC, PWI and CGSS were 0.006 (17.4%), 0.012 (31.0%), 0.050 (92.9%), respectively. ∗∗∗Significant difference (p < 0.01).
Discussion
Numerous empirical reports have demonstrated the enhanced vocational outcomes of IPS among participants with severe mental illness in the USA and other Western countries [9,10]. Based on the review by Bond of nine randomized controlled trials of the IPS programme, 56% of participants on average obtained competitive employment [19]. The employment rate of the present IPS participants (61.5%) is similar to that found in the review and in the Wong et al. study in Hong Kong [26], which further suggests the fidelity of implementation of the present study and the superiority of IPS over conventional vocational programmes. We obtained an employment rate of 82.8% among the ISE participants up to 39 months, which was significantly higher than that of the IPS participants (61.5%). This is the first piece of evidence to show that the long-term employment rate for the ISE programme was higher than that for the IPS prototype after 39 months of service. The long-term superiority of the employment rate of the ISE participants is probably due to the efforts of the employment specialists to upgrade job interview skills of the participants during the follow up. In addition, the impressive results may be due to the fact that employers put more weight on social skills in Hong Kong and Chinese culture when they make hiring decisions [25]. In this case, the job interview skills of the participants would have played a crucial role in influencing the decision to hire among the employers involved in the present study. Our hypothesis that the ISE programme would result in a higher employment rate and have a long-lasting affect has been supported by the present results.
It was found that the job tenure of ISE participants averaged 47 weeks, which was 10 weeks longer than that of the IPS participants towards the end of the 39 months. The hypothesis that ISE outperforms IPS in terms of its ability to enhance job maintenance is also supported. The major difference between ISE and IPS is that the social competence of ISE participants is enhanced by a skills training module, which is then maintained by skills generalization throughout the follow-up process [27]. Meanwhile, the employment specialists worked together with the participants to set and upgrade individualized behavioural goals so as to help the participants solve interpersonal conflicts. When participants handled interpersonal difficulties successfully, reinforcement was given as encouragement. Although IPS participants received similar services during the follow-up period, they lacked the skills training element, which was an important difference between the IPS and ISE protocols.
Job termination data provided further support on the advantages of using social skills training with supported employment. Many problems leading to job termination pertained to interpersonal difficulties [28]. Significant difference in term of interpersonal difficulty as the reason for job termination between IPS and ISE participants was achieved in the long-term follow up. In other words, ISE participants were less perplexed by interpersonal conflicts and were better able to resolve them. As to the number of job terminations, however, no significant difference was found between ISE and IPS participants. This suggests that social competence at the workplace may not be the only factor that affects number of job termination. Other impairments such as neurocognitive and social cognition also play a significant role in this aspect [29,30]. Further studies are therefore needed to gain better understanding on this issue.
For other vocational outcomes including job title, job nature, and salary, no significant results were found throughout the study. The explanation may be that the present participants in IPS and ISE shared similar socioeconomic characteristics (i.e. low educational level, lack of professional qualifications etc.), which are significant predictors of these aspects.
In terms of the non-vocational outcomes, the present findings suggested that both ISE and IPS participants obtained significantly higher scores on general self-efficacy than the TVR participants at 15 month follow up. Research evidence indicates that vocational outcomes are positively associated with self-efficacy [1,31], because the act of working creates and facilitates changes in individuals’ self-efficacy [32]. It is surprising, however, to discover a non-significant difference in the self-efficacy between these two groups. We think that the lack of significance is mainly due to the small effect size between the two conditions. If larger samples are used in future studies of a similar kind, a significant difference may be observed.
Despite encouraging results from the present study, there are limitations that need to be considered. First, the small and heterogeneous sample used is a problem for generalization. Second, the severely impaired individuals were excluded according to the inclusion criteria. This limits generalization of the findings to this subgroup. Third, the methodology would have been more vigorous if we evaluated fidelity to WSST by therapists and acquisition of skills by participants, and use of those skills in the workplace for the ISE condition. Fourth, data would be more valid if there were more local validation data of some of the instruments. Fifth, the results would have been more compelling if improvement in the social skills of the participants of the ISE condition had been demonstrated by a validated social skills assessment. Sixth, the educational level of the participants was not controlled for in the data analysis. Although it has previously been shown that educational level is a significant predictor of employment outcomes [33], prior systematic review on the predictors of employment outcomes reported controversial results on whether educational level is a significant predictor of employment among individuals with severe mental illness (Tsang HWH, Leung Ay, Chung RCK, Bell MD, Cheung WM: unpublished data, 2009). Further studies to clarify the relationship between demographic variables such as educational level and vocational outcomes are needed. Finally, the possible effect of allegiance [34,35] should be noted given that the employment specialists were not blind to the treatment assignment of the subjects.
Conclusion
The long-term effectiveness of the ISE programme in enhancing employment rates and job tenures among individuals with severe mental illness has been well demonstrated in the present randomized controlled trial. To our knowledge, this is the first of its kind in the world. In contrast, the hypothesis on non-vocational outcomes was not supported by the findings. Further studies with larger sample sizes and better control of confounding variables need to be conducted in the future. Because the ISE programme is a newly developed service protocol, there is a need to promote it among various community-based mental health settings. Nevertheless, clinicians should be encouraged to adopt the ISE model as the approach for vocational rehabilitation among people with severe mental illness, thereby possibly further strengthening the benefits of IPS.
Footnotes
Acknowledgements
This study was funded by a grant from the Health Services Research Committee (HHSRF Project No. 03040031) of the Hong Kong Government. We are grateful to the Richmond Fellowship of Hong Kong, the Baptist Oi Kwan Social Service, and the Occupational Therapy Depart ment of Kwai Chung Hospital from which we recruited the subjects. Thanks also go to Professor Gary Bond, Professor Bob Drake, Professor Robert R. Lieberman, and Ms Debbie Becker for their advice and support.