Abstract
Therapeutic homework activities have largely been used as a component of cognitive behavioural therapy (CBT) for anxiety and mood disorders, with a positive relationship demonstrated between the amount of homework completed and patient outcomes [1]. Similarly, a positive relationship has also been established between the quality of homework completed and outcome [2, 3]. With homework being considered a common factor within psychotherapies [4], further research is required to examine the relationship between homework and outcome for other clinical problems and different therapeutic approaches.
Homework is increasingly being promoted for use with severe mental illness (SMI) [5–8], but to our knowledge only two studies have examined the relationship between homework and outcomes for this group. The first study involved 29 patients engaged in individual CBT for psychosis [9] and the second, 32 older persons diagnosed with schizophrenia who were engaged in group CBT [10]. Results from these studies were mixed, with preliminary conclusions suggesting that homework may be important in the development of general skill acquisition rather than in promoting cognitive insight. An area that has not been explored is the relationship between homework and outcomes when homework is administered by mental health case managers.
Homework has been suggested as a useful clinical tool for case managers [8], and clinician surveys indicate that case managers regularly use homework [11]. Of concern is that only 15% of case managers report implementing homework in a systematic fashion [11]. Systematic homework administration involves providing clear instructions to patients regarding the frequency, duration, location of where the task should be completed and providing a written note of the homework assignment for the patient [12, 13]. The use of a systematic approach has been suggested as a means to facilitate patient homework adherence and improve the quality of the homework completed. It has also been recommended for individuals with SMI due to the motivational and memory problems commonly associated with these disorders. To date research has not examined the relationship between case managers’ use of systematic homework implementation and clinical outcomes.
The current study is correlational and examines the use of homework by mental health clinicians working within a recovery-orientated case management approach. Case managers were instructed to develop homework assignments that targeted areas of unmet need and addressed the patient's recovery goals. As such, it was expected that the use of homework would broadly produce improvements across both functional and recovery domains. It was hypothesized that (i) the total number of homework assignment sheets administered for each individual would be positively related to improved treatment outcomes; and (ii) how well the homework was completed (i.e. compliance with the assigned task) would also be positively related to improved treatment outcomes.
Method
Participants
Subject characteristics
†The most recent clinical diagnosis was taken from the person's file. The person's treating psychiatrist typically made this diagnosis.
Measures
The Health of the Nation Outcomes Scale (HoNOS) is a 12-item clinician-rated measure of general health and social functioning [16]. The internal reliability of the scale in the current study was acceptable (Cronbach α = 0.72).
The Kessler 10 (K10) is a patient-rated 10-item screening battery for non-specific psychological symptom distress [17]. Within the present study the K10 demonstrated good internal reliability (Cronbach α = 0.89).
The Recovery Assessment Scale (RAS) is a patient-rated measure that examines aspects of psychological recovery, with a special focus on hope, meaning and self-determination [18]. Within the current study the RAS demonstrated good internal reliability (α = 0.85).
The Mental Health Recovery Measure (MHR) is also a patient-rated measure of an individual's psychological recovery process [19]. Like the RAS, the MHR is based on the ‘recovery’ construct. In the current study the MHR demonstrated good internal reliability (Cronbach α = 0.89).
The homework assignment pad was developed for the present study and is based on the empirical recommendations of Shelton and Levy [13]. It included sections to (i) describe the homework assignment; (ii) record the treatment goal targeted by the homework; and specify (iii) when, (iv) where and (v) how long the task should take; it also included (vi) a section for patients to provide a percentage rating of their confidence to complete the task. A similar version of the administration component of the homework sheet can be found in the Kazantzis et al. study [20]. The sheet used in the present study, however, was designed to record only one homework assignment per page. To promote homework review, the first two items of the Homework Rating Scale were also included on the homework assignment sheet [21]. The items were ‘how much homework did you do since the last session?’ rated from 0 (none) to 4 (all); and ‘how well did you do the homework?’ rated from 0 (not at all) to 4 (extremely). Both patients and therapists provided independent ratings of homework completion at the following session. See Figure 1 for an example of the homework assignment pad.
Homework assignment sheet.
Procedure
Case managers were trained in the collaborative recovery model (CRM) [14]. As a component of the training, 2 h of training was provided on systematic homework implementation. Case managers recruited willing patients from their current caseload. A record of total numbers of patients approached by case managers was not recorded. Case managers were required to use the CRM in their work with patients [14, 22].
The homework aspect of the CRM required case managers to (i) administer homework in a systematic fashion by using the assignment pad; (ii) administer homework on at least a fortnightly basis; (iii) link the homework to the individual's recovery goals; (iv) obtain a percentage rating from the patient regarding their confidence to complete the assignment (based on the procedures outlined by Kazantzis et al. [23] case managers were instructed to renegotiate the task until the patient rated their confidence at ≥70% to complete the homework); (v) provide a copy of the completed homework sheet to the patient; and (vi) review the homework assignment at the following homework session by completing the Homework Rating Scale. Patients made ratings of the homework completion at the beginning of the session. Case managers were instructed to make their homework ratings following the completion of the session, using information collected during the appointment. Consequently, case manager ratings were in part based on the ratings provided by the patient. Case managers were not required to discuss their ratings with the patient.
Each participant was informed that participation in the study was voluntary. The research protocols received ethical review and approval from the University of Wollongong Human Ethics Committee.
Data analysis
Outcome measures were completed at intake, and at 3, 6, 9 and 12 months. A mean total outcome score was calculated for each of the outcome measures and was used in the analysis. The total scores for both the functional (HoNOS, K10) and recovery (RAS, MHR)-based measures were used as dependent variables. To examine if the homework associations were over and above the expected improvements associated with time, the 3 monthly measurement periods (time) were entered as an independent variable. To examine the relationship between homework use and outcome, the total number of homework sheets assigned at each 3 month period (total homework assigned) and ratings of the quality of homework completion (how well) were also used as independent variables.
A two-level hierarchical model for repeated measures was used. Individual patients represented the level 2 unit (between-individual variation), where we entered the independent variables, which may predict the variation within each individual's outcome scores. Repeated measurement on each outcome measure represented the level 1 unit (within-individual variation). A series of multilevel analyses were conducted to examine the extent to which the variation in each of the dependent variables could be predicted by the independent variables. Each dependent variable was analysed separately. Both fixed and random effects were examined. In the current study, examination of the random effects indicates the degree to which the effect of time, total homework assigned and how well differ across individuals.
Results
The average number of homework assignments administered to each person was 8.98 (SD = 17.25). A Kruskal–Wallis ANOVA indicated that there was a statistically significant difference between the average number of homework sheets administered to patients from Queensland (mean = 12.56, mean rank = 77.78), Victoria (mean = 6.7, mean rank = 72.29) and New South Wales (mean = 4.09, mean rank = 47.40; H = 20.29, df = 2, n = 129, p = 0.00). Post-hoc analysis indicated a significant difference between Queensland (mean rank = 45.43, n = 53) and New South Wales (mean rank = 73.84, n = 69; U = 977.00, z = − 4.42, p = 0.00). A Mann–Whitney U-test indicated a significant difference between the average number of homework assignments administered to patients within public mental health services (mean = 12.73, mean rank = 75.19, n = 59) and non-government services (mean = 5.60, mean rank = 56.42, n = 70; U = 1464.00, z = − 2.86, p < 0.01).
Both the case manager and the patient provided independent ratings of how much and how well the homework assignment was completed. Over the 12 month period, case managers rated the average amount of homework completed as mean = 2.52 (SD = 1.06) and the average quality of homework completion as 2.38 (SD = 0.07) on the 4 point rating scale. Similarly, patients rated the amount of homework completed as mean = 2.50 (SD = 1.01) and the quality of homework completed as mean = 2.35 (SD = 0.95). There were no statistically significant differences between patient and case manager ratings of how much or how well the homework was completed (both p > 0.05). As such, case manager ratings were used in the following analysis, but we found that using the patient ratings also produced similar results.
Does total homework assigned or how well homework was completed predict individual improvement?
HoNOS
Multilevel repeated measurement model of HoNOS (n = 129; regression coefficients (SE))
AGLS, arbitrary–distribution generalized least squares; HoNOS, Health of the Nation Outcome Scale. ∗p ≤ 0.05
K10
Multilevel repeated measurement model of the Kessler 10 (n = 129; regression coefficients (SE))
AGLS, arbitrary–distribution generalized least squares. ∗∗ p ≤ 0.05.
RAS and MHR
The same procedure was used to examine the relationship between the quality of homework completed and the recovery measures (RAS and MHR), although there were no significant effects.
Although the current study focuses on those patients who were given homework (n = 129), during our analysis we also examined the relationship between total number of homework sheets assigned and outcomes for the total sample (n = 242). Using hierarchical linear modelling a statistically significant relationship was found between total homework assigned and improvement on the HoNOS scale (p < 0.05). Statistically significant differences were not found on the K10 or the other recovery measures.
Discussion
The total number of homework sheets assigned was positively related to improved treatment results on the functional outcome measures. Ratings of how well patients completed their respective homework assignments were also found to predict total HoNOS scores. Although not significant at the 0.05 level, the relationship between the K10 and how well the homework was completed also approached significance. There were no random effects, indicating that these homework effects were consistent across patients and that effects are equally strong across patients. It is important to note that these relationships are correlational in nature and cannot confirm causality. The results are consistent, however, with both theoretical formulations [13, 24] and empirical evidence that suggests that the use of homework is positively related to improved treatment outcomes [1].
The total number of homework sheets assigned to each individual and how well the homework was completed did not predict scores on the recovery measures. It is not clear why we found effects on the more functional measures and not on the recovery-based scales. It is possible that the relatively new measures of recovery do not provide adequate sensitivity to measure change [25], a more extended period of time is required for recovery-oriented change to occur, or the homework assignments used by case managers may not have sufficiently addressed the recovery domains. With recovery being increasingly promoted within mental health policy and practice, further research is required with longer follow-up periods to examine this relationship. Additionally, the current study relied on global measures of patient outcome across multiple domains. Future research might benefit from using more specific measures (e.g. subscale scores) to identify areas of patient change. This would be particularly useful when examining the impact of specific types of homework.
Limitations
Results should be interpreted with caution due to a number of limitations. Recruitment of both case manager and patient participants was on a voluntary basis, and as such represented a convenience sample. Only 53% of individuals involved in the broader AIMhi study were assigned written homework using the research protocol forms. The limited use of homework appears to be associated with a failure of case managers to implement homework, rather than resistance from individual patients [26]. As a result, it is not clear to what extent these results are transferable to the wider population of individuals with SMI. When homework was assigned, an average of only nine homework sheets was administered to each individual during the 12 month study. This was substantially below what was recommended as part of the research protocol. It is possible that homework assignments were completed by patients but were not recorded for research purposes. For example, homework may have been verbally discussed, but not recorded on the homework assignment sheet (something that case managers reported). Alternatively, patients may have spontaneously completed between-session activities of their own accord because they viewed it as helpful to their own recovery. The basis for clinician decisions regarding the frequency of homework administration for individual patients is not clear. Survey data from case managers at the conclusion of the project, however, indicated that the most common reason they gave for not using the homework assignment pad was because they forgot to administer homework or neglected to bring the pad to the appointment. Other reasons included perceived patient resistance, concerns about overloading the patient and a lack of time. There were differences between States regarding the amount of homework assigned to individual patients. Similarly, on average, patients attending public mental health services were given more homework assignments than patients attending non-government services. This may highlight differences in funding and/or staffing levels between States and individual services that reduce available time to implement homework systematically. It may also reflect differences in the characteristics of staff in public versus non-government organizations where there is likely to be a higher proportion of university-trained staff in the public sector. Future research with larger samples may be able to further tease out the factors that may contribute to the amount of homework assigned.
With such a low ‘dose’ of systematic homework and the possibility that homework was completed by patients but not recorded for research purposes, it is notable that the total number of homework sheets assigned was still significantly related to the functional outcome measures. It is important that further research identify strategies to encourage case managers to use systematic homework administration procedures on a more regular basis.
Conclusions
The present study provides preliminary support for the use of systematic homework administration for individuals with SMI. A positive relationship was identified between the total number of homework assignments administered using the homework assignment pad and outcome on the HoNOS and K10. Similarly, how well the homework task was completed by the patient also predicted positive outcomes on the HoNOS. Although the correlations are small for the quantity and quality of homework completed, it is promising to find effects in a group with such high needs and with such a relatively low dose of homework.
Footnotes
Acknowledgements
Preparation of this article was supported by the Australian Integrated Mental Health Imitative (AIMhi) – High Support Stream funded by the National Health Medical Research Committee (NHMRC) Health Partnerships grant number 219327. Support for the project was provided by Dr Russel Familar, Dr Peter Caputi, Dr David Steele, Assoc. Professor Robert King and Professor David Kavanagh.