Abstract
Estimates based on census and epidemiological data and rates of parents accessing adult mental health services indicate that at least 1 million children in Australia have a parent with a mental illness [1]. Research indicates that children of parents with a mental illness (‘copmi’) are at greater risk of poorer outcomes than their peers [2], with higher rates of mental illness [3] and poorer development in behavioural [4], [5], social [6], and academic [7], [8] domains. In addition, copmi often provide caregiving support to their parent [9]. (The abbreviation ‘copmi’ is used for ease of expression, but it should not be confused with the organization COPMI that has the same acronym.) Young people in such caregiving roles are more likely to experience poorer outcomes than non-caregiving peers [10]. The observation that not all copmi report poor outcomes led to the development of a resilience model for this population, whereby the potential harmful effects of risk factors are removed or mitigated by the influence of protective factors [11], [12]. The identification of specific risk and protective factors related to characteristics of the individual, the parent's illness, and the family and social environments has, in turn, guided the development of interventions for copmi.
Consistent with the resilience framework, interventions developed for copmi have mostly utilized a model of preventative and early intervention, focused on the modification of risk and protective factors [13], [14]. These interventions typically use a group format and target risk and protective factors such as isolation, independence and various psychosocial skills [13], [14]. A review of studies that have evaluated the effectiveness of interventions for copmi showed some support for these interventions in achieving increases in mental illness knowledge [14]. The authors concluded, however, that there was insufficient evidence supporting the effectiveness of these interventions.
Common limitations of research into the effectiveness of interventions for copmi include lack of control groups, use of inappropriate outcome measures, lack of follow up, neglect of young caregiving experiences, and small sample size. Within Australia, few of the interventions for copmi have been formally evaluated [13], and the existing evaluations demonstrate limitations as described here. Within Australia there are no published evaluations of interventions for copmi that utilize a comparison or control group.
The purpose of the present study was to evaluate the effectiveness of an intervention for copmi currently used in Australia called the Koping Adolescent Group Program (KAP). KAP is a peer support intervention for copmi aged 12–18 years. Given that KAP is similar to many other Australian interventions that address parental mental illness, an evaluation of the effectiveness of this program will contribute to the evidence base for this group of interventions. KAP is provided by the Child and Youth Mental Health Service, within the Royal Children's Hospital and Health Service District of Queensland Health.
KAP adheres to a resilience framework, and consequently is designed to improve adjustment outcomes for copmi by modifying risk factors such as social isolation and inadequate mental health literacy and by strengthening protective factors such as an adequate repertoire of coping skills and intact peer relationships. We hypothesized that compared to control group participants, youth who participated in the KAP intervention would report more beneficial changes in risk and protective factors targeted by KAP, specifically, increased mental health literacy, social connectedness, and repertoire of coping skills. We also predicted that compared to a control group, KAP participants would report greater improvements in psychological and emotional adjustment. Finally, the impact of KAP on the caregiving experiences of copmi was also explored. Given that the intervention did not target caregiving and that prior studies have not examined the impact of interventions on the caregiving experiences of copmi, no predictions were made regarding the impact of KAP on caregiving outcomes.
Method
The effectiveness of KAP was examined using a treatment and waitlist-control group with a pre- and post-treatment, and 8 week follow-up design. Three groups of dependent variables were examined: intervention targets (mental health literacy, connectedness, coping strategies), adjustment outcomes (depressive symptomatology, life satisfaction, prosocial behaviour, emotional/behavioural difficulties), and caregiving experiences. This study received ethics approval from both the University of Queensland and the Royal Children's Hospital and Health Service District Ethics Committees.
Participants and recruitment
Participants in the treatment group were recruited via KAP. Referral to KAP can be made by any individual or organization, with the majority of referrals coming from Child and Youth Mental Health Services (57.8%), and the remainder coming from other youth services (15.5%), a parent or relative (9.8%), schools (8.5%), Adult Mental Health Services (5.6%), and the Department of Child Safety (2.8%). Inclusion criteria for KAP include: age 12–18 years and presence of a parent with a mental illness. Young people referred to each of the six KAP groups held in 2006 participated in the study. Control group participants were recruited from the KAP waitlist. Written parental consent was obtained for all participants.
There were 44 participants; 27 in the treatment and 17 in the control groups. All 27 treatment group participants completed the pretreatment measures, 25 (93%) completed post-treatment assessment, and 19 (70%) completed the follow up. All 17 control group participants completed the pretreatment and 15 (88%) completed the post-treatment assessments. The control condition did not include a follow-up assessment. Characteristics of participants are summarized in Table 1. Participants were aged 12–17 years (mean = 13 years, SD = 1.58), and 61% were female (39% male). Also included in Table 1 is a summary of the parental mental disorders, with the most common being depression (43%), psychosis (including schizophrenia; 32%), and bipolar disorder (25%).
Participant characteristics
Measures
Mental health literacy
Mental health literacy was assessed in two areas: knowledge of mental illness, and awareness of parent's mental illness. Knowledge was assessed by five items. Three open-ended items assessed knowledge about mental illness signs and symptoms. Responses to each item were scored from 0 to 4 according to the numbers of correct facts identified (0 = no knowledge demonstrated, 1 = one correct fact, 2 = two facts, 3 = three facts, 4 = four+ facts). The scoring of the open-ended items was completed according to the criteria set by the first author and independently re-scored by another clinician to ensure scoring consistency and accuracy. The fourth knowledge item presented the names of four mental illnesses and required participants to indicate those illnesses they recognized. Participants scored 1 point for each endorsed mental illness. The fifth knowledge item referred to the prior four mental illnesses and asked respondents ‘What do they mean to you?’. Responses indicating that the words described specific mental illnesses scored 1 point. Scores for each of the five items were summed to give an overall knowledge score (range = 0–9).
Awareness of mental illness was assessed by four items that required participants to respond yes or no to questions related to their awareness of their parent's mental illness (e.g. ‘Have you noticed how your parent/s’ illness affects them?’). Scores for each item (yes = 1, no = 0) were summed to give an overall awareness score (range = 0–4).
Connectedness
The degree to which participants felt connected to others was measured by the 20-item Social Connectedness Scale [15]. Respondents rate their level of agreement with each item on a 6 point scale (1 = strongly disagree, 6 = strongly agree).
Coping strategies
Coping strategies were measured by the Responses to Stress Questionnaire–Family Stress Version [16]. This 57-item scale was designed for adolescents and consists of five subscales: primary control engagement, secondary control engagement, disengagement, involuntary engagement, and involuntary disengagement. Respondents rate on a 4 point scale how often they utilize each coping strategy when experiencing family stress (1 = not at all, 4 = a lot).
Depressive symptomatology
The 10-item standardized Children's Depression Inventory–Short Form requires respondents to select one of three options for each item regarding how they have been feeling over the previous 2 weeks [17]. Although the scale is a measure of depressive symptomatology, the term ‘depression’ is used from hereon.
Life satisfaction
The five-item Satisfaction with Life Scale [18] was used to measure global life satisfaction. Respondents rate how much they agree with each statement on a 7 point scale (1 = strongly disagree, 7 = strongly agree).
Strength/difficulties
The 25-item standardized Strengths and Difficulties Questionnaire assesses adolescents’ behaviour across five subscales: emotional symptoms, conduct problems, hyperactivity/inattention, peer problems and prosocial behaviour [19]. In addition, the measure provides a total difficulties score. Respondents rate each statement with respect to how true it is for them on a 3 point scale (0 = not at all, 2 = certainly true).
Caregiving experiences
The Young Caregiver of Parents Inventory (YCOPI) [10] was used to assess participants’ caregiving experiences related to caring for their parent with mental illness. The YCOPI has two parts: part A inquires about the respondent's contributions to the family and parental tasks and functions, whereas part B focuses on the respondent's caregiving to the disabled/ill parent. The YCOPI has eight reliable and valid subscales: five in part A (caregiving responsibilities, perceived maturity, worry about parents, activity restrictions, isolation) and three in part B (caregiving compulsion, caregiving discomfort, caregiving confidence). Items consist of statements that are rated on a 5 point scale of agreement (1 = strongly disagree, 4 = strongly agree).
Participant satisfaction
Satisfaction with the KAP intervention was measured using six items (e.g. ‘The group helped me to feel like I am not alone’) that were rated on a 4 point scale (1 = strongly agree, 4 = strongly disagree), as well as three open-ended questions that enquired about what participants liked least and most, and what else they would have liked included in the program.
Intervention
KAP goals are to increase mental health literacy, connectedness with peers, and repertoire of coping skills. The KAP content was reviewed by the first author and also via consultation with previous facilitators and KAP participants to ensure that content was developmentally appropriate and relevant to the goals of intervention, and that the proportion of activities devoted to each goal was balanced. The revised KAP intervention was successfully piloted with a group of six participants. KAP consists of three 6 h group sessions held fortnightly. Intervention strategies included psychoeducation, coping skills training, peer support, group discussion, quizzes, and other activities (e.g. creative art and craft, videos, games). Each session is devoted to a particular theme: session 1 ‘Connecting and learning’, focused on building rapport between group members and provided psychoeducation about mental illness; session 2 ‘Stress in the family’, explored participants’ experiences of having a parent with a mental illness; session 3 ‘A brighter future’, concentrated on exploring ways of coping with stress. KAP was implemented according to a facilitator manual. Each KAP group consisted of two facilitators who were mental health clinicians, and between four and six participants. The Koping Program Coordinator acted as the primary facilitator for each of the six KAP groups conducted during the treatment phase, ensuring continuity of the implementation of the program protocol.
Procedure
Treatment group participants completed pretreatment measures at the beginning of the first KAP session. Post-treatment measures were completed 4 weeks later at the end of the final session. Follow up occurred 8 weeks after the final session (mean = 19 weeks, SD = 11.19; range = 8–42 weeks). Control group participants completed questionnaires at two time points equivalent to the pre- and post-treatment phases. Following completion of the final assessment, control group participants attended the next available KAP intervention.
Results
Data preparation
Missing data points were estimated using the participant's mean score for the relevant scale, and were not estimated if the percentage of missing items for a scale was >50%. Missing scale values were imputed using linear regression for participants who did not complete an assessment point. Post-treatment values were predicted separately for treatment and control groups using the best fit model of pretreatment values, age, and gender. Follow-up values were predicted for the treatment group using pretreatment values, post-treatment values, age, and gender. When the follow-up prediction equation used post-treatment values, follow-up values were not imputed for participant's whose post-treatment values had already been imputed. A group average was used to impute data when the prediction equation was not an improvement on using the group average.
Depression and Strengths/Difficulties descriptive analyses
Using relevant norms, the proportions of participants within the treatment and control groups who at pretreatment assessment scored within the clinical range for depression [17] and Strengths/Difficulties, were calculated [19] (Table 2). Using χ2 analysis, no significant differences in proportion of participants within the clinical range were found between the treatment and control groups for depression or Strengths/Difficulties.
Percentage of participants within clinical cut-off range
Pre- to post-treatment group comparisons
Preliminary analyses showed that the treatment and control groups did not differ at pretreatment assessment on demographics and all dependent variables. With respect to parental diagnosis, comparisons between groups on diagnoses of mothers and fathers separately showed no differences. When parents were combined and compared across groups, however, there was a weak trend for the treatment group to report more parental psychosis diagnoses, χ2(1) = 5.05, p = 0.03. It should be noted that due to the number of comparisons and the low numbers in many of the cells these analyses may not be reliable.
Comparisons between the treatment and control groups on changes in the dependent variables from pre- to post-treatment assessment were examined using 2 (group: treatment, control)×2 (time: pre, post) mixed ANOVAs (social connectedness, depression, life satisfaction) and MANOVAs (mental health literacy, Strengths/Difficulties, YCOPI part A and part B). Significance was set at p < 0.05 given the small sample size and the need to explore all potential intervention effects because KAP had not been evaluated previously. Group means and standard deviations for dependent variables where there was a significant effect of time are summarized in Table 3 along with F values. Contrary to predictions, none of the group comparisons produced a significant group×time interaction.
Main effects of time
YCOPI-A, Young Caregiver of Parents Inventory–Part A; YCOPI-B, Young Caregiver of Parents Inventory–Part B.; †df = (1,41); ‡df = (1,42); §df = (1, 40); ¶df = (2,48); ††using Huynh-Feldt corrected values, df = (1.72,41.19); ‡‡df = (2, 52); §§df = (2, 44); ¶¶using Huynh-Feldt corrected values, df = (1.06,24.38); †††df = (2, 46).; ∗p < 0.05, ∗∗p < 0.01.
Intervention targets
Analyses performed on social connectedness, coping strategies and mental health literacy indicated that there was a significant main effect for time on mental health literacy (F(2,40) = 6.53, p = 0.004, Wilks’ λ = 0.75, η2=0.25); both groups increased significantly from pre- to post-treatment assessment on knowledge and awareness of mental illness.
Adjustment outcomes
There was a significant main effect for time on life satisfaction but not depression: life satisfaction increased from pre- to post-treatment assessment for both groups. The omnibus F-value for the MANOVA performed on the Strengths/Difficulties subscales was marginally significant (F(5,38) = 2.03, p = 0.096, Wilks’ λ = 0.79, η2=0.21); both groups reported an increase in peer problems from pre- to post-treatment assessment.
Caregiving experiences
Regarding the YCOPI-Part A, a marginally significant group×time interaction emerged on caregiving responsibilities (F(1,40) = 3.61, p = 0.065, η2=0.08). Treatment group participants reported a decrease in caregiving responsibilities from pre- to post-treatment assessment, whereas control group participants reported an increase. There was also a marginally significant main effect for time on the YCOPI-Part A subscales (F(2,36) = 2.39, p = 0.057, Wilks’ λ = 0.75, η2=0.25); both groups showed a decrease in worry about their parents from pre- to post-treatment assessment. A significant main effect for time emerged on the YCOPI-Part B subscales (F(3,40) = 2.90, p = 0.047, Wilks’ λ = 0.82, η2=0.18); both groups reported a decrease in caregiving compulsion from pre- to post-treatment assessment.
Pre-, post-, and follow-up treatment group changes
Changes on intervention targets, adjustment and caregiving experiences between pre- and post-treatment assessment, and follow up for the KAP group were investigated using a 1 (group: treatment) ×3 (time: pre, post, follow up) design. Significant effects were followed up with repeated measures t-tests.
Intervention targets
Although social connectedness did not change over time, a significant effect for time emerged on mental health literacy (F(4,21) = 4.49, p = 0.009, Wilks’ λ = 0.54, η2=0.46). Knowledge (t(26) = 3.47, p = 0.002) and mental illness awareness (t(26) = 2.33, p = 0.028) increased from pre- to post-treatment assessment, and from pretreatment assessment to follow up (knowledge t(26) = 4.27, p < 0.001; awareness t(24) = 3.32, p = 0.003).
A marginally significant effect for time emerged on coping strategies (F(10, 15) = 2.39, p = 0.062, Wilks’ λ = 0.39, η2=0.62). Use of disengagement (t(24) = −2.50, p = 0.020) and involuntary disengagement coping (t(24) = −2.70, p = 0.013) decreased from pretreatment assessment to follow up, and from post-treatment assessment to follow up (disengagement t(24) = −2.60, p = 0.016, involuntary disengagement t(24) = −2.56, p = 0.017). Use of involuntary engagement coping decreased from post-treatment assessment to follow up (t(24) = −2.88, p = 0.008).
Adjustment outcomes
A marginally significant effect for time emerged on depression (p = 0.055); depression decreased from pretreatment assessment to follow up (t(26) = −2.48, p = 0.020). Life satisfaction also evidenced a significant effect for time: life satisfaction increased from pre- to post-treatment assessment (t(26) = 3.49, p = 0.002), and from pretreatment assessment to follow up (t(24) = 3.18, p = 0.004). A significant effect for time emerged on the Strengths/Difficulties subscales (F(10,15) = 5.21, p = 0.002, Wilks’ λ = 0.22, η2=0.78). Prosocial behaviour increased from post-treatment assessment to follow up (t(24) = 3.01, p = 0.006), and increased marginally from pretreatment assessment to follow up (t(24) = 2.04, p = 0.053), whereas emotional symptoms decreased from post-treatment assessment to follow up (t(24) = −2.28, p = 0.032).
Caregiving experiences
The YCOPI-Part A subscales demonstrated a significant effect for time (F(10,13) = 3.71, p = 0.015, Wilks’ λ = 0.26, η2=0.74); scores on the worry about parents, activity restrictions, and isolation subscales decreased over time. Worry about parents decreased from pre- to post-treatment assessment (t(26) = −3.06, p = 0.005), and from pretreatment assessment to follow up (t(26) = −2.01, p = 0.055). Activity restrictions and isolation decreased from pretreatment assessment to follow up (activity restrictions, t(24) = −4.13, p < 0.001; isolation, t(25) = −4.59, p < 0.001) and from post-treatment assessment to follow up (activity restrictions, t(24) = − 2.66, p = 0.014; isolation, t(25) = − 3.79, p = 0.001). Analyses on the YCOPI-Part B subscales showed that caregiving discomfort decreased from pretreatment assessment to follow up (t(24) = −2.60, p = 0.016), and from post-treatment assessment to follow up (t(24) = −3.24, p = 0.003).
Reliable change
Clinically significant change from pre- to post-treatment assessment on intervention targets and adjustment outcomes for treatment and control group participants was examined using the Reliable Change Index (RCI), in accord with recommendations by Jacobson and Truax [20]. The RCI is an indicator of clinically significant change for an individual. RCI is calculated by comparing a participant's actual change in score to the expected spread of the distribution of change scores if no actual change had occurred. An RCI was calculated for each participant to indicate change from pre- to post-treatment assessment on each dependent variable. The percentages of participants who had shown reliable change, expressed as either increased or decreased scores, were calculated for both groups (Table 4). χ2 analyses to determine if the groups differed significantly in the proportion of participants who exhibited reliable change from pre- to post-treatment assessment could not be performed due to the large number of matrices with expected cell counts of <5.
Percentage of participants with reliable change
Intervention targets
None of the control group participants showed reliable changes in mental illness knowledge and awareness, whereas 14.8% and 3.7% of KAP participants demonstrated reliable increases on these variables, respectively. Both the treatment and control groups had similar proportions of participants who demonstrated a reliable increase in connectedness. No one in either group demonstrated a reliable decrease in connectedness.
The treatment group had a higher proportion of participants with a reliable decrease in primary control engagement coping than the control group; none of the control group participants demonstrated reliable change on this variable. Regarding reliable change on the other coping strategies (secondary control engagement, disengagement, involuntary engagement and involuntary disengagement), compared to the control group, the treatment group had a higher proportion of participants who demonstrated an increase and a lower proportion of participants who demonstrated a decrease in their use of these coping strategies.
Adjustment outcomes
A greater proportion of the treatment group demonstrated a reliable decrease in depression and a reliable increase in life satisfaction compared to control group participants, who had no reliable change on these variables.
Regarding Strength/Difficulties, only the total difficulties and prosocial behaviour scores were subjected to RCI analyses. Compared to the control group, the treatment group had a greater proportion of participants with reliable increases and decreases in total difficulties and prosocial behaviour, respectively.
Participant satisfaction
Frequencies determined from the satisfaction data showed that the majority of participants agreed/strongly agreed that they: did not feel alone (88%), felt better about themselves (92%), felt that they were better able to cope with things (96%), felt more able to talk about their feelings and problems (92%), were able to talk about how they felt during KAP (88%), and learnt more about mental illness (92%). Regarding participants’ responses to the open-ended satisfaction questions, the most frequently reported aspects of KAP that were liked most were: learning, having fun, doing activities, and peer interactions. The only program aspect of KAP least liked was completing questionnaires. Most responses indicated that there were no additional issues that participants wanted addressed during the intervention.
Discussion
Overall, results from statistical analyses did not support the hypothesis that, compared to control group participants, KAP participants would report increased mental health literacy and social connectedness, an expanded repertoire of coping skills, and greater improvements in psychological and emotional adjustment. Statistical comparisons between the two groups showed that both groups improved before to after treatment in life satisfaction and knowledge and awareness of mental illness, and that both groups also showed an increase in peer problems. The treatment group participants, however, had statistically significant increases in mental health literacy, prosocial behaviour and life satisfaction and decreases in depression and emotional symptoms from pretreatment assessment to follow up. Contrary to predictions, the treatment group also had a decrease in use of most coping strategies at follow up.
With regards to differences between the treatment and control groups in reliable clinical change from before to after treatment, results indicate that compared to the control group a higher proportion of the treatment group had increased mental health literacy and life satisfaction, and decreased depression. In contrast, compared to the control group a higher proportion of the treatment group had increased behavioural and emotional difficulties and decreased prosocial behaviour before to after treatment. The control group demonstrated a reliable decrease in reliance on most coping strategies before compared to after treatment. The statistical and RCI coping data for the treatment group suggest that reliance on most coping strategies increased for the duration of KAP but tended to decrease after the intervention.
Overall, group comparisons failed to show statistically significant intervention effects, but the RCI data suggest that compared to the control group, markedly more KAP participants had clinically significant improvements in mental health literacy, depression, and life satisfaction. The follow-up data suggested that these treatment gains were maintained 8 weeks after treatment. Consistent with these findings, the satisfaction data showed that KAP participants believed that they had benefited from the intervention and were satisfied with it.
In contrast, treatment group participants also reported an increase in emotional and behavioural difficulties during the intervention, but analyses of follow-up data suggested that some improvements in these areas occurred between post-treatment assessment and follow up. For example, results indicated that KAP participants reported an increase in emotional symptoms after treatment, and a decrease at follow up. With respect to the latter, it is possible that the intervention sensitized participants to their difficulties through helping participants to identify and discuss their emotions and difficulties. In support of this explanation treatment satisfaction data showed that participants felt that the intervention helped them to talk about their feelings and problems.
In regards to exploring the impact of KAP on participants’ caregiving experiences, statistical analyses showed that there was a marginally significant group×time interaction on caregiving responsibilities such that treatment group participants reported a decrease in caregiving responsibilities from before to after treatment, whereas control group participants reported an increase. In addition, treatment group participants reported significant decreases from pretreatment assessment to follow up in caregiving discomfort, activity restrictions and isolation. These findings suggest that KAP may serve to decrease the adverse impacts of young caregiving.
Statistically significant changes in outcomes in similar intervention studies that were also observed in the present study as clinically reliable changes were increased knowledge of mental illness [21], and reduced depressive symptoms [22]. Differences in methodology may explain why the present study failed to show the statistically significant intervention effects found in some previous studies. Most prior studies that have examined the effectiveness of intervention for copmi have not utilized a control group, and no Australian studies in this area have utilized a control group. Results of statistical comparisons in the present study suggest that both the treatment and control groups changed at a similar rate. It is possible that had prior uncontrolled intervention studies utilized a control group, support for the effectiveness of the interventions in these studies would not be as strong. Further, it should be noted that Fraser et al., in their review of interventions for copmi, concluded that there was insufficient evidence supporting the effectiveness of these interventions [14].
The lack of support for the statistical effects of the intervention in group comparisons on the pre- to post-treatment change data may be due to the small sample size and the trend for the treatment group to have more serious parental mental illnesses. In addition, it is possible that control group participants were primed for change, in that being on the KAP waitlist may have fostered participants’ curiosity about mental illness, which may have led to participants engaging in research and discussion about mental illness. Additionally, these participants may have received support from another agency.
Limitations of the study include the lack of randomization and blinding and the utilization of a waitlist control. Additionally, the lack of follow-up data on the control group means that it is possible that the control group would have demonstrated improvements at follow up similar to those shown by the treatment group. Despite these limitations, the present findings have important implications. As discussed here, findings of prior intervention studies in this field that have not used a control group may overestimate intervention effectiveness. The discrepancy between participants’ satisfaction and subjective perceptions of the intervention and the quantitative change data suggests that the KAP intervention had beneficial impacts on domains (e.g. self confidence) not assessed by the measures used in the present study. It is important that future studies obtain both qualitative and quantitative data. Given the modest support for the effectiveness of KAP it is important that this and other similar interventions continue to be revised and undergo rigorous evaluation.
Footnotes
Acknowledgements
We thank Denisse Best, Shirley Anastasi, Thy Meddick, and the Child and Youth Mental Health Service, within the Royal Children's Hospital and Health Service District of Queensland Health, for their support of this research, and the young people and families who participated in this study.