Abstract
Keywords
Depression during adolescence is a serious and relatively common disorder that impairs academic achievement, social functioning and family life [1]. Clinical trials of treatment for adolescent depression have found empirical support for the efficacy of cognitive-behavioural therapy (CBT) [2–4], antidepressant drug therapy [5–8] and interpersonal therapy [9]. Part of a thorough investigation into the efficacy of a treatment is the evaluation of the consumer satisfaction or social acceptability of the treatment. Measurement of consumer satisfaction and treatment acceptability are a form of social validation and has been defined as ‘the judgement about the treatment procedures by non-professionals, laypersons and other potential consumers of treatment’ [10], p.259]. Even a highly effective treatment will be of limited use unless it is acceptable to individuals and society [11]. Consumer satisfaction measures have been used for more than 20 years in mental health services [12].
Although it is acknowledged that frequent use ofa treatment is tacit evidence of consumer satisfaction, little research has been conducted into the consumer satisfaction with treatment for child and adolescent psychopathology [13]. From the small published works on the topic, it has been found that ratings of consumer satisfaction and social acceptability can vary between treatments[14] and a general positive response bias in reports of satisfaction levels has been recognized [12]. Furthermore, the difficulty in avoiding sampling bias because of those who do not complete the treatment phase has been widely acknowledged [15].
Of the published works in the area of child and adolescent psychopathology, few clinical trials of treatment for adolescent depression have investigated consumer satisfaction. Of the few, Wood et al. [16] measured consumer satisfaction of CBT using a nine-point Likert scale of satisfaction (0= ‘no help’ to 8= ‘helped greatly’) at post-treatment and 9-month follow-up assessments. Both adolescents and parents completed the scale. A significant difference was found on young person-rated consumer satisfaction at post-treatment, favouring CBT over relaxation. Another study examining treatment acceptability was conducted by Brent et al. [4]. They assessed treatment satisfaction in young people and their parents who were treated with CBT, family or supportive therapy for adolescent depression. A measure was administered three times: after session 1, 6 and at the termination of treatment. Parents' acceptance of treatment satisfaction significantly improved over time, particularly for CBT in comparison to family and supportive therapies. From the published works to date, CBT is suggested to be associated with higher levels of satisfaction and acceptability for young people experiencing depression as opposed to other psychological therapies, for example, family or supportive therapy.
When assessing consumer satisfaction, attention needs to be given to the effect that an individual's age can have on their response style/preference. Typically, individuals are asked to rate, on a Likert-type scale, the level at which they agree or disagree with statements relating to themselves. For adults, Likert scales vary in the number of response choices, usually 5, 7 or 10, whereas young children can generally only discriminate between a couple ofresponses [17]. Research examining response styles of younger children acknowledges their tendency to think dichotomously and generally respond in an extreme manner (e.g. young children tend to endorse the choices at each end of the response continuum, ‘strongly agree’ or ‘strongly disagree’ and ignore the midpoints) [18]. As they develop, young children's ability to experience and report a varying degree of feelings is increased. Developmentally, children around the age of 12 years and onwards are able to acknowledge a range of emotions along a continuum, and thus able to report more subtle levels of intensity [19]. Taken together, these results suggest older children (12+ years) are capable of reporting on a Likert scale with the number of response choices falling in between that of adults and young children.
The aim of the present study was twofold: to examine the psychometric properties of a newly developed measure of consumer satisfaction and, in using this measure, to investigate the level of treatment satisfaction in a sample of depressed adolescents and parents participating in a randomized clinical trial of treatment for adolescent depression.
Method
Participants
Sixty-two adolescents aged 12–18 years participated in the randomized trial. Of the 62 participants, only 42 were given the consumer satisfaction questionnaire. The questionnaire was introduced after the trial had begun. Once introduced, the questionnaire was given to every participant. Of the 42 participants who received the questionnaire, consumer satisfaction data was gathered on 38 adolescents (67% female) with a mean age of 15.3 years and 37 parents (68% mothers) who completed the questionnaire. All adolescents (n=38) had a diagnosis of either major depressive disorder (62.8%); dysthymic disorder (18.6%) or depressive disorder not otherwise specific (NOS) (18.6%). Sixty-three per cent had conditions comorbid to depression, primarily anxiety disorders and parent–child relational problems. Data was collected from two children and adolescent mental health services in Victoria, Australia, in outer metropolitan Melbourne and the regional centre of Geelong.
Instrument
A consumer satisfaction questionnaire was designed and trialled in this study. The Adolescent Depression Treatment Satisfaction Questionnaire (ADTSQ) is a self-report measure consisting of three sections:(i) Treatment Satisfaction −11 questions covered satisfaction with the program and the clinicians involved (appropriateness of goal, logic of approach, confidence in treatment, confidence in long-term outcome, approach to treatment, structure, care and understanding, psychoeducation, providing ideas, problems-solving and confidence in recommending the program). These items were answered on a five-point Likert scale (‘not at all’, ‘a little bit’, ‘somewhat’, ‘a lot’ and ‘very very much’). Given the review by Chambers and Johnston regarding acceptable response style formats for individuals across the age range [18], this format was considered developmentally appropriate for our study. One item was negatively worded and required reverse scoring (structure). (ii) Treatment Preference –four treatment approaches for adolescent depression (medication, individual counselling, group therapy, family therapy) were ranked in order of preference. These treatments were selected as they are reflective of currently available treatments for adolescent depression. (iii) Qualitative/Written Comments –written responses were incorporated as previous research [20] has found that participants are more likely to express dissatisfaction when given the opportunity to express their thoughts in response to an open-ended qualitative question. Two questions pertaining to treatment gains and further comments provided an opportunity for written responses. Separate versions of the questionnaire existed for adolescents and parents, however, item content was identical. The instrument took approximately 5 minutes to complete.
Procedure
The questionnaire was given to participants at the end of the final treatment session for voluntary completion before the post-treatment assessment. As the trial had already begun before the implementation of the questionnaire, the number of potential participants was reduced to a subset (n=42) of the entire sample. The rate of return of the questionnaire was 90.5% which is a high response rate and compares favourably with those reported in the published works [15]. All participants had received treatment for adolescent depression within a randomized comparison of CBT, antidepressant medication using sertraline (SRT) and a combination of CBT and SRT (COMBINED). Cognitive-behavioural therapy treatment was provided individually to the young person and parents using a dual clinician model. Although parents were usually involved, young people were not excluded if parent involvement was not possible. The treatment program included 12 sessions consisting of goal-setting, communication training, problem-solving, relaxation training, pleasant events scheduling and cognitive therapy. Those allocated to SRT received sertraline for 12–24 weeks with a weekly to fortnightly review with a physician to monitor response, dose and side-effects. Those allocated to the COMBINED treatment received a combination of these two therapies. SPSS V 11 was used for data analyses.
Results
Preliminary analyses
As the sample analysed was a subset of a larger sample, independent sample t-tests were conducted to test the representativeness of the subsample. Results showed no significant differences between those participants who completed the questionnaire and those who did not on the following variables: adolescent's sex, age, level of depression and anxiety, treatment group allocation, parents' marital status and depression level of the parent group. The subset, therefore, is not unrepresentative of all the participants who received treatment in the program.
Psychometric properties
Given the instrument is a newly developed measure, the psychometric properties were examined. Cronbach's alpha coefficient for the 11 Treatment Satisfaction items on the ADTSQ is 0.85, indicating a high level of internal consistency.
Exploratory factor analysis was performed to examine the underlying structure of the measure. Maximum likelihood factor analysis using Promax rotation detected three components with eigenvalues greater than 1.0 (see Table 1). These three components each showed strong positive loadings (<0.40) on all items except two (structure and approach). Components were labelled Factor 1 –‘Treatment and Therapist Technique’, Factor 2 –‘Skills Gained’ and Factor 3 –‘Future Confidence’. One of the items (difference) loaded on both ‘Treatment and Therapist Technique’ and ‘Future Confidence’. This item was included on the ‘Future Confidence’ factor as it appeared to fit better with this theme. The three-factor solution explained a total of 70.85% of the total variance.
Adolescent Depression Treatment Satisfaction Questionnaire (ADTSQ) mean scores, standard deviations and exploratory analysis item loadings >0.40
Only loadings over 0.40 shown; Tx, treatment; Tst, Therapist. Numbers next to items indicate their order in the questionnaire.
Using this three-factor structure, subsequent analyses explored two aspects of treatment satisfaction, namely a comparison of the parent and adolescent groups and a comparison of the three treatment conditions. Total and subscale scores were calculated by summing the relevant item scores (see Table 2 for score ranges).
Comparison of mean satisfaction scores between parent and adolescent group and treatment type (CBT, SRT or COMB)
∗Significant at p<0.05; ∗∗significant at p<0.01; CBT, cognitive behavioural therapy; SRT, sertraline; COMB, combined CBT and SRT.
Treatment satisfaction
High levels of treatment satisfaction were reported. Independent samples t-tests were conducted to explore differences between the parent and adolescent groups. Results found no significant differences between parents and adolescents on total treatment satisfaction scores. Both parents and adolescents reported high scores on ‘Treatment and Therapist Technique’ and ‘Future Confidence’ factors. However, there was a significant difference (p< 0.01) between the two groups on ‘Skills Gained’ with adolescents reporting more satisfaction with the techniques learnt than parents.
One-waybetween-groups anova was conducted to explore the difference between the three treatment groups and levels of satisfaction. There was no significant difference between the treatment groups on total treatment satisfaction or on ‘Treatment and Therapist Technique’ and ‘Future Confidence’ factors. Factor 2 –‘Skills Gained’ was found to be significantly different between treatment groups (p<0.05). Posthoc comparisons using the Tukey least significant difference technique indicated that the CBT treatment group reported higher satisfaction on this factor compared to the SRTgroup. Similarly, those who receivedthe COMBINED treatment also reported higher satisfaction with regards to ‘Skills Gained’ than the SRT group. However, there was no significant difference between the CBT and COMBINED treatment groups.
Treatment preference
The parents and adolescents were asked to rank their most preferred treatment. For the adolescents, first preference ratings were as follows: 66% rated individual counselling; 29% rated medication; 5% rated group therapy; and none rated family therapy. Similarly, the parents rated individual counselling as their first preference (74%), followed by 17% rating medication, 6% rating group therapy and 3% rating family therapy. Therefore, most adolescents and parents preferred individual counselling as their first choice of intervention followed by medication, with group and family therapy being least preferred.
Similarly, when examining the three treatment groups in terms of treatment preference, a consistent pattern emerged. For those who received the CBT treatment, inspection of the results indicated 75% rated individual counselling, 18% rated medication, 7% rated group therapy and none rated family therapy. For those in the SRT group, results indicated 57% rated individual counselling, 33% rated medication, 10% rated group therapyand none rated family therapy. Finally, for those who received the COMBINED treatment, the most preferred treatment option was individual counselling (75%), followed by medication (21%), family therapy (4%) and none rated group therapy. Overall, participants from the three treatment groups rated individual counselling as their first choice of intervention followed by medication, with group and family therapy being least preferred.
Qualitative/written comments
The final part of the questionnaire invited qualitative comments from participants. The first question related to skills learned in the program. Comments were grouped into similar themes and the top five responses provided. For the parent group, the most frequently rated in terms of skills learnt was (i) communication skills (32%), followed by (ii) increased awareness and knowledge about adolescent depression (24%), (iii) improving family relationships and lowering conflict (18%), (iv) learning about happiness and that people care (13%) and (v) problem solving (13%). For the adolescent group, the most frequently rated in terms of skills learnt was also (i) communication skills (34%), followed by (ii) improving family relationships and lowering conflict (24%), (iii) problem solving (18%), (iv) coping with situations (13%) and (v) helpful thinking (11%).
When asked for any general comments, a number of parents (42%) and adolescents (53%) had no further remarks. Of those that did, most general comments reported (79% parents and 95% adolescents) found the program and staff helpful, felt more optimistic about the future and used the space to record their appreciation. There were a few comments pertaining to suggested improvements to the treatment format, whereas others voiced their preference for receiving another treatment type other than the one they had been allocated; particularly CBT rather than SRT (21% parents). Overall, most responses from both parents and adolescents were positive.
Discussion
The ADTSQ was shown to be a useful measure of consumer satisfaction in an Australian sample of depressed adolescents and their parents. The questionnaire showed good psychometric properties with high internal consistency. As the measure was newly constructed, exploratory factor analysis was performed to examine the underlying structure. A three-factor solution best fit the data and were suitably named ‘Treatment and Therapist Technique’, ‘Skills Gained’ and ‘Future Confidence’. Two items (‘Approach’ and ‘Structure’) were omitted because of poor loadings and were not included in the subsequent analyses. It is interesting to note that these two items were worded differently compared to the other items. The ‘Approach’ item refers to opinions about what is deemed to be a good treatment approach, which is inconsistent with the other items that request an evaluation of the program itself. The ‘Structure’ item is the only negatively worded item. Therefore, the wording of these items may explain or have contributed to the poor loadings. In addition to loading problems, the ‘Difference’ item which asks ‘How confident are you that this program will make a difference for you further down the track?’ loaded above 0.40 on both the ‘Treatment and Therapist Technique’ and ‘Future Confidence’ factors. As the latter pertains to level of confidence in the future the item was included on that factor. In terms of the response format of the Likert scale, anecdotally the responses of the young people in our study appeared to range across the five response choices, suggesting the format design of the instrument was acceptable.
In the current study, total consumer satisfaction as measured by the ADTSQ has been found to be highly satisfying, as reported by both young people and parents, and similarly high across the three treatments offered. Analysis of the factor scores revealed a predictable difference with the SRT group reporting lower scores than both treatments with CBT on the ‘skills gained’ factor. This difference was expected as the SRT treatment contained no skills training as opposed to the CBT, which is a skills-based treatment. A difference also existed on this factor between parents and young people. Although, both groups rated their skills gained in the moderate to high range, young people rated theirs as being significantly higher. This difference may be explained by the fact that the parents of young people in the SRT treatment group receivedno skills training as a specification ofthat treatment allocation, but their son or daughter receivedweekly supportive sessions in conjunction with a review of the medication and side-effects. These findings indicate that for those who completed the treatment program, satisfaction levels were generally high in both adolescents and parents and did not vary between treatments. These findings provide further evidence of the social acceptability of these treatments.
In terms of treatment preference across the treatment groups, individual counselling was consistently rated as the first treatment preference followed by medication. It was perhaps expected that these were the two most favoured treatments considering theywere the two treatments that were offered and may have a potential bias.
The adolescents and parents made a number of qualitative comments about skills and knowledge gained other than depression management from the program. Both reported a number of skills they had learnt, improvement in communication being the most frequently reported general benefit. Where participants were asked to make any further general comments, most responses were positive in nature and illustrated their appreciation for the program. A few suggestions were made pertaining to improvements; however, the overwhelming theme was that the program had a high impact regarding their ability to cope with daily challenges and improving their interpersonal relationships.
Although this paper has shown the ADTSQ to be a useful measure of consumer satisfaction, there are four limitations that require acknowledgment. First, it could be argued that the participants involved in a randomized trial may not be representative of the general community. Individuals who participate in clinical trials, such as the present study, are often required to accept structured treatment protocols that involve random allocation to treatment, specified treatment length and completion of numerous assessment measures. It is possible that research participants who are willing to accept such structured treatment protocols may be more inclined to be easily pleased and likely to report higher levels of consumer satisfaction than those who do not accept such treatment protocols. Second, program clinicians were aware of the collection ofthe consumer satisfaction data and this presumably encouraged the maintenance of high professional standards. Third, a general bias towards consumers reporting high levels of consumer satisfaction has been recognized [12]. Presumably, patients who have received the complete treatment phase/episode are likely to have a significant positive response bias. However, Perreault et al.'s [20] finding that expression of dissatisfaction was often found in a qualitative context must also be taken into consideration. In the current study when respondents were given the opportunity to make any further comments, favourable statements predominated. Finally, this study is further limited by the fact that data was only collected for those who completed the treatment. Although completers and non-completers were comparable on pretreatment clinical measures and there were no significant differences, this does equate similar levels of satisfaction. It reflects that demographic and clinical differences such as sex, age, severity of symptoms or treatment group did not impact on questionnaire completion. Given our data represents only those who completed the treatment, our sample is biased and it is likely that a percentage of the non-completers terminated because of less satisfaction.
Future studies examining consumer satisfaction should consider ways of maximizing participant response rates and also involving non-completers of treatment in their measurement of consumer satisfaction. It would also be of interest to examine the relationship between treatment outcome and level of satisfaction. Because of sample size and long-term follow-ups in our study, this has not been examined at this stage and will be an area to examine at the end of the research trial.
In summary, both CBT and SRT are highly satisfying treatments for adolescent depression as reported by depressed adolescents and their parents. Those treatments that contained cognitive behavioural therapy were rated as superior in terms of satisfaction with ‘skills gained’ compared with sertraline alone. This further supports practice guidelines recommendation that CBT should be first line oftreatment for youth depression [21].
Furthermore, individual counselling was rated as the first choice of treatment by the vast majority of participants irrespective of their treatment allocation in the program. For the effective use of a treatment, it needs to be considered acceptable by consumers, otherwise its use is limited [11]. Despite the limitations acknowledged in assessing consumer satisfaction, it remains an important part of a comprehensive treatment evaluation.