Abstract
Background:
Isolating a therapeutic procedure might be a powerful way to enhance our understanding of how cognitive behavior therapy (CBT) works. The present study explored new methods to isolate cognitive procedures and to study their direct impact on hypothesized underlying processes and CBT outcome.
Method:
The effects of a cognitive therapy skill acquisition procedure (n = 36) were compared to no procedure (n = 36) on cognitive therapy skills, dysfunctional thinking, distress, and mood in response to induced distress following a social stress test in healthy participants.
Results:
Participants reported more cognitive therapy skills after the procedure that focused on the acquisition of cognitive therapy skills compared to no procedure, but there were no differences in dysfunctional thinking, distress, and mood between the groups.
Conclusions:
By demonstrating an experimental approach to investigate mechanisms of change, including the pitfalls that come along with it, the present experiment provides a blueprint for other researchers interested in the underlying mechanisms of change in CBT for depression.
Keywords
Introduction
Cognitive behavior therapy (CBT) is the most extensively studied and widely recognized treatment for depression (for a review, see Butler, Chapman, Forman, & Beck, 2006; Cuijpers et al., 2013). However, about 47% of the individuals suffering from depression show no response to CBT (Cuijpers et al., 2014) and its effectiveness might have been overestimated (Cuijpers, Smit, Bohlmeijer, Hollon, & Andersson, 2010). Based on these findings, it seems necessary to explore how CBT could be optimized to achieve better treatment outcomes. A first step toward optimizing treatment outcomes would be to understand through which pathways CBT actually works (Hollon et al., 2002; Kazdin, 2007; Kazdin, 2009).
CBT for depression aims to tackle dysfunctional thinking by challenging negative thought patterns. This process is also referred to as cognitive change. Cognitive change can be described as the change of the content and structure of dysfunctional cognitions that are often measured using a self-report questionnaire that focuses on the patient’s dysfunctional beliefs. Accordingly, previous research showed a strong association between CBT and cognitive change and cognitive change and depressive outcomes (Beevers & Miller, 2005; Garrat, Ingram, Rand, & Sawalani, 2007; Lorenzo-Luaces, German, & DeRubeis, 2014; Quilty, McBride, & Bagby, 2008; Segal et al., 2006). However, most of these studies were unable to demonstrate a causal relationship between cognitive change and depressive outcome (Lorenzo-Luaces et al., 2014). Specifically, these studies showed that change in dysfunctional beliefs was associated with a greater reduction in depressive symptoms not only after CBT but also after other forms of psycho- or pharmacotherapy (Cristea et al., 2015; Garratt, Ingram, Rand, & Sawalani, 2007; Lorenzo-Luaces et al., 2014; Quilty et al., 2008). Hence, whether cognitive change is a direct result of cognitive procedures that are responsible for the change in depressive outcomes or a more general epiphenomenon of recovery from depression as a result of treatment remains an important question (Craighead, Sheets, & Bjornsson, 2005; Huibers & Cuijpers, 2015; Strunk, Brotman, & DeRubeis, 2010).
It is also possible that it is not the process of cognitive change but the acquisition or enhancement of cognitive therapy skills is responsible for the effects of CBT (Barber & DeRubeis, 1989; Strunk, DeRubeis, Chiu, & Alvarez, 2007; Strunk, Hollars, Adler, Goldstein, & Braun, 2014). Cognitive therapy skills have been defined as the ability to reevaluate the accuracy of one’s dysfunctional beliefs and engage proactively in pleasurable activities (Strunk et al., 2014). It has been suggested that learning cognitive therapy skills might be the primary process in CBT that subsequently leads to cognitive change. Circumstantial evidence for the hypothesis that cognitive therapy skills might act as a mechanism of change has been provided in a number of studies showing that the frequency and quality of cognitive therapy skills improve during treatment (Hundt, Mignogna, Underhill, & Cully, 2013; Lorenzo-Luaces et al., 2014). Moreover, cognitive therapy skills may explain why provocation of sad mood was associated with increased dysfunctional thinking in patients who received antidepressants compared to patients who received CBT, while both treatments have been associated with cognitive change (Barber & DeRubeis, 1989; Barber & DeRubeis, 2001; Garrat et al., 2007; Segal, Gemar, & Williams, 1999; Segal et al., 2006). However, the exact nature of the association between cognitive procedures, cognitive therapy skills, and treatment outcome is unclear.
At least two reasons might explain why previous studies were unable to confirm hypotheses about mechanisms of change in CBT beyond terms of association. First, the majority of studies were not able to show statistical mediation and simultaneously establish a time line between the mediator (the proxy variable that is often tested to account for the relationship between the independent and dependent variables) and outcome at the same time. This makes it impossible to determine whether change in the mediator actually preceded change in the outcome. Second, although direct manipulation of the mediator is assumed to be one of the most powerful ways to enhance our understanding of how psychotherapy works (Kazdin, 2007, 2009), this method has not been used to investigate mechanisms of change in CBT for depression. In fact, treatment experiments have been more common in the field of anxiety disorders, where the results of early (animal) experiments on the original learning theory are still seen as the theoretical basis for the behavioral treatment of anxiety disorders (Clark, 2004; Lissek et al., 2005; Mineka & Oehlberg, 2008). However, a recent special issue of Behavior Research and Therapy (Craske, 2016) reviewed the latest advances in experimental psychopathology and demonstrated how using an experimental approach might help to understand the processes underlying therapeutic change across a variety of psychological disorders. More specific, one way to increase insights into mechanisms of change might be to make the distinction between therapeutic procedures and change processes, as was recently illustrated with findings on the effects, and modification, of anxiety-linked attentional biases (Macleod & Grafton, 2016). For example, such a distinction might help to identify under what conditions therapeutic procedures will result in change in the hypothesized underlying processes and investigate how existing therapeutic procedures can be optimized (Macleod & Grafton, 2016).
An unprecedented example study in the field of depression was conducted by Teasdale and Fennel (1982). In this study, five patients treated with CBT for their depression received in random order a session focusing on changing their dysfunctional thoughts versus merely exploring their dysfunctional thoughts (without attempting to change thoughts). The results showed that the patients demonstrated a larger improvement in mood and less belief in the dysfunctional thought after a session with cognitive change procedures, compared to a session of only exploring the dysfunctional thought. These findings support the hypothesis that modifying dysfunctional thinking can directly reduce symptoms of depression and might explain how CBT really works. Moreover, it shows that the most straightforward way to investigate the direct role of cognitive change or skills in CBT might be to isolate a cognitive procedure that is expected to lead to immediate changes in the mediator and the subsequent outcome.
The present experiment examined whether we can isolate cognitive procedures to investigate how CBT for depression works. Showing that the cognitive procedure leads to changes in the mediator and the outcome, compared to the control procedure, would strengthen the notion that the hypothesized process functions as a mechanism of change. An illustration of the hypothesized cognitive procedures and processes in CBT for depression is presented in Figure 1.
Hypothesized cognitive procedures and processes in CBT for depression.
Effects of cognitive therapy skill acquisition on cognitive skills, dysfunctional thinking, and mood following a social stress test
The present experiment compared a procedure that focused on the acquisition of cognitive therapy skills to no procedure on measures of cognitive therapy skills (process), dysfunctional thinking (process), and mood (outcome) in a healthy student sample. As a way to activate dysfunctional thinking, the current experiment used a social stress test. Stress is a phenomenon that is highly related to both depression and anxiety (Hammen, 2005, 2006) and as an earlier experiment reported levels of dysfunctional thinking similar to that of a nondepressed sample after a sad mood induction in a healthy sample (de Graaf, Roelofs, & Huibers, 2009; Renner, Schwarz, Peters, & Huibers, 2014), we expected that for a student sample with low depression severity, a stress test would be more effective in inducing dysfunctional thinking. We expected that participants who received the procedure would report more cognitive therapy skills after the cognitive skill acquisition and show less dysfunctional thinking, less distress, and better mood after exposure to a social stress test than participants that did not receive the procedure.
Method
Design and participants
The present study had a 2 (procedure, between participants) × 3 (time, within participants) design. Participants were randomly assigned to a written cognitive therapy skill acquisition procedure (CTSAP; n = 36) or a control group that did not receive the procedure (n = 36) before exposing them to a social stress test. Seventy-two undergraduate healthy students (9 males and 63 females) ranging in the age from 18 years to 26 years (CTSAP: mean 21.22, SD 4.63 and control: mean 19.56, SD 1.64) participated in this study.
Therapeutic procedure: CTSAP
We developed a short procedure that was based on the protocol of Beck, Rush, Shaw, and Emery (1979). The goal of the procedure was to teach students a small set of cognitive therapy skills to challenge automatic negative thoughts using a written assignment. The first part of the procedure consisted of an introduction on the relation between thinking, behavior and mood, and the definition and examples of automatic negative thoughts and asked the participant to make a list of one’s own automatic negative thoughts that he or she would commonly encounter in daily life. Subsequently, the second part of the procedure focused on teaching participants two different ways to challenge their own automatic negative thoughts. The first technique showed participants that automatic negative thoughts can be challenged by searching for evidence that the automatic negative thought is true or not true, while the second technique focused on questioning the functionality of the thought (“how is having this thought helping you right now?”). After a description of the techniques, participants were asked to apply them to two of their own automatic negative thoughts they had identified during the first part of the procedure and two additional scenarios that included the presence of automatic negative thoughts in a situation similar to the social stress test (i.e., dysfunctional beliefs for scenarios 1 and 2 were “I am a loser” and “I will never be okay,” respectively). The total procedure took a maximum of 60 min.
Procedure
Ethics approval for this study was given by the scientific ethics board of VU Amsterdam, and informed consent was obtained from all participants. Participants were included if they were not in the treatment for a psychological problem as defined in the Diagnostic and Statistical Manual of Mental Disorders (DSM)-IV-TR. The experiment took place on two subsequent days and a follow-up 6–8 days later. Phones were turned off during the whole experiment. During Day 1, baseline measures were completed, and the experimental group completed the CTSAP. Afterward, both groups completed the cognitive therapy skill test. Day 2 started with a 15-min relax period. During this period, participants were provided with popular magazines and given the opportunity to bring their own reading. After relaxation, the experimental group received a reminder of the CTSAP (i.e., participants were given six questions to retrieve information from the CTSAP: (1) Give a summary of the CTSAP, (2) “Describe the relationship between behavior, thinking, and emotions,” (3) What is the role of negative thinking in the relation between thinking, behavior, and emotions?, (4) Can you give an example from daily life?, (5) “How would you cope with automatic negative thoughts?,” and (6) “Describe an example of a automatic negative thought you might encounter during a job interview and describe how you would cope with this thought”). Subsequently, participants received a short summary containing the correct answers and examples on these questions and participants from both groups were introduced to the Trier social stress test (TSST). The TSST has repeatedly shown to be a valuable tool for investigating stress responses in a laboratory setting (Birkett, 2011; Gaab et al., 2003; Kirschbaum, Pirke, & Hellhammer, 1993). After relaxing for 15 min, participants were introduced to the TSST by first showing them the interview room. They were then asked to prepare a speech in the next 10 min. After 10 min, they returned to the interview room to give a speech for the next 5 min. The speech was followed by a mental arithmetic task (5 min) in front of a jury consisting of two people. During data collection, three participants dropped out because they felt too stressed to perform the speech. After completing the TSST, participants were asked to complete questionnaires. Six to eight days after the first day, participants returned to the lab to complete follow-up questionnaires and for a debriefing. Participants received course credit or monetary compensation (13 euro) for their participation.
Measures
All measures were completed at baseline, after the TSST, and 6–8 days later.
Process measures
Cognitive therapy skill test
Cognitive therapy skills were tested using two written scenarios that included the presence of a automatic negative thought and asked participants what they would do in these situations. An English version of the cognitive therapy skill test is provided in Appendix 1. For each technique (i.e., searching for evidence and investigating the functionality of the thought) that was used to cope with the automatic negative thoughts in the two hypothetical situations, participants received 1 point (i.e., dichotomous variable: 0 = technique is present, 1 = technique is present) resulting in a minimum score of 0 and a maximum score of 4 (i.e., when both techniques were used for each scenario). Manipulation checks were scored by two independent raters that were blind to condition. In the case of different ratings, a third rater decided on the final rating.
Cognition checklist
The Cognition Checklist (CCL) investigates automatic negative thoughts and cognitions related to anxiety and depression (Steer, Beck, Clark, & Beck, 1994). The scale consists of 26 items that are rated on a 5-point Likert-type scale ranging from 1 = never to 5 = always and is divided in a depression (CCL-D, 14 items) and anxiety (CCL-A, 12 items) subscale. In previous research, average scores of depressed patients were 15.25 (SD = 8.23) and 24.04 (SD = 10.95) on the CCL-A and CCL-D, respectively. The coefficients α for the CCL-D and CCL-A were, respectively, .93 and .91 for the patients with a variety of DSM III diagnoses and convergent, and discriminant validity was supported (Steer et al., 1994).
Outcome measures
Positive Affect and Negative Affect Scale
The Positive Affect and Negative Affect Scale (PANAS) consists of 20 words that reflect positive affect (10 items) and negative affect (10 items) (Watson, Clark, & Tellegen, 1988). Participants are asked to rate each item on a 5-point Likert-type scale from 1 = not at all to 5 = extremely. Means and standard deviations (SD) for depressed patients are 17.1 (SD = 7.2) on the positive affect subscale and 34.3 (SD = 8.7) on the negative affect subscale, respectively (Boon & Peeters, 1999). Cronbach’s α for the Dutch version of the negative and positive affect subscale was .83 and .79, respectively (Peeters, Ponds, & Vermeeren, 1996). Construct validity was supported (Watson et al., 1988).
State-Trait Anxiety Inventory
The State-Trait Anxiety Inventory (STAI) consists of two distinct anxiety concepts: state and trait anxiety (Spielberger, Gorsuch, & Lushene, 1970). For this experiment, we only used the 20-item state anxiety subscale that measures a temporary feeling of tension and autonomic nervous system activation. Items are scored on a 4-point Likert-type scale, ranging from 1 = not at all to 4 = very much. Baseline mean scores of college students vary from 35 to 40 on the state scale and increase to a range of 43 to 60 after experiencing a stressful situation. Cronbach’s α ranged from .82 to .93 (Spielberger et al., 1970). Construct and concurrent validity were supported (Spielberger et al., 1970). Previous studies showed that state anxiety increases during the TSST (Birkett, 2011).
Data analyses
First, means and standard deviations of the baseline measurements were reported. Also, for the process and outcome measures, the Cronbach’s α in the current sample was reported. Second, the differences between the groups on the cognitive therapy skill test were tested using an independent t-test. Third, we tested group by time interactions using repeated-measures analysis of covariance. All analyses were conducted using SPSS (version 21.0; IBM Corp., 2012), and for all analyses, significance was tested at α < .05 (two-tailed). Levene’s test was used to test the homogeneity of variances.
Results
Baseline characteristics
Baseline characteristics can be found in Table 1. Mean and standard deviations for each group before and after the TSST, and at 1-week follow-up can be found in Table 2. Data of one participant were missing in the control group on the PANAS-positive and CCL at baseline. For the measurements after the TSST and at follow-up, data of three participants (two in the control group and one in the experimental group) on the PANAS, data of four participants on the STAI (two in each group), and of five participants (one participant in the experimental group and four in the control group) on the CCL were missing.
Baseline characteristics for each group (means and standard deviations).
Note. CCL = Cognition Checklist; PANAS = Positive and Negative Affect Scale; STAI = State-Trait Anxiety Inventory. Groups differed significantly on age at baseline: Participants in the experimental group were somewhat older than participants in the control group: t(43.69) = 2.03, p = .048.
*p < .05.
Means and standard deviations on the process and outcome measures in the experimental and control group.
Note. TSST: Trier social stress test; CCL = Cognition Checklist; CTS = Cognitive therapy skills; PANAS = Positive and Negative Affect Scale; NA = not applicable; STAI = State-Trait Anxiety Inventory. None of the differences between the procedures were significant. Note that cognitive therapy skills were only tested before the TSST (but after the skills acquisition procedure) and that state anxiety was not measured at follow-up.
Process measures
Cognitive therapy skill acquisition
The experimental group (M = .86, SD = .76) used significantly more techniques on the cognitive therapy skill test compared to the control group (M = .35, SD = .54), t (68) = 3.20, p < .01.
Dysfunctional beliefs
There were no differences between the experimental group and the control group in change in CCL scores over time (group by time interaction effects, respectively: F(2, 64) = .66; p = .52, F(2, 65) = 1.58; p = .21). Mean and standard deviations for each group can be found in Table 3. Cronbach’s α of the scale during the first measurement was .88. Inspection of the raw data showed no differences that point to clinical relevance.
Considerations for future experimental depression research.
Outcome measures
State anxiety
The TSST significantly induced distress in both groups, but there were no differences between the experimental and control groups in change of scores on the STAI, time: F(1, 66) = 186.76 p < .001; group by time interaction effect: F(1, 66) = .09; p = .76. Cronbach’s α of the scale during the first measurement was .88.
Mood
There were no differences between the experimental and control group in change of positive and negative affect on the PANAS, group by time interaction effects, respectively: F(2, 66) = .17; p = .84, F(2, 66) = .36; p = .69. Inspection of the raw data showed no differences that point to clinical relevance. Cronbach’s α of the scale during the first measurement was .85 and .82 for the positive and negative scale, respectively.
Discussion
The present experiment was one of the first to examine whether it is feasible to isolate cognitive procedures that are part of the CBT protocol for depression and to investigate whether these would lead to direct changes in the hypothesized underlying processes and outcomes. Results showed that participants who received a CTSAP reported better cognitive therapy skills before attending the social stress test but did not show less distress, dysfunctional thinking, or better mood after the test compared to participants that did not receive the procedure.
Before we conclude that CTSAPs do not lead to cognitive change and (subsequent) depression change, we would like to present several alternative explanations that might explain the lack of significant results.
First, our procedure may not have been sufficiently powerful to activate cognitive change or changes in mood: Participants showed a difference in use of cognitive therapy skills, but the mean level of skill use was low (means below 1) for both groups. This means that, even in the experimental group, a part of the participants did not use any of the skills that were provided during cognitive skill acquisition. The clinical relevance of this difference is therefore limited. Also, although a previous study that showed that cognitive change or better mood may be achieved with very short procedures (i.e., in the study of Teasdale & Fennel, 1982, five depressed patients, already in treatment, showed cognitive change and reduced depression even after a short 30-min cognitive change procedure), this study (Teasdale & Fennel, 1982) used a sample of moderately to severe depressed patients.
Second, our measures might have been insensitive for measuring cognitive change or cognitive therapy skills. In other words, the automatic negative thoughts related to depression and anxiety that are measured with the CCL may not be applicable to each individual. Instead of measuring cognitive change with standardized self-report questionnaires as was done in the current experiment, another strategy may be to measure the participants’ belief in personal, idiosyncratic dysfunctional thinking and related feelings before and after the procedure and use these differences as an outcome measure. Moreover, a stronger test of cognitive therapy skills might be to directly test the use of cognitive therapy skills by asking participants to write down how they would cope with their own dysfunctional beliefs following the social stress test, instead of giving example scenario’s that include automatic negative thoughts and measuring them before the social stress test. In this way, future studies might be able to test whether a procedure focused on cognitive therapy skill acquisition leads to the development of cognitive therapy (CT) skills and decreases dysfunctional thinking that is unique to a given individual. In addition, it seems important to investigate under what conditions cognitive procedures might affect different levels of dysfunctional thinking as these can be distinguished into the following: (1) automatic negative thoughts (i.e., as were used in the present experiment) that can be described as negative statements that are linked to a specific negative situation (e.g., “I guess I will fail the exam.”), (2) statements that include rules, expectations, attitudes, and assumptions that are more general than automatic negative thoughts (e.g., “If I’m not perfect, nobody will like me.”), and (3) core cognitive structures, also called schemas (e.g., “I am a loser.”) (Beck et al., 1979; McBride, Farvolden, & Swallow, 2007). Whereas automatic negative thoughts are easily accessible, assumptions and attitudes are less malleable, and schemas are hypothesized to rely outside of overt awareness and to underlie the presence of the dysfunctional thoughts (McBride et al., 2007; Young, Klosko, & Weishaar, 2003). It is possible that different levels of dysfunctional thinking require a different dose of cognitive skill acquisition (“different strokes for different folks”) to lead to successful use of CT skills and the reduction of dysfunctional thinking.
Third, a sad mood induction might be better suited for investigating the effects of cognitive change procedures on dysfunctional thinking and depressed mood. Although stress is a phenomenon that is highly related to depression (Hammen, 2005, 2006), and the social stress test did lead to an increase in feelings of distress, in contrast to our expectations, it did not lead to higher levels of dysfunctional thinking or worsened mood in a healthy student sample. Future studies should investigate whether sad mood inductions can lead to an increase in idiosyncratic automatic negative thoughts, attitudes, and schemas in healthy samples or use (sub)clinical samples to allow for change in outcomes and processes.
Taken together, our pilot study leads to six important considerations for future experimental depression research, which are presented in Table 3.
Concluding remarks
Against the background that experimental studies on mechanisms of change in cognitive behavioral therapy for depression have been scarce and even lacking, the present pilot study might serve as a first step toward a more experimental approach to investigate mechanisms of cognitive change in the field of depression. We reported on the design and results of a hypothesis-driven experiment as well as on the pitfalls that came along with them. Before we can conclude that cognitive procedures do not lead to change in underlying processes and (subsequent) depression outcomes, more experimental research in this area is highly needed. Future experimental depression treatment research might benefit from the abovementioned recommendations.
Footnotes
Authors’ contributions
Sanne J. E. Bruijniks, Marit Sijbrandij, and Marcus J. H. Huibers contributed to the design. Sanne J. E. Bruijniks conducted the data collection. Sanne J. E. Bruijniks performed data analysis and wrote the manuscript. All authors read, contributed, and approved the final manuscript.
Ethics approval
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, and its most recent revision.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.