Meta-Analysis Shows Trauma Memories in Posttraumatic Stress Disorder Lack Coherence: A Response to Taylor et al. (2022)

Abstract

Taylor et al. reported that in healthy participants, memories of traumatic and comparison films did not differ in coherence. The lack of a group diagnosed with posttraumatic stress disorder (PTSD) and limitations of the trauma-film paradigm mean that their design is unable to directly test predictions made by clinical theories of PTSD. Contrary to what Taylor et al. claimed, there is convincing evidence for trauma memories in PTSD being incoherent or disorganized. Meta-analysis demonstrated a strong positive association between PTSD and memory incoherence/disorganization, moderated by the effect of the methods chosen to assess disorganization.

In a recent study in Clinical Psychological Science, Taylor et al. (2022) tested the hypothesis that memories of a traumatic film would be less coherent than memories of a nontraumatic film, citing predictions derived from clinical theories (Brewin, 2016; van der Kolk & Fisler, 1995). It is important to clarify that these theories, like most clinical theories, are based on a diathesis-stress model in which stressor exposure (e.g., to events involving high levels of fear or horror and uncontrollability, fear of death or injury) interacts with individual vulnerability factors to increase the risk of developing conditions such as posttraumatic stress disorder (PTSD). Contrary to what some (Porter & Birt, 2001; Shobe & Kihlstrom, 1997) have implied, clinical theories have not addressed the issue of whether memories of traumatic events differ from memories of other events in people with no (or low levels of) posttraumatic symptoms. This is a separate question that may or may not throw light on the processes involved in PTSD. Because the Mechanical Turk workers used by Taylor et al. were not assessed for the presence of disorders such as PTSD, their study cannot directly test clinical theories (see also Zoellner & Bittenger, 2004).

In the article, Taylor et al. (2022) also suggested that therapists might encourage patients to fill gaps in their memories to resolve incoherence and promote recovery. There is little evidence that resolving coherence is related to recovery (Jones et al., 2007), and this is rarely mentioned as an aim of trauma-focused therapies. Rather, therapies seek primarily to overcome avoidance and reduce the intense negative emotions that are associated with specific aspects of the traumatic experience (Brewin et al., 2010; Ehlers & Clark, 2000). Additional material from memory may return spontaneously as part of this process, but filling gaps in memory is not part of the treatment rationale.

The study by Taylor et al. (2022) nevertheless raises some interesting issues of methodology and interpretation. van der Kolk and Fisler (1995) suggested that

if trauma is defined as the experience of an inescapable stressful event that overwhelms one’s existing coping mechanisms, it is questionable whether findings of memory distortions in normal subjects exposed to videotaped stresses in the laboratory can serve as meaningful guides to understanding traumatic memories. (p. 506)

Reviewers of the trauma-film paradigm have concurred, noting that it is an analogue of trauma and one very different from an actual traumatic event (Holmes & Bourne, 2008). Except under very specific and limited circumstances, viewing pictures of events onscreen would not be a sufficiently traumatic exposure to qualify for a potential diagnosis of PTSD in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2013).

Despite this caveat, the trauma-film paradigm has been very productive (James et al., 2016), but mainly in situations that demonstrate significant effects among participants all exposed to such a film. For example, it has provided evidence of a robust dissociation between the effects of a visuospatial intervention on voluntary and involuntary memory of traumatic scenes (Lau-Zhu et al., 2019). When findings are null, however, as in Taylor et al. (2022), the paradigm is vulnerable to the criticism that this may be because it does not adequately model exposure to a traumatic event.

Another interesting issue is the decision by Taylor et al. (2022) to match their stimulus materials on “storyness,” the degree to which it is possible for viewers to follow the events depicted. Is storyness a confound, as their design implies, or is it an aspect of some potentially traumatic events that accounts for why the memories people form may sometimes be incoherent? In the context of PTSD, traumatic events are often described as shocking, overwhelming, or violating assumptions, and common reactions include being cognitively overloaded, unable to grasp what is happening (Massazza et al., 2021). Controlling for storyness may have the effect of minimizing some of the qualities that make an event potentially traumatic—this could be addressed by future research.

There is a key theoretical controversy over whether the evidence favors trauma memories being more disorganized or incoherent in samples with PTSD than in samples with no disorder, which is predicted by clinical theories of PTSD (Brewin, 2014), or whether there are no differences between such samples, which is predicted by episodic-memory theories of PTSD grounded in cognitive psychology (Rubin, Deffler, et al., 2016). Previous narrative reviews have not come to any definite conclusion (Crespo & Fernandez-Lansac, 2016; O’Kearney & Perrott, 2006). One reason is that the methods used to assess disorganization/coherence have been very variable. Many have involved programs automatically computing indices only indirectly related to coherence, such as the Flesch reading-ease formula. Their validity in this context has not been established empirically and is highly questionable (Gray & Lombardo, 2001).

There are two distinct approaches to measuring disorganization/incoherence with acceptable face validity. One, based on therapy with PTSD patients, is to elicit a very detailed trauma narrative that includes the worst moments and have judges rate individual utterance units for markers of disorganization, such as repetition and nonconsecutive chunks (Foa et al., 1995). This is a measure of local incoherence involving neighboring clauses in a text (Habermas & Bluck, 2000). The other approach, employed by Taylor et al. (2022), among others, has judges or participants themselves make ratings of the entire memory or narrative. Ratings have adopted two wording directions, assessing either the degree of disorganization/incoherence or the degree of organization/coherence, assuming low levels of the latter are equivalent to high levels of the former. A specific example of a coherence measure, referred to by Habermas and Bluck (2000) as “global coherence,” is the narrative-coherence coding scheme (Reese et al., 2011), which has three dimensions: context (where and when an event took place), chronology (the order in which the actions within an event took place), and theme (the point of a story, typically with a high point and a resolution). In this approach, the way the narrative is elicited does not typically include instructions to include the worst moments of the traumatic event.

A previous commentary on Rubin, Deffler, et al. (2016) proposed that evidence for fragmentation and incoherence in PTSD trauma memories depended on the choice of method, which were usually found when the Foa et al. (1995) procedures were used but not when studies employed self-report or judges to rate the entire narrative (Brewin, 2016). It was also pointed out that many studies not finding group differences suffered from insufficient power. The views expressed in this commentary were, however, rejected as “outdated beliefs” that contrasted with “scientific evidence,” and it was suggested that they resulted from “a pattern of selective omissions” (Rubin, Berntsen, et al., 2016).

We set out to decide between these different interpretations of the literature using meta-analysis to aggregate all the available studies. Following Brewin (2014), we required studies to compare trauma-exposed samples meeting criteria for PTSD or its precursor, acute stress disorder (determined using diagnoses or validated symptom thresholds on PTSD symptom scales), with equivalent trauma-exposed samples not meeting criteria for any psychiatric disorder. Studies additionally had to report data allowing samples to be compared on self-ratings or judge ratings of coherence or disorganization of their memory of a single traumatic event (i.e., one meeting Criterion A of the DSM-5; American Psychiatric Association, 2013). Computer-based indices were excluded because of a lack of demonstrated validity.

Following an updated literature search and a discussion of the eligibility of specific studies for inclusion (McNally et al., 2022), 17 studies were judged to fully meet these criteria. Two of the studies discussed in McNally et al. (2022) did not fully meet these criteria, either because the samples did not use a validated threshold to determine PTSD (Rubin et al., 2008) or because the memories sampled were not always traumatic (Rubin et al., 2011), but were still included in the analyses. Studies were coded on whether they used samples of adults (n = 15) versus children or adolescents (n = 4). Studies were also coded according to whether Foa et al.’s (1995; FOA) guidelines were used to elicit a detailed narrative and rate disorganization in individual utterance units. In those that did (FOA: n = 8; Evans et al., 2007; Halligan et al., 2003; Harvey & Bryant, 1999; Jelinek et al., 2009, 2010; Jones et al., 2007; McGuire et al., 2021; Salmond et al., 2011), effect sizes were grouped according to whether they were based on the ratings of the individual utterance units (detailed) or on additional global narrative or questionnaire ratings (global). In all of these additional ratings, disorganization was reflected in higher scores. In those studies that did not use the Foa et al. guidelines (not-FOA: n = 11), effect sizes were grouped according to the wording direction of the global narrative or questionnaire ratings; high scores indicated either disorganization or organization (Berntsen et al., 2003; Hagenaars et al., 2009; McKinnon et al., 2017; Murray et al., 2002; O’Kearney et al., 2007, 2011; Rubin, 2011; Rubin, Deffler, et al., 2016; Rubin et al., 2004, 2008, 2011). Figure 1 shows the aggregated effect size within each study organized by study classification.

Fig. 1.

Forest plot of studies included in meta-analysis.

A random-effects meta-analysis was conducted using a three-level multilevel model in which effect sizes are nested within studies (full details of the methods and results and a modeling of the effects of selection bias are provided in the Supplemental Material available online, and data and quarto files are available on OSF at https://osf.io/597hr/). For the overall model ignoring predictors, there was a strong positive association between disorganization and PTSD that was significant, θˆ = 0.32, 95% confidence interval [CI] = [0.17, 0.46], t(17) = 4.59, p < .001. There was significant heterogeneity, Q_E(79) = 208.81, p < .001. The moderation effect attributable to the sample age group was close to zero and not significant, θˆ = −0.07, 95% CI = [–0.36, 0.23], t(16) = −0.49, p = 0.630 (see Table S1 in the Supplemental Material).

Table S2 in the Supplemental Material shows that the association between disorganization and PTSD was large and significantly different from 0 for both effect sizes derived from the FOA studies (FOA, detailed: θˆ = 0.47; FOA, global: θˆ = 0.76) and for effect sizes derived from the not-FOA studies in which high scores indicated disorganization (θˆ = 0.39). The effect size was virtually zero in the not-FOA studies in which high scores indicated organization (θˆ = 0.01).

The overall moderation effect was significant, F(3, 14) = 28.96, p < .001, indicating that effect sizes were significantly different across the four categories. Table S3 in the Supplemental Material shows the parameter estimates of this model. Compared with effect sizes based on detailed coding of disorganization in the FOA studies, effect sizes were (a) significantly larger when based on global ratings in the FOA studies, (b) significantly smaller in not-FOA studies when high scores indicated organization, and (c) not significantly different to not-FOA studies when high scores indicated disorganization.

These results suggest that incorrect conclusions might be drawn from research that included both types of global measure but did not distinguish between them, and this secondary hypothesis was tested in the subset of six studies authored by Rubin, Berntsen, and their colleagues (Berntsen et al., 2003; Rubin, 2011; Rubin et al., 2004, 2008, 2011, 2016). The effect of wording direction was strong but only just significant (k is small), βˆ = 0.35, 95% CI = [0.00, 0.69], t(4) = 2.79, p = .049. Table S4 in the Supplemental Material shows that the effect was large and significantly different from 0 when high scores indicated disorganization/incoherence (θˆ = 0.26) but was close to zero and nonsignificant when high scores indicated organization/coherence (θˆ = −0.08).

These analyses indicate several important things. First, the overall results clearly support predictions from clinical theories over episodic-memory theories of PTSD derived from cognitive psychology. Once direction of item wording is taken into account, the link between disorganization and PTSD is confirmed even within data collected by critics of clinical theories, such as Rubin and Berntsen. Statements such as “the incoherence of the trauma memory has little support as a component of PTSD” (Rubin, Deffler, et al., 2016, p. 23) or “we and others have found no evidence of incoherence in traumatic memories” (Taylor et al., 2022, p. 425) need to be reconsidered in light of this evidence.

Second, the inclusion or exclusion of specific studies is much less important to the question of whether trauma memories are incoherent or fragmented than the methodological procedures used. The assumption that measures of organization/coherence reflect the same latent constructs as measures of disorganization/incoherence, as relied on by many authors, including Taylor et al. (2022), is inconsistent with our analyses. Similar observations about the effect of item wording on responses have been found in research on memory (Brewin et al., 2019) and clinical disorders such as social anxiety (Rodebaugh et al., 2007). Studies of PTSD indicate that other constructs that appear to be bipolar opposites, such as positive and negative social support, have different correlates and implications for psychopathology (Andrews et al., 2003).

Future articles need to explicitly acknowledge the importance of the different measurement strategies in any discussion of disorganization/incoherence. What is less clear is whether there is any advantage to narratives being assessed at the level of local rather than global coherence, as suggested by Brewin (2016). Data from the small number of available studies contradicted this hypothesis, but the good performance of global measures in this context may reflect the instruction to provide a very detailed narrative. The alternative possibility is that investigators simply need to select measures that directly address the variables of interest, whether at the local or global level.

The findings of the meta-analysis should come as no surprise given that high levels of stress are often associated with impaired memory in animals and humans (Kim & Diamond, 2002; Metcalfe et al., 2019; Morgan et al., 2004; Sandi & Pinelo-Nava, 2007) and have adverse effects on the functioning of brain structures, such as the hippocampus, that are centrally involved in episodic memory (Diamond et al., 2007; Nadel & Jacobs, 1998). Moreover, there is evidence that memory for the sensory-perceptual aspects of a negative event is strengthened at the expense of memory for associations between the content and context in which the event took place, a process that may have implications for PTSD (Bisby et al., 2020). To date, however, the concepts of coherence and fragmentation have been only vaguely specified in clinical theories. What is now needed is to translate the insights resulting from the analysis into a better understanding of memory in PTSD.

One suggestion is to recognize that there is never one trauma narrative but, as for other significant autobiographical memories, a set of narratives developed for different functional and social purposes (Alea & Bluck, 2003). It has been proposed that accounts of trauma may be varied according to the audience, for example, to communicate with friends and family, casual acquaintances, and others, such as law-enforcement or immigration officers (Brewin, 2016). One reason for this variation is because narratives are influenced by the familiarity of the listener and by goals such as intimacy regulation (Alea & Bluck, 2003). Individuals may be prepared to go into more detail of very personal experiences with close friends and family than with acquaintances. Fragmentation may be present in only a subset of these narratives.

A related observation is that traumatic events often include intensely distressing moments, including shameful elements that individuals prefer to hide, as illustrated by the reluctance of refugees and asylum seekers to disclose experiences of sexual violence (Bögner et al., 2007) and by the not infrequent omission of important elements in trauma narratives elicited in research (Ehring, 2004; Evans et al., 2007). The treatment of PTSD is largely based on overcoming high levels of avoidance, typically shown in a reluctance to think or talk about the most upsetting aspects. In therapy, people with PTSD are often required to focus on these moments. It is therefore plausible that disorganization and fragmentation occur only during specific moments that are characterized by extremely high fear and may correspond to periods of dissociation (Brewin, 2016; Ehlers et al., 2004).

This has clear implications for future studies. One strategy would be to contrast the effects of different instructions for eliciting narratives (including vs. not mentioning the importance of worst moments) on different measures of both coherence and disorganization. Another would be to conduct a detailed study of the experiences of patients who are deliberately retrieving avoided aspects of their traumatic memories for the first time. This should additionally consider whether fragmentation and disorganization necessarily reflect differences in the way memories are encoded or, rather, the effect of high levels of arousal during retrieval (Brewin, 2014; Zoellner & Bittenger, 2004).

Supplemental Material

sj-docx-1-cpx-10.1177_21677026241240456 – Supplemental material for Meta-Analysis Shows Trauma Memories in Posttraumatic Stress Disorder Lack Coherence: A Response to Taylor et al. (2022)

Supplemental material, sj-docx-1-cpx-10.1177_21677026241240456 for Meta-Analysis Shows Trauma Memories in Posttraumatic Stress Disorder Lack Coherence: A Response to Taylor et al. (2022) by Chris R. Brewin and Andy P. Field in Clinical Psychological Science