All that glitters is not gold: Why randomized controlled trials are bronze standard (at best) in determining best practices for the treatment of posttraumatic stress disorder

Abstract

The randomized controlled trial (RCT) is considered the gold standard for evaluating the effectiveness of interventions or treatments across fields of physical and psychological medicine. While the RCT is well-placed for issues best addressed with a medical model approach—its origin story, the study and treatment of scurvy, an ideal exemplar—its stature and use must be challenged in cases of experiential phenomena, such as posttraumatic stress disorder (PTSD), where there is no one answer, treatment, or approach that suits all persons. Uncontested, its evidentiary superiority threatens to have deleterious effects on determining best practices for the treatment of PTSD.

While the randomized controlled trial (RCT) is ubiquitously considered the gold standard for evaluating the effectiveness of interventions or treatments in medicine, psychology, and a plethora of other fields, it is critically important for researchers and practitioners to recognize where its gold stature holds. The RCT is an effective approach for conditions that can be understood and treated using the medical model, which assumes all illnesses have a biological or physiological basis, and the symptoms can be objectively measured and treated with medical interventions, such as medication or surgery. Its origin story—the study and treatment of scurvy—is an ideal exemplar. But the applicability of the RCT is not golden in all cases, particularly for effectively evaluating health conditions that are borne of experience and comparatively more complex in their development and consequent treatment, such as posttraumatic stress disorder (PTSD). This distinction is essential for researchers and practitioners given that the incessant, undisputed use of the RCT as gold standard in all cases has arguably had adverse effects on the development of PTSD treatment guidelines, and so on PTSD treatment and practice. By revisiting the origin of the RCT, this commentary aims to provide the reader an evidence-informed reminder of why RCTs were crowned gold in the first place, and why they are bronze, at best, for determining best practices for the treatment of PTSD.

Re-evaluating the evaluative powers of the RCT

Historically, scurvy had devastating effects on seamen, who developed and regularly died from scurvy on long voyages at sea. In 1747, on a ship plagued by scurvy, James Lind carried out what is considered to be the first reported controlled clinical experiment in the history of medicine, particularly because of its use of control groups (Baron, 2009). Lind divided twelve scurvy-afflicted sailors into six groups of two, where each group consumed the same diet daily, plus one unique supplement of either cider, sulfuric acid, vinegar, seawater, barley water, or two oranges and one lemon (Lind, 1753); therein, Lind systematically studied the effects of supplementing the diet with citrus fruit (Carlisle, 2004). Within 6 days, the two sailors who had eaten the oranges and lemons were fit for duty (Lind, 1753).

Irrespective of all other factors, scurvy is a physical illness that has a clear biological cause, and its symptoms can be objectively measured and treated with one intervention, Vitamin C supplementation—a textbook case for the use of a medical model approach (i.e. an RCT). In contrast, PTSD is experiential; it is a mental health condition predicated on the experience of trauma(s), such as physical or sexual assault in childhood or adulthood, military combat, serious accidents, natural disasters, or terrorist incidents (Mayo Clinic, 2023). Its development, the severity, and persistence of its accompanying symptoms (e.g. re-living or re-experiencing the event; avoiding people, situations, or circumstances that remind them of the event; experiencing negative changes in beliefs and feelings; and, exhibiting various forms of hyperarousal (American Psychiatric Association, 2013: 270–280; National Centre for PTSD, 2023), its treatment, and its dynamic, non-linear process of recovery (St. Cyr et al., 2022), are comparatively more complex and affected not by one factor alone, but by a multiplicity of factors, unique to each individual. This cannot be sufficiently controlled for, randomized, blinded, or truthfully reproduced when studying PTSD and its treatment via RCTs.

Still, the medical model persists, and the RCT is upheld as the gold standard for evaluating the efficacy of treatments for PTSD, despite its obvious shortcomings in this context. RCTs do not—cannot—sufficiently address the context in which trauma(s) occurs; the consequent social, emotional, mental, physical, occupational, economic, cultural, and environmental impacts the trauma(s) and illness have on the individual; nor why a treatment or approach is or is not effective. It is critical that researchers and practitioners not only recognize, acknowledge, and account for these limitations, but also for the adverse effects these limitations threaten to have on aptly guiding practice.

Expanding evidence to better guide practice

Meant to serve as aids in clinical decision-making, clinical practice guidelines (CPGs) have been developed to provide evidence-based recommendations for the treatment of PTSD (American Psychological Association, 2017a) and play an increasingly prominent role in directing health intervention priorities and practice. However, the methodological design of these CPGs limits their evidence base to meta-analyses and systematic reviews comprised of RCTs, thereby excluding studies (hence, therapeutic approaches) not studied by RCTs. For this, the CPGs have been highly criticized for limiting the integration of clinical expertise, and patient values, preferences, and characteristics into their appraisal of the research evidence; neglecting to include all current and novel therapies used to treat PTSD; endorsing only a handful of psychotherapies for PTSD based on insufficient evidence; and not reflecting the results of other research approaches, including “research that examines how processes work within therapies, and qualitative research that explores clients’ and therapists’ experiences of therapy” (American Psychological Association, 2017b). The shared fear is that the CPGs will be taken as gospel “by the outside world,” where these omissions have calamitous impacts on the availability and support of treatments not endorsed within (American Psychological Association, 2017b).

To gather evidence that supports effective treatments for PTSD, multiple traditions of research should be considered and incorporated into the CPGs, including process-outcome research, qualitative research that explores clients’ and therapists’ experiences and perspectives, and prospective longitudinal studies that could determine whether the concerns expressed by people with PTSD are modified by time, therapeutic interventions, severity of PTSD symptoms, persistence of comorbidities, changes in their medical or health status, or any other factors outside of traditional service or therapeutic provision.

Lind concluded “oranges and lemons were the most effectual remedies for this distemper at sea” (Lind, 1753). In cases like scurvy, where one objectively measured cure exists, the RCT holds its gold stature; but where the phenomena under study are experiential, as in the case of PTSD, the RCT is limited in its capacity to effectively generate the highest standard of evidence. To determine best practices for the treatment of PTSD, researchers and practitioners must lean into complexity and value less quantified approaches to solving complex health problems—recognizing that, in these cases, the RCT is bronze, at best.