Comment on: History repeating: guidelines to address common problems in psychedelic science

Dear David Taylor, Editor-in-Chief of Therapeutic Advances in Psychopharmacology

The call for rigor by van Elk and Fried ¹ is a welcome contribution to psychedelic research. The authors emphasized the well-known principles of internal, external, and construct validities, highlighting that conclusions should not extend beyond robust statistical inferences. The examples of previous inadequacies and mistakes regarding some of these principles, including oversight by peer reviewers, are valuable, and future research can indeed benefit from more rigor.

However, the 21 studies scrutinized amount to a small portion of the 320 clinical trials and over 5800 studies retrieved on PubMed for ‘psychedelics OR hallucinogens NOT cannabis’ in the past 10 years. Therefore, the judgment that there are ‘serious doubts on the inferences that have been drawn in research carried out in the last decade’ must be interpreted with care. Furthermore, problems were classified as easy, moderate, or hard to solve, but it is difficult to understand why conflicts of interest were rated as easy. Pharmaceuticals are one of the most lucrative businesses on the planet, and bad practices have become pervasive in medicine, to the detriment of the scientific literature. This includes ‘regulatory capture’, through which the industry’s vested interests critically influence norms, shaping many of the problems rated as easy and moderate, such as standards for adverse event reporting, sample size estimations, lack of long-term research, and restricted access to clinical trial data.

On the other end of their spectrum, placebo effect, causal mechanisms, and unblinding were rated as hard problems, based on an unqualified claim that randomized controlled trials (RCTs) ‘are considered the gold standard’ and are ‘usually double-blind’. Such strong statements demand rigorous epistemological justification and support in empirical data. However, among 200 randomly selected RCTs in five of the top medical and four of the top psychiatric journals, only 7% and 9%, respectively, reported blinding assessments. ² In the broader literature, blinding assessments were reported between 2% ³ and 8% ⁴ of hundreds of randomly selected studies. Psychedelic studies are above this range, with 17%, ⁵ which is unfortunately also low, likely because of the removal of blinding assessment recommendations in guidelines such as CONSORT in 2010. While it is plausible that actual unblinding is higher with psychoactive drugs, these pervasive failures across biomedicine undermine claims for the method’s epistemic authority. It is thus important to ponder if we are facing a gold or a double standard: many studies are accepted despite unblinding, while others are rejected because of unblinding. Regarding psychedelics, clinical benefits were disqualified in the 1960s precisely because of unblinding. ⁶ More than half a century later, history repeated itself.

But is this kind of skepticism epistemically warranted? Does unblinding invalidate reported improvements, or does it raise questions and reveal some of the many causal factors involved in the multiple levels of mechanisms implicated in psychiatric disorders and treatments? Hypothetically, how could an experimental intervention, compared to a control, cause considerably larger improvements in subjective feelings without patients eventually breaking blind correctly guessing their treatment allocations based on their feelings? The situation is acutely problematic in psychiatry and doubly serious in psychopharmacology: because changing subjectivities is a specific drug effect, the attempt to separate those from unspecific ‘placebo effect’ is epistemically unwarranted. Moreover, RCTs are not neutral devices, and double-blinding introduces other biases: ambivalence, passivity, confusion, resentful demoralization, and voluntary submission. ⁷ Notably, double-blinding is intended to increase internal validity to strengthen causal inferences. On the other hand, it decreases external validity because clinical practices do not resemble the passivity, frustrations, and uncertainties imposed on patients and clinicians by double-blinding. But external validity, usually neglected in research, arguably matters most to patients, clinicians, and policymakers, who need to know if what worked in clinical trials will also work outside RCTs. ⁸ In the case of psychedelic therapy, blinding experimental groups becomes nearly impossible because the drugs specifically affect self-awareness and agential stances, which are directly related to the purported psychological mechanisms of psychedelic therapy. Furthermore, almost all suggested methods for trying to improve blinding, such as the use of low doses, fewer drug administrations, minimal support conditions, incomplete disclosure, deception, and even general anesthesia, are likely to result in less benefit to patients, and may also cause potential ethical violations and unintended harms.

A solution to this conundrum is to acknowledge that in psycho-therapeutic interventions, including psycho-active drugs in psychiatry, expectancy, therapeutic alliance, self-narratives, interpersonal behavior, and even beliefs, poorly defined as ‘placebo effect’, are not biases (noise), but treatment factors (signal). By definition, biases, or confounders, are unrelated to the therapeutic intervention, whereas factors are related to the intervention. All should be measured and reported, but always considering if each one is derailing or supporting multicausal pathways underlying clinical improvements in psychiatry. Misclassifying supportive factors as biases would lead, logically, to poorer treatment outcome estimates. Thus, epistemological concerns have important consequences.

The really hard problem, then, is for biomedical researchers to acknowledge that the epistemic authority attributed to the double-blind standard is ill-founded in a philosophical bias. ⁹ This introduces an epistemic bias in biomedical research: the outright rejection of clinical improvements requiring patient’s agency, insight, and knowledge. ¹⁰ It originates in a problematic reductionism, ill-suited for psychiatry and for understanding how self-aware human beings purposefully modify their own lived experiences, especially when using psycho-active drugs and psychotherapy. Rigorous research also requires avoiding common misunderstandings about what can be achieved with RCTs, recognizing that methodological appropriateness depends on the phenomena under investigation and that ‘the gold standard or “truth” view does harm when it undermines the obligation of science to reconcile RCTs results with other evidence in a process of cumulative understanding’. ⁸ For example, other methodological and statistical approaches can be used to incorporate ‘Real-World Evidence’ into decision-making and regulatory approvals in addition to RCTs. ¹¹ In the case of complex and multifaceted treatments such as psychedelic-assisted therapy, the supposed rigor of blinding is too narrow in scope and thus inadequate to study the dynamic cognitive processes involving patient’s agency and self-awareness during non-ordinary states of consciousness.