Abstract
Background:
The classical psychedelics, psilocybin, peyote, ayahuasca/N,N-dimethyltryptamine, and lysergic acid diethylamide are considered promising new treatments for psychiatric illnesses, such as depression, anxiety, addiction, and obsessive-compulsive disorders. However, their profound and characteristic subjective effects raise concern for distinctive biases in randomized clinical trials.
Methods:
We performed a systematic literature search to identify all clinical trials on classical psychedelics with patient populations to examine descriptive data and determine the risk of bias. Two independent reviewers searched three databases (PubMed, Embase, and APA PsycNet) and extracted information on study design, study population, use of active or inactive placebo, dropouts, evaluation of blinding of intervention, and reporting of expectancy and therapeutic alliance.
Results:
We included 10 papers reporting on 10 unique trials. The trials generally included populations that were predominantly white and highly educated. The trials had small samples and considerable dropout. Blinding was either unsuccessful or not reported regardless of type of placebo. Few trials published protocols, statistical analysis plans (SAPs), and outcomes relating to psychotherapy fidelity. All trials but one were rated as high risk of bias.
Conclusion:
Successful blinding of intervention is a significant challenge in this field. To better accommodate this, we suggest that future trials use a parallel-group design and utilize an active placebo on a psychedelic-naïve population. Future trials should publish trial protocol and SAPs, use clinician-rated outcomes accessed by a blinded rater, evaluate blinding of intervention, and consider measuring expectancy and therapeutic fidelity.
Introduction
Classical psychedelics, such as psilocybin, lysergic acid diethylamide (LSD), N,N-dimethyltryptamine (DMT)/ayahuasca, and mescaline, are serotonin 2A receptor agonists that produce an altered state of consciousness (ASC) in a dose-dependent manner. An ASC can include changes in sensory perception, thought, mood, and sense of self-reality. These substances were extensively studied during the 1950s and 1960s when a minimum of 10,000 patients were prescribed psychedelics (Passie, 1997) and some thousand received psychedelics in a research setting (Carhart-Harris and Goodwin, 2017; Cohen, 1960; Krebs and Johansen, 2012; Larsen, 2021; Nichols, 2016). This research declined during the 1970s (Clark and Oram, 2020), and recommenced with Strassman’s research on DMT in the early 1990s (Strassman et al., 1994) after a two-decade hiatus. This milestone was the spark for what is deemed a “psychedelic renaissance” and now basic, translational, and clinical research has its resurgence.
Clinical trials have been conducted on classical psychedelics for indications of alcohol use disorder (Bogenschutz et al., 2015), obsessive-compulsive disorder (OCD; Moreno et al., 2006), nicotine dependence (smoking) (Johnson et al., 2014), anxiety and depression in life-threatening illness, and major depression disorder (Carhart-Harris et al., 2016, 2018, 2021; Davis et al., 2021; Griffiths et al., 2006; Grob et al., 2011; Ross et al., 2016). Most trials have published promising and profound results, as have systematic reviews and meta-analyses (Galvão-Coelho et al., 2021; Goldberg et al., 2020a, 2020b; ; Li et al., 2022; Luoma et al., 2020; Vargas et al., 2020; Yu et al., 2021, 2022).
However, the apex of evidence-based experiments, randomized clinical trials (RCTs), are likely challenged by the occurrence of pronounced effects such as the ASC induced by classical psychedelics. The design aims to reduce outcome impacting biases, such as performance bias and detection bias, by blinding and randomly allocation participants between interventions arms (Peace and Chen, 2010). However, the evaluation of blinding of intervention is rarely reported or discussed in trials of psychiatric medicine in general. Moreover, when evaluated, reviews of psychiatric research find blinding to be ineffective in over 50% of trials (Baethge and Baldessarini, 2013), which is probably because psychopharmaceuticals often cause recognizable effects and especially recognizable adverse effects. In reference to this, a Cochrane systematic review of conventional antidepressive medicine found unblinding to be a possible inflator of effect size (Moncrieff et al., 2004). Furthermore, did another Cochrane meta-analysis found that the use of active placebo in trials on conventional antidepressive medicine was associated with a smaller effect size, when compared to trials who used standard placebo (Laursen et al., 2023).
Unblinding of participants represents a substantial source of bias, as participants expectations of treatment may cause experience bias (Greenberg and Fisher, 1994). One explanation could be that side effects often correlate with therapeutic effects. Another possible explanation is that pronounced side effects enhance the placebo effect in participants allocated to the active group (Thomson, 1982).
The subjective effects of psychedelics challenge blinding of intervention for participants and personnel in RCTs in a way unlike any other substance. It is questionable whether established blinding methods, such as placebo tablets, work as intended in RCTs with classical psychedelics, or whether participants can deduce which condition they have been allocated to. Such unmasking could cause significant response bias as many outcomes are participant reported in psychiatry research (Roe et al., 2022; Sartorius, 2014). Active placebo, which aims to mimic the subjective effects of the drug being tested, has been employed as a means to reduce this bias. However, a recent Cochrane meta-analysis that compared trials utilizing inactive and active placebo failed to find any difference in drug effects. However, the results were imprecise, and the confidence interval (CI) was compatible with a difference ranging from important to irrelevant, and in the sensitivity analysis of low risk trials did they found a difference in drug effects (Laursen et al., 2023).
In addition, there is currently a “hype” surrounding the field of psychedelic medicine, with an extensive amount of positive coverage by the media. This attention could cause further bias by influencing participants to have wishful expectations and poor generalizable trial populations consisting of participants especially interested in psychedelics.
We further find it important to evaluate patient’s expectancy and therapeutic alliance, hence both could be possible mediators of a placebo response.
Expectancy is regarded as a key contributing factor to placebo response in RCTs. This has been shown in a meta-analysis to influence the estimation of the average treatment effect in RCTs on conventional antidepressive medicine (Rutherford et al., 2009). This meta-analysis compared placebo-controlled trials with trials that compared an antidepressant with another antidepressant and found the odds of responding to medication in a placebo trial to be near twice those in a placebo-controlled trial. The authors of the meta-analysis suggested that the participants’ expectations in the trials with an active placebo could be higher, which, in turn, could lead subjects to continue treatment during periods of clinical worsening or increased adverse effects and report less severe symptoms (Rutherford et al., 2009). Furthermore, other research found that the greater the probability of receiving a placebo in a trial, the lesser the placebo response (Papakostas and Fava, 2009). This could also be regarded as indirect evidence of how expectancy could contribute to the effect of an intervention, as the participants would know if they had a greater chance of being allocated the trial drug. Similarly, another meta-analysis of patients undergoing psychotherapy found that patients’ pre-treatment expectancy of a positive outcome predicted treatment efficacy (Constantino et al., 2018), and a recent trial of psychedelic microdosing in a naturalistic setting found that a positive expectancy score at baseline predicted improvement of well-being during the trial (Kaertner et al., 2021).
In psychotherapy without the application of psychedelics, the therapeutic alliance formed between patient and therapist is regarded as paramount and is one of the important nonspecific factors shared by several schools of psychotherapy (Hatoor and Krupnick, 2001; Zilcha-Mano et al., 2019). It is evident that development of a solid therapeutic alliance is predictive of effective conventional psychotherapy (Flückiger et al., 2018), and it has been suggested that it also affects treatment outcomes in psychopharmacotherapy (Zilcha-Mano et al., 2019). In the field of psychedelic medicine, Kaertner et al. (2021) assessed therapeutic alliance and found that its strength in the treatment condition predicted pre-session rapport, resulted in greater emotional breakthrough and increased mystical-type experience. In addition, a better therapeutic alliance predicted a more effective therapeutic effect on depressive symptoms (Kaertner et al., 2021).
We aim to conduct a systematic review to gain information about the quality of the clinical research on classical psychedelics and to point out probable biases that should be considered when planning future research.
We will apply the risk-of-bias tool RoB 2.0 by the Cochrane Collaboration (Higgins et al., 2021) to the existing RCTs investigating classical psychedelics in patient populations. Furthermore, we will review blinding of intervention and to what extent clinical trials reported on expectancy and therapeutic alliance. Following this, bias, challenges, and methodological shortcomings in the available research will be discussed.
Materials and methods
The review was registered in the Open Science Framework (DOI 10.17605/OSF.IO/J6RBM).
Data acquisition
We attempted to identify all human studies on classical psychedelics in a clinical setting available for review from January 1, 1990 until November 7, 2022. We chose 1990 as our starting year since it was when Rick Strassman became the first in 20 years to legally administer psychedelics to human subjects since the early 1970s.
Search and screening strategy
Searches were performed on PubMed, Embase, and APA PsycNet databases from the inception of 1990 until November 7, 2022. The primary search was conducted in January 2022 and was updated November 7, 2022 (see Appendix A for the complete search strategy). Findings were imported to Covidence (www.covidence.org) online software, which identified and removed duplicates.
Using Covidence, ORH and EDP screened all titles and abstracts independently to determine their relevance, with discrepancies resolved by a consensus decision.
Eligibility criteria
Retrieved papers were checked against the following inclusion criteria: (1) complete articles published in peer-reviewed scientific journals, which were reporting on (2) placebo-controlled clinical trials, (3) utilizing one of the classical psychedelics psilocybin, DMT/ayahuasca, LSD. or mescaline, and (4) published in English or Danish.
Qualitative studies, abstracts, letters, reviews, meta-analyses, secondary analyses, conference abstracts, comments, trial protocols, editorials, and papers reporting on microdosing were excluded. Microdosing was considered as doses of upwards of, for example, 20 μg of LSD or 1 mg of psilocybin, based on the principle that a microdose is 5–10% of the active doses utilized in clinical trials (Marschall et al., 2022).
After an independent evaluation of inclusion and exclusion criteria by ORH and EDP, with discrepancies resolved by a consensus decision, the remaining papers were included in the final sample.
Data extraction
From the final sample of articles, ORH and EDP independently extracted descriptive data such as year of publication, trial design, sample size, and type of intervention. ORH further extracted data on evaluation of blinding (as accessed in any way the trialist saw fit), reporting on therapeutic alliance, reporting on participant expectancy (as accessed in any way the trialist saw fit), and trial population. In addition, ORH extracted data regarding published protocol or statistical analysis plan (SAP) through a search of it being mentioned in the published original research paper or by searching clinicaltrials.gov and the European Union Drug Regulating Authorities Clinical Trials Database. He lastly extracted data on whether an independent academic research group or a commercial organization undertook the trial. EDP verified these findings. Data on results were not extracted as the different primary results were subject to high heterogenicity (regarding diagnosis and outcomes), and review of results was not the aim of this review. No secondary publications (including trial registers) were included in the present analysis.
Study quality assessment
The quality of the RCTs in the final sample was assessed using the Risk of Bias (RoB) 2.0 and implementation Excel tool of the Cochrane Collaboration (Higgins JPT, 2021) to assess RCTs’ quality, for the primary outcome only. We further evaluated markers of treatment fidelity. Two authors (ORH and EDP) evaluated each included trial independently the risk of bias domains as being at “low risk,” “some concerns,” or “high risk.” Following this initial rating, each trial was rated according to its highest risk of bias in any domain. We rated studies with “some concerns” in multiple domains as “high risk” of bias, per the RoB 2.0 instructions (Higgins JPT, 2021). Any disagreements were resolved via consensus discussion. Further quality assessment was done narratively.
Results
Study selection
A total of 3909 papers were identified in the search. After removing duplicates, 2896 papers remained and were screened for eligibility. Of these, 162 papers were selected for full-text evaluation of inclusion and exclusion criteria, resulting in the identification of ten primary papers that reported on 10 unique trials (Bogenschutz et al., 2022; Carhart-Harris et al., 2021; Gasser et al., 2014; Goodwin et al., 2022; Grob et al., 2011; Holze et al., 2022; Johnson et al., 2014; Moreno et al., 2006; Palhano-Fontes et al., 2019; Ross et al., 2016) (see Figure 1 for the PRISMA flow diagram). One trial utilized a single-blinded design (Moreno et al., 2006), and nine utilized a double-blinded design (Carhart-Harris et al., 2021; Gasser et al., 2014; Goodwin et al., 2022; Grob et al., 2011; Holze et al., 2022; Johnson et al., 2014; Griffiths et al., 2016; Palhano-Fontes et al., 2019; Ross et al., 2016). No trial utilized a triple-blinded design. Two were crossover (Johnson et al., 2014; Ross et al., 2016), and three were within-subject designs where patients served as their own controls (Grob et al., 2011; Holze et al., 2022; Moreno et al., 2006), while five (Bogenschutz et al., 2022; Carhart-Harris et al., 2021; Gasser et al., 2014; Goodwin et al., 2022; Palhano-Fontes et al., 2019) had a parallel-group design. Two trials utilized a standard placebo (Holze et al., 2022; Palhano-Fontes et al., 2019) and the remaining trials used utilized an active placebo (see Table 1 for further descriptive details). Trials that utilized an active placebo applied a low dose of the intervention drug or niacin. Most subjects had college-/university-level education and were predominantly white. One trial included patients with alcohol use disorder (Bogenschutz et al., 2022), another included patients with OCD (Moreno disordered al., 2006), and the remaining eight trials included patients with either primary mood disorders or mood disorders secondary to somatic illness. Eight studies inquired about participants’ prior use of any psychedelic. Seven trials provided an evaluation of blinding of intervention (Bogenschutz et al., 2022; Gasser et al., 2014; Grob et al., 2011; Holze et al., 2022; Griffiths et al., 2016; Palhano-Fontes et al., 2019; Ross et al., 2016). This was accessed by making either the psychedelic therapist and/or the patient guess the assigned condition or described narratively.
PRISMA 2020 flow diagram with reasons for exclusion.
Data regarding trial design, sample size, diagnostic group, prior psychedelic use, expectancy and alliance, allocation concealment, protocol, and SAP.
WSD: within-subjects design; PGD: parallel group design; C: crossover; SAP: statistical analysis plan; C-R: cancer related; OCD: obsessive-compulsive disorder; TR: treatment resistant; LD: low dose; LSD: lysergic acid diethylamide; Study staff: could be therapists, raters, or other investigators; “–”: no reporting; “+”: reported.
Active placebo.
Nausea inducing placebo liquid (water, yeast, citric acid, zinc sulfate, and caramel colorant).
Inactive placebo.
Only assessed prior LSD use.
Staff guessed wrong at one of two placebo administrations for a single patient.
Data missing for two patients.
Among these were blinding of poor success, and it is not possible to summarize differences in successful blinding of intervention between active and inactive placebos.
No studies published data on expectancy, and only one trial published data on therapeutic alliance (Carhart-Harris et al., 2021). Four trials published a protocol and SAP on clinicaltrials.org or as supplementary materials to the main article (Bogenschutz et al., 2022; Goodwin et al., 2022). Independent academic research groups conducted all trials except Goodwin et al. (2022).
Risk of bias
All trials except Goodwin et al. (2022) were assessed to be at high risk of bias in the overall judgment as these had high risk of bias in one or more domains. Goodwin et al. (2022) were rated as low risk of bias. All trials except Goodwin et al. (2022) were at least rated as high risk of bias in the domain “Measurement of the outcome” partly due to unsuccessful blinding or lack of reporting of blinding of outcome assessor. Goodwin et al. (2022) were rated to have low risk of bias for the primary outcome in this domain based on the use of external and remote outcome accessors, who were explicitly stated to be unaware of the details of the trial and the trial-group assignments. The RoB 2.0 tool assesses the outcome assessors who in this instance are deemed sufficiently blinded; however, participants and trial personnel were likely not. Every trial with crossover rated at a high risk of bias in the domain “Period and carryover effects” due to substantial risk of carryover effects (see Figure 2 for specific results).
RoB 2.0 assessment of RCTs utilizing classical psychedelics in patient populations.
Discussion
A substantial amount of research has been conducted on classical psychedelics since the 1990s. We salute this pioneering work but also acknowledge that such early research into a complex subject can have certain bias and methodological shortcomings. We wish for the best practice of clinical research into classical psychedelics and believe that there is a need for increased focus on the bias in this field.
We found that the included studies had a considerable risk of bias. The greatest contributor to this overall risk of bias was domain 2, “Deviations from intended interventions,” and domain 4, “Measurement of the outcome” of the RoB, where all trials except Goodwin et al. (2022) had substantial issues with blinding. Seven (Bogenschutz et al., 2022; Gasser et al., 2014; Grob et al., 2011; Holze et al., 2022; Griffiths et al., 2016; Palhano-Fontes et al., 2019; Ross et al., 2016) trials published on the success of blinding, and it was generally found to be unsuccessful both concerning patients and study personnel which is in line with previous investigations of the successfulness of blinding in psychopharmacological trials (Hróbjartsson et al., 2007).
Domains 2 and 4 were challenging to score for the trials that did not evaluate blinding of intervention, and it is therefore debatable as to what extent assessors (the patient, for self-reported outcomes) are blinded to the condition. We rated the domain to the most stringent extent. Thus, if evaluation on blinding suggested it, or if data were unavailable, we deemed that outcome assessors were aware of the assigned condition and that it could have affected their assessment.
Goodwin et al. (2022) were, as mentioned, the only trial rated as having a low risk of bias in the “Measurement of the outcome” domain due to the use of an external, remote outcome accessor explicitly stated to be unaware of the details of the trial and the trial-group assignments. However, if one were to rate the same domain of the RoB for another outcome which were self-reported by the patients, would the score have been poorer as Goodwin et al. (2022) do not publish data on the success of patient blinding of intervention. Therefore, based on the approach in this paper, it would have been rated as a “high risk of bias.” The general unblinding of patients and trial personnel is evaluated in domain 2 on the RoB 2.0. Herein the RoB tool first asks if patients and personnel were aware of the intervention, which we, as mentioned, only had reason to believe. This leads to the next question, which asks whether deviations from the intended intervention arose because of the trial context. We had no reason to believe so, and this led Goodwin et al. (2022) to be rated as low risk of bias for this domain.
Limited reporting on and unsuccessful application of blinding are not unique shortcomings for clinical trials on psychedelic therapy as it is observed in other psychopharmacological trials (Baethge and Baldessarini, 2013; Colagiuri et al., 2019; Fergusson et al., 2004; Hróbjartsson et al., 2007; Scott et al., 2022). The poor reporting of blinding of participants and personnel in clinical trials on psychedelics and clinical science, in general, could be explained by several possible reasons; Baethge et al. (2013) speculate that a reluctance to assess blinding stems from trialist’s concern that unsuccessful blinding could cast doubt on positive trial outcomes. Second, it has been pointed out that industry sponsorship is associated with lesser reporting on blinding success (Colagiuri et al., 2019; Scott et al., 2022). Furthermore, there is a lack of a standard to assess the success of blinding. Patients are often asked to guess their treatment at the end of the trial, but such data may be subject to recall bias (Hemilä, 2005; Moher et al., 2015; Sackett, 2007). Lastly, there is no standard for incorporating success with blinding into data analysis, although emerging methods such as the correct guess rate curve exist and have been applied in clinical research of psychedelics (Szigeti, 2022).
Overall, the blinding of participants and personnel appears unsuccessful in most of the clinical trials. A contributing factor to this could be the predominant inclusion of patients familiar with the altered states of consciousness characteristic of the classical psychedelics. This may lead to subject bias in patient-reported outcomes due to this population possibly having influential pre-inclusion expectancy and hopes regarding the potential future of psychedelic therapy.
Adequate blinding of participants and personnel might not be possible in this research field due to the profound subjective effects of the classical psychedelics. Nevertheless, we advise that future research assess and report their efforts to do so as more evidence is needed.
We further suggest that future research employs a clinician-administered outcome accessed by a blinded rater and omits patient-reported outcomes as the primary outcome, as applied in the trial by Goodwin et al. (2022). Lastly, we suggest that trials with a randomized design that compares psychedelic therapy to current treatment regimens, and qualitative designs, could be a valuable supplement to evidence from placebo-controlled trials.
Different types of active placebos were utilized in our sample. The studies utilizing the active condition in a low dose could, in reality, be regarded as dose–response parallel-group designs if one—as the proponent of the field of “psychedelic microdosing”—believes that the low dose is therapeutic (Muthukumaraswamy et al., 2021). Currently, there is not any convincing evidence that suggests that psychedelic microdosing has any clinical effect (Szigeti et al., 2021), but, as was pointed out by Muthukumaraswamy et al. (2021), the open-label trial of psilocybin for depression by Carhart-Harris et al. (2016) showed an effect from administration of only 10 mg psilocybin. In that trial, the Hamilton Depression Rating Scale (HAM-D) scores dropped from 21.4 to 10.7 following administration of the first “microdose” of 10 mg psilocybin, and after the full dose (25 mg) these dropped to 7.4. Thus, 76% of the therapeutic effect on the HAM-D was elicited by the smaller dose. It is debatable at what threshold a psychedelic “microdose” becomes a psychedelic “minidose.” However, this dosage was possible in relation to the outcome, too high to be a therapeutically inactive placebo, and the therapeutic effect experienced in the trial might possible not have been entirely due to placebo effect and to participation in a trial with psychotherapy. Likewise, Goodwin et al. (2022) found that a dose of 1 mg psilocybin entailed noticeable reduction in depressive symptoms. Suppose this had been the case in an RCT with active placebo psilocybin and perfect blinding of participants and personnel, this could have led to an underestimation of the effect size of the active condition since the low dose was causing a treatment effect (Muthukumaraswamy et al., 2021).
We found little difference in effectiveness between active and inactive placebo. However, it is essential to highlight the lack of data on this issue, and the overall lack of research into the possible therapeutic effect of active placebo (Laursen et al., 2020). We encourage the continued use of an active placebo and regard it as the least flawed method to ensure effective blinding of participants and personnel in trials.
In addition to discussion blinding of intervention and placebo choice regarding psychedelic medicine, we wish to point out more generic issues emerging from our risk of bias assessment:
(a) Most trials were judged as low risk of bias in the domain “Randomization process.” The trial, which was rated as “some concerns,” had an imbalance in baseline measures which suggested an imbalance in the randomization process, in addition to a lack of details regarding the randomization process.
(b) Many trials had small sample sizes, leaving them vulnerable to participant dropout. For example, we rated Davis et al. (2021) and Gasser et al. (2014) as “high risk of bias” in domain 2 “Deviations from intended interventions” due to substantial dropout of patients (1 and 3 patients, more than 5% as per Cochrane guidelines), who were then excluded from the analysis, as per protocol. The small samples reflect that these studies were primarily done in academic settings, presumably on a small budget, and with limited resources overall, compared to conventional phase 2 or phase 3 studies typically carried out by the medical industry. In addition, these small samples are not able to control or adjust for confounders; they are inadequate to conduct mediation analysis to control for confounders such as expectancy and therapeutic alliance (Muthukumaraswamy et al., 2022).
(c) Two trials (Grob et al., 2011; Griffiths et al., 2016) failed to report information on missing outcomes, including a statement about no information on missing outcome data, how missing data were treated, and how data were analyzed (intention-to-treat or per-protocol).
(d) Four trials (Gasser et al., 2014; Grob et al., 2011; Griffiths et al., 2016; Ross et al., 2016) utilized a randomized crossover or randomized within-subjects design. These were rated on the “Period and carryover effects” domain of the RoB 2.0 for crossover trials and were all at a high risk of bias. One of the main psychedelic hypotheses is that a single treatment—that occasions a mystical type experience—can lead to permanent positive clinical changes. The crossover trial design also has problems due to the possibility that participants’ beliefs about the condition they have been assigned will change as they progress in the trial.
(e) Four (Bogenschutz et al., 2022; Carhart-Harris et al., 2021; Goodwin et al., 2022; Ross et al., 2016) trials published study protocols and SAPs. This led the remaining trials except Holze et al. (2022) to be rated “Some concerns” in domain 5, “Risk of bias in selection of the reported result” due to a lack of information on the trial design and planned statistical analysis. None had published a protocol in advance in a peer-reviewed scientific journal.
We advise that investigators of future trials seek to publish their trial protocol in an appropriate peer-reviewed scientific journal before the finalization and publication of their trial. This will allow the scientific community to evaluate whether the eventual analysis and results are consistent with the investigators’ original intent. The peer-review process will further ensure outside input on trial design which might otherwise be missing since most clinical trials on classical psychedelics have been funded through philanthropic donations of ordinary citizens, and not large grants from foundations which perform thorough peer reviews of grant proposals. Furthermore, publication of a trial protocol will add accountability for authors to report their results in a timely and accurate manner, thereby decreasing publication bias. Likewise, publication of trial protocols informs the public and the scientific community about which trials are being conducted, which helps avoidance of unintended duplication and betters coordinated research efforts (Ohtake and Childs, 2014; Smith and Martin, 2011; West, 2012). We note that some of the upcoming clinical trials on psychedelics have published protocols (Spriggs et al., 2021) and advise that all upcoming clinical trials of psychedelics publish a study protocol and SAP beforehand.
(f) None but one trial (Murphy et al., 2022) assessed the psychotherapeutic treatment fidelity measures. This constitutes a possible risk of bias since the therapeutic utilization of psychedelics includes necessary elements of psychotherapy administered to both arms, where the psychedelic is regarded as an adjunct to psychotherapy. Treatment fidelity is thus necessary to maintain validity and to ensure a fair comparison between psychotherapeutic interventions (Moncher and Prinz, 1991). In the absence of fidelity evaluation, one cannot ascertain whether or not a treatment effect is due to the intervention or to nonspecific treatment factors (Cook and Campbell, 1979). Concerning this, only one trial assessed the working/therapeutic alliance (Murphy et al., 2022). We believe it is important to assess fidelity and therapeutic alliance in RCTs with classical psychedelics to quantize possible systematic differences in the therapeutic alliance between groups. Assessment can be done with standardized instruments such as the Scale to Assess the Therapeutic Relationship Patient Version (McGuire-Snieckus et al., 2007) or the Working Alliance Inventory (Horvath and Luborsky, 1989).
Relating to this did none of the included trials assessed patient expectancy. We find it essential to assess patients’ expectancy, and suggest that future research assesses participant expectations prior to randomization to evaluate the possible contribution of expectancy. Standardized instruments such as the Credibility/Expectancy Questionnaire (Younger et al., 2012) could be utilized for this purpose.
(g) The trials examined in this review generally had a homogeneous population and included patients with primarily mood disorders and patients with symptoms of depression and anxiety secondary to somatic illness. Of the studies that reported the participants’ education level, the majority had college-/university-level education. Furthermore, the included patients were predominantly white, although many studies did not report patients’ ethnicity or race. We suspect that the patients included in the trials are not representative of real-world patient populations which might lead to self-selection bias. Concerning this, we found that in most trials only a small fraction of the participants evaluated for inclusion were included. Many potential participants were excluded due to psychiatric comorbidity, in line with current guidelines for safety in clinical trials on psychedelics, which advise that patients with a personal or family history of psychotic illness should be excluded. We suggest that future trials seek to employ fewer exclusion criteria and include more participants with coexisting personality disorders, attention disorders, or other non-psychotic psychiatric illnesses commonly found in real-life clinical populations. We further suggest that future trials seek to include patients with a more heterogeneous demographic profile, such as patients from minorities (Brown et al., 2014) (including LGBTQ+ populations) and those who have a more diverse profile regarding education level. This will help to ensure the generalizability of the results obtained and counteract the sampling bias typically seen in psychiatric research (Brown et al., 2014; George et al., 2020).
(h) One trial was industry sponsored (Goodwin et al., 2022), and this could possible lead to the so-called “Industry Sponsorship bias (Holman et al., 2019),” which refers to the tendency of studies to support the interests of the trial’s financial sponsor. This tendency is supported by a 2017 Cochrane meta-analysis which compared industry-sponsored research with otherwise funded research, and found that industry-sponsored research were more likely to favor the sponsor’s products (relative risk = 1.27; 95% CI 1.17–1.37) and that authors’ conclusions were more favorable (relative risk = 1.34; 95% CI 1.19–1.51) (Lundh et al., 2017).
A limitation of this systematic review is that we did not contact the corresponding authors to inquire about any unreported findings related to the aim of our review. It is possible that such data exist and could have influenced our assessment of the quality of the studies and our discussions.
In summary, we found considerable bias in the published clinical research on psychedelic therapy with patient populations. Based on the present review, we suggest the following: The traditional parallel-group, placebo-controlled design is best for investigating classical psychedelics in clinical samples. Due to the distinct subjective effects of classical psychedelics and lack of reporting on blinding of intervention, we recommend application of an active placebo to best ensure blinding. An active placebo must have subjective effects like those of classical psychedelics, and at present, a low dose of a classical psychedelic seems as the best active placebo. This deception should be easier to ensure in a sample of psychedelic-naïve participants. However, a low dose of a psychedelic substance as a placebo could be problematic as it might have therapeutic effects. This solution should not be regarded as perfect but as the least flawed solution known at present. We further suggest that researchers employ clinician-administered outcomes rated by blinded accessors, even though this, again, represents the least flawed solution known at present. Trialists could further evaluate the blinding of participants, and preferably, this might be done multiple times during a trial to track changes in participants’ beliefs, and we suggest doing this after each administration of psychedelic or placebo. We further suggest that future trials publish protocols and SAPs prior to finalization and emphasize reporting on expectancy, participants’ prior psychedelic use, therapeutic alliance, and success of allocation concealment.
Footnotes
Appendix A: Search strategy
Author contributions
ORH and EDP conceptualized the review. ORH and EDP carried out literature screening and full-text reading, data extraction, and tabulation. ORH was responsible for writing the first draft manuscript. OJS and SA contributed with comments and guidance. The manuscript was finalized by ORH and EDP. All authors have discussed, reviewed, and approved the manuscript.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: ORHs PhD which is not related to the present paper is financed by the publicly funded Region Zealand Mental Health Services, and the not-for-profit organization TrygFonden and the A.P. Møller Fonden, Fonden til Lægevidenskabens Fremme. EMDs pregraduate research stipend is granted by the Lundbeck Foundation. SA and OJSs professorships are internally funded by the Region Zealand Mental Health Services. SA and OJS have not received any pharmaceutical-industry sponsorships.
