Sage Journals: Discover world-class research

Abstract

Randomized controlled trials (RCTs) are considered the gold standard for demonstrating the efficacy of a treatment or intervention; however, the RCT study design alone is insufficient to prevent all forms of bias. In this paper, we review conflicts of interest and other biases that pose challenges to the methodological rigor of psychedelic RCTs, and explore recommended bias mitigation measures. We focus on biases of special relevance to psychedelic RCTs and methods of mitigating bias that we believe are actionable and urgent practices to implement in psychedelic research: (1) independent data assessment, (2) multisite trials, (3) quantitative bias analysis, and (4) open science practices. Although we advocate for the adoption of these practices in psychedelic research, they are not psychedelic specific. We suggest that the challenges posed by psychedelic research offer an opportunity for researchers to creatively find and implement solutions across the study lifecycle to reduce bias in other fields of clinical research.

Keywords

Bias randomized controlled trials independent assessment multisite trials open science

Introduction

Randomized controlled trials (RCTs) are considered the gold standard for demonstrating the efficacy of a treatment or intervention. Randomizing participants to experimental or control condition(s) balances both observed and unobserved participant characteristics between groups, in addition to ensuring that participant allocation to each group is unpredictable, such that differences in study outcomes can be causally attributed to the study drug/intervention (Hariton and Locascio, 2018). The re-emerging field of psychedelic¹ research has conducted numerous RCTs testing if various psychedelic compounds can improve symptoms of mental disorders including depression and anxiety (e.g. Raison et al., 2023; Goodwin et al., 2022; Carhart-Harris et al., 2021; Griffiths et al., 2016). Yet despite its considerable strengths, the RCT study design alone is insufficient to prevent all forms of bias, both in psychedelic research and in general.

Bias and its impact on study results recently became the center of an FDA Advisory Committee discussion appraising the safety and efficacy of midomafetamine (MDMA) capsules for PTSD, which the FDA declined to approve (Reardon, 2024). Methodological problems in the sponsor’s clinical trials included functional unblinding, as 90% of participants randomized to MDMA were able to guess their study group assignment (MDMA or placebo). As a result, biases such as detection bias (when groups in a trial differ in the way outcomes are collected or verified), performance bias (differences that occur due to knowledge of intervention allocation), and expectation bias (when people’s expectations about the experiment can influence the results) may have affected study results (FDA, 2024; Mustafa et al., 2024; van Elk & Fried, 2023). Furthermore, concerns were raised that investigators, therapists, and participants allegedly held strong prior beliefs about the benefits of MDMA-assisted therapy. It was alleged that some trial therapists had reportedly discouraged negative reports by participants, and encouraged positive ones, thereby potentially introducing demand effects and resulting in skewed reports of harms and benefits (Mustafa et al. 2024). These, and other issues, have led some to doubt the validity of the data and the accuracy of reported harms and benefits from the sponsor’s trials (Mustafa et al., 2024). It has proven frustratingly difficult to understand whether such biases were present, and if so, their scope, magnitude, or effect on the data.

Concerns about bias in clinical trials are, of course, neither new nor specific to psychedelic research. Jacques Normand, study director of an Institute of Medicine panel that investigated needle exchange programs, noted that three conditions may contribute to concerns around bias in clinical research: “(a) a sensitive topic (e.g. something illegal or controversial); (b) ideological differences concerning the outcomes being examined; and (c) the research contains uncertainty” (Bachrach and Newcomer, 2002). Psychedelics may fall under many, if not all of these conditions, as these substances were outlawed in the United States by the Controlled Substances Act of 1970 and demonized by the media in the 1960s and 1970s (Siff, 2008; Siff, 2015). Uncertainty, Normand argues, may also be present if for “ethical or logistical reasons, the science is unable to implement the gold standard of a double-blind, randomized controlled trial,” (Bachrach and Newcomer, 2002) and de-blinding is known to occur in psychedelic studies (Marks, 2024; Muthukumaraswamy et al., 2021). A number of other factors (e.g. financial conflicts of interests, investigator biases, intellectual conflicts of interests, etc) can also impact data collection, rigor, and research integrity.

Researcher-level biases in clinical trials are important to address, as they violate the principle of equipoise. Theoretical equipoise (also known as personal equipoise) requires the investigator to be in a state of genuine uncertainty about the comparative effectiveness of each arm of the trial (Fried, 1974), while clinical equipoise argues that the condition of equipoise is satisfied if the medical community is in a state of genuine uncertainty (Freedman, 1987). Violations of equipoise can lead to bias, that in turn may account for a proportion of the effect of the intervention (i.e. a proportion of the effect seen is due to bias and not the intervention itself) (Cook and Sheets, 2011). Equipoise is held as an ethical justification for randomization in clinical trials, and if a trial demonstrates a consistent strong effect of the intervention (entailing that there may no longer be equipoise), all participants must be unblinded and receive the treatment (Deichmann et al., 2016). By reducing researcher-level biases, equipoise can be better maintained in clinical trials.

To address the issue of researcher-level bias, a number of measures have been proposed for clinical trials, and more specifically for psychedelic clinical trials, but have generally been unevenly applied (Buchman et al., 2024; Hovmand et al., 2023; Bradley et al., 2020). Some potential solutions include Open Science practices (methods that are aimed at introducing transparency into the research process, increasing accessibility of the data and sharing of methodology), and the application of bias-mitigation policies and regulations that the pharmaceutical industry is subject to. Uptake of Open Science practices have been steadily increasing, but are not yet the norm in many fields (Armeni et al., 2021; Christensen et al., 2020). Similarly, pharmaceutical clinical trials in the United States are subject to certain regulations that may help to address bias (Cashin et al., 2023). However, these practices are rarely combined in academic research settings—in this paper, we advocate for their application, using academic psychedelic clinical trials as an example.

We focus here on researcher-level biases that have been recently discussed as posing challenges to the methodological soundness of psychedelic clinical trials. Using adverse events reporting as a case study, we examine the impact of biases on psychedelic science. We then review measures that have been previously recommended in the literature to address these factors, then argue for four methods of reducing bias and conflicts of interest that we believe are particularly actionable and urgent practices to implement: (1) independent data assessment, (2) multisite trials, (3) quantitative bias analysis, and (4) greater use of Open Science practices. Crucially, we believe that these practices will help respond to growing calls to promote independence and insert independent assessment practices within the scientific process (Klein and Kesselheim, 2024; van Elk and Fried, 2023). We detail the rationale for these and current clinical trial bias-reduction standards and then analyze their application specifically in psychedelic research contexts. We advocate for the adoption of these measures, along with improved science communication and transparency. Overall, we aim to increase awareness of the real and perceived sources of bias in psychedelic clinical trials, and articulate ways that they can be addressed in their design, conduct, and reporting.

Researcher biases and conflicts of interests in clinical trials

Recent work in the research ethics literature has pointed to several factors that can bias the analyses and reporting of clinical trial data. Financial conflicts of interests (COIs) (see Table 1 for an overview of various COIs) have long been recognized as a potential factor leading to unethical, illegal, or unprofessional behavior on the part of the researcher (Field and Lo, 2009), with COIs in drug and treatment studies sponsored by pharmaceutical companies strongly associated with the reporting of positive findings (Friedman and Richter, 2004). In psychiatric clinical trials, financial COIs are common, with nearly half of all studies reporting at least one author with a COI (Perlis et al., 2005). Within psychedelic clinical trials, recent analyses suggest that financial relationships with industry and philanthropic funders may similarly risk the autonomy and integrity of the research (although such a risk is applicable to all research) (Buchman and Rosenbaum, 2024). Given the predominance of funding from philanthropic sources, industry, and individual donors, and the historic lack of federal funding for psychedelic research, financial COI may be of special relevance in psychedelic science (Petranker et al., 2020).

Table 1.

Types of conflicts of interest.

Conflict of interest
Type	Definition	Reference
Financial	A financial conflict of interest exists when the institution’s designated official(s) reasonably determines that an investigator’s significant financial interest (SFI) could directly and significantly affect the design, conduct, or reporting of the (NIH-funded) research.	NIH Financial Conflict of Interest Definition https://grants.nih.gov/grants/policy/coi/index.htm
Non-financial	Each and every condition other than financial, that can generate, or be perceived as generating bias at any stage in the scientific process. Non-financial COIs (sometimes called “private interests”) can be personal, political, academic, ideological, or religious	PLoS Medicine Editors (2008). Making sense of non-financial competing interests. PLoS Med. 5, e199.
Intellectual	Academic activities that create the potential for an attachment to a specific point of view that could unduly affect an individual’s judgment about a specific recommendation	Guyatt Et al. (2010). The vexing problem of guidelines and conflict of interest: A potential solution. Annals of Internal Medicine 152(11):738–741.
Publication	Failure to publish the results of a study on the basis of the direction or strength of the study findings	Dickersin K, Min YI . Publication bias: the problem that won’t go away. Ann N Y Acad Sci 1993;703:135–48.
Competing interests	Financial and non-financial interests that could directly undermine, or be perceived to undermine the objectivity, integrity and value of a publication, through a potential influence in the judgements and actions of authors with regard to objective data presentation, analysis and interpretation	Nature Portfolio Editorial Policies. Competing Interests. https://www.nature.com/nature-portfolio/editorial-policies/competing-interests

Adverse Events Reporting: A potential outcome of neglecting biases is the inaccurate collection and reporting of adverse events³ (AE) data. Historical evidence suggests that harms are underreported in clinical trial publications across the majority of medical fields (Ioannidis and Lau, 2001). The underreporting of adverse events threatens the validity of systematic reviews and meta-analyses of harms, and has significant implications for clinicians, patients, regulators and other stakeholders who rely upon the published literature to make evidence-based decisions (Golder et al., 2016).
AE Reporting in Psychedelic Clinical Trials: While the underreporting of adverse events (which are expected to be documented throughout the trial) is an issue for all clinical trials (Bachrach and Newcomer, 2002), it has been raised as a specific concern with regard to psychedelic trials. The assessment of adverse events in psychedelic studies can be particularly challenging given the difficulty in differentiating between side effects and therapeutic effects of the psychedelic experience (Calder and Hasler, 2024). A 2022 mixed-methods systematic review of clinical studies of psychedelic drugs found that two out of the twenty included psilocybin studies, two out of the three included ayahuasca studies and one out of the eleven included MDMA studies did not report adverse events (Breeksema et al., 2022). This may be at least in part because investigators, potentially balancing the roles of researcher, clinician, and sponsor, may have conflicts of interest as they determine the relevance of the adverse event to the study drug (IOM, 2006).
Addressing bias leading to the underreporting or misrepresentation of adverse events is critical for all research, given that rates of adverse event reporting can be quite low in other fields. For example, adverse event recording and reporting for psychotherapy interventions is generally weak - one study found that none of the 82 National Institute of Health Research (NIHR) funded trials of psychological interventions from 1995-2013 mentioned the occurrence of AEs in their final report (Duggan et al., 2014). While high levels of scrutiny have been aimed at psychedelic research, it is worth noting that rates of AE collection in psychedelic studies are higher than in other clinical research domains - in a systematic review of classic psychedelic clinical trials, Hinkle et al. found that 53.3% of trials reported some AE data, compared to 46% for trials of health interventions (Hinkle et al., 2024). A systematic review of psychotherapy research (85 RCTs) found that AEs (or the absence of) were reported in only 35.3% of trials (Klatte et al., 2023). This is an area for improvement across all of science.
Proposed Solutions: Transparently and forthrightly collecting and publishing all adverse events data can help to improve trust in the field, combat the perception that psychedelic researchers are overstating the benefits without addressing harms, and reduce the hype around psychedelics (Meling et al., 2024). Elk and Fried suggest that safety and adverse events should be included as primary outcome measures, with an independent arbiter, instead of researchers, deciding whether or not an adverse event is drug-related (van Elk and Fried, 2023). Breeksema et al additionally propose that in order to combat the inadequate assessment of adverse events, studies should describe the timing and severity of effects more extensively, and to draw upon qualitative research methods to help better understand the nature of challenging experiences (Breeksema et al., 2022). Similarly, Hinkle and colleagues (2024) argue for clear and systematic methods to ascertain adverse events and their severity. Determining adverse events of special interest (AESIs) may help promote standardized data capture and reporting practices for adverse events that are more specific to or prevalent in psychedelic research (Cheung et al., 2024; Ehrenkranz et al., 2024; Hinkle et al., 2024).

More attention has recently been paid to non-financial COIs—such as personal relationships, political activity, or the desire to please a funder—and the harms that they can cause (Grundy, 2021; Wiersma, 2018; Yusuf & Bosch, 2006). Financial and non-financial COIs are frequently intertwined; for example, pharmaceutical companies may encourage collaborations with physicians through both financial incentives and non-financial ones (for example, public recognition or endorsement of an individual’s expertise). Some effort has been made to assess the impact of non-financial COIs on biomedical research: Viswanathan and colleagues, in their investigation of strategies to mitigate non-financial COIs in systematic reviews, argue that ignoring non-financial interests may cast doubt on the impartiality of a review (Viswanathan et al., 2014). In academic clinical trials, flawed or inaccurate results may arise due to unduly strong affinity for a specialty, to a scientific theory, or to academic appearance (Østengaard et al., 2020).

A key subset of non-financial COIs are intellectual COIs. Although it may be difficult to delineate when a viewpoint becomes an intellectual COI, or whether all interests are conflicts of interest (see Bero and Grundy, 2016 for discussion), preliminary evidence has pointed to the potential bias associated with intellectual COIs. For example, one study found that primary authors of studies included in a meta-analysis were more likely to interpret the results of a meta-analysis as indicating a strong association compared to independent methodologists (Panagiotou and Ioannidis, 2012). In the context of clinical practice guideline development, Norris and colleagues found that the recommendations given for mammography screening guidelines may be affected by the specialty and intellectual interests of the guideline’s authors (Norris et al., 2012).

It is important to note that a risk of bias can still be present even if researchers do not deliberately manipulate or misrepresent their data (i.e. commit fraud), as even when trying to be objective, researchers’ judgements may still be subject to unconscious biases (Dana and Loewenstein, 2003). One source of bias for all researchers is their disciplinary perspective and the theoretical models embedded within, especially when research starts with a narrow conceptual framework. Multidisciplinary collaborations may be needed to see how researchers’ perspectives influence their work (Bachrach and Newcomer, 2002). Environmental influences may additionally be a subtle source of investigator bias (e.g. if an investigator’s university has particularly close ties with industry) (Bachrach and Newcomer, 2002). Other factors that may affect clinical trials include a number of subconscious biases such as confirmation bias—the tendency to believe what one wants or expects to believe—and anchoring bias, or the tendency to cling to a belief despite accumulating contrary evidence (Kahneman, 2011; Tversky and Kahneman, 1974).

In psychedelic research (and other fields), bias can stem from sources other than the primary investigator, such as from industry sponsors and trial therapists. As with other fields, industry involvement in psychedelic clinical trials may impact safety outcome data and trial results in a variety of ways, for example, through the selective reporting of data and the favoring of study designs that may not detect adverse events (Koning et al., 2024). Regarding non-financial COIs, as one example, the MDMA trials reviewed by the FDA (discussed above) were sponsored by a single advocacy organization, raising concerns about the influence of ideological biases on trial results (Bedi et al., 2023). Various stakeholders in psychedelic research, such as trial therapists, may also hold intellectual COIs, for example, by believing in the benefits of psychedelics to the extent of ignoring all contradictory evidence (Petranker et al., 2020). As one such case, allegations of trial therapists holding strong positive beliefs about MDMA-assisted therapy in the FDA-reviewed trials were raised, which may have led to inaccurate reporting of adverse events and outcomes (Marks, 2024). Some allege that personal use of psychedelics by trial therapists can also threaten objectivity and may be fairly common, with one survey of facilitators associated with a Phase II psychedelic trial finding that 88% had experience with at least one kind of psychedelic (although limited by a small sample size, n = 32) (Aday et al., 2023). Although personal experience with psychedelics may have benefits (Rosenbaum et al., 2024), it may also bring with it a risk that the therapist might introduce personal beliefs or interpretations to the participant which may then bias study results (Johnson, 2020).

At the researcher-level, concerns have been raised that personal use of psychedelic substances by psychedelic researchers may compromise researcher objectivity and research integrity, with one study finding that researcher associations with psychedelic culture may decrease public confidence in research quality and validity (Forstmann et al., 2021; Kious et al., 2023)². Other worries involve the coupling of research with advocacy (e.g. researchers supporting the legalization of psychedelics), which some argue may also present a threat to equipoise (Poppe and Repantis, 2024).

Finally, functional unblinding, or the discerning of the assigned treatment by the researcher or participant, is an oft-encountered source of bias within psychedelic clinical trials due to the drug’s distinctive psychoactive effects (Muthukumaraswamy et al., 2021). This threat to equipoise may be compounded by other sources of bias that may be present, for example, if researchers already hold strong convictions about the beneficial effects of psychedelics.

In attending to the various biases and conflicts of interest in psychedelic clinical trials, it is important to recognize the diverse sources from which they may originate. In Box 1, we explore one potential consequence of ignoring biases in research that is of particular interest and importance in psychedelic trials: the inaccurate reporting of adverse events, efficacy, and other outcomes. Case Study: Inaccurate Adverse Events Reporting - A Risk of Ignoring Biases in Clinical Trials, and Proposed Solutions. In the following section, we review measures that have been proposed to mitigate bias in clinical trials.

Brief review of measures proposed to reduce bias in clinical trials

Various measures have been proposed and implemented to mitigate these biases in general, particularly within the context of industry-sponsored trials (for a select summary, see Table 2). In recognition of the biases potentially introduced by financial COIs, the majority of biomedical journals and funders (estimated to be over 90% of journals) have adopted financial COI policies and guidelines for researchers (Resnik, 2023). For example, the Public Health Service and the National Science Foundation in the United States requires funded institutions to adopt written policies for the disclosure and managing of funded investigators’ financial interests (National Science Foundation [NSF], 2005; National Institutes of Health [NIH], 2022), whilst many journals have modeled their policies on those published by organizations like the International Committee of Medical Journal Editors (ICMJE) (Resnik, 2023; International Committee of Medical Journal Editors [ICMJE], 2024). Moreover, the ICMJE has recently updated its policies to classify the intentional non-disclosure of COIs as a form of scientific misconduct (International Committee of Medical Journal Editors [ICMJE], 2018). Guidelines for disclosing financial COIs within the context of clinical trials and research have also been proposed and developed (Gottlieb, 2020). However, it has been suggested that COI disclosures may not be the most effective method of avoiding biases, and instead provide a false sense of comfort as they may not actually achieve their goals of preventing actual or apparent bias, nor of helping the reader to assess the influence of the COI present (Dunn et al., 2016; Goozner, 2010; Tresker, 2022; Yusuf and Bosch, 2006). De Melo-Martin and Intemann, drawing upon evidence showing that COI disclosure policies do not discourage researchers from forming financial relationships with industry, argue that COI disclosures do not help readers and reviewers identify bias in published articles (de Melo-Martin and Intemann, 2009), pointing to the need for other measures to help mitigate the impact of financial COIs.

Table 2.

Select summary of measures proposed to address biases.

General bias-reduction measures proposed (non-psychedelic specific)
Issue	Solution	References
Non-disclosure, or incomplete disclosure, of COIs	Mandatory disclosure of COIs	Ross et al. (2012)
Publication bias, selective reporting	Preregistration of trials e.g. on ClinicalTrials.gov	McDonnell et al. (2021)
Publication bias, selective reporting	ORCID (unique, persistent identifiers for researchers)	McDonnell et al. (2021)
Intellectual COIs	Include a diversity of viewpoints and disciplines, for example, through adversarial collaborations	Bachrach and Newcomer (2002), Clark and Tetlock (2023), Sniderman et al. (2012)
Underreporting of adverse events	Improvement of recording and reporting methods	Lineberry et al. (2016)
Viewpoint bias	Adversarial collaboration	Clark et al. (2023)
Underreporting of adverse events	Standardization of selection criteria and methods of collecting AE data	Cornelius et al. (2013)
Bias-reduction measures proposed for psychedelic clinical trials
Intellectual COIs	Inclusion of researchers without personal or financial ties to psychedelic medicine	McNamee et al. (2023)
COIs	Transparent reporting of COIs	van Elk and Fried (2023)
COIs	Preregistration	van Elk and Fried (2023)
COIs	Application of Open Science standards	Petranker et al. (2020)
Adverse events	Use of an independent arbiter to decide whether an adverse event is drug-related	van Elk and Fried (2023)
Adverse events	Inclusion of safety and adverse events as primary outcomes measures	van Elk and Fried (2023)
Adverse events	More extensive descriptions of timing and severity of adverse events effects	Breeksema et al. (2022)
Adverse events	Improve fieldwide standard for adverse events	Calder and Hasler (2024), Ehrenkranz et al. (2024), Hinkle et al. (2024), Palitsky et al. (2024)

A number of policies for scientific grant review and other research activities have been created to address non-financial COIs, while an increasing number of journals have begun to adopt policies regarding non-financial COI disclosures (Resnik, 2023). For example, the ICMJE’s guidelines on COI disclosure explicitly mention that interests such as personal relationships, academic competition or intellectual beliefs may represent or be perceived as COIs, and should be disclosed (International Committee of Medical Journal Editors [ICMJE], 2024). Similarly, in 2018, Nature began requiring authors to disclose non-financial conflicts of interest, including unpaid memberships in advocacy organizations and acting as an expert witness (Nature, 2018). More generally, Sniderman et al. and Bachrach and Newcomer both advocate for the inclusion of a diversity of viewpoints to help combat intellectual COIs, arguing that fostering an open contest amongst different perspectives is one of the best protections against biases stemming from our viewpoints (Bachrach and Newcomer, 2002; Sniderman et al., 2012).

Adversarial collaborations may be one particularly well-suited method to accomplish this (Mellers et al., 2001). Engaging in adversarial collaboration restricts a researcher’s ability to design a study that preemptively favors their desired outcome (Clark and Tetlock, 2023). However, the adoption of measures within clinical trials to mitigate non-financial COIs, (e.g. employing an independent trial statistician or establishing an additional independent data safety and monitoring board) can be varied.

There have also been recent proposals in the literature specific to the psychedelic clinical trial context that aim to address COIs and researcher biases by inserting independence within the research process. McNamee and colleagues call for researchers “without personal or financial ties to psychedelic medicine” to, amongst other suggestions, join the research teams of clinical trials (McNamee, Devenot & Buisson, 2023), while Barnett and colleagues advise being “dispassionate and objective in the study of psychedelics, avoiding the quasi-religious stance that has come to characterize some elements of the ‘psychedelic movement’” (Barnett et al., 2024, p. 2). Elk and Fried similarly argue for the inclusion of industry-independent experts in all stages of the clinical trial process for industry-sponsored trials (e.g. design, data collection, writing).

To address bias from functional unblinding, employing dose-response or active comparator study designs, or using active placebos, can help mitigate this issue. Additionally, symptoms could be assessed by independent raters who would be ideally unaware of the treatment’s nature (Butler et al., 2022; Wen et al., 2024). In addition, recording participants’ and facilitators’ past experience with psychedelics, or limiting the proportion of participants with prior psychedelic use, may be helpful. Improving reporting procedures for blinding, and doing so in a quantitative and systematic fashion, will also be important for the field (Wen et al., 2024).

Other suggestions to help address COIs in psychedelic science include the transparent reporting of COIs in a publicly accessible manner (e.g. not behind a paywall in journals, upfront at conferences), and the implementation of measures that can help safeguard against COIs, such as preregistration and blinding of group membership during analysis (van Elk and Fried, 2023). Preregistration, adherence to Open Science principles, increased replication efforts and reporting constraints on generalizability to limit hype are also called for by Petranker et al. (2020). Finally, we call for increased federal funding in the psychedelic research field to help mitigate both financial and non-financial COIs for researchers.

Proposals to safeguard and insert independence within the scientific process

In the following section, we detail four methods that we argue will help respond to the above-described calls to insert independent assessment within the scientific process (e.g. for adverse events reporting and for the review of prior research) (Klein and Kesselheim, 2024; McNamee et al., 2023; van Elk and Fried, 2023; Petranker et al., 2020). The methods described below introduce independent assessors or sites to help reduce the influence of researcher-level biases, with quantitative bias analysis and Open Science measures adding an additional stage where the potential influence of bias can be checked and mitigated.

Independent assessors and sites— if sufficiently resourced, free from conflicts of interest and separate from the institution running the trial/experiment— can add a valuable layer of oversight to trial procedures, especially those which include a subjective component (Dodd et al., 2008). Their use can promote objectivity in the running and analysis of the trial, reduce bias, and protect participant safety, all of which may contribute to promoting the trial’s credibility and trustworthiness (Schöffski, 2021)—one survey from the Pew Research Center (2019) found that open access to data and independent review increased Americans’ trust in research findings. Although more work is needed to describe and standardize the concept of independence and how it can be established for the assessor/site, such methods can contribute to improving the rigor of research conducted not just in psychedelic science, but in all fields.

Proposal 1: Independent assessment of clinical trial data

Independent/external assessments of clinical trial data (e.g. of primary outcomes, adverse events) are one method that can help combat researcher biases and conflicts of interest, improve clinical trial data’s rigor and accuracy, and ensure objectivity (Ross et al., 2012). Such an assessment could consist of “an independent evaluation of the data conducted by professionals with no ties to, or financial compensation from, the sponsor or its surrogates. . .” and “accompany the published abstract in medical journals” (Haines and Miklos, 2011, p. 10). As one example of this in practice, JAMA requires industry-sponsored studies to have data analysis conducted by an independent biostatistician (Deangelis and Fontanarosa, 2010). Specific calls from psychedelic researchers for the injection of more independence include suggestions to hire independent auditors to investigate and share results from clinical trials to help rebuild trust, and for independent arbiters to determine whether adverse events are related to the treatment in question (Marks, 2024; van Elk and Fried, 2023).

Independent assessment and oversight methods are not new, and have been implemented for clinical trials in various medical areas. Independent oversight of research frequently occurs via Institutional Review Boards (IRBs), while Independent Data Monitoring Committees (IDMCs) are commonly used in both academic and industry-sponsored trials to help protect the interests of the patient, and to ensure participants are not being exposed to unnecessary risk. IDMCs may additionally analyze interim data for large studies, with a focus on the safety of the patients enrolled (Sartor and Halabi, 2015; Walters, 1993). In response to concerns about investigator bias and honest reporting of adverse event data in industry-sponsored trials, Auerbach and colleagues propose that an Independent Clinical Events Committee (CEC) could help to review and adjudicate clinical events data accordingly (Auerbach et al., 2013). They found that for an industry-sponsored spine surgery clinical trial, an independent CEC reclassified 37% of adverse events (the large majority being an update in severity)—such a committee thus could potentially help to mitigate investigator bias and facilitate an accurate assessment of safety data (Auerbach et al., 2013). Another study found that for a Phase III oncology clinical trial, investigators tended to overestimate response rates when compared with an Independent Review Committee, pointing to the benefit of their inclusion for reducing investigator bias (Tang et al., 2010).

For psychedelic clinical trials, implementing an independent assessment of data post-publication could take the form of making more data publicly available, following the principles of Open Science, with appropriate measures to secure participant confidentiality, thereby allowing independent researchers to perform their own analysis. Pre-data publication, independent assessment could entail having an independent panel or committee analyze interim data (including video recordings of experimental procedures) at prespecified time points to ensure that data is being collected accurately and that patient safety is not compromised. This analysis and review could also be conducted by a Contract Research Organization. Following van Elk and Fried (2023) and Auerbach and colleagues, an independent committee could also make decisions regarding the relatedness and severity of adverse events.

Proposal 2: Multisite trials

Multisite trials are another method that may reduce the risk of bias and improve the rigor of clinical trial data. Such trials can reduce personal biases from researchers in running the study, given the variety of sites and researchers (Das, 2022). Moreover, they provide opportunities to draw upon various methodological, statistical and clinical expertises and perspectives, given the participation of a larger number of investigators (Fuller et al., 1994). There may also be a decreased risk of data manipulation and fishing, while having diverse sites also allows for comparison across site groups and the detection of variation (Das, 2022). Multisite trials additionally may lead to more diverse study samples, as they offer opportunities for subject recruitment from broader and more representative sampling pools or geographic regions, and may lead to more rapid recruitment of participants (Flynn, 2009).

In psychedelic clinical trials, multisite studies may reduce the likelihood of researcher-level bias as each site could be run by different investigators (e.g. academic sites and industry sites), who may have different backgrounds and training and would be unlikely to share the same potential biases or COIs. Lack of diverse ethnoracial inclusion is common in psychedelic studies (and clinical research generally), but multisite trials provide more opportunities to recruit and enroll larger samples of diverse, representative participants (Hughes et al., 2024; Thrul et al., 2021). Currently, a variety of therapeutic and therapeutic support modalities are used in psychedelic clinical trials (Cavarra et al., 2022), but conducting a multisite psychedelic study would create an additional need and motivation to standardize therapeutic procedures and reduce variability between sites. Comparing results and adverse events between sites can help identify any issues with study procedures and implementation, and support the development of best practices that can be shared publicly to benefit future studies. The independence of each site would be a key component to reducing bias, and criteria must be established in order to define and ensure the independence of each site.

Proposal 3: Quantitative bias analysis

Quantitative bias analysis (QBA) is a set of methods that can be used to evaluate the sensitivity of study results to systematic error (i.e. bias) and to assess the effects of bias on the direction and magnitude of effect estimates (Fox and Lash, 2020). QBA can address multiple forms of bias, including information bias (e.g. misclassification), selection bias (e.g. differential loss to follow up between treatment arms), and confounding (Lash et al., 2009). To address bias, QBA can be performed at any stage of study design or analysis (Fox and Lash, 2020). Conducting QBA is of particular importance when seeking to understand causal relationships and conducting analyses that may inform policy decisions or clinical practice (Lash et al., 2009).

Especially given the concerns around bias influencing the results of psychedelic clinical trials, QBA may help address any bias that may have occurred during the conduct or analysis of a study, including investigator-level cognitive biases. Specific examples include using QBA to address underreporting of adverse events, overly positive outcome scores, and non-representative participant demographics. Including QBA during the study design phase may help design an internal validity substudy to address bias and/or determine which data will be most important to collect because of the likely effect on study results (Lash et al., 2009). Conducting QBA once study results are known can delineate the magnitude and direction of bias on the results, thereby providing an understanding of what the effects would have been had there been no bias. Pre-registering and publishing QBA plans and results demonstrates transparency and can promote informed discussion within the field about reducing bias in clinical trials.

Proposal 4: Open Science in clinical trials

Discussions of bias in clinical trials and their accompanying mitigation strategies have generally proceeded separately to discussions about Open Science practices and the “replicability crisis” (that published research findings frequently cannot be replicated). Although no formal definition exists, characteristics of Open Science include transparency at all stages of the research process (Ramjoué, 2015), accessibility of data, results and publications (Bisol et al., 2014), sharing of methods and intermediate stages of the research (Grubb and Easterbrook, 2011), and being collaboratively developed (Hormia-Poutanen and Forsström, 2016). The importance of independent replication of results and reproducibility has also been re-emphasized by the Open Science movement (Schooler, 2014).

Several Open Science practices have been proposed, with the goal of reducing bias and improving the methodological rigor of clinical trials and in research more generally. Pre-registration, the sharing of clinical trial data and code, and registration at ClinicalTrials.gov promote the replication of analyses and greater transparency. In the United States, the mandatory registration of certain clinical trials (i.e. at ClinicalTrials.gov) by the FDA Amendments Act of 2007, along with primary and secondary outcomes, has helped to establish a record of research and to prevent publication bias (Bradley et al., 2020); however, many other non-drug related academic studies do not use this platform and registration can be incomplete (e.g. not uploading informed consent forms within the mandated period, see Axson et al., 2024). In some other countries, Sunshine Acts, which require the disclosure of physicians’ financial interests, have been implemented (Fabbri et al., 2018). Other suggestions aimed at reducing bias in research include establishing an accessible and regularly updated database of COIs from researchers, institutions, physicians and patient advocacy groups (Bradley et al., 2020).

We suggest that, if not already implemented, psychedelic clinical trials in academic settings could benefit from greater use of Open Science practices (e.g. preregistration, open sharing of data, and sharing study materials, replication), in addition to adopting the methods discussed above of reducing bias such as the independent assessment of data. These bias reduction methods should be acknowledged as related to and contributing toward Open Science, as they aim to reduce researcher-level and other biases in research, improve transparency, and help to establish trust in research. One approach to encourage the use of such practices may be to create Badges (akin to Open Science Badges) (Figure 1—Open Science badges) that acknowledge and incentivize the use of these methods (Kidwell et al., 2016). Another promising avenue for independent replication of psychedelic research are programs such as the National Institutes of Health’s recently established Replication Initiative, which provides funding for researchers to replicate their results with a third-party Contract Research Organization (NIH, 2025).

Figure 1.

Badges to acknowledge open practices, Center for Open Science, CC-by attribution 4.0 international, https://osf.io/tykzf. While Open Science badges exist for preregistration, open data and open materials, we suggest additional badges be available for those who choose to implement the methods we describe, such as a badge for independent statistical analysis.

Implementing certain Open Science methods within psychedelic research may be difficult, given the sensitive nature of the data involved (e.g. video recordings of dosing sessions, psychiatric profiles). Without careful consideration and thought, open data sharing could potentially violate participants’ safety, privacy, and confidentiality. While the open sharing of video recordings of dosing sessions may not be feasible without violating the privacy of participants, the sharing of other study materials, such as protocols and summary data, may also contribute toward the goal of transparency while preserving participants’ privacy. In general, assessing the risk of disclosure and re-identification will be crucial in deciding which forms of data to share (Alter and Gonzalez, 2018).

Conclusion: Summary and recommendations

In the previous sections, we have reviewed various types of researcher-level biases and their specific application to psychedelic research. To help facilitate an in-depth exploration and discussion, we focused here on measures that may help to address real or perceived researcher biases and conflicts of interest—however, it is outside the scope of this paper to discuss other issues that may affect the accuracy of study results such as fraud, poor study design, and the external validity of study results. Other approaches will be needed to improve the rigor and validity of the research conducted, and to help ensure equipoise is maintained.

We described and advocated for four specific measures that can be implemented to reduce the risk of biases (both real and perceived) in psychedelic clinical trials: (1) independent assessment of the data, (2) multisite trials, (3) quantitative bias analysis, and (4) increasing the uptake and use of Open Science practices. Crucially, many of these potential solutions rely upon the concept of independence and independent review, which requires commitment and effort to introduce and maintain. Although we suggest that these methods be applied to psychedelic clinical trials, they are not psychedelic-specific measures and may be good practice for all clinical trials. We acknowledge that such practices require varying amounts of time, effort, and resources to incorporate, and some may be unfeasible given the resources and funding available. These barriers do present obstacles to their implementation: nonetheless, conducting rigorous, internally valid and trustworthy research is critical to good science and advancing the field of psychedelics. Establishing these practices as norms for the field and increasing investment in these practices will be essential to future psychedelic science.

Finally, while our discussion of these issues were motivated by the current challenges within psychedelic research, they are not unique and are present in many other scientific fields. By looking to and learning from both Open Science and methods that have been used to reduce bias in clinical trials (in industry-sponsored and academic research settings), psychedelic research can work toward improving its rigor and transparency in addition to suggesting improvements that could be more widely adopted in science more generally. We believe that the challenges in the psychedelic field offer an opportunity to creatively find and implement solutions that can then be applied to improve the rigor of clinical research more generally.

Footnotes

Acknowledgements

We are grateful to the two anonymous reviewers for their comments that greatly improved the content of the manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

D.B.Y. reported research funds from the National Institutes of Health (NIH) and Heffter Institute as well as support support through the Hopkins Center for Psychedelic and Consciousness Research funded by Tim Ferriss, Matt Mullenweg, Craig Nerenberg, Blake Mycoskie, and the Steven and Alexandra Cohen Foundation. He has delivered paid lectures to the Integrative Psychiatry Institute.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Ethics approval

The authors declare that research ethics approval was not required for this study.

ORCID iDs

Katherine Cheung

Rebecca Ehrenkranz

Notes

References

Aday

Skiles

Eaton

, et al. (2023) Personal psychedelic use is common among a sample of psychedelic therapists: Implications for research and practice. Psychedelic Medicin. 1(1): 27–37.

Alter

Gonzalez

(2018) Responsible practices for data sharing. American Psychologist 73(2): 146.

Armeni

Brinkman

Carlsson

, et al. (2021) Towards wide-scale adoption of open science practices: The role of open science communities. Science and Public Policy 48(5): 605–611.

Auerbach

McGowan

Halevi

, et al. (2013) Mitigating adverse event reporting bias in spine surgery. JBJS 95(16): 1450–1456.

Axson

Ramachandran

Lisenby

, et al. (2024) Consent form reporting on clinicaltrials. Gov, 2013-2023. JAMA Network Open 7(6): e2418895.

Bachrach

Newcomer

(2002) Addressing bias in intervention research: Summary of a workshop. Journal of Adolescent Health 31(4): 311–321.

Barnett

Vest

Delatte

, et al. (2024) Practical considerations in the establishment of psychedelic research programs. Psychopharmacology 4: 1–7.

Bedi

Cotton

Guerin

, et al. (2023) MDMA-assisted psychotherapy for post-traumatic stress disorder: The devil is in the detail. Australian & New Zealand Journal of Psychiatry 57(4): 476–481.

Bero

Grundy

(2016) Why having a (nonfinancial) interest is not a conflict of interest. PLoS Biology 14(12): e2001221.

10.

Bradley

DeVito

Lloyd

, et al. (2020) Reducing bias and improving transparency in medical research: A critical overview of the problems, progress and suggested next steps. Journal of the Royal Society of Medicine 113(11): 433–443.

11.

Breeksema

Kuin

Kamphuis

, et al. (2022) Adverse events in clinical treatments with serotonergic psychedelics and MDMA: A mixed-methods systematic review. Journal of Psychopharmacology 36(10): 1100–1117.

12.

Buchman

Rosenbaum

(2024) Psychedelics in PERIL: The commercial determinants of health, financial entanglements and population health ethics. Public Health Ethics 17: phae002.

13.

Butler

Jelen

Rucker

(2022) Expectancy in placebo-controlled trials of psychedelics: If so, so what?. Psychopharmacology 239(10): 3047–3055.

14.

Calder

Hasler

(2024) Validation of the Swiss psychedelic side effects inventory: Standardized assessment of adverse effects in studies of psychedelics and MDMA. Journal of Affective Disorders 365: 258–264.

15.

Carhart-Harris

Giribaldi

Watts

, et al. (2021) Trial of psilocybin versus escitalopram for depression. New England Journal of Medicine 384(15): 1402–1411.

16.

Cashin

Richards

DeVito

, et al. (2023) Registration of health and medical research. BMJ Evidence-Based Medicine 28(1): 68–72.

17.

Cavarra

Falzone

Ramaekers

, et al. (2022) Psychedelic-assisted psychotherapy—A systematic review of associated psychological interventions. Frontiers in Psychology 13: 887255.

18.

Cheung

Ehrenkranz

Hinkle

, et al. (2024) Commentary: A framework for assessment of adverse events in psychedelic research. Journal of Psychopharmacology 4: 02698811241309623.

19.

Christensen

Wang

Levy Paluck

, et al. (2020) Open science practices are on the rise. The State of Social Science (3S) Survey.

20.

Clark

Tetlock

(2023) Adversarial collaboration: The next science reform. In Frisby

C. L.

Redding

R. E.

O’Donohue

W. T.

Lilienfeld

S. O.

(Eds.), Political Bias in Psychology: Nature, Scope, and Solutions (pp. 905–927). New York: Springer.

21.

Cook

Sheets

(2011) Clinical equipoise and personal equipoise: Two necessary ingredients for reducing bias in manual therapy trials. Journal of Manual & Manipulative Therapy 19(1): 55–57.

22.

Cornelius

Sauzet

Williams

, et al. (2013) Adverse event reporting in randomised controlled trials of neuropathic pain: Considerations for future practice. Pain 154(2): 213–220.

23.

Dana

Loewenstein

(2003) A social science perspective on gifts to physicians from industry. JAMA 290(2): 252–255.

24.

Deangelis

Fontanarosa

(2010) The importance of independent academic statistical analysis. Biostatistics 11(3): 383–384.

25.

Deichmann

Krousel-Wood

Breault

(2016) Bioethics in practice: Considerations for stopping a clinical trial early. Ochsner Journal 16(3): 197–198.

26.

Dodd

Korn

Freidlin

, et al. (2008) Blinded independent central review of progression-free survival in phase III clinical trials: Important design element or unnecessary expense?. Journal of Clinical Oncology 2008 26(22): 3791–3796.

27.

Duggan

Parry

McMurran

, et al. (2014) The recording of adverse events from psychological treatments in clinical trials: Evidence from a review of NIHR-funded trials. Trials 15: 1–9.

28.

Dunn

Coiera

Mandl

, et al. (2016) Conflict of interest disclosure in biomedical research: A review of current practices, biases, and the role of public registries in improving transparency. Research Integrity and Peer Review 1: 1–8.

29.

Ehrenkranz

Agrawal

Nayak

, et al. (2024) Adverse events should not be surprising in psychedelic research. Psychedelic Medicine 3(1): 59–62.

30.

Fabbri

la Santos

Mezinska

, et al. (2018) Sunshine policies and murky shadows in Europe: Disclosure of pharmaceutical industry payments to health professionals in nine European countries. International Journal of Health Policy and Management 7: 504.

31.

FDA (2024) Briefing Document for June 4th Meeting of the Psychopharmacologic Drugs Advisory Committee Meeting. Available at: https://www.fda.gov/media/178984/download

32.

Freedman

(1987) Equipoise and the ethics of clinical research. In Human Experimentation and Research (pp. 427–431). Routledge.

33.

Fried

(1974, 2016) Medical Experimentation: Personal Integrity and Social Policy: New edition. Oxford University Press.

34.

Flynn

(2009) The benefits and challenges of multisite studies: Lessons learned. AACN Advanced Critical Care 20(4): 388–391.

35.

Forstmann

Sagioglou

(2021) How psychedelic researchers’ self-admitted substance use and their association with psychedelic culture affect people’s perceptions of their scientific integrity and the quality of their research. Public Understanding of Science 30(3): 302–318.

36.

Fox

Lash

(2020) Quantitative bias analysis for study and grant planning. Annals of Epidemiology 43: 32–36.

37.

Friedman

Richter

(2004) Relationship between conflicts of interest and research results. Journal of General Internal Medicine 19: 51–56.

38.

Fuller

Mattson

Allen

, et al. (1994) Multisite clinical trials in alcoholism treatment research: Organizational, methodological and management issues. Journal of Studies on Alcohol, Supplement 12: 30–37.

39.

Golder

Loke

Wright

, et al. (2016) Reporting of adverse events in published and unpublished studies of health care interventions: A systematic review. PLoS Medicine 13(9): e1002127.

40.

Goodwin

Aaronson

Alvarez

, et al. (2022) Single-dose psilocybin for a treatment-resistant episode of major depression. New England Journal of Medicine 387(18): 1637–1648.

41.

Goozner

(2010) NIH proposes new conflict-of-interest Rules for investigators, JNCI: Journal of the National Cancer Institute 102: 1006–1007.

42.

Gottlieb

(2020). Financial conflicts of interest in clinical trials. Principles and Practice of Clinical Trials: 1–17.

43.

Griffiths

Johnson

Carducci

, et al. (2016) Psilocybin produces substantial and sustained decreases in depression and anxiety in patients with life-threatening cancer: A randomized double-blind trial. Journal of Psychopharmacology 30(12): 1181–1197.

44.

Grubb

Easterbrook

(2011) On the lack of consensus over the meaning of openness: An empirical study. PloS One 6(8): e23420.

45.

Grundy

(2021) A politics of objectivity: biomedicine’s attempts to grapple with “non-financial” conflicts of interest. Science and Engineering Ethics 27(3): 37.

46.

Guyatt

Akl

Hirsh

, et al. (2010) The vexing problem of guidelines and conflict of interest: A potential solution. Annals of Internal Medicine 152(11): 738–741.

47.

Haines

Gabor Miklos

(2011) Time to mandate data release and independent audits for all clinical trials. Medical Journal of Australia 195(10): 575–577.

48.

Hariton

Locascio

(2018) Randomised controlled trials—the gold standard for effectiveness research. BJOG: An International Journal of Obstetrics and Gynaecology 125(13): 1716.

49.

Das

(2022) Multicenter studies: relevance, design and implementation. Indian Pediatrics 59(7): 571–579.

50.

Department of Health and Human Services, “Clinical trials registration and results information submission”, 42 CFR Part 11. https://www.federalregister.gov/documents/2016/09/21/2016-22129/clinical-trials-registration-and-results-information-submission

51.

Hinkle

Graziosi

Nayak

, et al. (2024) Adverse events in studies of classic psychedelics: A systematic review and meta-analysis. JAMA Psychiatry 81(12): 1225–1235.

52.

Hormia-Poutanen

Forsström

(2016) Collaboration at international, national and institutional level–Vital in fostering open science. LIBER Quarterly: The Journal of the Association of European Research Libraries 26(1): 3–12.

53.

Hovmand

Poulsen

Arnfred

, et al. (2023) Risk of bias in randomized clinical trials on psychedelic medicine: A systematic review. Journal of Psychopharmacology 37(7): 649–659.

54.

Hughes

Garcia-Romeu

(2024) Ethnoracial inclusion in clinical trials of psychedelics: A systematic review. EClinicalMedicine 74: 102711.

55.

Institute of Medicine (2006) Preventing Medication Errors. The National Academy of Science Press, Washington, DC.

56.

International Committee of Medical Journal Editors (ICMJE) (2024) Disclosure of financial and non-financial relationships and activities, and conflicts of interest. Available at: https://www.icmje.org/recommendations/browse/roles-and-responsibilities/author-responsibilities–conflicts-of-interest.html

57.

International Committee of Medical Journal Editors (2018). Recommendations. http://www.icmje.org/recommendations/

58.

Ioannidis

Lau

(2001) Completeness of safety reporting in randomized trials: An evaluation of 7 medical areas. JAMA 285(4): 437–443.

59.

Johnson

(2020) Consciousness, religion, and gurus: Pitfalls of psychedelic medicine. ACS Pharmacology & Translational Science 4(2): 578–581.

60.

Kahneman

(2011) Thinking, Fast, and Slow. New York, NY: Farrar, Straus, and Giroux.

61.

Klatte

Strauss

Flückiger

, et al. (2023) Adverse events in psychotherapy randomized controlled trials: A systematic review. Psychotherapy Research 1: 1–6.

62.

Klein

Kesselheim

(2024). Revisiting the potential of “money blinds”: Lessons from Lykos’s MDMA research. Health Affairs Forefront.

63.

Norris

Burda

Holmer

, et al. (2012) Author’s specialty and conflicts of interest contribute to conflicting guidelines for screening mammography. Journal of Clinical Epidemiology 65(7): 725–733.

64.

Kidwell

Lazarević

Baranski

, et al. (2016) Badges to acknowledge open practices: A simple, low-cost, effective method for increasing transparency. PLoS Biology 14(5): e1002456.

65.

Kious

Schwartz

Lewis

(2023) Should we be leery of being Leary? Concerns about psychedelic use by psychedelic researchers. Journal of Psychopharmacology 37(1): 45–48.

66.

Koning

Solmi

Brietzke

(2024) The influence of stakeholder interests on safety outcome reporting in psychedelic research and implications for science communication. Trends in Psychiatry and Psychotherapy. Online ahead of print.

67.

Lash

Fox

Fink

(2009) Applying quantitative bias analysis to epidemiologic data. American Journal of Epidemiology 170(10): 1316–1318.

68.

Lineberry

Berlin

Mansi

, et al. (2016) Recommendations to improve adverse event reporting in clinical trial publications: A joint pharmaceutical industry/journal editor perspective. BMJ 2016; 355.

69.

Marks

(2024) Psychedelic therapy scrutinized by FDA advisory committee? JAMA. Published online July 30, 2024. DOI: 10.1001/jama.2024.13370.

70.

McDonnell

Dalton

Ahern

, et al. (2021) Methods to mitigate industry influence in industry sponsored research. Clinical Spine Surgery 34(4): 143–145.

71.

McNamee

Devenot

Buisson

(2023) Studying harms is key to improving psychedelic-assisted therapy—participants call for changes to research landscape. JAMA Psychiatry 80(5): 411–412.

72.

Mellers

Hertwig

Kahneman

(2001) Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration. Psychological Science 12(4): 269–275.

73.

Meling

Ehrenkranz

Nayak

, et al. (2024) Mind the psychedelic hype: Characterizing the risks and benefits of psychedelics for depression. Psychoactives 3(2): 215–234.

74.

de Melo-Martín

Intemann

(2009) How do disclosure policies fail? Let us count the ways. The FASEB Journal 23(6): 1638–1642.

75.

Mustafa

McQueen

Nikitin

, et al. (2024) MDMA-Assisted Psychotherapy for Post-Traumatic Stress Disorder: Effectiveness and Value; Evidence Report. Institute for Clinical and Economic Review. Available at https://icer.org/assessment/ptsd-2024/#overview

76.

Muthukumaraswamy

Forsyth

Lumley

(2021) Blinding and expectancy confounds in psychedelic randomized controlled trials. Expert Review of Clinical Pharmacology 14(9): 1133–1152.

77.

National Science Foundation (NSF) (2005) Grantee standards. Available at: https://www.nsf.gov/pubs/manuals/gpm05_131/gpm5.jsp

78.

National Institutes of Health (NIH) (2022) Managing conflict of interest in NIH peer review of grants and contracts. Available at: https://grants.nih.gov/policy/peer/peer-coi.htm

79.

National Institutes of Health (NIH) (2025) Replication to Enhance Research Impact Initiative. Available at: www.commonfund.nih.gov/replication-initiative

80.

Nature (2018) Nature journals tighten rules on non-financial conflicts. Nature 554(7690): 6.

81.

Office of Human Research Protections (OHRP) (2007) Unanticipated problems involving risks and adverse events guidance. United States Department of Health and Human Services. Available at: https://www.hhs.gov/ohrp/regulations-and-policy/guidance/reviewing-unanticipated-problems/index.html#Q2 (accessed August 2024).

82.

Østengaard

Lundh

Tjørnhøj-Thomsen

, et al. (2020) Influence and management of conflicts of interest in randomised clinical trials: Qualitative interview study. BMJ 371: m3764.

83.

Palitsky

Kaplan

Perna

, et al. (2024) A framework for assessment of adverse events occurring in psychedelic-assisted therapies. Journal of Psychopharmacology 38(8): 690–700.

84.

Panagiotou

Ioannidis

(2012) Primary study authors of significant studies are more likely to believe that a strong association exists in a heterogeneous meta-analysis compared with methodologists. Journal of Clinical Epidemiology 65(7): 740–747.

85.

Perlis

, et al. (2005) Industry sponsorship and financial conflict of interest in the reporting of clinical trials in psychiatry. American Journal of Psychiatry 162(10): 1957–1960.

86.

Petranker

Anderson

Farb

(2020) Psychedelic research and the need for transparency: Polishing Alice’s looking glass. Frontiers in Psychology 11: 1681.

87.

Pew Research Center (2019) Trust and Mistrust in Americans’ Views of Scientific Experts. Available at: https://www.pewresearch.org/science/wp-content/uploads/sites/16/2019/08/PS_08.02.19_trust.in_.scientists_FULLREPORT-1.pdf

88.

Poppe

Repantis

(2024). Ethical aspects of psychedelic-assisted treatments: An overview. In: Current Topics in Behavioral Neurosciences. Springer, Berlin, Heidelberg.

89.

Raison

Sanacora

Woolley

, et al. (2023) Single-dose psilocybin treatment for major depressive disorder: A randomized clinical trial. JAMA 330(9): 843–853.

90.

Ramjoué

(2015) Towards open science: The vision of the European Commission. Information Services & Use 35(3): 167–170.

91.

Reardon

(2024) MDMA therapy for PTSD rejected by FDA panel. Nature. Available at: https://www.nature.com/articles/d41586-024-01622-3

92.

Resnik

(2023) Disclosing and managing non-financial conflicts of interest in scientific publications. Research Ethics 19(2): 121–138.

93.

Rosenbaum

Hare

Hapke

, et al. (2024) Experiential training in psychedelic-assisted therapy: A risk-benefit analysis. Hastings Center Report 54(4): 32–46.

94.

Ross

Gross

Krumholz

(2012) Promoting transparency in pharmaceutical industry–sponsored research. American Journal of Public Health 102(1): 72–80.

95.

Ross

Potter

Barratt

, et al. (2020) “Coming out”: Stigma, reflexivity and the drug researcher’s drug use. Contemporary Drug Problems 47(4): 268–285.

96.

Sartor

Halabi

(2015) Independent data monitoring committees: An update and overview. Urologic Oncology: Seminars and Original Investigations 33(3): 143–148.

97.

Schöffski

(2021). Importance and role of independent data monitoring committees (IDMCs) in oncology clinical trials. BMJ Open 11(10): e047294.

98.

Schooler

(2014) Metascience could rescue the ‘replication crisis’. Nature 515: 9.

99.

Siff

(2008). Henry Luce’s Strange Trip: Coverage of LSD in Time and Life, 1954–68. Journalism History 34(3): 126–134.

100.

Siff

(2015) Acid Hype: American News Media And The Psychedelic Experience. University of Illinois Press.

101.

Sniderman

Furberg

(2012) Pluralism of viewpoints as the antidote to intellectual conflict of interest in guidelines. Journal of Clinical Epidemiology 65(7): 705.

102.

Stull

Smith

Vest

, et al. (2022) Potential value of the insights and lived experiences of addiction researchers with addiction. Journal of Addiction Medicine 16(2): 135–137.

103.

Tang

Pond

Chen

(2010) Influence of an independent review committee on assessment of response rate and progression-free survival in phase III clinical trials. Annals of Oncology 21(1): 19–26.

104.

Thrul

Garcia-Romeu

(2021) Whitewashing psychedelics: Racial equity in the emerging field of psychedelic-assisted mental health research and treatment. Drugs: Education, Prevention and Policy 28(3): 211–214.

105.

Tresker

(2022) Unreliable threats: Conflicts of interest disclosure and the safeguarding of biomedical knowledge. Kennedy Institute of Ethics Journal 32(1): 103–126.

106.

Tversky

Kahneman

(1974) Judgment under Uncertainty: Heuristics and Biases: Biases in judgments reveal some heuristics of thinking under uncertainty. Science 185(4157): 1124–1131.

107.

van Elk

Fried

(2023) History repeating: Guidelines to address common problems in psychedelic science. Therapeutic Advances in Psychopharmacology 13: 20451253231198466.

108.

Viswanathan

Carey

Belinson

, et al. (2014) A proposed approach may help systematic reviews retain needed expertise while minimizing bias from nonfinancial conflicts of interest. Journal of Clinical Epidemiology: 67(11): 1229–1238.

109.

Walters

(1993) Data monitoring committees: The moral case for maximum feasible independence. Statistics in Medicine 2(5-6): 575–580.

110.

Wen

Singhal

Jones

, et al. (2024) A systematic review of study design and placebo controls in psychedelic research. Psychedelic Medicine 2(1): 15–24.

111.

Wiersma

Kerridge

Lipworth

(2018) Dangers of neglecting non-financial conflicts of interest in health and medicine. Journal of Medical Ethics 44(5): 319–322.

112.

Yusuf

Bosch

(2006) Independent design and conduct of clinical trials. Clinical Trials 3(6): 503–507.

Enhanced independence: De-biasing processes in psychedelic research and beyond

Abstract

Keywords

Introduction

Researcher biases and conflicts of interests in clinical trials

Brief review of measures proposed to reduce bias in clinical trials

Proposals to safeguard and insert independence within the scientific process

Proposal 1: Independent assessment of clinical trial data

Proposal 2: Multisite trials

Proposal 3: Quantitative bias analysis

Proposal 4: Open Science in clinical trials

Conclusion: Summary and recommendations

Footnotes

Acknowledgements

Declaration of conflicting interests

Funding

Consent to participate

Consent for publication

Ethics approval

ORCID iDs

Notes

References