Beyond RCTs—Better Research Designs and Methods for Psychosocial Efficacy Studies in Mental Health

Abstract

Background

Classic randomized controlled trials (RCTs) are considered the gold standard in mental health research but, given their multiple limitations and threats to external validity, alternative trial designs may be more suitable in many circumstances.

Objectives

This narrative review explores research designs and methodologies that can be useful to researchers interested in investigating the effects of psychosocial or other non-medical interventions for psychiatric conditions.

Results

New or better-tailored methods have been developed over the years, including: improved RCTs with adaptive designs (e.g., microRCT, SMART, RAR, target-emulation, adaptive dose-response), randomized baseline single case experimental designs; experience sampling method; E-technology (e.g., sensors for movement, social media monitoring with natural language processing); matched cohort studies and cluster randomized trials; and partially randomized preference trials, as well as mixed-methods and hybrid effectiveness/implementation trials.

Conclusion

By expanding the scope of recommended methods, above and beyond classic RCTs, the field can more easily determine the impact of psychosocial treatments in mental health research.

Plain Language Summary

Better Ways to Study Psychosocial Treatments for Mental Health

Psychosocial interventions play an important role in supporting people living with various mental health conditions, yet it can be difficult to study their effects using traditional research methods. Randomized controlled trials are often seen as the best way to test treatments, but they are not always well suited to psychosocial approaches, which depend on individual preferences, engagement, real-world settings, and flexible delivery. This review describes a range of alternative research designs that can help better evaluate non-medical treatments and programs for people with mental health conditions who can be difficult to recruit for large studies, such as those with schizophrenia or psychosis. These include studies that closely track changes within individuals over time, methods that capture experiences in daily life using smartphones or digital sensors, approaches that compare groups receiving different services in real-world settings, and newer trial designs that adapt to participants’ needs or preferences as the study progresses. By using research methods that are better aligned with how psychosocial treatments are delivered and experienced, researchers can generate more meaningful and applicable evidence. Expanding the range of study designs may ultimately improve the development, evaluation, and implementation of psychosocial interventions, leading to more effective and person-centred care for people affected by mental health conditions.

Introduction

Randomized controlled trials (RCTs) are considered the gold standard method in clinical research. Under ideal conditions—representative samples, credible controls, and double-blinding—RCTs provide strong causal evidence of a treatment's efficacy. When studying psychosocial treatments in people with mental health conditions who are difficult to recruit, such as people with psychotic disorders, these conditions are unfortunately rarely met. In this narrative review, we will first briefly describe the limitations of using traditional RCTs for psychosocial treatments in mental health research. We will follow with a presentation of modified or adaptive RCT designs that may be viable alternatives to more traditional RCTs; single case experimental designs (SCEDs); Experience sampling method (ESM); E-technology (e.g., sensors for movement, social media monitoring via natural language processing); retrospective cohort studies (matched); partially randomized patient preference trials (RPPTs); target emulation trials; and mixed methods hybrid intervention effectiveness trials.

Limitations of Traditional RCTS

Traditional RCT designs, which were developed for pharmacological trials, are sub-optimal in several ways in psychosocial treatment research, resulting in poor generalizability of the results. For instance, selecting people with certain characteristics (e.g., without any comorbid conditions) can improve the homogeneity of the sample but can limit the generalizability of the results since comorbid presentation in schizophrenia and psychosis, for instance, is the norm, not the exception. ¹ The choice of a valid placebo is also an issue as it is virtually impossible for the treatment provider to be unaware of the treatment offered, and for the participant, of the treatment they are receiving. Also, novel treatments are typically more attractive to participants than sham treatments offered as controls, which can lead to an imbalance in dropouts across conditions. To compensate for this dropout problem, it is recommended to compare two active treatments. However, it has been proven that bona fide (i.e., “good enough”) psychosocial treatments, offered by competent therapists, tend to obtain good results, ² at least initially. This occurs for a number of reasons, explored in detail elsewhere. ² Even a non-skills-based intervention (such as “befriending,” which is a common control in psychotherapy trials) involves consistent positive social interactions with a caring professional, and is likely inherently therapeutic, instilling a sense of social connection. This typically results in smaller differences at the “end of treatment period” between the active and control conditions. To demonstrate a medium effect size, with equivalence or non-inferiority trials (involving an active control condition), larger samples are needed, and can exceed 1,000 participants. ³ Furthermore, in psychotherapy trials, the level of participant engagement in therapy and their homework completion often impact the magnitude of change—disengaged participants can make an intervention appear ineffective. As most treatments are better than wait-lists or “treatment as usual” conditions, an RCT without a good placebo or comparison treatment will unlikely be funded or published. In psychotherapy trials, the treatment's efficacy depends on the fidelity to the treatment protocol, as well as on the therapist's competence and allegiance to an intervention. These aspects can moderate the treatment's efficacy, are scarcely assessed in RCTs, and are difficult to control for statistically. Table 1 lists several issues with classical RCTs reported in the literature.^4–9

Table 1.

RCT Issues in Mental Health Psychosocial Research.

RCT Component	Issue
Patient representativeness/generalizability	Most mental health disorders are heterogeneous disorders with multiple comorbidities—for statistical power, homogeneous sampling is preferred—limiting generalizability. Recruitment for RCTs is difficult with certain subgroups (e.g., people with psychosis)—high rate of refusal—can take years to recruit a sample of sufficient statistical power (i.e., extremely costly). High drop-out rates.
Placebo or equivalent treatment	Good active control treatments with competent therapists will rarely perform significantly worse than the novel treatment tested and typically show improvements from baseline due to non-specific therapy factors. Therapeutic benefits in psychotherapy require engagement in treatment, homework completion, etc. These are greater when patient receive their preferred treatment. Having disengaged participants in the active treatment arm could artificially reduce effect sizes. Equivalent control treatments are rarely equivalent—novel treatments rarely have a real comparison. There are few equivalent or non-inferiority trials (which need significantly larger samples). Almost all interventions work better than waitlist or TAU.
Blinding	No real blinding—therapists know what they are delivering and participants know what they are receiving. Researcher or therapist allegiance to a treatment is a known moderator (rarely assessed in RCTs). Assessors can often guess who received what—creating a detection bias.
Effects due to treatment or delivery	Treatment is not distinct from treatment delivery method—difficult to distinguish the effects of the treatment from those of the therapist's competence. Treatment fidelity is an essential moderator and not always measured.
Reflect clinical practice	Most clinical settings consider the person's preferences—RCTs do not consider preferences (impact on recruitment and generalizability). Treatment delivered in an RCT is unlikely to reflect the same treatment delivered in routine clinical practice.

RCT = randomized controlled trial.

Table 2.

Summary of Key Strengths and Limitations of Psychosocial Efficacy Study Designs.

Design	Strengths	Limitations and Challenges
Adaptive designs	Fewer participants directed through less promising treatments Greater statistical power due to dynamic participant allocation Variants of adaptive designs can enrich for important outcomes or investigate precision medicine	Can be more computationally and methodologically challenging, especially variants such as micro-randomized trials or sequential multiple assessment randomized trials Blinding across multiple rounds of randomization may not be feasible Variants that involve algorithmic randomization to different treatment arms (e.g., nested-precision randomized controlled trial) need to be calibrated carefully to avoid underpowered or wasteful treatment arms
Single case experimental design	Within-subjects analyses allow for greater statistical power All participants receive the intervention without a “waitlist,” responding to important clinical needs	Order effects from condition sequence must be considered (i.e., contamination effects) Expectancies may influence responses on subjective ratings and confound treatment effects of interest.
Experience sampling method	Intensive sampling procedure can increase power for within and between-person analyses Emphasizes ecological validity Designs can integrate high temporal precision to explore dynamic or brief effects Reduces recall bias Can flexibly complement other designs	Depending on the sampling design and survey length, participant burden can be high and result in notable missing data Limited survey duration can result in key constructs being assessed through a limited number of items (typically 1–3) Relatively limited guidance on appropriate survey duration, frequency, or administration period (e.g., how long should a survey be, how many should be administered per day, how many days of surveys are most appropriate) Although variables can be summarized on a person-level basis, use of daily observations requires more sophisticated modelling that accounts for within-subject correlations and contextual factors
E-technology	Objective measures of behaviour circumvent subjective biases Minimizes participant burden through passive data collection Can collect an extensive number of features of interest helpful for machine learning and similar approaches Can flexibly complement other designs	Data privacy must be carefully considered due to the sensitivity of the features collected (including GPS coordinates, phone records, language samples) Although objective, the construct validity of passive measures should be critically evaluated; several common indices have papers evaluating their validity Collection and analysis of passive variables can require specialized and costly) technology or applications and relatively advanced computational processes Data collection can depend on participant devices, potentially introducing biases related to sociodemographic characteristics
Matched cohort and cluster randomized trials	Administrative data can allow for large, representative samples Allows for research questions around real-world clinical effectiveness and implementation Retrospective data can represent a larger timeframe than feasible in other designs	Administrative data can be limited in the variables available and often suffers from missingness regarding psychological constructs, constraining available outcomes Especially in retrospective data analysis, important confounds may not be addressable (e.g., equal opportunity to participate in different treatments) Especially in prospective trials, long-term follow-up necessitates long-term funding Quasi-experimental nature restricts causal inference
Partially randomized patient preference trials	Can improve engagement and outcomes by allowing for participant preferences May be more ethical in responding to participant treatment goals and preferences Retains randomization among participants without a preference	Cell size for undesired treatments could be small; however, this alone may be an important marker of treatment acceptability Participants may also be wary of novel treatment, creating systematic differences in who requests allocation to such conditions Statistical control for baseline characteristics enhances internal validity, but may not be equivalent to control through randomization
Target emulation trial	Pragmatic, focusing on real-world effectiveness Retains many strengths of RCTs while operating with real-world data	Due to use of existing clinical data, blinding and important controls may be absent Statistical control for baseline characteristics enhances internal validity, but may not be equivalent to control through randomization
Mixed methods hybrid intervention effectiveness trials	Adopts principles from implementation science to investigate treatment implementation and effectiveness Mixed method designs offer rich data regarding trial implementation and outcomes	Blending of implementation science requires significant resources, including specialized staff and careful considerations of staff buy-in and study design May have a more restricted set of outcomes than other designs or standard RCT Adequate sample size may require use of administrative data

Note. RCT = randomized controlled trial.

List of strengths and limitations is not intended to be exhaustive, see the main text for further details for each design. All designs also share certain potential limitations in common with RCTs, such as the need to collect representative samples, designing comparison conditions, and challenges related to participant and clinician blinding.

Given these pitfalls, other methods for determining treatment efficacy have been developed and are relevant for mental health research.

Improved RCTS

Adaptive designs: Adaptive designs are essentially flexible RCTs that use results from interim data analyses to make decisions on the ongoing trial, without undermining its integrity or validity.^10,11 Making adaptations to a trial along the way might seem like breaking some of the cardinal rules of RCTs, yet it allows for more efficient trials because fewer individuals will be randomised to a less promising treatment; fewer people are needed to demonstrate the treatment's efficacy; the trial has fewer chances of being underpowered; dose–response can more easily be determined; population specifics of who is most likely to benefit from a treatment may be identified. There are several types of adaptive RCT designs, and though most still require large samples and complex data analysis, they are more cost-effective than traditional RCTs.

Adaptive enrichment trials: Adaptive enrichment trials do not randomize participants equally but will instead deliver the intervention according to baseline characteristics in order to allow for either: (a) prognostic enrichment, by selecting those who are at greater risk of developing a disorder, or (b) predictive enrichment, to select those more likely to benefit from the treatment. ¹² We could imagine, for example, a trial with a stratified risk-based allocation design using NAPLS-2 risk calculator to offer the most intensive treatment to people considered at greater risk of conversion to psychosis. ¹³

Micro-randomized trials: Unlike traditional RCTs, micro-randomized trials (MRTs) enable researchers to isolate and evaluate the effects of individual components at multiple decision points—thus revealing when, for whom, and under which conditions a given element of treatment is most effective.^14–16 A defining feature of MRTs is their focus on proximal outcomes—short-term effects measured shortly after a component is delivered. MRTs are not meant to assess the overall efficacy of a treatment but are well suited for evaluating the short-term impact of individual treatment components, or mechanisms through which improvements are achieved. ¹⁵ For instance, in a physical activity intervention, the number of steps taken within 30 min of a prompt may serve as a proximal outcome, while gains in physical fitness represent a distal one. ¹⁴ MRTs also allow researchers to examine how these short-term effects vary across contexts—such as time of day, location, mood, or cognitive state—insights on moderators that are critical for designing interventions that remain relevant and minimally burdensome.^14–16 At each decision point, participants are randomized only if they meet pre-established availability criteria—for example, they are not currently hospitalized. The repeated randomization process—often involving dozens or even hundreds of decision points per participant—enables researchers to estimate the causal effects of specific intervention components while accounting for contextual and temporal variability. MRTs were developed for Just-in-Time Adaptive Interventions, digitally delivered interventions that adapt in real time to an individual's changing internal states and environmental contexts (often using mobile technologies with “push” or notifications that are initiated automatically ¹⁴ ) and can be used to assess any psychosocial intervention that brings quick gains, as well as longer term impact (e.g., Thomas and colleagues used this design for the study of an ACT based app for distressed college students ¹⁷ ).

Sequential multiple assignment randomized trial (SMART): SMART designs ¹² allow participants to be randomized at multiple decision points based on their early response (or non-response) to treatment. The goal is to assess the efficacy of sequences of interventions over time, adjusting to the person's needs. An example is the Staged Treatment in Early Psychosis sequential multiple assignment randomized trial from McGorry et al. ¹⁸ In this study participants all received an initial brief intervention. After a given duration, the non-responders were then randomized to continue receiving the intervention or received a new one (CBT). The non-responders in the CBT arm continued receiving the intervention but were then randomized to receiving one of two medications. This design enables us to test a stepped-care or decision-tree model which can be used for a range of mental health conditions.

Platform trials also allow the continuous evaluation of multiple interventions within a single master trial. ¹² New treatments can be added or removed over time. Participants are randomized to one of several intervention arms (e.g., CBT, peer support, cognitive remediation (CR), acceptance and commitment therapy, or an active control condition). Arms can be stopped (if ineffective) or added (if new interventions emerge in the literature) without restarting the trial. This “master protocol” design allows the researchers to evaluate multiple psychosocial interventions across sites or countries ¹⁹ (e.g., CBT vs. metacognitive therapy vs. virtual reality-based therapy) but is only feasible across large funded networks or consortiums.

Response-adaptive randomization (RAR): RAR ¹² alters the probability of treatment allocation during the trial based on interim outcomes—participants are increasingly allocated to better-performing arms. This method requires regular interim analyses and, according to the results (i.e., effect size), the randomization ratio changes (e.g., from 1:1 to 2:1) favouring the superior treatment arm. A Bayesian adaptive model is a similar approach but uses a Bayesian model, uploaded after each batch of data, to determine the probability of one treatment being superior to the other and guiding the randomization ratio change (as well as early stoppage of one arm if required). Recent advances in artificial intelligence (AI) can allow for (AI)-driven adaptive trials ²⁰ where, instead of one large trial, series of “mini-trials” are conducted where the results of each feed into the next. At each step, AI methods are used to determine the effectiveness of the interventions under evaluation and alter the proportion of participants allocated to each intervention in the next mini-trial. The sequence of mini-trials stops as soon as the estimates of the intervention effects are demonstrated. As far as we know, this method has yet to be used in psychosocial trials in psychiatry.

Nested-precision RCTs (npRCTs) are a novel adaptive trial design developed to address limitations of traditional RCTs in the context of precision medicine. ²¹ The npRCT combines a standard RCT phase, in which participants are randomized to one of two interventions, with a second phase in which a personalization algorithm is developed to identify prescriptive variables that may inform optimal treatment allocation. If such variables are identified, the trial transitions to a precision phase: new participants are either allocated to treatments based on the developed algorithm or continue to be randomized. This design was introduced to overcome the predominance of retrospective algorithm development (for machine learning) and the lack of prospective evaluation. ²¹ The design also supports ongoing refinement and empirical evaluation of personalization algorithms as more data become available. ²¹ A hypothetical example of an npRCT comparing CBT and ACT for anxiety disorders would involve an initial traditional RCT phase randomizing participants to either treatment to estimate average comparative efficacy and identify prescriptive baseline variables associated with differential treatment response, such as intolerance of uncertainty, experiential avoidance, psychological flexibility, or comorbid depressive symptoms. These data would then be used to build a precision algorithm predicting whether a patient is more likely to benefit from CBT or ACT. In the second phase, new participants would be randomized either to algorithm-guided treatment selection or to usual random allocation to CBT or ACT. This design would permit evaluation of both overall treatment efficacy and the added value of precision-based treatment assignment within a single integrated trial. To our knowledge, the npRCT has not yet been used in psychosocial mental health research. Limits include blinding issues (i.e., experimenter and expectation biases), switching too early (not enough power) or too late (waste of time and resources) to the precision arms, and considering the effect size allocation ratio before starting the trial, to avoid oversampling one arm and leaving the other empty, which would make this design useless.

Single Case Experimental Design

As in RCTs, SCEDs target specific behaviours assessed with standardized measures but typically include more numerous and continuous measurements throughout each phase of the study.^22,23 SCEDs are “within-subject analyses” that necessitate repeated assessments of the dependent variables across each phase of the design to draw valid inferences regarding the effect of the independent variable (i.e., the intervention) on the dependent variables. SCEDs can be conducted with a small number of participants with multiple baselines of a dependent variable before, during, and after the implementation of an independent variable. ²⁴ The introduction of the independent variable is staggered in time across participants to control for time, maturation, and history. ²⁵ This allows to determine if the behavioural or cognitive changes measured are linked to changes targeted by the program. The best practice guidelines for SCEDs include randomizing participants to the number of assessments during the baseline (pre-intervention phase), and staggering the introduction of the intervention across the sample. This approach provides rigorous control over confounding variables while being considered more ethical, in that all vulnerable individuals who are interested in the intervention receive it, with only the timing of the treatment delivery being randomized.^22,26,27

If a treatment is hypothesized as only efficacious when used (e.g., a mindfulness-based treatment), an A-B-A model could be of interest (see Figure 1). ABA is useful when effects are expected to be reversible and withdrawal is ethical.

Figure 1.

Visual results from A-B-A type SCED.

Cross-over SCEDs can also be very instructive when treatment contamination over time is low (e.g., yoga vs. cardio training). Here, following baseline assessments (multiple), treatment A is randomly offered to a number of participants, and treatment B to the others. After treatment completion, everyone receives the other treatment. SCEDs also have the advantage of allowing the researchers to measure specific components of treatment, and even testing the impact of introducing a component at different stages of treatment (e.g., adding home journaling to a psychotherapy). According to Smith ²² multiple baseline SCEDs should ideally show replication of the effects across three conditions: subjects (a sample size of 30 is considered very good), settings (multiple), and behaviours (more than one dependent variable is targeted). Today, SCEDs are considered a valid method of determining the efficacy of a treatment or intervention, ²³ and requiring fewer participants than RCTs. However, the potential for convenience sampling bias is a criticism of this design. SCEDs are ideal when the repeated measure is objective (e.g., minutes spent interacting with others), rather than a subjective rating (e.g., self-esteem).

Experience Sampling Method (or Ecological Momentary Assessment)

ESMs, including ecological momentary assessments, are brief empirically validated structured diary assessments that use signals to trigger data collection to reflect, as much as possible, the measured concepts in daily life. ²⁸ ESM can be used to collect moment-to-moment changes in experiences, behaviours, mood, or other psychological constructs via multiple daily recordings, enabling sufficient within subject statistical power, allowing for much smaller samples. Participants repeatedly complete short questionnaires, often on their mobile phone—following a prompt. Like SCEDs, ESM is a within-subject approach that allows researchers to detect subtle changes over time, with the possibility of getting multiple daily evaluations before, during, and after a treatment. It is also possible to compare changes in ESM data of those receiving a treatment to that of a group of people not receiving it or receiving an alternative treatment. ²⁹ Given that symptoms may vary over time, often within a single day, ESM offers a more realistic picture of change during a treatment period (or control condition) than classic RCT designs. As such, ESM can be used to evaluate changes in positive emotions linked to social contact—which could be an explanatory variable or mediator of change enhancing the effects of antidepressants in a trial. ³⁰

Data collection typically involves 3 to 10 prompts per day for one week to one month. This method is particularly interesting to understand links between events or concepts over time (e.g., links between social interactions and depressive symptoms), while controlling for intrasubject correlations and avoiding recall biases. ²⁸ As such, it can be used to determine predictors of treatment outcomes, explore models of treatment mediation or moderation, or assess how an intervention modifies links between putative causal variables within a single day and over time. ESM has been used extensively in mental health research, including studies assessing the clinical efficacy of psychotherapies. ³¹ ESM has its limitations- the number of prompts can lead to a high number of missing, automatic or invalid responses, linked to participation burden. Also, it is essential to control for within-subject correlations and assess any potential contextual factor that might influence the results.

E-Technology

Whereas the ESM method uses self-report surveys, similar technology can also be used to collect data “passively.” Although this could be done through specialized peripherals (e.g., mobile electrocardiograph, pedometers), most current applications rely on sensor data built into common consumer electronics (e.g., smartwatch heart rate or accelerometry sensors, phone geolocation, call records, screen time). ³² Within psychiatric research, passive sensor data is often collected as part of digital phenotyping (the use of active and passive digital data to determine behavioural phenotypes with clinical relevance) to advance precision medicine, ³² relapse prediction, ³³ and risk calculation for the development of critical events like a developing a psychotic episode. ³⁴ It can also be used to evaluate treatment outcomes, such as determining if a psychosocial treatment for social anxiety helps participants leave the house more (using geolocation).

Passive sensor data also provides intensive longitudinal data, namely the continuous measurement of movement, screen activation, physical location, social activity, heart rate, and others depending on the sensors used. Because of this data resolution, sensor data can facilitate highly powered within-subject analyses. Sensor data can be outcome measures for psychosocial trials (e.g., measuring sleep time during CBT for insomnia), proxies for symptoms,^35,36 and used to validate mental or physical health apps,^37,38 including in remote trials. ³⁹

Natural Language Processing

Beyond sensors, natural language processing (NLP) offers significant promise for studies seeking to use extract treatment outcomes from routine clinical notes and practice (e.g., to determine change in psychiatric symptoms in medical charts in those having received a specific treatment vs. those who haven’t), identifying themes in electronic health records. ⁴⁰ Similar analyses can use other sources of text or speech (e.g., negative-emotion terms likely linked to depression), like social media posts, ⁴¹ but require careful attention to privacy and other ethical concerns. ⁴² E-technologies are often proxies for psychosocial outcomes, and have risks of lost data, inadequately used technology, broken or lost material, involve high cost, and would require addressing ethical and data protection concerns.

Matched Cohort Studies and Cluster Randomized Trials

Cohort studies involve large samples of individuals for whom administrative data, or clinical interviews and self-report measures, are available across different groups of people who were exposed to a given program, treatment or context contrasted with those who weren’t. To determine the impact of the program or treatment, those exposed can be compared a posteriori to others who are recruited based on their similarity on a set of criteria (typically: age, sex, years of education, socioeconomic status, and diagnosis)—this would be considered a matched cohort retrospective study. Alternatively, the participants can also be recruited simultaneously in different settings, using the same measures—a matched prospective cohort study. On a smaller scale, this may be considered a quasi-experimental design, but with larger samples, such matched groups yield naturalistic cohorts which are representative of the heterogeneity seen in typical clinical settings. In cohort studies, data typically cover samples recruited over many years and allow researchers to compare the outcomes of those who received a treatment program to those who didn’t. A good example is the Maine and Vermont longitudinal studies that collected data over three decades and demonstrated that the psychiatric rehabilitation programs offered in Vermont led to better functioning and social recovery and fewer symptoms than the traditional care offered in similar settings in Maine. ⁴³ In prospective, as well as in quasi-experimental designs, the samples might differ slightly given they might be living in different contexts (as in Maine and Vermont). A retrospective example could involve the comparison of a set of variables of a group receiving a novel treatment compared to matched data of those treated in the same setting before that treatment was offered, or who were simply not exposed to the treatment for other reasons. Cohort studies are one of the most robust forms of clinical research to look at larger phenomena, such as the efficacy of services implemented in specific regions or states but are less suitable for specific or brief time-limited treatments. They require long follow-up periods and necessitate long-term funding.

Cluster randomized trial designs take a similar approach, where a number of sites are recruited to a study, but the interventions are only offered at some of them (the site selection is done randomly). If the participant profiles are comparable across sites, then interested participants can be recruited to the clinical interventions, and improvements from routine clinical practice vs the intervention can be contrasted across sites for matched participants. This design, although limited by the difficulty in demonstrating perfect equivalence between sites and participants, was used to demonstrate that supported employment programs resulted in better employment outcomes for people with severe mental illness compared to day hospitals. ⁴⁴

Partially RPPTs

Typical RCTs do not consider participant preference. However, to a greater extent than a medication trial, psychotherapy is not effective without engagement, interest and active participation. Meta-analyses of psychotherapy trials (e.g., Swift and Callahan ⁴⁵ ) indicate that participants receiving their preferred intervention can improve outcomes by 58% in psychotherapy trials and reduce dropouts by half. Partially RPPTs ⁴⁶ ask all potential participants if they have a strongly preferred treatment option; those who do are allocated to their preferred treatment, and those who do not have a strong preference are randomized across the treatment conditions (similar to a typical RCT). Internal validity is evaluated by ensuring that there are no significant baseline symptoms or demographic differences between the arms in the subgroup that had strong preferences, as well as between the overall samples. Following this, the assumptions and analyses can be carried out similar to an RCT (assessors remain blind to preference and condition).

This design has not been widely used in trials offering multiple psychotherapy or services options. In psychosis research, Haddock et al. ⁴⁷ used a similar design to determine the effects, and preferred means of different CBT for psychosis delivery options (treatment as usual, self-help, telephone-delivered, intensive individual psychotherapy and group therapy). Unfortunately, only three participants accepted to be randomized, and few wished to try a new intervention beyond their services as usual. This underlines a clear limitation that occurs if participants are wary of randomization or of novel treatments, since the design only works when there are enough participants willing to be randomized.

Target Emulation Trial

There may be conditions, such as interventions to treat acute suicidality, where a non-specific placebo (like “befriending”), might be considered unethical and studying the long-term outcomes of an intervention may be desired. Target emulation trials ⁴⁸ involve the application of design principles from randomized trials to the analysis of observational data while following the analytical approaches of a clinical trial that it may be emulating. For instance, a study of an intervention for auditory hallucinations might involve people receiving the intervention in routine clinical practice. However, they may model their outcome measurement, timing and intervention on a previous trial. This may be particularly relevant for key outcomes that might be relatively rare events (like auditory hallucinations or suicidal behaviour). It also allows to compare the magnitude of these changes to previous smaller or shorter duration RCTs. We might alternatively start by designing a hypothetical randomized trial—the “target trial”—which includes key elements such as eligibility criteria, treatment strategies, time zero, outcomes, and a data analysis plan based on other RCTs with the same target population. The next step is to emulate this trial using observational data from routine clinical care by identifying eligible individuals, assigning treatments based on available data, and analyzing outcomes while adjusting for baseline confounding factors to approximate the conditions of randomization. ⁴⁸ This approach may be comparable to a single-arm study and may be relevant for replication trials, larger naturalistic trials, and longer-term follow-ups. Target emulation trials are pragmatic trials (measuring real-world effectiveness) and are meant for treatments offered in naturalistic settings. They are however limited by their lack of control conditions, and could be influenced by several confounding variables, such as comorbidities, parallel treatments, and so on.

Mixed Methods Hybrid Intervention Effectiveness Trials

Once a clinical intervention has been validated and demonstrated as beneficial, a crucial next step is evaluating its effectiveness in routine clinical practice. Hybrid intervention-effectiveness trials offer a way to evaluate this stage. ⁴⁹ It is a relatively recently developed trial design that combines elements of implementation science (such as understanding barriers and facilitators to the implementation of an intervention in routine clinical practice) with effectiveness trials (i.e., real-world efficacy). These pragmatic trials aim to use a clinically representative sample (with few exclusion criteria), and a large enough sample size to identify small to moderate effect sizes. The outcome measures are typically simple and clinically relevant, and the assessment and treatment are typically delivered by the site's clinicians or mental health professionals. While trialists might advocate for randomization, it is possible to integrate a patient preference design (if enough people are willing to be randomized) within a hybrid intervention-effectiveness trial (while maintaining blinded assessments, for instance).

Hybrid trials are divided into 3 types. Type 1 focuses mainly on evaluating “real-world” effectiveness of the intervention, while considering the implementation context; Type 2 equally weighs out both components; while Type 3 evaluates the implementation strategies with effectiveness as a secondary outcome.

An example from a type 2 design is, an ongoing current study by Lepage et al. ⁵⁰ examining the effectiveness of virtual CR and metacognitive training for participants with psychosis. Participants’ changes in cognition, cognitive biases and symptoms are measured by blinded assessors (effectiveness) whereas barriers and facilitators to administering and receiving these interventions as well as ways to ensure continued access to them are also being assessed (implementation). By far the most ecologically informative method, these hybrid trials can be expensive, and very demanding in terms of resources, site collaborations, and complexity of assessments (often utilizing mixed methods—incorporating qualitative interviews alongside quantitative assessments).

Discussion

This review aimed at demonstrating that several research designs and methodologies can be used, above and beyond traditional RCTs, to help determine the effects, efficacy or effectiveness of psychosocial interventions for mental health diagnoses. These different methods each have their advantages and limitations but manage to overcome some of the issues and threats to external validity seen in classical RCT designs.

To help guide the use of these methods, here is a summary of their main advantages and limitations (Table 2):

Although we presented these methods separately, they can be combined within a single program of research (e.g., embedding ESM and/or passive sensor outcomes within SCEDs or MRTs, or leveraging NLP from electronic health records to complement cohort and pragmatic designs). Given the advantages of these approaches and the well-documented limitations of traditional RCTs in psychotherapy trials, we encourage researchers to explicitly justify the choice of a conventional RCT for psychosocial treatment research when alternative designs may be more ethical, feasible, and ecologically valid. Moving forward, the goal should not be to replace RCTs, but to adopt a “fit-for-purpose” design mindset: selecting methods that match the intervention's mechanisms, the clinical context, and the realities of engagement, heterogeneity, and implementation. Doing so can accelerate evidence generation, improve generalizability, and support the development of scalable, person-centred psychosocial interventions that are both scientifically rigorous and clinically meaningful.