Sage Journals: Discover world-class research

Abstract

Pilot and feasibility randomized controlled trials (RCTs) help determine whether to proceed to a full-scale trial and identify necessary modifications. Qualitative research embedded in these trials can offer insight into feasibility, acceptability, and implementation of interventions, which is not always evident through quantitative data alone. It remains unclear how qualitative findings have influenced decisions to progress to full-scale RCTs. A scoping review was conducted, guided by Joanna Briggs Institute methodology. MEDLINE was searched for pilot and feasibility RCTs of health intervention published between 2017 and 2022 in four target journals, focusing on those using qualitative research to inform progression decision-making. Eligible studies were published in English with freely available full-text results. Titles and abstracts were screened independently by two reviewers, with full texts screened by one reviewer and a random 20% sample double-screened by a second reviewer. Data were extracted on trial characteristics, progression criteria, qualitative methods, and decision-making processes. Of the 932 records screened, 69 trials (across 94 publications) met inclusion criteria. Two approaches to using qualitative research in progression decision-making were identified: the application of prespecified progression criteria and the integration of qualitative and quantitative data. Of 18 trials which reported prespecified progression criteria, five included qualitative progression criteria. Eight trials employed mixed-methods integration techniques. Three used both integration techniques and qualitative progression criteria. The findings highlight limited but promising examples of meaningful integration. Clearer guidance is needed to support the consistent and transparent use of qualitative research in informing progression decisions in pilot and feasibility RCTs.

Keywords

pilot projects feasibility studies randomized controlled trials progression criteria qualitative research

Background

To facilitate smoother implementation of a full-scale randomized controlled trial (RCT), researchers may undertake a pilot or feasibility RCT in which components such as acceptability, intervention delivery, and recruitment and retention are first evaluated on a smaller scale (Bond et al., 2023; Eldridge et al., 2016; NIHR; Skivington et al., 2021). These preliminary studies help researchers and funders determine whether progression to a full-scale RCT is justified and what modifications, if any, are necessary prior to doing so (Leon et al., 2011). Prespecified progression criteria are recommended in pilot and feasibility RCTs to guide these decisions and assess feasibility (Skivington et al., 2021). However, such criteria are often underreported, and where they are presented, the basis for their development, justification, and use in evaluating study processes and interventions is frequently unclear (Mbuagbaw et al., 2019; Mellor et al., 2021).

Moreover, progression criteria tend to be based primarily on quantitative data, with less attention given to the contribution of qualitative findings (Baldeh et al., 2020). Qualitative research can illuminate issues that may not be captured through quantitative methods alone, such as whether an intervention is acceptable to prospective participants or whether trial processes are feasible in practice. For example, qualitative research has shown how potential participants view hypothetical trial scenarios and how these views can indicate that a planned RCT may not be feasible (Lavender & Kingdon, 2009). The Qualitative Research in Trials (QUART) study similarly demonstrated that qualitative methods can clarify whether aspects of an intervention or trial design require modification before progressing to a full-scale RCT (O’Cathain et al., 2014). Although pilot and feasibility RCTs have a different purpose from full-scale RCTs, the potential contributions of qualitative research to understanding feasibility and acceptability are equally important in these early-phase studies.

Current guidance encourages the integration of qualitative research within pilot and feasibility RCTs to help explain how and why improvements to the study may be required, which would not be possible with quantitative data alone (O’Cathain et al., 2014, 2015; Pearson et al., 2020; Thabane et al., 2010). More recently, Aschbrenner et al. (2022) proposed five planning steps for optimizing mixed methods in pilot and feasibility studies. These include (1) identifying the feasibility domains to be examined with mixed methods; (2) aligning quantitative and qualitative data sources; (3) determining the timing of data collection; (4) planning integrative analyses using joint displays; and (5) drawing meta-inferences about feasibility. This framework offers practical guidance for integrating data and highlights where and when qualitative methods can help interpret quantitative progression criteria. However, other guidance on progression criteria remains focused on quantitative metrics. While understanding how qualitative research enhances the interpretation of progression criteria is valuable, specific guidance on using qualitative findings in pilot and feasibility study progression decision-making is lacking.

Currently, the underreporting of qualitative findings in pilot and feasibility RCTs limits understanding of their impact on trial progression decision-making. For example, a review found that while 71% of external pilot trials reported plans for qualitative research, only 51% reported the findings of that research (Mellor et al., 2021). Additionally, in only one-third of the studies, the qualitative findings, like intervention acceptability, informed progression decision-making. Following from these findings, recommendations for progression criteria in external pilot trials were developed (Mellor et al., 2023). These recommendations also supported the use of qualitative findings in pilot and feasibility study progression decision-making, but specific guidance on how to apply qualitative findings into these decisions remains lacking.

As a first step in addressing this gap, the aim of this scoping review was to explore how qualitative research has informed decisions about progression from pilot and feasibility RCTs to full-scale RCTs. The objectives were to:

(1) Identify pilot and feasibility RCTs that have utilized qualitative research findings to inform decisions about progression to a full-scale RCT.

(2) Explore which study components were assessed using qualitative methods.

(3) Describe how decisions about progression from pilot and feasibility RCTs to full-scale RCTs have been made, including through the use of qualitative findings.

Methods

A protocol for this scoping review was registered on the Open Science Framework: osf.io/vyfgx (Björklund et al., 2023). The review was informed by Joanna Briggs Institute guidance (Peters et al., 2015, 2020) and is reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews (PRISMA-ScR) (Tricco et al., 2018).

Pilot and feasibility RCTs of any health intervention were eligible for inclusion if qualitative research methods were used to inform decision-making about a future full-scale RCT. Studies were limited to full-text, freely available English-language publications from the last five years of the search date (July 8, 2022). Exclusion criteria were (a) pre-pilot studies or trials not designed to inform a future full-scale RCT (as defined by the UK Medical Research Council; Craig et al., 2008); (b) studies that were not randomized or controlled; and (c) studies involving interventions not targeting patients. Protocols and full-scale RCTs were also excluded (see Table 1).

Table 1.

Inclusion Criteria

Inclusion criteria	Exclusion criteria
Results publication of a pilot or feasibility RCT	Pre-pilot studies and trials
	Protocols or full-scale RCTs
	Not randomized or controlled
Qualitative research methods were used to inform decision-making about a future full-scale RCT	Qualitative research methods not used
	Qualitative methods not used to inform trial feasibility or accessibility
Full-text publication freely available in English	Intervention did not target patients
Published within the last five years	Not informing a future full-scale RCT

In line with previous methodologically focused reviews (Mbuagbaw et al., 2019; Mellor et al., 2021), four journals were searched via MEDLINE: BMJ Open, Pilot and Feasibility Studies, Trials, and PLOS One. These journals were selected as they had the highest number of citations with the terms ‘pilot’, ‘trials’, and ‘qualitative’ in the title, based on a preliminary MEDLINE search. Reference lists of included pilot and feasibility RCTs were screened for additional publications relating to the same study, for example, where qualitative findings were reported separately from primary trial results. Search terms included ‘pilot’, ‘feasibility’, ‘trial’, ‘study’, and ‘qualitative’ (see Supplemental File 1 for full search strategy).

Duplicates and publications with ‘protocol’ in the title were removed in EndNote (The EndNote Team, 2013). The remaining records were uploaded to Rayyan (Ouzzani et al., 2016) for screening. All titles and abstracts were screened independently by two reviewers (MB and NC). Uncertainties or disagreements were resolved through discussion with a third reviewer (CJ). Full-text screening followed, using an eligibility form developed by the review team (MB, NC, MH, and CJ). One reviewer (MB) screened all full texts, with a random 20% of texts being double-screened by another reviewer (NC, MH, or CJ). Any disagreements were discussed until consensus was reached. Publications relating to the same pilot and feasibility RCT were combined and reported as a single study.

Data extraction was conducted by one reviewer (MB), with data from a random 20% of studies also extracted by another reviewer (NC, MH, or CJ) to ensure accuracy. Extracted data included general study characteristics (e.g., country and intervention type), qualitative research methods, progression criteria, trial progression recommendations (e.g., how decisions were made), and the decision-making process. Findings are presented through a descriptive summary and synthesis. Feasibility domains were determined inductively during data analysis and grouped according to common areas of focus in pilot and feasibility RCTs. Domains such as fidelity and implementation were distinguished by whether the extracted data related to the delivery of the intervention during the pilot or feasibility RCT (fidelity) or to broader issues related to the context of the trial (implementation).

Results

In total, 932 articles were identified. There were no duplicates. This reduced to 202 after title and abstract screening. A further 129 articles were excluded when full-text articles were assessed for eligibility. The most common reason for exclusion was that the studies were not pilot or feasibility RCTs (n = 93) or not informing a future RCT (n = 30). The rationale for this exclusion was that the decision-making process would not be relevant to report for these studies. Twenty-one additional articles were identified through bibliography searching of included studies. In total, 69 pilot and feasibility RCTs were included in the review, reported in 94 publications. Twenty-two of the included studies reported results in more than one article, including 11 that reported the qualitative findings separately from the main results publication (see Figure 1).

Figure 1.

PRISMA-ScR flowchart

Characteristics of Included Pilot and Feasibility RCTs

The characteristics of included pilot and feasibility RCTs are summarized in Table 2 and Supplemental File 2. Most commonly, pilot and feasibility RCTs were conducted in the United Kingdom (n = 45, 65%), followed by Ireland (n = 5, 7%) and Canada (n = 5, 7%). The sample size ranged from 13 to 360 participants, with larger sample sizes being included in cluster RCTs. The therapeutic area of included pilot and feasibility RCTs varied, including mental health (n = 12, 17%), health promotion (n = 8, 12%), and obstetrics and gynaecology (n = 8, 12%). The most common type of interventions tested were behaviour change interventions (e.g., educational interventions to promote a healthier lifestyle or self-management of symptoms) (n = 31, 45%) and therapeutic interventions (e.g., psychotherapy and mindfulness meditation) (n = 23, 33%). The majority of qualitative methods used by pilot and feasibility RCTs included interviews (n = 53, 77%) or a combination of interviews and focus groups (n = 11, 16%).

Table 2.

Study Characteristics

Study characteristics	N (%)
Country
United Kingdom	45 (65)
Ireland	5 (7)
Canada	5 (7)
United States	3 (4)
Australia	2 (3)
Bangladesh	1 (1)
Brazil	1 (1)
Chile	1 (1)
Denmark	1 (1)
Germany	1 (1)
Italy	1 (1)
Malawi	1 (1)
New Zealand	1 (1)
Sweden	1 (1)
Study terminology
Feasibility RCT	28 (41)
Pilot RCT	22 (32)
Feasibility trial	3 (4)
Pilot trial	2 (3)
Other	14 (20)
Therapeutic area
Mental health	10 (14)
Health promotion	8 (12)
Obstetrics and gynaecology	6 (9)
Stroke	6 (9)
Oncology	5 (7)
Respiratory	4 (6)
Neurology	3 (4)
Diabetes	3 (4)
Cardiology	2 (3)
Chronic pain	2 (3)
Dementia	2 (3)
Emergency medicine	2 (3)
Nephrology	2 (3)
Orthopaedics	2 (3)
Paediatrics	2 (3)
Autism	1 (1)
Chronic fatigue syndrome	1 (1)
Dermatology	1 (1)
Dietetics	1 (1)
Ear, nose, and throat	1 (1)
Gastroenterology	1 (1)
Geriatrics	1 (1)
Multiple sclerosis	1 (1)
Rheumatology	1 (1)
Surgical	1 (1)
Study intervention
Behaviour change	27 (39)
Therapeutic	27 (39)
Surgical	4 (6)
Pharmacological	3 (4)
Other	8 (12)
Participants/study population targeted by qualitative research
Patients/participants only	33 (48)
Participants and intervention deliverers	19 (28)
Participants and staff	12 (17)
Healthcare staff/intervention deliverers only	5 (7)
Trial sample size
≤30	28 (41)
31–60	19 (27)
61–100	13 (19)
≥101	9 (13)

Study Components Assessed Using Qualitative Research Methods

In the included pilot and feasibility RCTs, qualitative research was mainly used to collect data related to the study intervention, such as its acceptability (n = 44, 64%), feasibility (n = 32, 46%), and views about specific intervention components (n = 8, 41%). Qualitative methods were also used to assess participants’ and study staff’s overall views, experiences, and perspectives of having taken part in the study (n = 44, 64%). Other data collected through qualitative methods included the acceptability or feasibility of the recruitment methods (n = 20, 29%), acceptability of outcome data collection methods (n = 13, 19%), randomization (n = 10, 14%), retention (n = 8, 12%), adherence (n = 4, 6%), intervention fidelity (n = 2, 3%), and consent procedures (n = 1, 1%).

Approaches to Decision-Making About Trial Progression Using Qualitative Research

In line with the review eligibility criteria, all included pilot and feasibility RCTs reported using qualitative research to some extent to inform decisions about trial progression. Two approaches to decision-making were evident: (a) use of prespecified progression criteria and (b) use of data integration strategies. These two approaches were not always used together, and data integration was not always explicitly stated as being used to inform trial progression decision-making.

Use of Prespecified Progression Criteria

Eighteen pilot and feasibility RCTs included prespecified progression criteria. Of these 18, 13 included quantitative progression criteria only. As shown in Table 3, the most common domains to be included in quantitative progression criteria were recruitment (n = 12), outcome data collection (n = 12), and retention (n = 11). The reported methods for evaluating quantitative progression criteria included percentage thresholds (n = 14) (e.g., ≥50% of eligible participants will consent for participation in the study) and a traffic light system (n = 3) (three thresholds indicating go, amend, or stop, e.g., consent rates of eligible women: green if at least 25% of eligible women agree to participate; amber if rate between 11% and 24% women agree to participate; and red if <10% of eligible women agree to participate). The values for thresholds varied across included studies depending on the sample size and type of progression criteria. One study (McKenzie et al., 2020) did not specify how the progression criteria were assessed or specify what the progression criteria were but reported in the Discussion section that ‘progression to a definitive trial was supported based on predetermined criteria met’.

Table 3.

Domains Included in Prespecified Progression Criteria Within the 18 Studies That Included Them

Domain	RCTs that included quantitative progression criteria only (n = 13)	RCTs that included quantitative and qualitative progression criteria (n = 5)
Domain		Quantitative criteria	Qualitative criteria
Acceptability	4	1	5
Adherence	6	2	1
Efficacy	1	2	0
Fidelity	0	1	1
Funding	0	1	0
Implementation	0	0	1
Randomization	0	2	1
Recruitment	12	4	1
Retention	10	3	2
Outcome data collection	6	5	1
Other/unclear	1	2	2

Five pilot and feasibility RCTs (Artom et al., 2019; Dombrowski et al., 2020; Griffin et al., 2019; Pennington et al., 2019; Watson et al., 2021) included both qualitative and quantitative prespecified progression criteria. The domains included in qualitative progression criteria were acceptability (n = 5) and recruitment (n = 2). No studies included qualitative progression criteria only. As shown in Table 4, methods for evaluating qualitative progression criteria were often ambiguous (e.g., ‘There will be positive opinions from participants regarding an acceptably positive experience of taking part in the intervention’ (Artom et al., 2019)). Two studies applied a quantitative value to qualitative evidence (e.g., ‘Seventy-five per cent of children recruited to the treatment group and their parents rate therapy via Skype as acceptable’ (Pennington et al., 2019)).

Table 4.

Feasibility and Pilot RCTs With Combined Quantitative and Qualitative Progression Criteria

	Quantitative progression criteria	Qualitative progression criteria
Artom et al. Pilot and Feasibility Studies (2019) 5: 145	≥50% of eligible participants will consent for participation in the study	There will be positive opinions from the therapists regarding the feasibility of delivering the intervention
	≤10% missing data in each completed questionnaire booklet	There will be support from HCPs [healthcare professional] regarding the feasibility of implementation of the intervention
	90% of participants will read all the sessions of the manual; ≥15 min per week will be spent completing tasks in relation to the intervention	There will be positive opinions from participants regarding an acceptably positive experience of taking part in the intervention
	≥80% of those consented will start the intervention; ≥70% of participants who start will complete all 8 therapist support sessions; ≤20% of participants will withdraw from the intervention support sessions; ≥70% of participants will complete baseline and three-month follow-ups
Dombrowski et al. BMJ Open 2020; 10: e032653.	Willingness to be randomized	Acceptability of the intervention and the control group (by the majority of the target group)
	Feasibility of recruiting 105 men in four months
	12-month outcomes on at least 72% of men randomized per group, consistent with a recent UK weight management trial in men and systematic reviews of male obesity literature
	Evidence of mean weight loss of at least 3% of baseline weight at 12 months in any intervention group
	Commitment by, for example, government or National Health Service (NHS)/local authorities to fund the incentive intervention to ensure translation and sustainability
Griffin et al. BMJ Open 2019; 9: e033534.	Randomization occurs and more than 80% of those assessed accept randomization	Intervention is acceptable to a majority of fathers and families from differing black, Asian, and minority ethnic groups and socioeconomic backgrounds
	Recruitment of at least 68 out of the planned 90 fathers (75%) within the four-month time frame
	Intervention implemented with fidelity in 75% of observations
	Attendance: 70% attending at least five of nine of the planned sessions
	More than 70% follow-up at three and six months
	Mean weight loss in the intervention arm of ≥3 kg
Pennington et al. BMJ Open 2019; 9: e024233.	75% identified children agree to be randomized to dysarthria therapy or usual treatment	75% children recruited to the treatment group and their parents rate therapy via Skype as acceptable and no insurmountable problems are reported by families*
	75% children allocated to the usual treatment group are retained for the duration of the study	* To assess acceptability of delivery of the therapy via Skype and acceptability of the study design, we analyzed transcribed interview data using an inductive thematic analysis
	85% recorded sessions reach criterion of 120 behaviours produced across four exercises
	67% in the Skype therapy condition and their parents rate therapy as at least somewhat effective at 12 weeks after therapy
Watson et al. BMJ Open 2021; 11: e043364.	70% of recruitment targets attained for all research components	Participants, SOs [significant other], and audiologists evaluate acceptability of the ACE and RCT positively
	Study consent/retention rates and proposed sample sizes indicate delivery of the full RCT is plausible within a five-year study period
	90% of ACE groups of five to seven consented participants formed within the intervention window with participants attending three of five sessions
	Economic, acceptability, outcome measure, and fidelity evaluation data successfully collected

In pilot and feasibility RCTs where prespecified progression criteria were used, trial progression decision-making was based on whether those criteria were met or not (e.g., ‘A future RCT using the recruitment strategy reported here is not likely to be feasible. The fundamental problem was in accessing sufficient struggling HA [hearing aid] users at follow-up’ (Watson et al., 2021)). In those studies, the qualitative findings were used to varying degrees to understand why those criteria were (or were not) met. In pilot and feasibility RCTs where progression criteria were assessed using a traffic light system, decisions about trial progression and changes to the trial were more clearly on both quantitative and qualitative findings; one study (Amaefule et al., 2022) found that the quantitative progression criteria for adherence were within the amber zone and through qualitative findings were able to determine what the barrier was and what the participants preferred (i.e., a preference for the intervention supplement to be delivered in a tablet form instead of powder).

Data Integration Strategies

Two categories of integration of quantitative and qualitative data were identified: (1) description and/or discussion of quantitative and qualitative findings in the results or discussion section of the paper and (2) reported use of specific methods to integrate qualitative and quantitative data.

Description of Findings in Results or Discussion

Forty-six (67%) of the included 69 pilot and feasibility RCTs integrated the qualitative and quantitative data to varying degrees through simple description and/or discussion of the findings. Some studies used sequential description and/or discussion (e.g., discussing quantitative findings about study recruitment first and then discussing qualitative findings relating to study recruitment separately). Other studies discussed trial domains of interest (e.g., study recruitment) more generally and referred to both types of data throughout the discussion. Although the qualitative findings were discussed in some capacity, how they impacted the trial progression decision-making was sometimes unclear, particularly in the absence of prespecified progression criteria.

Specific Methods to Integrate Data

Eight (12%) of the 69 pilot and feasibility RCTs reported using specific methods to integrate the qualitative and quantitative data. Reported integration techniques used included data triangulation, a mixed-methods matrix, meshing, comparing and contrasting the data, and concurrent presentation of the data. While these studies reported the method of integration, integrated findings were not always presented in the results section or discussed. For example, Agarwal et al. (2019) reported using meshing to integrate the qualitative and quantitative data: ‘Meshing quantitative and qualitative methods was intended to provide a more multi-dimensional understanding of the feasibility of this study’. However, qualitative findings were presented in their own paragraph in the results section and discussed separately to the quantitative findings (Agarwal et al., 2019).

Only three of the included pilot and feasibility RCTs included both prespecified progression criteria and specific methods to integrate data (4%). Young et al. (2020) used a joint display showing the progression criteria and the qualitative findings relating to them side by side (see Figure 2). Taylor et al. (2022) reported the progression criteria in the main results publication and discussed the integration of qualitative and quantitative data into a single framework in a separate qualitative publication. However, a visual display of integration was not provided. A third study by Dombrowski et al. (2020) reported using triangulation and matrix coding to integrate qualitative and quantitative data but did not explicitly relate data integration to the progression criteria or provide a visual display of the integration.

Figure 2.

Example of a joint display table by Young et al. BMJ Open 2020; 10: e041227 (Young et al., 2020)

Decision-Making About Trial Progression

How decisions about trial progression were made (including if changes to the RCT were required) was not explicitly reported in the majority of pilot and feasibility RCTs (n = 61, 88%) (see Table 5). Eight (12%) (Dombrowski et al., 2020; Giovannetti et al., 2020; Griffin et al., 2019; Kyte et al., 2022; Perra et al., 2021; Rondung et al., 2022; Tarrant et al., 2021; Young et al., 2020) of the studies included some statement about how decisions on trial progression were made (e.g., ‘An independent study steering committee agreed that the Game of Stones study had demonstrated acceptability and feasibility agreeing that overall the prespecified progression criteria were sufficiently met to support a full three-arm multisite RCT’ (Dombrowski et al., 2020)).

Table 5.

Trial Progression Decision-Making Process

Study ID	Trial progression decision-making process
Dombrowski et al. 2020	“An independent study steering committee agreed that the Game of Stones study had demonstrated acceptability and feasibility agreeing that overall, the prespecified progression criteria were sufficiently met to support a full three-arm multisite randomised controlled trial (RCT).”
Giovannetti et al. 2020	“The Steering Committee and an international expert panel (four persons) jointly discussed the study findings in two dedicated meetings and outlined the multicentre RCT evaluation of the Italian READY for MS program.”
Griffin et al. 2019	“Progression criteria, agreed on by the funding panel and the Study Steering Committee, were predefined to help evaluate whether the feasibility trial should be recommended to progress to a fully powered RCT.”
Kyte et al. 2022	“An independent steering committee was convened to provide guidance to the trial management group and to review feasibility data during the trial.”
Perra et al. 2021	“A steering group that included practitioners working in neonatal services and a parent representative, as well as the study group, met approximately every 2 months to monitor progress of the study.”
Rondung et al. 2022	“A process for Decision-making after Pilot and feasibility Trials (ADePT) was used to inform necessary amendments in order to move on with the trial more efficiently.”
Tarrant et al. 2021	“The current pilot trial, accompanying qualitative interviews, a post-trial feedback session with facilitators, and researcher reflections constitute additional actions taken to optimise the intervention and trial procedures prior to a definitive trial.“
Young et al. 2020	“Judgement regarding feasibility was based on a set of a priori progression criteria. For each criterion, the development of ‘stop’ (indicating when there are issues with the trial that cannot be resolved) and ‘go’ thresholds (when there are no issues that may impede the success of a trial) were co-produced by patients, clinicians and researchers. Results falling between these thresholds indicated that adaptation to trial procedures may render a definitive RCT viable.”

Discussion

This scoping review aimed to explore how qualitative research has informed decision-making about progression from pilot and feasibility RCTs to full-scale RCTs. In total, 69 studies were included in the review that were conducted in different countries and tested a variety of interventions in different therapeutic areas. Within the included studies, qualitative research methods were most commonly used to assess intervention acceptability, feasibility, and components of the interventions.

Two broad approaches to using qualitative data in trial progression decisions were identified: the use of prespecified progression criteria and the integration of qualitative and quantitative findings. While progression criteria were more frequently reported, they tended to be quantitative in nature. Integration strategies were used but were often underutilized and rarely made explicit in relation to progression decision-making. Furthermore, the actual processes through which decisions were made about progression to a full-scale RCT were poorly reported.

Only 18 (26%) of the included studies reported prespecified progression criteria. On the one hand, this is surprising given the presence of such criteria in methodological guidance on pilot and feasibility trials (Skivington et al., 2021). On the other hand, this finding reflects prior reviews that also noted a lack of consistent reporting criteria (Mbuagbaw et al., 2019; Mellor et al., 2021). Prespecified criteria serve a critical role as they offer a structured approach for assessing feasibility and acceptability of both trial processes and interventions. However, their largely quantitative nature may limit their ability to capture the complex, context-dependent aspects of trial acceptability that qualitative methods can highlight when meaningfully integrated with those findings (Aschbrenner et al., 2022).

Previous reviews have primarily examined the reporting of progression criteria (Mbuagbaw et al., 2019; Mellor et al., 2021) or whether qualitative research has been reported in pilot and feasibility studies (Baldeh et al., 2020). These reviews consistently found that both remain underreported despite methodological guidance highlighting their importance. What has not yet been sufficiently explored is how qualitative research is used to inform decisions about trial progression, which is what this review addresses. This is a critical area of inquiry, particularly as guidance continues to recommend the use of qualitative research in evaluating complex interventions and informing future study design (Skivington et al., 2021).

In current practice, progression criteria have become closely associated with quantitative thresholds, for example, recruitment rates, retention levels, or adherence measures. However, these kinds of measures may be insufficient for evaluating less distinct but equally important aspects of feasibility, such as how acceptable participants find the intervention or trial procedures (Mellor et al., 2023). In this context, the term ‘progression criteria’ may not adequately reflect the interpretive nature of qualitative data, which offer a different kind of understanding not easily determined through numerical thresholds alone. Five studies in this review demonstrated that qualitative findings contributed to judgements about feasibility or acceptability in ways that influenced the progression decisions. These examples illustrate how qualitative data can play a substantive role when it is considered alongside quantitative data, although it is not clear whether this was due to the inclusion of qualitative progression criteria or explicit data integration methods, or both.

Across the included studies, qualitative research supported a range of contributions to progression decisions, including identifying modifications to intervention delivery (e.g., Amaefule et al., 2022), clarifying barriers to recruitment (Watson et al., 2021), and helping to interpret mixed-methods feasibility outcomes (Dombrowski et al., 2020; Young et al., 2020). These examples show how qualitative findings can shape decisions even in studies without prespecified qualitative criteria or formal integration techniques. This aligns with findings from the QUART study, which identified examples of qualitative research influencing intervention refinement, recruitment approaches, outcome measures, and the interpretation of trial findings (O’Cathain et al., 2014).

Among the studies that included both types of data, the majority (n = 46, 67%) described some form of integration methods, typically through narrative synthesis or descriptive comparison. However, only eight studies (12%) employed formal data integration. This is notable given recent guidance that calls for deliberate and transparent integration strategies to maximize the value of mixed methods in pilot and feasibility studies (Aschbrenner et al., 2022). When used meaningfully, these integration approaches can facilitate deeper understanding of the relationships between feasibility, acceptability, and contextual factors, thereby providing a more holistic foundation for progression decisions.

Notably, 93% of studies did not report the actual process by which progression decisions were made, limiting the field’s understanding of how both qualitative and quantitative findings are used in practice. In the few studies (n = 3) that did use both prespecified progression criteria and specific data integration methods, the influence of qualitative findings on progression decision-making was more explicit. In these cases, integration enhanced transparency and provided clear justifications for whether and how to proceed to a full-scale RCT. Such clarity is essential for research teams, funding bodies, and trial steering committees who evaluated the feasibility for larger-scale trials. Furthermore, these examples serve as potential examples for future trials aiming to embed qualitative research more meaningfully within progression decision-making processes.

The findings from this review suggest that researchers may benefit from clearer guidance on how to plan for, conduct, and report qualitative research in the context of pilot and feasibility study progression decision-making. There remains an important need to better understand why qualitative research is or is not used in these decision-making processes and how its integration with quantitative findings can be made more systematic and transparent. Methodological research to address these issues could support more rigorous and impactful pilot and feasibility studies. Such work could also ensure that the full potential of qualitative inquiry is realized in optimizing intervention development and trial design.

Strengths and Limitations

This review contributes to the literature by offering the first, to our knowledge, focused examination of how qualitative research has been used to inform decisions about whether to progress from pilot and feasibility to full-scale RCTs. It complements and extends prior reviews by examining not just whether qualitative research was used, but how it was applied within the context of trial development and progression decision-making. A particular strength is the inclusion of multiple publications linked to single studies, allowing for a more comprehensive capture of qualitative findings, especially those reported separately from the main trial findings. This approach enabled a more holistic view of how qualitative data contributed to the overall decision-making process.

The review also highlights three exemplar studies that integrated both prespecified progression criteria and structured data integration techniques, demonstrating how mixed-methods approaches can be optimally employed to support progression decision-making. These studies may serve as useful reference points for researchers aiming to better integrate qualitative research into pilot and feasibility studies.

As a scoping review, this study did not synthesize findings across all relevant literature or assess the quality of included studies. Instead, its aim was to map key themes, gaps, and patterns in the use of qualitative research for trial progression decisions. Another limitation is the review’s focus on randomized and controlled pilot and feasibility studies. While this ensured a manageable number of included studies, it excluded other forms of feasibility research. Nonetheless, the focus on qualitative contributions to progression decision-making means that many of the findings here may be transferable to other types of pilot and feasibility studies beyond RCTs.

Restricting the search to four journals may have limited the pool of eligible results papers, as pilot and feasibility RCT findings can be published across a broader range of journals. This may have reduced the diversity of included studies. However, the journals selected were deemed to publish a considerable number of pilot and feasibility RCTs, and the restriction contributed to maintaining a manageable scope for the review. As such, this is unlikely to affect the main conclusions of the review.

Conclusions

This review identified pilot and feasibility RCTs that used qualitative research to help determine whether and how to progress to a full-scale RCT. Approaches to supporting trial progression included prespecified progression criteria, integration of qualitative and quantitative data, or a combination of both. However, most studies lacked transparent reporting of these processes, and the combined use of progression criteria and integration strategies was rare. While guidance for both elements exists, current practice remains inconsistent. The three exemplar studies identified here offer valuable examples for integrating qualitative and quantitative findings to support more robust, transparent, and holistic progression decisions. Embracing such approaches could help ensure that qualitative research contributes more meaningfully to progression decision-making in pilot and feasibility studies. This would ultimately support better-informed intervention development and more effective health research.

Supplemental Material

Supplemental Material - The Role of Qualitative Research in Progression Decision-Making in Mixed-Methods Pilot and Feasibility Randomized Controlled Trials: A Scoping Review

Supplemental Material for The Role of Qualitative Research in Progression Decision-Making in Mixed-Methods Pilot and Feasibility Randomized Controlled Trials: A Scoping Review by Matilda Björklund, Nicola Cornwall, Melanie Ann Holden, and Clare Jinks in Qualitative Health Research

Footnotes

ORCID iD

Matilda Björklund

Ethical Considerations

Our study did not require an ethical board approval because it did not directly involve humans or animals.

Author Contributions

MB: Conceptualization, data curation, formal analysis, investigation, methodology, visualization, and writing – original draft. NC: Conceptualization, data curation, formal analysis, investigation, methodology, visualization, supervision, writing – review and editing, and validation. MH: Conceptualization, data curation, formal analysis, investigation, methodology, visualization, supervision, writing – review and editing, and validation. CJ: Conceptualization, data curation, formal analysis, investigation, methodology, visualization, supervision, writing – review and editing, and validation.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This PhD was funded by the Faculty of Medicine and Health Sciences Doctoral Fellowship Scheme at Keele University. NC and CJ are part funded by the National Institute for Health Research and Care Applied Research Collaboration West Midlands.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Supplemental Material

Supplemental material for this article is available online.

References

Agarwal

Gaber

Richardson

Mangin

Ploeg

Valaitis

Reid

G. J.

Lamarche

Parascandalo

Javadi

O'Reilly

Dolovich

(2019). Pilot randomized controlled trial of a complex intervention for diabetes self-management supported by volunteers, technology, and interprofessional primary health care teams. Pilot and Feasibility Studies, 5, 118. https://doi.org/10.1186/s40814-019-0504-8

Amaefule

C. E.

Drymoussi

Gonzalez Carreras

F. J.

Pardo Llorente

M. D. C.

Lanz

Dodds

Sweeney

Pizzo

Thomas

Heighway

Daru

Sobhy

Poston

Khalil

Myers

Harden

Hitman

Khan

K. S.

Zamora

Thangaratinam

(2022). Myo-inositol nutritional supplement for prevention of gestational diabetes (EMmY): A randomised, placebo-controlled, double-blind pilot trial with nested qualitative study. BMJ Open, 12(3), Article e050110. https://doi.org/10.1136/bmjopen-2021-050110

Aschbrenner

K. A.

Kruse

Gallo

J. J.

Plano Clark

V. L.

(2022). Applying mixed methods to pilot feasibility studies to inform intervention trials. Pilot and Feasibility Studies, 8(1), Article 217. https://doi.org/10.1186/s40814-022-01178-x

Artom

Czuber-Dochan

Sturt

Proudfoot

Roberts

Norton

(2019). Cognitive-behavioural therapy for the management of inflammatory bowel disease-fatigue: a feasibility randomised controlled trial. Pilot and Feasibility Studies, 5, Article 145. https://doi.org/10.1186/s40814-019-0538-y

Baldeh

MacDonald

Kosa

S. D.

Lawson

D. O.

Stalteri

Olaiya

O. R.

Alotaibi

Thabane

Mbuagbaw

(2020). More pilot trials could plan to use qualitative data: A meta-epidemiological study. Pilot and Feasibility Studies, 6(1), Article 164. https://doi.org/10.1186/s40814-020-00712-z

Björklund

Cornwall

Holden

Jinks

(2023). How has qualitative research been used in pilot and feasibility studies to inform the decision-making around progression to full-scale randomised controlled trials? A scoping review protocol. Retrieved from osf.io/vyfgx.

Bond

Lancaster

G. A.

Campbell

Chan

Eddy

Hopewell

Mellor

Thabane

Eldridge

(2023). Pilot and feasibility studies: Extending the conceptual framework. Pilot and Feasibility Studies, 9(1), Article 24. https://doi.org/10.1186/s40814-023-01233-1

Craig

Dieppe

Macintyre

Michie

Nazareth

Petticrew

Medical Research Council Guidance . (2008). Developing and evaluating complex interventions: The new Medical Research Council guidance. BMJ, 337, Article a1655. https://doi.org/10.1136/bmj.a1655

Dombrowski

S. U.

McDonald

van der Pol

Grindle

Avenell

Carroll

Calveley

Elders

Glennie

Gray

C. M.

Harris

F. M.

Hapca

Jones

Kee

McKinley

M. C.

Skinner

Tod

Hoddinott

(2020). Game of Stones: Feasibility randomised controlled trial of how to engage men with obesity in text message and incentive interventions for weight loss. BMJ Open, 10(2), Article e032653. https://doi.org/10.1136/bmjopen-2019-032653

10.

Eldridge

S. M.

Chan

C. L.

Campbell

M. J.

Bond

C. M.

Hopewell

Thabane

Lancaster

G. A.

PAFS consensus group . (2016). CONSORT 2010 statement: Extension to randomised pilot and feasibility trials. BMJ, 355, Article i5239. https://doi.org/10.1136/bmj.i5239

11.

Giovannetti

A. M.

Quintas

Tramacere

Giordano

Confalonieri

Messmer Uccelli

Solari

Pakenham

K. I.

(2020). A resilience group training program for people with multiple sclerosis: Results of a pilot single-blind randomized controlled trial and nested qualitative study. PLoS One, 15(4), Article e0231380. https://doi.org/10.1371/journal.pone.0231380

12.

Griffin

Sun

Sidhu

Adab

Burgess

Collins

Daley

Entwistle

Frew

Hardy

Hurley

Jones

McGee

Pallan

Young

Morgan

Jolly

(2020). Healthy Dads, Healthy Kids UK, a weight management programme for fathers: feasibility RCT. BMJ Open, 9(12), Article e033534. https://doi.org/10.1136/bmjopen-2019-033534

13.

Kyte

Anderson

Bishop

Bissell

Brettell

Calverts

Chadburn

Cockwell

Dutton

Eddington

Forster

Hadley

Ives

N. J.

Jackson

L. J.

O'Brien

Price

Sharpe

Stringer

Verdi

Waters

Wilcockson

(2022). Results of a pilot feasibility randomised controlled trial exploring the use of an electronic patient-reported outcome measure in the management of UK patients with advanced chronic kidney disease. BMJ Open, 12(3), Article e050610. https://doi.org/10.1136/bmjopen-2021-050610

14.

Lavender

Kingdon

(2009). Primigravid women’s views of being approached to participate in a hypothetical term cephalic trial of planned vaginal birth versus planned cesarean birth. Birth, 36(3), 213–219. https://doi.org/10.1111/j.1523-536X.2009.00325.x

15.

Leon

A. C.

Davis

L. L.

Kraemer

H. C.

(2011). The role and interpretation of pilot studies in clinical research. Journal of Psychiatric Research, 45(5), 626–629. https://doi.org/10.1016/j.jpsychires.2010.10.008

16.

Mbuagbaw

Kosa

S. D.

Lawson

D. O.

Stalteri

Olaiya

O. R.

Alotaibi

Thabane

(2019). The reporting of progression criteria in protocols of pilot trials designed to assess the feasibility of main trials is insufficient: A meta-epidemiological study. Pilot and Feasibility Studies, 5(1), Article 120. https://doi.org/10.1186/s40814-019-0500-z

17.

McKenzie

Dallos

Stedmon

Hancocks

Vickery

P. J.

Barton

Vassallo

Myhill

Chynoweth

Ewings

(2020). SAFE, a new therapeutic intervention for families of children with autism: a randomised controlled feasibility trial. BMJ Open, 10(12), Article e038411.

18.

Mellor

Albury

Dutton

S. J.

Eldridge

Hopewell

(2023). Recommendations for progression criteria during external randomised pilot trial design, conduct, analysis and reporting. Pilot and Feasibility Studies, 9(1), Article 59. https://doi.org/10.1186/s40814-023-01291-5

19.

Mellor

Eddy

Peckham

Bond

C. M.

Campbell

M. J.

Lancaster

G. A.

Thabane

Eldridge

S. M.

Dutton

S. J.

Hopewell

(2021). Progression from external pilot to definitive randomised controlled trial: A methodological review of progression criteria reporting. BMJ Open, 11(6), Article e048178. https://doi.org/10.1136/bmjopen-2020-048178

20.

NIHR . Glossary. https://www.nihr.ac.uk/glossary/

21.

O’Cathain

Goode

Drabble

S. J.

Thomas

K. J.

Rudolph

Hewison

(2014). Getting added value from using qualitative research with randomized controlled trials: A qualitative interview study. Trials, 15(1), Article 215. https://doi.org/10.1186/1745-6215-15-215

22.

O’Cathain

Hoddinott

Lewin

Thomas

K. J.

Young

Adamson

Jansen

Y. J.

Mills

Moore

Donovan

J. L.

(2015). Maximising the impact of qualitative research in feasibility studies for randomised controlled trials: Guidance for researchers. Pilot and Feasibility Studies, 1(1), Article 32. https://doi.org/10.1186/s40814-015-0026-y

23.

O’Cathain

Thomas

K. J.

Drabble

S. J.

Rudolph

Goode

Hewison

(2014). Maximising the value of combining qualitative research and randomised controlled trials in health research: The QUAlitative Research in Trials (QUART) study – A mixed methods study. Health Technology Assessment, 18(38). https://doi.org/10.3310/hta18380

24.

Ouzzani

Hammady

Fedorowicz

Elmagarmid

(2016). Rayyan - A web and mobile app for systematic reviews. Systematic Reviews, 5(1), Article 210. https://doi.org/10.1186/s13643-016-0384-4

25.

Pearson

Naylor

P.-J.

Ashe

M. C.

Fernandez

Yoong

S. L.

Wolfenden

(2020). Guidance for conducting feasibility and pilot studies for implementation trials. Pilot and Feasibility Studies, 6(1), Article 167. https://doi.org/10.1186/s40814-020-00634-w

26.

Pennington

Stamp

Smith

Kelly

Parker

N. K.

Stockwell

Aluko

Othman

Brittain

Vale

(2019). Internet delivery of intensive speech and language therapy for children with cerebral palsy: A pilot randomised controlled trial. BMJ Open, 9(1), Article e024233. https://doi.org/10.1136/bmjopen-2018-024233

27.

Perra

Wass

McNulty

Sweet

Papageorgiou

K. A.

Johnston

Bilello

Patterson

Alderdice

(2021). Very preterm infants engage in an intervention to train their control of attention: results from the feasibility study of the Attention Control Training (ACT) randomised trial. Pilot and Feasibility Studies, 7(1), Article 66. https://doi.org/10.1186/s40814-021-00809-z

28.

Peters

M. D.

Godfrey

C. M.

Khalil

McInerney

Parker

Soares

C. B.

(2015). Guidance for conducting systematic scoping reviews. International Journal of Evidence-Based Healthcare, 13(3), 141–146. https://doi.org/10.1097/xeb.0000000000000050

29.

Peters

M. D. J.

Marnie

Tricco

A. C.

Pollock

Munn

Alexander

McInerney

Godfrey

C. M.

Khalil

(2020). Updated methodological guidance for the conduct of scoping reviews. JBI Evidence Synthesis, 18(10), 2119–2126. https://doi.org/10.11124/jbies-20-00167

30.

Rondung

Leiler

Sarkadi

Bjärtå

Lampa

Löfving

S. G.

Calam

Oppedal

Keeshin

Warner

(2022). Feasibility of a randomised trial of Teaching Recovery Techniques (TRT) with refugee youth: results from a pilot of the Swedish UnaccomPanied yOuth Refugee Trial (SUPpORT). Pilot and Feasibility Studies, 8(1), Article 40. https://doi.org/10.1186/s40814-022-00998-1

31.

Skivington

Matthews

Simpson

S. A.

Craig

Baird

Blazeby

J. M.

Boyd

K. A.

Craig

French

D. P.

McIntosh

Petticrew

Rycroft-Malone

White

Moore

(2021). A new framework for developing and evaluating complex interventions: Update of Medical Research Council guidance. BMJ, 374, Article n2061. https://doi.org/10.1136/bmj.n2061

32.

Tarrant

Carter

Dean

S. G.

Taylor

Warren

F. C.

Spencer

Adamson

Landa

Code

Backhouse

Lamont

R. A.

Calitri

(2021). Singing for people with aphasia (SPA): Results of a pilot feasibility randomised controlled trial of a group singing intervention investigating acceptability and feasibility. BMJ Open, 11(1), Article e040544. https://doi.org/10.1136/bmjopen-2020-040544

33.

Taylor

Cairns

Glass

B. D.

(2022). Feasibility, accessibility and acceptability a pharmacist-led ear health intervention at rural community pharmacies (LISTEN UP): A mixed-methods study in Queensland, Australia. BMJ Open, 12(4), Article e057011. https://doi.org/10.1136/bmjopen-2021-057011

34.

Thabane

Chu

Cheng

Ismaila

Rios

L. P.

Robson

Thabane

Giangregorio

Goldsmith

C. H.

(2010). A tutorial on pilot studies: The what, why and how. BMC Medical Research Methodology, 10(1), Article 1. https://doi.org/10.1186/1471-2288-10-1

35.

The EndNote Team . (2013). EndNote. In (Version EndNote 2025) [64 bit]. Clarivate.

36.

Tricco

A. C.

Lillie

Zarin

O’Brien

K. K.

Colquhoun

Levac

Moher

Peters

M. D. J.

Horsley

Weeks

Hempel

Akl

E. A.

Chang

McGowan

Stewart

Hartling

Aldcroft

Wilson

M. G.

Garritty

Straus

S. E.

(2018). PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467–473. https://doi.org/10.7326/m18-0850

37.

Watson

Coleman

Jackson

Bell

Maynard

Hickson

Forster

Fairhurst

Hewitt

Gardner

Iley

Gailey

Thyer

N. J.

(2021). Randomised controlled feasibility trial of an active communication education programme plus hearing aid provision versus hearing aid provision alone (ACE To HEAR). BMJ Open, 11(4), Article e043364. https://doi.org/10.1136/bmjopen-2020-043364

38.

Young

H. M. L.

March

D. S.

Highton

P. J.

Graham-Brown

M. P. M.

Churchward

D. C.

Grantham

Goodliffe

Jones

Cheung

M. M.

Greenwood

S. A.

Eborall

H. C.

Conroy

Singh

S. J.

Smith

A. C.

Burton

J. O.

(2020). Exercise for people living with frailty and receiving haemodialysis: A mixed-methods randomised controlled feasibility study. BMJ Open, 10(11), Article e041227. https://doi.org/10.1136/bmjopen-2020-041227

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.68 MB