Although the randomized controlled trial (RCT) is the most reliable design to infer causality, evidence suggests that it is vulnerable to biases that weaken internal validity. In this paper, we review factors that introduce biases in RCTs and we propose quantitative and qualitative strategies for colleting relevant data to strengthen internal validity. The factors are related to participants’ reactions to randomization, attrition, treatment perceptions, and implementation of the intervention. The way in which these factors operate is explained and pertinent empirical evidence is synthesized. Quantitative and qualitative strategies are described. Researchers can plan to assess these factors and examine their influence, providing evidence of what actually contributed to the interventions’ causal impact.
AbrahaI.CherubiniA.CozzolinoF.De FlorioR.LuchettaM. L.RimlandJ. M.FollettiI.MarchesiM.GermaniA.OrsoM.EusebiP.MontedoriA. (2015).
Deviation from intention to treat analysis in randomised trials and treatment effect estimates: Meta-epidemiological study. BMJ,
350(may27 2), h2445–h2445. https://doi.org/10.1136/bmj.h2445
2.
BarbosaC. D.BalpM.-M.KulichK.GermainN.RofailD. (2012).
A literature review to explore the link between treatment satisfaction and adherence, compliance, and persistence. Patient Preference and Adherence,
6, 39–48.
3.
BeasleyM. J.Ferguson-JonesE. A.MacfarlaneG. J.; on Behalf of the MUSICIAN Study Team. (2017).
Treatment expectations but not preference affect outcome in a trial of CBT and exercise for pain. Canadian Journal of Pain = Revue Canadienne de la Douleur,
1(1), 161–170. https://doi.org/10.1080/24740527.2017.1384297
4.
Bradley-GilbrideJ.BradleyC. (2010). Partially randomized preference trial design. In N. J. Salkind (Ed.), Encyclopedia of research design (Vol.
2, pp. 1009–1015). SAGE.
5.
CarneyC. E.BuysseD. J.Ancoli-IsraelS.EdingerJ. D.KrystalA. D.LichsteinK. L.MorinC. M. (2012).
The consensus sleep diary: Standardizing prospective sleep self-monitoring. Sleep,
35(2), 287–302. https://doi.org/10.5665/sleep.1642
6.
CarrollC.PattersonM.WoodS.BoothA.RickJ.BalainS. (2007).
A conceptual framework for implementation fidelity. Implementation Science,
2(1), 40–48.
7.
ChengK. K. F.MetcalfeA. (2018).
Qualitative methods and process evaluation in clinical trial context: Where to head?International Journal of Qualitative Methods,
17(1), 160940691877421–160940691877424. https://doi.org/10.1177/160940691
8.
DalkinS. M.GreenhalghJ.JonesD.CunninghamB.LhussierM. (2015).
What’s in a mechanism? Development of a key concept in realist evaluation. Implementation Science,
10, 49–55. https://doi.org/10.1186/s13012-015-0237-x
9.
DillerJ. W.BrownR. M.PatrosC. H. G. (2013).
Revisiting Kazdin (1980): Contemporary treatment acceptability for problem behavior in children. International Journal of Psychology and Psychological Therapy,
13(2), 225–231.
10.
DubergA.MöllerM.SunvissonH. (2016). “
I feel free”: Experiences of a dance intervention for adolescent girls with internalizing problems. International Journal of Qualitative Studies on Health and Well-Being,
11, 31946–31960. http://dx.doi.org/10.3402/qhw.v11.31946
11.
FernandezE.SalemD.SwiftJ. K.RamtahalN. (2015).
Meta-analysis of dropout from cognitive behavioral therapy: Magnitude, timing, and moderators. Journal of Consulting & Clinical Psychology,
86(6), 1108–1122. https://doi.org/10.1037/ccp000044
12.
FriedenT. R. (2017).
Evidence for health decision making—Beyond randomized, controlled trials. The New England Journal of Medicine,
377(5), 465–475.
13.
FrisaldiE.ShaibaniA.BenedettiF. (2017).
Why we should assess patients’ expectations in clinical trials. Pain and Therapy,
6(1), 107–104.
14.
GlasgowR. E.MagidD. J.BeckA.RitzwollerD.EstabrooksP. A. (2005).
Practical clinical trials for translating research to practice: Design and measurement recommendations. Medical Care,
43(6), 551–559.
15.
GrossC. P.KrumholzH. M.Van WyeG.EmanuelE. J.WendlerD. (2006).
Does random treatment assignment cause harm to research participants?PLoS Medicine,
3(6), e188.https://doi.org/10.1371/journal.pmed.0030188
16.
HansenH. P.Tjørnhøj-ThomsenT. (2016).
Meeting the challenges of intervention research in health science: An argument for a multimethod research approach. The Patient,
9(3), 193–200. https://doi.org/10.1007/s40271=015-0153-9
17.
HernánM. A.RobinsJ. M. (2017).
Per-protocol analyses of pragmatic trials. The New England Journal of Medicine,
377(14), 1391–1398. https://doi.org/10.1056/NEJMsm1605385
18.
HolmM.AlvarizaA.FürstC.-J.WengströmY.ÅrestedtK.ÖhlenJ.GoliathI. (2017).
Recruiting participants in a randomized controlled trial testing an intervention in palliative cancer care—The perspectives of health care professionals. European Journal of Oncology Nursing: The Official Journal of European Oncology Nursing Society,
31, 6–11. https://doi.org/10/1016/j.ejon2017.09.001
19.
HolmeØ.LøbergM.KalagerM.BretthaueM.HernánM. A.AasE.EideT. J.SkovlundE.SchneedeJ.TveitK. M.HoffG. (2014).
Effect of flexible sigmoidoscopy screening on colorectal cancer incidence and mortality: A randomized clinical trial. JAMA,
312(6), 606–615. https://doi.org/10.1001/jama.2014.8266
20.
HorwitzR. I.Hayes-ConroyA.CaricchioR.SingerB. H. (2017).
From evidence based medicine to medicine based evidence. The American Journal of Medicine,
130(11), 1246–1250.
21.
JohnsonR. B.SchoonenboomJ. (2016).
Adding qualitative and mixed methods research to health intervention studies: Interacting with differences. Qualitative Health Research,
26(5), 587–602.
22.
KendraM. S.WeingardtK. R.CucciareM. A.TimkoC. (2015).
Satisfaction with substance use treatment and 12-step groups predicts outcomes. Addictive Behaviors,
40, 27–32. https://doi.org/10.1016/j.addbeh.2014.08.003
23.
KerrC.RobinsonE.StevensA.BraunholtzD.EdwardsS.LilfordR. (2004).
Randomisation in trials: Do potential trial participants understand it and find it acceptable?Journal of Medical Ethics,
30(1), 80–84.
24.
KomatsuH.YagasakiK.YamaguchiT. (2016).
Effects of a nurse-led medication self-management programme in cancer patients: Protocol for a mixed-method randomised controlled trial. BMC Nursing,
15, 9–18. https://doi.org/10.1186/s12912-016-0130-1
25.
KozicaS. L.TeedeH. J.HarrisonC. L.KleinR.LombardC. B. (2016).
Optimizing implementation of obesity prevention programs: A qualitative investigation within a large-scale randomized controlled trial. The Journal of Rural Health: Official Journal of the American Rural Health Association and the National Rural Health Care Association,
32(1), 72–81.
LawlorC.SharmaB.KhondokerM.PetersE.KuipersE.JohnsL. (2017).
Service user satisfaction with cognitive behavioral therapy for psychosis: Associations with therapy outcomes and perceptions of the therapist. The British Journal of Clinical Psychology,
56(1), 84–102. https://doi.org/10.1111/bjc.12122
28.
LindheimO.BennettC. B.TrentacostaC. J.McLearC. (2014).
Client preferences affect treatment satisfaction, completion, and clinical outcome: A meta-analysis. Clinical Psychology Review,
34, 506–517.
29.
LodiS.SharmaS.LundgrenJ. D.PhillipsA. N.ColeS. R.LoganR.AganB. K.BabikerA.KlinkerH.ChuH.LawM.NeatonJ. D.HernánM. A.; INSIGHT Strategic Timing of AntiRetroviral Treatment (START) Study Group. (2016).
The per-protocol effect of immediate vs. deferred ART initiation in the START randomized trial. Aids (London, England),
30(17), 2659–2663. https://doi.org/10.1097/QAD.0000000000001243
30.
Meinich PetersonS.ZoffmannV.KjaergaardJ.SteensballsL. G.GreisenG. (2013).
Disappointment and adherence among parents of newborns allocated to the control group: A qualitative study of a randomized clinical trial. Trials,
15, 126–136.
MilosevicI.LevyH. C.AlcoladoG. M.RadomskyA. S. (2015).
The treatment acceptability/adherence scale: Moving beyond the assessment of treatment effectiveness. Cognitive Behaviour Therapy,
44(6), 456–469.
33.
MooneyT. K.Connolly GibbonsM. B. C.GallopR.MackR. A.Crits-ChristophP. (2014).
Psychotherapy credibility ratings: Patient predictors of credibility and the relation of credibility to therapy outcome. Psychotherapy Research: Journal of the Society for Psychotherapy Research,
24(5), 565–577. https://doi.org/10.1080/10503307.2013.847988
34.
MossA. S.ReibelD. K.GreesonJ. M.ThaparA.BubbR.SalmonJ.NewbergA. B. (2015).
An adapted mindfulness-based stress reduction program for elders in a continuing care retirement community: Quantitative and qualitative results from a pilot randomized controlled trials. Journal of Applied Gerontology: The Official Journal of the Southern Gerontological Society,
34(4), 518–538.
35.
OmerS.GoldenE.PriebeS. (2016).
Exploring the mechanism of a patient-centered assessment with a solution focused approach (DiALOG+) in the community treatment of patients with psychosis: A process evaluation within a cluster-randomised controlled trial. PLoS One,
11(2), e0148415. https://doi.org/10.1371/journal.pone.0148415
36.
ReichertsP.GerdesA. B. M.PauliP.WieserM. J. (2016).
Psychological placebo and nocebo effects on pain rely on expectations and previous experience. The Journal of Pain,
17(2), 203–214.
37.
RixonL.BaronJ.McGaleN.LorencattoF.FrancisJ.DaviesA. (2016).
Methods used to address fidelity of receipt in health intervention research: A citation analysis and systematic review. BMC Health Services Research,
16(1), 663–686. https://doi.org/10.1186/s12913-016-1904-6
38.
RobinsonE. J.BiggerstaffD.JenningsS.MaylorE. A. (2012).
Do the public share practitioners’ views about the best evidence?Patient Education and Counseling,
88(2), 325–329.
39.
SchoultzM.MacadenL.HubbardG. (2016).
Participants’ perspectives on mindfulness-based cognitive therapy for inflammatory bowel disease: A qualitative study nested within a pilot randomised controlled trial. Pilot and Feasibility Studies,
2, 3–16. https://doi.org/10.1186/s40814-015-0041-z
40.
SekhonM.CartwrightM.FrancisJ. J. (2017).
Acceptability of healthcare interventions: An overview of reviews and development of a theoretical framework. BMC Health Services Research,
17(1), 88–100. https://doi.org/10.1186/s12913-017-2031-8
41.
ShadishW. R.HuX.GlaserR. R.KownackiR.WongS. (1998).
A method for exploring the effects of attrition in randomized experiments with dichotomous outcomes. Psychological Methods,
3(1), 3–22.
42.
SheanG. (2014).
Limitations of randomized control designs in psychotherapy research. Advances in Psychiatry,
2014, 561452. http://dx.doi.org/10.1155/2014/561452
43.
SidaniS. (2015). Health intervention research: Advances in research design and methods.
SAGE.
44.
SidaniS.EpsteinD. R.FoxM. (2017c).
Psychometric evaluation of a multi-dimensional measure of satisfaction with behavioral interventions. Research in Nursing & Health,
40(5), 459–469.
45.
SidaniS.EpsteinD. R.MirandaJ.FoxM. (2018).
Psychometric properties of the treatment perception and preferences scale. Clinical Nursing Research,
27(6), 743–761. https://doi.org/10.1177/1054773816654137
46.
SidaniS.FoxM.CollinsL. (2017a).
Towards patient-centered clinical trial designs. European Journal for Patient-Centered Care,
5(3), 300–307.
47.
SidaniS.FoxM.EpsteinD. R. (2017b).
Contribution of treatment acceptability to acceptance of randomization: An exploration. Journal of Evaluation in Clinical Practice,
23(1), 14–20. https://doi.org/10.1111/jep.12423.
48.
SpillaneJ. P.ParejaA. S.DornerL.BarnesC.MayH.HuffJ.CamburnE. (2010).
Mixing methods in randomized controlled trials (RCTs): Validation, contextualization, triangulation, and control. Educational Assessment, Evaluation and Accountability,
22(1), 5–28. https://doi.org/10.1007/s11092-009-9089-8
49.
SteidtmannD.ManberR.ArnowB. A.KleinD. N.MarkowitzJ. C.RothbaumB. O.ThaseM. E.KocsisJ. H. (2012).
Patient treatment preference as a predictor of response and attrition in treatment for chronic depression. Depression and Anxiety,
29(10), 896–905.
50.
SwiftJ. K.GreenbergR. P. (2012).
Premature discontinuation in adult psychotherapy: A meta-analysis. Journal of Consulting and Clinical Psychology,
80(4), 547–559. https://doi.org/10.1037/a002822
51.
TohS.Hernández-DíazS.LoganR.RobinsJ. M.HernánM. A. (2010).
Estimating absolute risks in the presence of nonadherence: An application to a follow-up study with baseline randomization. Epidemiology (Cambridge, Mass.),
21(4), 528–539. https://doi.org/0.1097/EDE.0b013e3181df1b69
52.
YoungerJ.GandhiV.HubbardE.MackeyS. (2012).
Development of the Stanford expectations of treatment scale (SETS): A tool for measuring patient outcome expectancy in clinical trials. Clinical Trials (London, England),
9(6), 767–776. https://doi.org/10.1177/1740774512465064
