Blinding aims to minimize biases from what participants and investigators know or believe. Randomized controlled trials, despite being the gold standard to evaluate treatment effect, do not generally assess the success of blinding. We investigated the extent of blinding in back pain trials and the associations between participant guesses and treatment effects.
Methods:
We did a review with PubMed/OvidMedline, 2000–2019. Eligibility criteria were back pain trials with data available on treatment effect and participants’ guess of treatment. For blinding, blinding index was used as chance-corrected measure of excessive correct guess (0 for random guess). For treatment effects, within- or between-arm effect sizes were used. Analyses of investigators’ guess/blinding or by treatment modality were performed exploratorily.
Results:
Forty trials (3899 participants) were included. Active and sham treatment groups had mean blinding index of 0.26 (95% confidence interval: 0.12, 0.41) and 0.01 (−0.11, 0.14), respectively, meaning 26% of participants in active treatment believed they received active treatment, whereas only 1% in sham believed they received sham treatment, beyond chance, that is, random guess. A greater belief of receiving active treatment was associated with a larger within-arm effect size in both arms, and ideal blinding (namely, “random guess,” and “wishful thinking” that signifies both groups believing they received active treatment) showed smaller effect sizes, with correlation of effect size and summary blinding indexes of 0.35 (p = 0.028) for between-arm comparison. We observed uniformly large sham treatment effects for all modalities, and larger correlation for investigator’s (un)blinding, 0.53 (p = 0.046).
Conclusion:
Participants in active treatments in back pain trials guessed treatment identity more correctly, while those in sham treatments tended to display successful blinding. Excessive correct guesses (that could reflect weaker blinding and/or noticeable effects) by participants and investigators demonstrated larger effect sizes. Blinding and sham treatment effects on back pain need due consideration in individual trials and meta-analyses.
GBD 2017 Disease and Injury Incidence and Prevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet2018; 392(10159): 1789–1858.
2.
MartinBDeyoRMirzaS, et al. Expenditures and health status among adults with back and neck problems. JAMA2008; 299(6): 656–664.
3.
DagenaisSCaroJHaldemanS.A systematic review of low back pain cost of illness studies in the United States and internationally. Spine J2008; 8(1): 8–20.
4.
LuoXPietrobonRSunS, et al. Estimates and patterns of direct health care expenditures among individuals with back pain in the United States. Spine2004; 29(1): 79–86.
ChouRDeyoRFriendlyJ, et al. Nonpharmacologic therapies for low back pain: a systematic review for an American College of Physicians Clinical Practice Guideline. Ann Intern Med2017; 166(7): 493–505.
7.
ChouRQaseemASnowV, et al. Diagnosis and treatment of low back pain: a joint clinical practice guideline from the American College of Physicians and the American Pain Society. Ann Intern Med2007; 147(7): 478–491.
8.
QaseemAWiltTMcLeanR, et al. Noninvasive treatments for acute, subacute, and chronic low back pain: a clinical practice guideline from the American College of Physicians. Ann Intern Med2017; 166(7): 514–530.
9.
SchulzKFAltmanDGMoherD, et al. CONSORT 2010 changes and testing blindness in RCTs. Lancet2010; 375(9721): 1144–1146.
10.
HoptonAKMacphersonH.Assessing blinding in randomised controlled trials of acupuncture: challenges and recommendations. Chin J Integr Med2011; 17(3): 173–176.
11.
SchulzKFGrimesDA.Blinding in randomised trials: hiding who got what. Lancet2002; 359: 696–700.
12.
KolahiJBangHParkJ, et al. CONSORT 2010 and controversies regarding assessment of blindness in RCTs. Dental Hypotheses2010; 1(2): 99–105.
13.
GreenfieldMMhyreJMashourG, et al. Improvement in the quality of randomized controlled trials among general anesthesiology journals 2000 to 2006: a 6-year follow-up. Anesth Analg2009; 108(6): 1916–1921.
14.
BeggCChoMEastwoodS, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA1996; 276: 637–639.
15.
ReesJWadeTLevyD, et al. Changes in beliefs identify unblinding in randomized controlled trials: a method to meet CONSORT guidelines. Contemp Clin Trials2005; 26(1): 25–37.
16.
HowickJWebsterRReesJ, et al. TIDieR-Placebo: a guide and checklist for reporting placebo and sham controls. PLoS Med2020; 17(9): e1003294.
17.
BoutronIGuittetLEstellatC, et al. Reporting methods of blinding in randomized trials assessing nonpharmacological treatments. PLoS Med2007; 4(2): e61.
18.
BoutronIMoherDAltmanD, et al. Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med2008; 148(4): 295–309.
19.
BoutronIEstellatCGuittetL, et al. Methods of blinding in reports of randomized controlled trials assessing pharmacologic treatments: a systematic review. PLoS Med2006; 3(10): e425.
20.
Borg DebonoVZhangSYeC, et al. The quality of reporting of RCTs used within a postoperative pain management meta-analysis, using the CONSORT statement. BMC Anesthesiol2012; 12: 13.
21.
BoutronITubachFGiraudeauB, et al. Blinding was judged more difficult to achieve and maintain in nonpharmacologic than pharmacologic trials. J Clin Epidemiol2004; 57(6): 543–550.
22.
KolahiJBangHParkJ.Towards a proposal for assessment of blinding success in clinical trials: up-to-date review. Community Dent Oral Epidemiol2009; 37(6): 477–484.
23.
MorozAFreedBTiedemannL, et al. Blinding measured: a systematic review of randomized controlled trials of acupuncture. Evid Based Complement Alternat Med2013; 2013: 708251.
24.
FreedBAssallOPanagiotakisG, et al. Assessing blinding in trials of psychiatric disorders: a meta-analysis based on blinding index. Psychiatry Research2014; 219: 241–247.
25.
BaethgeCAssallOBaldessariniRJ.Systematic review of blinding assessment in randomized controlled trials in schizophrenia and affective disorders 2000–2010. Psychother Psychosom2013; 82(3): 152–160.
26.
BraithwaiteFWaltersJLiL, et al. Blinding strategies in dry needling trials: systematic review and meta-analysis. Physical Therapy2019; 99(11): 1461–1480.
27.
ColagiuriBSharpeLScottA.The blind leading the not-so-blind: a meta-analysis of blinding in pharmacological trials for chronic pain. J Pain2019; 20(5): 489–500.
28.
ArandjelovićO.A new framework for interpreting the outcomes of imperfectly blinded controlled clinical trials. PLoS One2012; 7(12): e48984.
29.
ArandjelovićO.Clinical trial adaptation by matching evidence in complementary patient sub-groups of auxiliary blinding questionnaire responses. PLoS One2015; 10(7): e0131524.
30.
LandsmanVFilleryMVernonH, et al. Sample size calculations for blinding assessment. J Biopharm Stat2018; 28(5): 857–869.
31.
ZhangZKotzRWangC, et al. A causal model for joint evaluation of placebo and treatment-specific effects in clinical trials. Biometrics2013; 69(2): 318–327.
32.
BangHNiLDavisCE.Assessment of blinding in clinical trials. Control Clin Trial2004; 25: 143–156.
33.
WalterSAwasthiSJeyaseelanL.Pre-trial evaluation of the potential for unblinding in drug trials: a prototype example. Contemp Clin Trials2005; 26(4): 459–468.
34.
JamesKEBlochDALeeKK, et al. An index for assessing blindness in a multi-centre clinical trial: disulfiram for alcohol cessation—a VA cooperative study. Stat Med1996; 15(13): 1421–1434.
35.
HouwelingAShapiroSCohenJ, et al. Blinding strategies in the conduct and reporting of a randomized placebo-controlled device trial. Clin Trials2014; 11(5): 547–552.
36.
HertzbergVChimowitzMLynnM, et al. Use of dose modification schedules is effective for blinding trials of warfarin: evidence from the WASID study. Clin Trials2008; 5(1): 23–30.
37.
MachadoLKamperSHerbertR, et al. Imperfect placebos are common in low back pain trials: a systematic review of the literature. Eur Spine J2008; 17(7): 889–904.
38.
BraithwaiteFWaltersJLiL, et al. Effectiveness and adequacy of blinding in the moderation of pain outcomes: systematic review and meta-analyses of dry needling trials. PeerJ2018; 6: e5318.
39.
MooreSNeylonCEveM, et al. “Excellence R Us”: university research and the fetishisation of excellence. Palgrave Communications2017; 3: 16105.
40.
LangTStroupD.Who knew? The misleading specificity of “double-blind” and what to do about it. Trials2020; 21: 697.
41.
ParkJWhiteARStevinsonC, et al. Who are we blinding? A systematic review of blinded clinical trials. Perfusion2001; 14: 296–304.
42.
ParkJBangHCanetteI.Blinding in clinical trials, time to do it better. Complement Ther Med2008; 16(3): 121–123.
43.
BangH.Random guess and wishful thinking are the best blinding scenarios. Contemp Clin Trials Commun2016; 3: 117–121.
44.
BangHFlahertySPKolahiJ, et al. Blinding assessment in clinical trials: a review of statistical methods and a proposal of blinding assessment protocol. Clin Res Regulat Affair2010; 27(2): 42–51.
45.
MathieuEHerbertRMcGeechanK, et al. A theoretical analysis showed that blinding cannot eliminate potential for bias associated with beliefs about allocation in randomized clinical trials. J Clin Epidemiol2014; 67(6): 667–671.
46.
AndersonTReidDBBeatonGH.Vitamin C and the common cold: a double-blinded trial. Can Med Assoc J1972; 107: 503–508.
47.
KadelRKipK.A SAS Macro to compute effect size (Cohen’s) and its confidence interval from raw survey data. In: Annual southeast SAS users group conference, Durham, NC, 14–16 October 2012.
48.
SchwarzerGCarpenterJRuckerG.Meta-Analysis with R. Springer, 2015.
49.
LandsmanVLandsmanDLiC, et al. Overdispersion models for correlated multinomial data: applications to blinding assessment. Statistics in Medicine2019; 38(25): 4963–4976.
50.
HrobjartssonAForfangEHaahrMT, et al. Blinded trials taken to the test: an analysis of randomized clinical trials that report tests for the success of blinding. Int J Epidemiol2007; 36(3): 654–663.
51.
FergussonDGlassKCWaringD, et al. Turning a blind eye: the success of blinding reported in a random sample of randomised, placebo controlled trials. BMJ2004; 328: 432.
52.
Ramond-RoquinABoutonCGobin-TempereauA, et al. Interventions focusing on psychosocial risk factors for patients with non-chronic low back pain in primary care—a systematic review. Fam Pract2014; 31(4): 379–388.
PhillipsC.Publication bias in situ. BMC Med Res Methodol2004; 4: 20.
55.
LeeHLambSBaggM, et al. Reproducible and replicable pain research: a critical review. Pain2018; 159(9): 1683–1689.
56.
CoeytauxRParkJ.Acupuncture research in the era of comparative effectiveness research. Ann Intern Med2013; 158(4): 287–288.
57.
KaptchukTFriedlanderEKelleyJS, et al. Placebos without deception: a randomized controlled trial in irritable bowel syndrome. PLoS One2010; 5(12): e15591.
58.
SchneiderTLuethiJMauermannE, et al. Pain response to open label placebo in induced acute pain in healthy adult males. Anesthesiology2020; 132(3): 571–580.
59.
MoustgaardHClaytonGJonesH, et al. Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study. BMJ2020; 368: 16802.
60.
GodleeF.Blinding may be unnecessary, but please divest. BMJ2020; 368: m255.
61.
Armijo-OlivoSFuentesJda CostaBR, et al. Blinding in physical therapy trials and its association with treatment effects: a meta-epidemiological study. Am J Phys Med Rehabil2017; 96(1): 34–44.
62.
MahajanRGuptaK.Adaptive design clinical trials: methodology, challenges and prospect. Indian J Pharmacol2010; 42(4): 201–207.
63.
Al-LameeRThompsonDDehbiH, et al. Percutaneous coronary intervention in stable angina (ORBITA): a double-blind, randomised controlled trial. Lancet2018; 391(10115): 31–40.
64.
BeardDReesJCookJ, et al. Arthroscopic subacromial decompression for subacromial shoulder pain (CSAW): a multicentre, pragmatic, parallel group, placebo-controlled, three-group, randomised surgical trial. Lancet2018; 391(10118): 329–338.
65.
CohenSWallaceMRauckR, et al. Unique aspects of clinical trials of invasive therapies for chronic pain. Pain Rep2019; 4(3): e687.
66.
WanMOrlu-GulMLegayH, et al. Blinding in pharmacological trials: the devil is in the details. Arch Dis Child2013; 98(9): 656–659.
67.
ShapiroS.Risks of estrogen plus progestin therapy: a sensitivity analysis of findings in the Women's Health Initiative randomized controlled trial. Climacteric2003; 6(4): 302–310; discussion 310.
68.
GarbeESuissaS.Issues to debate on the Women's Health Initiative (WHI) study. Hormone replacement therapy and acute coronary outcomes: methodological issues between randomized and observational studies. Human Reprod2004; 19(1): 8–13.
69.
PrasadVCifuAS.The necessity of sham controls. Am J Med2019; 132(2): e29–e30.
70.
FlumD.In reply. Sham surgery in clinical trials. JAMA2007; 297: 1546.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
