MetaBLIND is the largest meta-epidemiological study on the impact of blinding in randomized trials to date. We examined MetaBLIND data on the impact of blinding patients on patient-reported outcomes. 68 out of 132 included trials tested knowledge recall and had questionable relevance to clinical trials. In 17 out of 18 comparisons, the number of trials in the blinded or nonblinded group was 2 or lower. In several key trials, the blinding status was uncertain. Effect sizes compared in MetaBLIND appear to reflect random differences in study design and setting rather than the impact of blinding trial participants.
Agency for Healthcare Research and Quality (2008) Methods Guide for Effectiveness and Comparative Effectiveness Reviews. AHRQ Methods for Effective Health Care. Rockville, MD: Agency for Healthcare Research and Quality. Available at: http://www.ncbi.nlm.nih.gov/books/NBK47095/ (accessed 20 July 2021).
2.
AltanLBingölUKarakoçY, et al. (2001) Clinical investigation of methotrexate in the treatment of ankylosing spondylitis. Scandinavian Journal of Rheumatology30(5): 255–259.
3.
AnandRNorrieJBradleyJM, et al. (2020) Fool’s gold? Why blinded trials are not always best. BMJ368: l6228.
4.
Armijo-OlivoSFuentesJda CostaBR, et al. (2017) Blinding in physical therapy trials and its association with treatment effects: A meta-epidemiological study. American Journal of Physical Medicine & Rehabilitation96(1): 34–44.
5.
ArmstrongDParéPPericakD, et al. (2001) Symptom relief in gastroesophageal reflux disease: A randomized, controlled comparison of pantoprazole and nizatidine in a mixed patient population with erosive esophagitis or endoscopy-negative reflux disease. The American Journal of Gastroenterology96(10): 2849–2857.
6.
BachsLParésAElenaM, et al. (1989) Comparison of rifampicin with phenobarbitone for treatment of pruritus in biliary cirrhosis. Lancet1(8638): 574–576.
7.
BalkEMBonisPAMoskowitzH, et al. (2002) Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. The Journal of the American Medical Association287(22): 2973–2982.
8.
BateCMGreenJRBAxonATR, et al. (1997) Omeprazole is more effective than cimetidine for the relief of all grades of gastro-oesophageal reflux disease-associated heartburn, irrespective of the presence or absence of endoscopic oesophagitis. Alimentary Pharmacology & Therapeutics11(4): 755–763.
9.
BerkmanNDSantaguidaPLViswanathanM, et al. (2014) The Empirical Evidence of Bias in Trials Measuring Treatment Differences. AHRQ Methods for Effective Health Care. Rockville, MD: Agency for Healthcare Research and Quality. Available at: http://www.ncbi.nlm.nih.gov/books/NBK253181/ (accessed 24 June 2021).
10.
BradtJDileoCPotvinN (2013) Music for stress and anxiety reduction in coronary heart disease patients. Cochrane Database of Systematic Reviews2021: CD006577.
11.
BrinkWRRammelkampCHJrDennyFW, et al. (1951) Effect in penicillin and aureomycin on the natural course of streptococcal tonsillitis and pharyngitis. The American Journal of Medicine10(3): 300–308.
12.
BrumfittWSlaterJD (1957) Treatment of acute sore throat with penicillin; a controlled trial in young soldiers. Lancet272(6958): 8–11.
13.
ChapplePAFranklinLMPaulettJD, et al. (1956) Treatment of acute sore throat in general practice; therapeutic trial, with observations on symptoms and bacteriology. British Medical Journal1(4969): 705–708.
14.
CzarneckaDKlocekMBetkowska-KorpałaB, et al. (2000) Influence of hormone replacement therapy on the quality of life in postmenopausal women with hypertension. Przeglad Lekarski57(7–8): 397–401.
15.
DagnelieCFvan der GraafYDe MelkerRA (1996) Do patients with sore throat benefit from penicillin? A randomized double-blind placebo-controlled clinical trial with penicillin V in general practice. British Journal of General Practice46(411): 589–593.
16.
De MeyereMMervieldeYVerschraegenG, et al. (1992) Effect of penicillin on the clinical course of streptococcal pharyngitis in general practice. European Journal of Clinical Pharmacology43(6): 581–585.
17.
DennyFWWannamakerLWHahnEO (1953) Comparative effects of penicillin, aureomycin and terramycin on streptococcal tonsillitis and pharyngitis. Pediatrics11(1): 7–13.
18.
DevereauxPJChoiPTEl-DikaS, et al. (2004) An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods. Journal of Clinical Epidemiology57(12): 1232–1236.
19.
DruckerAMChanA-W (2020) Blindsided: Challenging the dogma of masking in clinical trials. BMJ368: m229.
20.
El-DaherNTHijaziSSRawashdehNM, et al. (1991) Immediate vs. delayed treatment of group: A beta-hemolytic streptococcal pharyngitis with penicillin V. The Pediatric Infectious Disease Journal10(2): 126–129.
21.
FeliuJGonzález-BarónMBerrocalA, et al. (1992) Usefulness of megestrol acetate in cancer cachexia and anorexia. A placebo-controlled study. American Journal of Clinical Oncology15(5): 436–440.
22.
FerreiraAYoungTMathewsC, et al. (2013) Strategies for partner notification for sexually transmitted infections, including HIV. Cochrane Database of Systematic Reviews10: CD002843.
23.
FlórezGGarcía-PortillaPAlvarezS, et al. (2008) Using topiramate or naltrexone for the treatment of alcohol-dependent patients. Alcoholism Clinical and Experimental Research32(7): 1251–1259.
24.
FlórezGSaizPAGarcía-PortillaP, et al. (2011) Topiramate for the treatment of alcohol dependence: Comparison with naltrexone. European Addiction Research17(1): 29–36.
25.
FreitagFGCollinsSDCarlsonHA, et al. (2002) A randomized trial of divalproex sodium extended-release tablets in migraine prophylaxis. Neurology58(11): 1652–1659.
26.
FujiwaraYHiguchiKNebikiH, et al. (2005) Famotidine vs. omeprazole: A prospective randomized multicentre trial to determine efficacy in non-erosive gastro-oesophageal reflux disease. Alimentary Pharmacology & Therapeutics21(Suppl. 2): 10–18.
27.
GardenALMerryAFHollandRL, et al. (1996) Anaesthesia Information—What Patients Want to know. Anaesthesia and Intensive Care24(5): 594–598.
28.
GarlyM-LBaléCMartinsCL, et al. (2006) Prophylactic antibiotics to prevent pneumonia and other complications after measles: Community based randomised double blind placebo controlled trial in Guinea-Bissau. BMJ333(7581): 1245.
29.
GhentCNCarruthersSG (1988) Treatment of pruritus in primary biliary cirrhosis with rifampin. Results of a double-blind, crossover, randomized trial. Gastroenterology94(2): 488–493.
30.
GodleeF (2020) Blinding may be unnecessary, but please divest. BMJ368: m255.
31.
Gonzalez-LopezLGarcia-GonzalezAVazquez-Del-MercadoM, et al. (2004) Efficacy of methotrexate in ankylosing spondylitis: A randomized, double blind, placebo controlled trial. The Journal of Rheumatology31(8): 1568–1574.
32.
HempelSSuttorpMJMilesJN, et al. (2011) Empirical Evidence of Associations Between Trial Quality and Effect Size. AHRQ Methods for Effective Health Care. Rockville, MD: Agency for Healthcare Research and Quality. Available at: http://www.ncbi.nlm.nih.gov/books/NBK56925/ (accessed 24 June 2021).
33.
HigginsJPTThomasJChandlerJ, et al. (2019) Cochrane Handbook for Systematic Reviews of Interventions, 2nd edn.Chichester: John Wiley & Sons.
34.
HróbjartssonAEmanuelssonFSkou ThomsenAS, et al. (2014a) Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies. International Journal of Epidemiology43(4): 1272–1283.
35.
HróbjartssonAThomsenASEmanuelssonF, et al. (2014b) Observer bias in randomized clinical trials with time-to-event outcomes: Systematic review of trials with both blinded and non-blinded outcome assessors. International Journal of Epidemiology43(3): 937–948.
36.
HróbjartssonABoutronI (2011) Blinding in randomized clinical trials: Imposed impartiality. Clinical Pharmacology and Therapeutics90(5): 732–736.
37.
HróbjartssonAThomsenASEmanuelssonF, et al. (2012) Observer bias in randomised clinical trials with binary outcomes: Systematic review of trials with both blinded and non-blinded outcome assessors. BMJ344: e1119.
38.
HróbjartssonAThomsenASEmanuelssonF, et al. (2013) Observer bias in randomized clinical trials with measurement scale outcomes: A systematic review of trials with both blinded and nonblinded assessors. CMAJ: Canadian Medical Association journal = journal de l’Association medicale canadienne185(4): E201–E211.
39.
HughesJRSteadLFHartmann-BoyceJ, et al. (2014) Antidepressants for smoking cessation. Cochrane Database of Systematic Reviews1: CD000031.
40.
JuulSGluudCSimonsenS, et al. (2021) Blinding in randomised clinical trials of psychological interventions: A retrospective study of published trial reports. BMJ Evidence-Based Medicine26(3): 109.
41.
KabraSKLodhaRHiltonDJ (2008) Antibiotics for preventing complications in children with measles. Cochrane Database of Systematic Reviews3: CD001477.
42.
KanieckiRG (1997) A comparison of divalproex with propranolol and placebo for the prophylaxis of migraine without aura. Archives of Neurology54(9): 1141–1145.
43.
KarelitzSChangCCMatthewsZE (1954) The prophylaxis and treatment of bacterial complications of measles with benzethacil and aqueous procaine penicillin G. The Journal of Pediatrics44(4): 357–363.
44.
KinnersleyPPhillipsKSavageK, et al. (2013) Interventions to promote informed consent for patients undergoing surgical and other invasive healthcare procedures. Cochrane Database of Systematic Reviews7: CD009445.
45.
KjaergardLLVillumsenJGluudC (2001) Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Annals of Internal Medicine135(11): 982–989.
46.
KlapperJ (1997) Divalproex sodium in migraine prophylaxis: A dose-controlled study. Cephalalgia: An International Journal of Headache17(2): 103–108.
47.
KleijnenJ (2021) Re: Impact of blinding on estimated treatment effects in randomised clinical trials: Meta-epidemiological study. Available at: https://www.bmj.com/content/368/bmj.l6802/rr (accessed 29 June 2021).
48.
KuvakiBKücükgüclüSSentürkZ, et al. (2000) Rectal premedication in children: Clonidine compared with midazolam. European Journal of Anaesthesiology17: 143.
49.
LandsmanJBGristNRBlackR, et al. (1951) “Sore throat” in general practice. British Medical Journal1(4702): 326–329.
50.
LindeMMullenersWMChronicleEP, et al. (2013) Valproate (valproic acid or sodium valproate or a combination of the two) for the prophylaxis of episodic migraine in adults. Cochrane Database of Systematic Reviews6: CD010611.
51.
LittlePWilliamsonIWarnerG, et al. (1997) Open randomised trial of prescribing strategies in managing sore throat. BMJ314(7082): 722–727.
52.
LoprinziCLEllisonNMSchaidDJ, et al. (1990) Controlled trial of megestrol acetate for the treatment of cancer anorexia and cachexia. Journal of the National Cancer Institute82(13): 1127–1132.
53.
MacdonaldTCWatsonIH (1951) Sulphonamides and acute tonsillitis; a controlled experiment in a Royal Air Force community. British Medical Journal1(4702): 323–326.
54.
MankowskiJLKingstonJMoranT, et al. (2009) Paracervical compared with intracervical lidocaine for suction curettage: A randomized controlled trial. Obstetrics and Gynecology113(5): 1052–1057.
55.
MartinottiGDi NicolaMRomanelliR, et al. (2007) High and low dosage oxcarbazepine versus naltrexone for the prevention of relapse in alcohol-dependent patients. Human Psychopharmacology22(3): 149–156.
56.
MartinottiGDi NicolaMTedeschiD, et al. (2010) Pregabalin versus naltrexone in alcohol dependence: A randomised, double-blind, comparison trial. Journal of Psychopharmacology24(9): 1367–1374.
57.
MathewNTSaperJRSilbersteinSD, et al. (1995) Migraine prophylaxis with divalproex. Archives of Neurology52(3): 281–286.
58.
MiddletonDBD’AmicoFMerensteinJH (1988) Standardized symptomatic treatment versus penicillin as initial therapy for streptococcal pharyngitis. The Journal of Pediatrics113(6): 1089–1094.
59.
MoherDPhamBJonesA, et al. (1998) Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?Lancet352(9128): 609–613.
60.
MoustgaardHClaytonGLJonesHE, et al. (2020a) Impact of blinding on estimated treatment effects in randomised clinical trials: Meta-epidemiological study. BMJ368: l6802.
61.
MoustgaardHJonesHESavovićJ, et al. (2020b) Ten questions to consider when interpreting results of a meta-epidemiological study-the MetaBLIND study as a case. Research Synthesis Methods11(2): 260–274.
62.
NastriCOLaraLAFerrianiRA, et al. (2013) Hormone therapy for sexual function in perimenopausal and postmenopausal women. Cochrane Database of Systematic Reviews6: CD009672.
63.
NüeschEReichenbachSTrelleS, et al. (2009) The importance of allocation concealment and patient blinding in osteoarthritis trials: A meta-epidemiologic study. Arthritis & Rheumatism61(12): 1633–1641.
64.
OsmanağaoğluMAAtasaralTBaltacıD, et al. (2006) Effect of different preparations of hormone therapy on sexual dysfunction in naturally postmenopausal women. Climacteric9(6): 464–472.
65.
ØstergaardLAndersenBMøllerJK, et al. (2003) Managing partners of people diagnosed with Chlamydia trachomatis: A comparison of two partner testing methods. Sexually Transmitted Infections79(5): 358–361.
66.
PageMJHigginsJPTClaytonG, et al. (2016) Empirical evidence of study design biases in randomized trials: Systematic review of meta-epidemiological studies. PLoS One11(7): e0159267.
67.
PaniPPTroguEPaciniM, et al. (2014) Anticonvulsants for alcohol dependence. Cochrane Database of Systematic Reviews2: CD008544.
68.
PetersenKPhillipsRSSoukupJ, et al. (1997) The effect of erythromycin on resolution of symptoms among adults with pharyngitis not caused by group A streptococcus. Journal of General Internal Medicine12(2): 95–101.
69.
PetersonPKSirrSAGrammithFC, et al. (1994) Effects of mild exercise on cytokines and cerebral blood flow in chronic fatigue syndrome patients. Clinical and Diagnostic Laboratory Immunology1(2): 222–226.
70.
PlanerDLevIElitzurY, et al. (2011) Bupropion for smoking cessation in patients with acute coronary syndrome. Archive of Internal Medicine171(12): 1055–1060.
71.
PodestaALopezPTergR, et al. (1991) Treatment of pruritus of primary biliary cirrhosis with rifampin. Digestive Diseases and Sciences36(2): 216–220.
72.
Rosti-OtajärviEMHämäläinenPI (2014) Neuropsychological rehabilitation for multiple sclerosis. Cochrane Database of Systematic Reviews2: CD009131.
73.
Ruiz GarciaVLópez-BrizECarbonell SanchisR, et al. (2013) Megestrol acetate for treatment of anorexia-cachexia syndrome. Cochrane Database of Systematic Reviews3: CD004310.
74.
SaltajiHArmijo-OlivoSCummingsGG, et al. (2018) Influence of blinding on treatment effect size estimate in randomized controlled trials of oral health interventions. BMC Medical Research Methodology18(1): 42.
75.
SavovićJJonesHAltmanD, et al. (2012) Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: Combined analysis of meta-epidemiological studies. Health Technology Assessment16(35): 1–82.
76.
SchmidtAPValinettiEABandeiraD, et al. (2007) Effects of preanesthetic administration of midazolam, clonidine, or dexmedetomidine on postoperative pain and anxiety in children. Paediatric Anaesthesia17(7): 667–674.
77.
SchmollE (1992) Risks and benefits of various therapies for cancer anorexia. Oncology49 (Suppl. 2): 43–45.
78.
SchmollEWilkeHTholeR, et al. (1991) Megestrol acetate in cancer cachexia. Seminars in Oncology18(1Suppl. 2): 32–34.
SchulzKFChalmersIHayesRJ, et al. (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. The Journal of the American Medical Association273(5): 408–412.
81.
SchulzKFGrimesDA (2002) Blinding in randomised trials: Hiding who got what. Lancet359(9307): 696–700.
82.
ShortenAShortenBKeoghJ, et al. (2005) Making choices for childbirth: A randomized controlled trial of a decision-aid for informed birth after cesarean. Birth32(4): 252–261.
83.
SigtermanKEvan PinxterenBBonisPA, et al. (2013) Short-term treatment with proton pump inhibitors, H2-receptor antagonists and prokinetics for gastro-oesophageal reflux disease-like symptoms and endoscopy negative reflux disease. Cochrane Database of Systematic Reviews5: CD002095.
84.
SolariAMottaAMendozziL, et al. (2004) Computer-aided retraining of memory and attention in people with multiple sclerosis: A randomized, double-blind controlled trial. Journal of Neurological Sciences222(1–2): 99–104.
85.
SpinksAGlasziouPPDel MarCB (2013) Antibiotics for sore throat. Cochrane Database of Systematic Reviews11: CD000023.
86.
StaceyDLégaréFColNF, et al. (2014) Decision aids for people facing health treatment or screening decisions. Cochrane Database of Systematic Reviews1: CD001431.
87.
SteckelbergAHulfenhausCHaastertB, et al. (2011) Effect of evidence based risk information on “informed choice” in colorectal cancer screening: Randomised controlled trial. BMJ342: d3193.
88.
SterneJAJüniPSchulzKF, et al. (2002) Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Statistics in Medicine21(11): 1513–1524.
89.
TangsiriwatthanaTSangkomkamhangUSLumbiganonP, et al. (2013) Paracervical local anaesthesia for cervical dilatation and uterine intervention. Cochrane Database of Systematic Reviews9: CD005056.
90.
VenablesTLNewlandRDPatelAC, et al. (1997) Omeprazole 10 milligrams once daily, omeprazole 20 milligrams once daily, or ranitidine 150 milligrams twice daily, evaluated as initial therapy for the relief of symptoms of gastro-oesophageal reflux disease in general practice. Scandinavian Journal of Gastroenterology32(10): 965–973.
91.
Von RoennJHArmstrongDKotlerDP, et al. (1994) Megestrol acetate in patients with AIDS-related cachexia. Annals of Internal Medicine121(6): 393–399.
92.
WeltonAJVickersMRKimJ, et al. (2008) Health related quality of life after combined hormone replacement therapy: Randomised controlled trial. BMJ337: a1190–a1190.
93.
WeltonNJAdesAECarlinJB, et al. (2009) Models for potentially biased evidence in meta-analysis using empirically based priors. Journal of the Royal Statistical Society Series A (General)172(1): 119–136.
94.
WhitfieldMJHughesAO (1981) Penicillin in sore throat. The Practitioner225(1352): 234–239.
95.
WittchenH-UHochEKlotscheJ, et al. (2011) Smoking cessation in primary care – A randomized controlled trial of bupropione, nicotine replacements, CBT and a minimal intervention. International Journal of Methods in Psychiatric Research20(1): 28–39.
96.
XanderCMeerpohlJJGalandiD, et al. (2013) Pharmacological interventions for pruritus in adult palliative care patients. The Cochrane Database of Systematic Reviews (6): CD008320. DOI: 10.1002/14651858.CD008320.pub2.
97.
YaziciFGArslanMBirbicerH, et al. (2003) Comparison of the intrauterine topical anaesthesia and paracervical block on pain in patients undergoing endometrial biopsy. The Pain Clinic15(3): 339–343.
98.
YonisATaylorRWarrenF, et al. (2021) Re: Impact of blinding on estimated treatment effects in randomised clinical trials: Meta-epidemiological study. Available at: https://www.bmj.com/content/368/bmj.l6802/rr-5 (accessed 29 June 2021).
99.
ZwartSRoversMMde MelkerRA, et al. (2003) Penicillin for acute sore throat in children: Randomised, double blind trial. BMJ327(7427): 1324.
100.
ZwartSSachsAPRuijsGJ, et al. (2000) Penicillin for acute sore throat: Randomised double blind trial of seven days versus three days treatment or placebo in adults. BMJ320(7228): 150–154.
