A recent paper uses Bayes factors to argue a large minority of rigorous, large-scale education RCTs are “uninformative.” The definition of uninformative depends on the authors’ hypothesis choices for calculating Bayes factors. These arguably overadjust for effect size inflation and involve a fixed prior distribution, disregarding the trials’ varied intentions.
BeardE.DienesZ.MuirheadC.WestR. (2016). Using Bayes factors for testing hypotheses about intervention effectiveness in addictions research. Addiction, 111(12), 2230–2247.
2.
DeatonA.CartwrightN. (2018). Understanding and misunderstanding randomized controlled trials. Social Science & Medicine, 210, 2–21.
3.
DienesZ. (2014). Using Bayes to get the most out of non-significant results. Frontiers in Psychology, 5, 781.
4.
DienesZ.MclatchieN. (2018). Four reasons to prefer Bayesian analyses over significance testing. Psychonomic Bulletin & Review, 25(1), 207–218.
5.
Lortie-ForguesH.InglisM. (2019). Rigorous large-scale educational RCTs are often uninformative: Should we be concerned?Educational Researcher, 48(3), 158–166.
6.
SimpsonA. (2017). The misdirection of public policy: Comparing and combining standardised effect sizes. Journal of Education Policy, 32(4), 450–466.
