Calls for improving transparency across the social science are increasing. One strategy that is gaining momentum in the quest to increase transparency is the practice of preregistering a study and analysis plan prior to conducting a study. In this article, we examine the potential of preregistration as a strategy for increasing transparency in education studies. We review existing registries within the social sciences and provide a rationale for why we need a registry specifically geared towards education studies. Finally, we introduce the Registry of Efficacy and Effectiveness Studies (REES), developed by the Society for Research on Educational Effectiveness (SREE). The goal of REES is to increase transparency for studies seeking to draw causal conclusions within the education research and evaluation community.
BanksG. C.RogelbergS. G.WoznyjH. M.LandisR. S.RuppD. E. (2016, September). Editorial: Evidence on questionable research practices: The good, the bad, and the ugly. Journal of Business and Psychology, 31(3), 323–338.
3.
BrodeurA.LéM.SangierM.ZylbergY. (2013). Star wars: The empirics strike back (IZA Discussion Paper No. 7268). Bonn, Germany: Institute for the Study of Labor (IZA).
4.
CaseyK.GlennersterR.MiguelE. (2012). Reshaping institutions: Evidence on aid impacts using a pre-analysis plan. Quarterly Journal of Economics, 127(4), 1755–1812.
5.
De AngelisC. D.DrazenJ. M.FrizelleF. A.HaugC.HoeyJ.… HortonR.International Committee of Medical Journal Editors. (2005). Is this clinical trial fully registered?—A statement from the International Committee of Medical Journal Editors. New England Journal of Medicine, 352(23), 2436–2438. https://doi.org/10.1056/NEJMe058127
6.
DunningT. (2016). Transparency, replication, and cumulative learning: What experiments alone cannot achieve. Annual Review of Political Science, 19(1), S1–S23. Retrieved from https://doi.org/10.1146/annurev-polisci-072516-014127
7.
FanelliD. (2009, May29). How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS one, 4(5), e5738. https://doi.org/10.1371/journal.pone.0005738
8.
FrancoA.MalhotraN.SimonovitsG. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), 1502–1505.
9.
GehlbachH.RobinsonC. D. (2018). Mitigating illusory results through preregistration in education. Journal of Research on Educational Effectiveness, 11(2), 296–315.
10.
GelmanA.LokenE. (2013). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Retrieved from http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf
11.
GerberA.MalhotraN.DowlingC. M.DohertyD. (2010). Publication bias in two political behavior literatures. American Politics Research, 38(4), 591–613. doi:http://dx.doi.org/10.1561/100.00008024
12.
HigginsJ. P.AltmanD. G.SterneJ. A. (2011). Chapter 8: Assessing risk of bias in included studies. In HigginsJ. P.GreenS. (Eds.), Cochrane handbook for systematic reviews of interventions (5.1.0 ed.). The Cochrane Collaboration. Retrieved from www.handbook.cochrane.org
13.
Institute of Education Sciences and National Science Foundation. (2013). Common guidelines for education research and development. Retrieved from http://ies.ed.gov/pdf/CommonGuidelines.pdf
14.
IoannidisJ. P. (2005, August). Why most published research findings are false. PLoS Med, 2(8), 0696–0701.
15.
IoannidisJ. P.MunafòM. R.Fusar-PoliP.NosekB. A.DavidS. P. (2014). Publication and other reporting biases in cognitive sciences: detection, prevalence and prevention. Trends in Cognitive Science, 18(5), 235–241. Retrieved from http://doi.org/10.1016/j.tics.2014.02.010
16.
JohnL. K.LoewensteinG.PrelecD. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532.
17.
KepesS.BennettA. A.McDanielM. A. (2014). Evidence-based management and the trustworthiness of our cumulative scientific knowledge: Implications for teaching, research, and practice. Academy of Management Learning & Education, 13, 446–466. Retrieved from http://dx.doi.org/10.5465/amle.2013.0193
18.
KerrN. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2, 196–217.
19.
KidwellM. C.LazarevićL. B.BaranskiE.HardwickeT. E.PiechowskiS.FalkenbergL.-S.. . . NosekB. A. (2016). Badges to acknowledge open practices: A simple, low-cost, effective method for increasing transparency. PLOS Biology, 14(5), e1002456. https://doi.org/10.1371/journal.pbio.1002456
20.
Leis-NewmanE. (2011). Securing tenure: On the tenure track? Here are four keys to making sure you get the ultimate prize. Monitor on Psychology, 42(5), 7.
21.
MiguelE.CamererC.CaseyK.CohenJ.EsterlingK. M.GerberA.. . . Van der LannM. (2014). Promoting transparency in social science research. Science, 343(30), 30–31.
NorrisS. L.MoherD.ReevesB. C.SheaB.LokeY.GarnerS.. . . WellsG. (2013, March). Issues relating to selective reporting when including non-randomized studies in systematic reviews on the effects of healthcare interventions. Research Synthesis Methods, 4(1), 36–47. doi:10.1002/jrsm.1062
24.
NosekB. A.SpiesJ. R.MotylM. (2012, November7). Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7(6), 615–631. doi:10.1177/1745691612459058
25.
OlkenB. A. (2015). Promises and perils of pre-analysis plans. Journal of Economic Perspectives, 29, 61–80.
26.
PigottT. D.ValentineJ. C.PolaninJ. R.WilliamsR. T.CanadaD. D. (2013). Outcome-reporting bias in education research. Educational Researcher, 42(8), 424–432.
27.
SimonsohnU.NelsonL. D.SimmonsJ. P. (2014). P-curve: A key to the file drawer. Journal of Experimental Psychology: General, 143, 534–547. doi:10.1037/a0033242
