Open practices, such as preregistration, registered reports, open materials, open data, open analytic code, replication, open peer review, open access, and conflict-of-interest and funding statements, support the transparency, accessibility, and reproducibility of research and other scholarship. The purpose of this review was to examine the prevalence of these open practices in the special education literature. We reviewed a randomly selected sample of 250 articles published in special education journals in 2020. Results indicated that conflict-of-interest and funding statements were present in most articles; a small but meaningful proportion of articles provided open materials and were open access; and preregistration, registered reports, open data, open analytic code, open peer review, and replication were rarely or never observed. Recommendations for researching and supporting the use of open practices in special education scholarship are provided.
Get full access to this article
View all access options for this article.
References
1.
AdewumiM. T.VoN.TritzD.BeamanJ.VassarM. (2021). An evaluation of the practice of transparency and reproducibility in addiction medicine literature. Addictive Behaviors, 112, 106560. https://doi.org/10.1016/j.addbeh.2020.106560
American Psychological Association. (2020). Publication manual of the American Psychological Association, 7th edition. Author.
4.
AndersonJ. M.WrightB.RauhS.TritzD.HornJ.ParkerI.BergeronD.CookS.VassarM. (2021). Evaluation of indicators supporting reproducibility and transparency within cardiology literature. Heart (British Cardiac Society), 107(2), 120–126. https://doi.org/10.1136/heartjnl-2020-316519
5.
AschaM.KatabiL.StevensE.GatherwrightJ.VassarM. (2022). Reproducible research practices in the plastic surgery literature. Plastic and Reconstructive Surgery, 149(4), 810e–823e. https://doi.org/10.1097/PRS.0000000000008956
BurgelmanJ. C.PascuC.SzkutaK.Von SchombergR.KaralopoulosA.RepanasK.SchouppeM. (2019). Open science, open data, and open scholarship: European policies to make science fit for the twenty-first century. Frontiers in Big Data, 2, 43. https://doi.org/10.3389/fdata.2019.00043
9.
ChhinC. S.TaylorK. A.WeiW. S. (2018). Supporting a culture of replication: An examination of education and special education research grants funded by the Institute of Education Sciences. Educational Researcher, 47(9), 594–605. https://doi.org/10.3102/0013189X18788047
CookB. G.FlemingJ. I.HartS. A.LaneK. L.TherrienW. J.van DijkW.WilsonS. E. (2022). A how-to guide for open-science practices in special education research. Remedial and Special Education, 43(4), 270–280. https://doi.org/10.1177/07419325211019100
12.
CookB. G.LloydJ. W.MellorD.NosekB. A.TherrienW. J. (2018). Promoting open science to increase the trustworthiness of evidence in special education. Exceptional Children, 85(1), 104–118. https://doi.org/10.1177/0014402918793138
13.
CookB. G.MagginD. M.RobertsonR. E. (2021). Registered reports in special education: Introduction to the special series. Remedial and Special Education, 42(3), 131–139. https://doi.org/10.1177/0741932521996459
14.
CookB. G.TherrienW. J.CoyneM. D. (Eds.). (2016). Replication in special education research [Special issue]. Remedial and Special Education, 37(4).
CoyneM. D.CookB. G.TherrienW. J. (2016). Recommendations for replication research in special education: A framework of systematic, conceptual replications. Remedial and Special Education, 37(4), 244–253. https://doi.org/10.1177/0741932516648463
17.
DechartresA.RavaudP.AtalI.RiverosC.BoutronI. (2016). Association between trial registration and treatment effect estimates: A meta-epidemiological study. BMC Medicine, 14. https://doi.org/10.1186/s12916-016-0639-x
18.
Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021). Investigating the replicability of preclinical cancer biology. Elife, 10, e71601. https://doi.org/10.7554/eLife.71601
19.
FladieI.EvansS.CheckettsJ.TritzD.NorrisB.VassarM. (2021). An evaluation of reproducibility and transparency indicators in orthopedic literature. Journal of Evidence-Based Medicine, 14, 281–284. https://doi.org/10.1111.jebm.12457
20.
FlemingJ. I.CookB. G. (2022). Open access in special education: A review of journal and publisher policies. Remedial and Special Education, 43(1), 3–14. https://doi.org.10.1177/0741932521996461
21.
FlemingJ. I.McLucasA. S.CookB. G. (2022). Review of four preregistration registries for special education researchers [Unpublished manuscript]. https://doi.org/10.35542/osf.io/7ujgs
22.
FlemingJ. I.WilsonS. E.EspinasD.van DijkW.CookB. G. (2022). Special education researchers’ knowledge, attitudes, and reported use of open science practices [Unpublished manuscript]. https://edarxiv.org/tyc36/
23.
FuD. Y.HugheyJ. J. (2019). Releasing a preprint is associated with more attention and citations for the peer-reviewed article. Elife, 8, e52646. https://doi.org/10.7554/eLife.52646
24.
GabelicaM.BojčićR.PuljakL. (2022). Many researchers were not compliant with their published data sharing statement: A mixed-methods study. Journal of Clinical Epidemiology, 150, 33–41. https://doi.org/10.1016/j.jclinepi.2022.05.019
25.
GehlbachH.RobinsonC. D. (2018). Mitigating illusory results through preregistration in education. Journal of Research on Educational Effectiveness, 11(2), 296–315. https://doi.org/10.1080/19345747.2017.1387950
26.
Gehlbach, H., & Robinson, C. D. (2021). From old school to open science: The implications of new research norms for educational psychology and beyond. Educational Psychologist, 56(2), 79–89. https://doi.org/10.1080/00461520.2021.1898961
27.
Gershenson, S., Polikoff, M. S., & Wang, R. (2020). When paywall goes AWOL: The demand for open–access education research. Educational Researcher, 49(4), 254–261.https://doi.org/10.3102/0013189X2090983
28.
HardwickeT. E.ThibaultR. T.KosieJ. E.WallachJ. D.KidwellM. C.IoannidisJ. P. (2022). Estimating the prevalence of transparency and reproducibility-related research practices in psychology (2014–2017). Perspectives on Psychological Science, 17(1), 239–251. https://doi.org/10.1177/1745691620979806
29.
HardwickeT. E.WallachJ. D.KidwellM. C.BendixenT.CrüwellS.IoannidisJ. P. (2020). An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014–2017). Royal Society Open Science, 7(2), 190806. https://doi.org/10.1098/rsos.190806
30.
HavenT. L.ErringtonT. M.GleditschK. S.van GrootelL.JacobsA. M.KernF. G.PineiroR.RosenblattF.MokkinkL. B. (2020). Preregistering qualitative research: A Delphi study. International Journal of Qualitative Methods, 19. https://doi.org/10.1177/1609406920976417
31.
HoutkoopB. L.ChambersC.MacleodM.BishopD. V.NicholsT. E.WagenmakersE. J. (2018). Data sharing in psychology: A survey on barriers and preconditions. Advances in Methods and Practices in Psychological Science, 1(1), 70–85. https://doi.org/10.1177/2515245917751886
32.
Institute of Education Sciences. (2021). Standards for excellence in education research. https://ies.ed.gov/seer/
33.
John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532. https://doi.org/10.1177/09567976114309
KaplanR. M.IrvinV. L. (2015). Likelihood of null effects of large NHLBI clinical trials has increased over time. PLoS ONE, 10(8), e0132382. https://doi.org/10.1371/journal.pone.0132382
36.
KleinO.HardwickeT. E.AustF.BreuerJ.DanielssonH.MohrA. H.NilsonneG.VanpaemelN.FrankM. C. (2018). A practical guide for transparency in psychological science. Collabra: Psychology, 4(1), 20. https://doi.org/10.1525/collabra.15
37.
LandisJ. R.KochG. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. https://doi.org/10.2307/2529310
38.
LemonsC. J.KingS. A.DavidsonK. A.BerryessaT. L.GajjarS. A.SacksL. H. (2016). An inadvertent concurrent replication: Same roadmap, different journey. Remedial and Special Education, 37, 213–222. https://doi.org/10.1177/0741932516631116
MakelM. C.HodgesJ.CookB. G.PluckerJ. A. (2021). Both questionable and open research practices are prevalent in education research. Educational Researcher, 50(8), 493–504. https://doi.org/10.3102/0013189X211001356
41.
MakelM. C.PluckerJ. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304–316. https://doi.org/10.3102/0013189X14545513
42.
MakelM. C.PluckerJ. A.FreemanJ.LombardiA.SimonsenB.CoyneM. (2016). Replication of special education research: Necessary but far too rare. Remedial and Special Education, 37(4), 205–212. https://doi/org/10.1177/0741932516646083.
MiguelE.CamererC.CaseyK.CohenJ.EsterlingK. M.GerberA.GlennersterR.GreenD. P.HumphreysM.ImbensG.LaitinD.MadonT.NelsonL.NosekB. A.PetersenM.SedlmayrR.SimmonsJ. P.SimonsohnU.Van der LaanM. (2014). Social science. Promoting transparency in social science research. Science, 343(6166), 30–31. https://doi.org/10.1126/science.1245317
45.
NorrisE.O’ConnorD. B. (2019). Science as behaviour: Using a behaviour change approach to increase uptake of open science. Psychology & Health, 34(12), 1397–1406. https://doi.org/10.1080/08870446.2019.1679373
46.
NosekB. A.BeckE. D.CampbellL.FlakeJ. K.HardwickeT. E.MellorD. T.van’t VeerA. E.VazireS. (2019). Preregistration is hard, and worthwhile. Trends in Cognitive Sciences, 23(10), 815–818. https://doi.org/10.1016/j.tics.2019.07.009
47.
NosekB. A.EbersoleC. R.DeHavenA. C.MellorD. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114
48.
OdomS. L.BrantlingerE.GerstenR.HornerR. H.ThompsonB.HarrisK. R. (2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71(2), 137–148. https://doi.org/10.1177/001440290507100201
49.
Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), 943–951. https://doi.org/10.1126/science.aac4716
50.
PapageorgiouS. N.XavierG. M.CobourneM. T.EliadesT. (2018). Registered trials report less beneficial treatment effects than unregistered ones: A meta-epidemiological study in orthodontics. Journal of Clinical Epidemiology, 100, 44–52. https://doi.org/10.1016/j.jclinepi.2018.04.017
PiwowarH.PriemJ.LarivièreV.AlperinJ. P.MatthiasL.NorlanderB.FarleyA.WestJ.HausteinS. (2018). The state of OA: A large-scale analysis of the prevalence and impact of open access articles. Peer Journal of Life and Environmental Sciences, 6, e4375. https://doi.org/10.7717/peerj.4375
53.
PluckerJ. A.MakelM. C. (2021). Replication is important for educational psychology: Recent developments and key issues. Educational Psychologist, 56(2), 90–100. https://doi.org/10.1080/00461520.2021.1895796
54.
RauhS.BowersA.RorahD.TritzD.PateH.FryeL.VassarM. (2022). Evaluating the reproducibility of research in obstetrics and gynecology. European Journal of Obstetrics & Gynecology and Reproductive Biology, 269, 24–29. https://doi.org/10.1016/j.ejogrb.2021.12.021
55.
ReichJ.GehlbachH.AlbersC. J. (2020). “Like upgrading from a typewriter to a computer”: Registered reports in education research. AERA Open, 6(2). https://doi.org/10.1177/2332858420917640
56.
Robinson-GarciaN.CostasR.van LeeuwenT. N. (2020). Open access uptake by universities worldwide. PeerJ, 8, e9410. https://doi.org/10.7717/peerj.9410
Scheel, A. M., Schijen, M. R., & Lakens, D. (2021). An excess of positive results: Comparing the standard psychology literature with registered reports. Advances in Methods and Practices in Psychological Science, 4(2).https://doi.org/10.1177/2515245921100746
59.
Schäfer, T., & Schwarz, M. A. (2019). The meaningfulness of effect sizes in psychological research: Differences between sub-disciplines and the impact of potential biases. Frontiers in Psychology, 10, 813. https://doi.org/10.3389/fpsyg.2019.00813
60.
SilberzahnR.UhlmannE. L.MartinD. P.AnselmiP.AustF.AwtreyE.BahníkŠ.BaiF.BannardC.BonnierE.CarlssonR.CheungF.ChristensenG.ClayR.CraigM. A.Dalla RosaA.DamL.EvansM. H.Flores CervantesI., . . . NosekB. A. (2018). Many analysts, one data set: Making transparent how variations in analytic choices affect results. Advances in Methods and Practices in Psychological Science, 1(3), 337–356. https://doi.org/10.1177/2515245917747646
61.
TedersooL.KüngasR.OrasE.KösterK.EenmaaH.LeijenÄPedasteM.RajuM.AstapovaA.LuknerH.KogermannK.SeppT. (2021). Data sharing practices and data availability upon request differ across scientific disciplines. Scientific Data, 8(1), 1–11. https://doi.org/10.1038/s41597-021-00981-0
62.
TraversJ. C.CookB. G.TherrienW. J.CoyneM. D. (2016). Replication research and special education. Remedial and Special Education, 37(4), 195–204. https://doi.org/10.1177/0741932516648462
63.
van DijkW.SchatschneiderC.HartS. A. (2021). Open science in education science. Journal of Learning Disabilities, 54(2), 139–152. https://doi.org/10.1177/0022219420945267
64.
VazireS. (2018). Implications of the credibility revolution for productivity, creativity, and progress. Perspectives on Psychological Science, 13(4), 411–417. https://doi.org/10.1177/1745691617751884
65.
WaltersC.HarterZ. J.WayantC.VoN.WarrenM.ChronisterJ.TritzD.VassarM. (2019). Do oncology researchers adhere to reproducible and transparent principles? A cross-sectional survey of published oncology literature. BMJ Open, 9(12), e033962. https://doi.org/10.1136/bmjopen-2019-033962
66.
WilkinsonM. D.DumontierM.AalbersbergI. J.AppletonG.AxtonM.BaakA.BlombergN.BoitenJ.Bonino da Silva SantosL.BourneP. E.BouwmanJ.BrookesA. J.ClarkT.CrosasM.DilloI.DumonO.EdmundsS.EveloC. T.FinkersR., . . . MonsB. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3(1), 1–9. https://doi.org/10.1038/sdata.2016.18
