This study is a systematic examination of the open access status of research in two flagship language testing and assessment journals: Language Testing and Language Assessment Quarterly. Coding and analysing 898 articles, we investigated (a) the prevalence of open access in four aspects—open manuscripts, open materials, open data, and open code, and (b) the relationship between open access and various characteristics of research, tests, and researchers. Our study revealed a positive trend in the adoption of open access over time, with open manuscripts and materials showing notable increases. Open code and data have remained scarce, though with a recent uptick from a low base. Notably, logistic regression results suggest inequitable participation in open access as authors from the Global South were less likely to have open manuscripts. Recognising the potential role of flagship journals as trend and standard setters, we call on the field to (a) shift towards more equitable open access models, (b) balance intellectual property concerns with validation needs, (c) recognise open code and open data with protected access via dedicated badges, and (d) adopt Research Transparency Statements, a new reporting structure inclusive of methodological and epistemological differences in open research practices.
AlferinkI.MarsdenE. (2023). OASIS: One resource to widen the reach of research in language studies. Innovation in Language Learning and Teaching, 17(5), 946–952. https://doi.org/10.1080/17501229.2023.2204100
2.
Al-HoorieA. H.CinagliaC.HiverP.HuenschA.IsbellD. R.LeungC.SudinaE. (2024). Open science: Considerations and issues for TESOL research. TESOL Quarterly, 58, 537–556. https://doi.org/10.1002/tesq.3304
3.
Al-HoorieA. H.HiverP. (2023). The postprint pledge—Toward a culture of researcher-driven initiatives: A commentary on “(why) are open research practices the future for the study of language learning?”Language Learning, 73(Suppl. 2), 388–391. https://doi.org/10.1111/lang.12577
4.
Al-HoorieA. H.HiverP. (2024). Open science in applied linguistics: An introduction to metascience. In PlonskyL. (Ed.), Open science in applied linguistics. Applied Linguistics Press.
5.
AndringaS.MosM.Van BeuningenC.GonzálezP.HornikxJ.SteinkraussR. (2024). Diamond is a scientist’s best friend: Counteracting systemic inequality in open access publishing. Dutch Journal of Applied Linguistics, 1–13. https://doi.org/10.51751/dujal18802
6.
BolibaughC.VanekN.MarsdenE. (2021). Towards a credibility revolution in bilingualism research: Open data and materials as stepping stones to more reproducible and replicable research. Bilingualism: Language and Cognition, 24(5), 801–806. https://doi.org/10.1017/s1366728921000535
7.
BrainardJ.KaiserJ. (2022). US to require free access to papers on all research it funds. Science, 377(6610), 1026–1027. https://doi.org/10.1126/science.ade6577
8.
BurtonJ. D. (2023). Reflections on the past and future of language testing and assessment: An emerging scholar’s perspective. Language Testing, 40(1), 24–30. https://doi.org/10.1177/02655322221126607
ChapelleC. A. (2020). Validity in language assessment. In WinkeP. M.BrunfautT. (Eds.), The Routledge handbook of second language acquisition and language testing (pp. 11–20). Routledge. https://doi.org/10.4324/9781351034784
11.
ChiarelliA.JohnsonR.PinfieldS.RichensE. (2019). Preprints and scholarly communication: An exploratory qualitative study of adoption, practices, drivers and barriers. F1000Research, 8(971), 1–78. https://doi.org/10.12688/f1000research.19619.2
12.
ChiwareE. R. T.SkellyL. (2023). Overcoming challenges to open research practices—A perspective from the global south: A commentary on “(why) are open research practices the future for the study of language learning?”Language Learning, 73, 392–396. https://doi.org/10.1111/lang.12576
GodfroidA.AndringaS. (2023). Uncovering sampling biases, advancing inclusivity, and rethinking theoretical accounts in second language acquisition: Introduction to the special issue SLA for all?Language Learning, 73, 981–1002. https://doi.org/10.1111/lang.12620
22.
HardingL.KremmelB. (2016). Teacher assessment literacy and professional development. In TsagariD.BanerjeeJ. (Eds.), Handbook of second language assessment (pp. 413–428). De Gruyter. https://doi.org/10.1515/9781614513827-027
HardwickeT. E.ThibaultR. T.KosieJ. E.WallachJ. D.KidwellM. C.IoannidisJ. P. A. (2022). Estimating the prevalence of transparency and reproducibility-related research practices in psychology (2014-2017). Perspectives on Psychological Science, 17(1), 239–251. https://doi.org/10.1177/1745691620979806
HiverP.NagleC. (2024). Complex adaptive interventions: The challenge ahead for instructed second language acquisition research. Annual Review of Applied Linguistics, 1–16. https://doi.org/10.1017/S0267190524000060
28.
HuangC.-K.NeylonC.HoskingR.MontgomeryL.WilsonK. S.OzaygenA.Brookes-KenworthyC. (2020). Evaluating the impact of open access policies on research institutions. eLife, 9, e57067. https://doi.org/10.7554/eLife.57067
29.
HughesA.PorterD. (1984). Editorial. Language Testing, 1(1), i–ii.
30.
IsaacsT.ChalmersH. (2023). Reducing “avoidable research waste” in applied linguistics research: Insights from healthcare research. Language Teaching, 1–18. https://doi.org/10.1017/S0261444823000411
31.
IsaacsT.WinkeP. (2024). Purposeful turns for more equitable and transparent publishing in language testing and assessment. Language Testing, 41(1), 3–8. https://doi.org/10.1177/02655322231203234
32.
IsbellD. R.BrownD.ChenM.DerrickD. J.GhanemR.ArvizuM. N. G.SchnurE.ZhangM.PlonskyL. (2022). Misconduct and questionable research practices: The ethics of quantitative data handling and reporting in applied linguistics. The Modern Language Journal, 106(1), 172–195. https://doi.org/10.1111/modl.12760
33.
IsbellD. R.KimJ. (2023). Developer involvement and COI disclosure in high-stakes English proficiency test validation research: A systematic review. Research Methods in Applied Linguistics, 2(3), 100060. https://doi.org/10.1016/j.rmal.2023.100060
KremmelB.HardingL. (2020). Towards a comprehensive, empirical model of language assessment literacy across stakeholder groups: Developing the language assessment literacy survey. Language Assessment Quarterly, 17(1), 100–120. https://doi.org/10.1080/15434303.2019.1674855
36.
KunnanA. J. (2014). Fairness and justice in language assessment: Principles and public reasoning. In DengX.SeowR. (Eds.), Alternative pedagogies in the English language and communication classroom (pp. 36–39). Centre for English Language Communication, National University of Singapore.
37.
KunnanA. J. (2018). Evaluating language assessments. Routledge.
38.
LaurinavichyuteA.YadavH.VasishthS. (2022). Share the code, not just the data: A case study of the reproducibility of articles published in the Journal of Memory and Language under the open data policy. Journal of Memory and Language, 125, 104332. https://doi.org/10.1016/j.jml.2022.104332
39.
LiuM. (2023). Whose open science are we talking about? From open science in psychology to open science in applied linguistics. Language Teaching, 56(4), 443–450. https://doi.org/10.1017/S0261444823000307
40.
LiuM.Al-HoorieA. H.HiverP., (2024). Open data, code, and materials for “Open access in language testing and assessment.”Open Science Framework. https://doi.org/10.17605/osf.io/vbjd6
41.
LiuM.ChongS. W.MarsdenE.McManusK.Morgan-ShortK.Al-HoorieA. H.PlonskyL.BolibaughC.HiverP.WinkeP.HuenschA.HuiB. (2023). Open scholarship in applied linguistics: What, why, and how. Language Teaching, 56(3), 432–437. https://doi.org/10.1017/S0261444822000349
42.
LiuM.De CatC. (2024). Open science in applied linguistics: A preliminary survey. In PlonskyL. (Ed.), Open science in applied linguistics. Applied Linguistics Press.
MarsdenE.MackeyA., (2014). IRIS: A nesw resource for second language research. Linguistic Approaches to Bilingualism, 4(1), 125–130. https://doi.org/10.1075/lab.4.1.05mar
45.
MarsdenE.Morgan-ShortK. (2023). (Why) are open research practices the future for the study of language learning?Language Learning, 73, 344–387. https://doi.org/10.1111/lang.12568
46.
MarsdenE.Morgan-ShortK.ThompsonS.AbugaberD. (2018). Replication in second language research: Narrative and systematic reviews and recommendations for the field. Language Learning, 68(2), 321–391. https://doi.org/10.1111/lang.12286
47.
MarsdenE.PlonskyL. (2018). Data, open science, and methodological reform in second language acquisition research. In GudmestadA.EdmondsA. (Eds.), Critical reflections on data in second language acquisition (pp. 219–228). John Benjamins.
48.
McManusK. (2022). Are replication studies infrequent because of negative attitudes? Insights from a survey of attitudes and practices in second language research. Studies in Second Language Acquisition, 44(5), 1410–1423. https://doi.org/10.1017/S0272263121000838
49.
NorouzianR. (2021). Interrater reliability in second language meta-analyses: The case of categorical moderators. Studies in Second Language Acquisition, 43(4), 896–915. https://doi.org/10.1017/S0272263121000061
50.
PageM. J.McKenzieJ. E.BossuytP. M.BoutronI.HoffmannT. C.MulrowC. D.ShamseerL.TetzlaffJ. M.AklE. A.BrennanS. E.ChouR.GlanvilleJ.GrimshawJ. M.HróbjartssonA.LaluM. M.LiT.LoderE. W.Mayo-WilsonE.McDonaldS.MoherD. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71
51.
PinfieldS.WakelingS.BawdenD.RobinsonL. (2020). Open access in theory and practice: The theory-practice relationship and openness. Routledge.
52.
PironF. (2018). Postcolonial open access. In HerbU.SchöpfelJ. (Eds.), Open divide: Critical studies on open access (pp. 1–8). Litwin Books.
53.
PiwowarH.PriemJ.LarivièreV.AlperinJ. P.MatthiasL.NorlanderB.FarleyA.WestJ.HausteinS., (2018). The state of OA: A large-scale analysis of the prevalence and impact of Open Access articles. PeerJ, 6. 1–23. https://doi.org/10.7717/peerj.4375
54.
PlonskyL. (2024). Open science in applied linguistics. Applied Linguistics Press.
55.
PorteG.McManusK. (2019). Doing replication research in applied linguistics. Routledge.
56.
R Core Team. (2023). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
57.
ŠimukovičE. (2018). Open access, a new kind of emerging knowledge regime? In HerbU.SchöpfelJ. (Eds.), Open divide: Critical studies on open access (pp. 31–40). Litwin Books.
58.
SuberP. (2012). Open access. MIT Press.
59.
UiT The Arctic University of Norway. (n.d.). TROLLing: Tromsø Repository of Language and Linguistics. https://site.uit.no/trolling
WallachJ. D.BoyackK. W.IoannidisJ. P. A. (2018). Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017. PLOS Biology, 16(11), Article e2006930. https://doi.org/10.1371/journal.pbio.2006930
WilkinsonM. D.DumontierM.Aalbersberg IjJ.AppletonG.AxtonM.BaakA.BlombergN.BoitenJ.-W.da Silva SantosL. B.BourneP. E. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3(1), 1–9.
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
