Open Science and the next generation of language testing research

Al-Hoorie A. H. Cinaglia C. Hiver P. Huensch A. Isbell D. R. Leung C. Sudina E. (2024). Open science: Considerations and issues for TESOL research. TESOL Quarterly, 58(1), 537–556. https://doi.org/10.1002/tesq.3304

Al-Hoorie A. H. Vitta J. P. (2019). The seven sins of L2 research: A review of 30 journals’ statistical quality and their CiteScore, SJR, SNIP, JCR Impact Factors. Language Teaching Research, 23(6), 727–744. https://doi.org/10.1177/1362168818767191

Bachman L. F. Palmer A. S. (1982). The construct validation of some components of communicative proficiency. TESOL Quarterly, 16(4), 449–465. https://doi.org/10.2307/3586464

Burton J. D. (2024). Evaluating the impact of nonverbal behavior on language ability ratings. Language Testing, 41(4), 729–758. https://doi.org/10.31219/osf.io/tc3qg

Byrnes H. (2013). Notes from the editor. Modern Language Journal, 97(4), 825–827.

Chapelle C. A. (2021). Argument-based validation in testing and assessment. Sage.

Chapelle C. A. Ockey G. (2024). Open Science in language assessment research contexts: A reply to Winke. Language Testing, 41(4), 882–885. https://doi.org/10.1177/02655322241239377

Dimova S. Yan X. Ginther A. (2022). Local tests, local contexts. Language Testing, 39(3), 341–354.

Douglas D. (2001). Performance consistency in second language acquisition and language testing research: A conceptual gap. Second Language Research, 17(4), 442–456.

Dudley A. Marsden E. J. Bovolenta G. (2024). A context-aligned two thousand test: Towards estimating high-frequency French vocabulary knowledge for beginner-to-low intermediate proficiency adolescent learners in England. Language Testing, 41(4), 759–791. https://doi.org/10.31219/osf.io/x6bzs

Gebril A. Bali M. (2024). A global south perspective on open science in language assessment: A response to Paula Winke. Language Testing, 41(4), 886–891. https://journals.sagepub.com/doi/10.1177/02655322241260121

Ha H. T. Nguyen D. T. B. Stoeckel T. (2024). What is the best predictor of word difficulty? A case of data mining using random forest. Language Testing, 41(4), 828–844. https://doi.org/10.1177/02655322241263628

Harding L. Winke P. (2021). Editorial 2021. Language Testing, 38(1), 3–5. https://doi.org/10.1177/0265532220965757

Harding L. Winke P. (2022). Innovation and expansion in “Language Testing” for changing times. Language Testing, 39(1), 3–6. https://doi.org/10.1177/02655322211053212

Hui B. Huntley E. (2020). Promoting open science practices: What can (should) graduate programs do? https://doi.org/10.31219/osf.io/vmtza

Isaacs T. Winke P. M. (2024). Purposeful turns for more equitable and transparent publishing in language testing and assessment. Language Testing, 41(1), 3–8. https://doi.org/10.1177/02655322231203234

Isbell D. R. Brown D. Chen M. Derrick D. J. Ghanem R. Arvizu M. N. G. Schnur E. Zhang M. Plonsky L. (2022). Misconduct and questionable research practices: The ethics of quantitative data handling and reporting in applied linguistics. The Modern Language Journal, 106(1), 172–195. https://doi.org/10.1111/modl.12760

Isbell D. R. Kim J. (2023). Developer involvement and COI disclosure in high-stakes English proficiency test validation research: A systematic review. Research Methods in Applied Linguistics, 2(3), 100060. https://doi.org/10.1016/j.rmal.2023.100060

Isbell D. R. Son Y.-A. (2022). Measurement properties of a standardized Elicited Imitation Test: An integrative data analysis. Studies in Second Language Acquisition, 44(3), 859–885. https://doi.org/10.1017/S0272263121000383

Kerr N. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196–217. https://doi.org/10.1207/s15327957pspr0203_4

Koizumi R. Maie R. Yanagisawa A. In’nami Y. (2024). Considerations to promote and accelerate Open Science: A response to Winke. Language Testing, 41(4), 892–897. https://doi.org/10.1177/02655322241239379

Kunnan A. J. (2018). Evaluating language assessments. Routledge.

Kyle K. Eguchi M. (2024). Evaluating NLP models with written and spoken L2 samples. Research Methods in Applied Linguistics, 3(2), 100120.

LaFlair G. T. (2024). An industry perspective on open science: A response to Winke. Language Testing, 41(4), 865–871. https://doi.org/10.1177/02655322241261716

Larsson T. Plonsky L. Sterling S. Kytö M. Yaw K. Wood M. (2023). On the frequency, prevalence, and perceived severity of questionable research practices. Research Methods in Applied Linguistics, 2(3), 100064. https://doi.org/10.1016/j.rmal.2023.100064

Liu M. Al-Hoorie A. H. Hiver P. V. (2024). Open access in language testing and assessment: The case of two flagship journals. Language Testing, 41(4), 703–728. https://doi.org/10.31219/osf.io/aedbu

Marsden E. Morgan-Short K. (2023). (Why) Are open research practices the future for the study of language learning? Language Learning, 73, 344–387. https://doi.org/10.1111/lang.12568

Marsden E. Morgan-Short K. Thompson S. Abugaber D. (2018). Replication in second language research: Narrative and systematic reviews and recommendations for the field. Language Learning, 68(2), 321–391. https://doi.org/10.1111/lang.12286

McNamara T. F. (1990). Item Response Theory and the validation of an ESP test for health professionals. Language Testing, 7(1), 52–76. https://doi.org/10.1177/026553229000700105

Nishizawa H. (2024). Authenticity of academic lecture passages in high-stakes tests: A temporal fluency perspective. Language Testing, 41(4), 792–816. https://journals.sagepub.com/doi/10.1177/02655322241262453

Pan J. Marsden E. (2024). Developing internet-based Tests of Aptitude for Language Learning (TALL): An open research endeavour. Language Testing, 41(4), 817–827. https://doi.org/10.1177/02655322241241849

Sage. (2023). Declared conflicts of interest for language testing editorial staff, 2023. Language Testing. https://journals.sagepub.com/pb-assets/cmscontent/hej/LT%20Editorial%20Staff%20COIs_LTJ%202023-1690793722.pdf

Taylor L. Banerjee J. (2023). Accommodations in language testing and assessment: Safeguarding equity, access, and inclusion. Language Testing, 40(4), 847–855. https://doi.org/10.1177/02655322231186221

Winke P. (2024). Sharing, collaborating, and building trust: How Open Science advances language testing. Language Testing, 41(4), 845–859. https://doi.org/10.1177/02655322231211159

Winke P. Brunfaut T. (Eds.). (2021). The Routledge handbook of second language acquisition and language testing. Routledge.

Yan J. Fan J. (2024). Open Science for language assessment research and practice in China: A response to Winke. Language Testing, 41(4), 877–881. https://doi.org/10.1177/02655322231223100