Although there is abundant evidence for the use of first-language (L1) knowledge by bilinguals when using a second language (L2), investigation into the impact of L1 knowledge in large-scale L2 language assessments and discussion of how such impact may be controlled has received little attention in the language assessment literature. This study examines these issues through investigating the use of L1-Japanese loanword knowledge in test items targeting L2-English lexical knowledge in the Reading section of EIKEN grade-level tests, which are primarily taken by Japanese learners of English. First, the proportion of English target words that have loanwords in Japanese was determined through analysis of corpus-derived wordlists, revealing that the distribution of such items is broadly similar to that in language in general. Second, the impact of loanword frequency in Japanese (and cognate status) was demonstrated through statistical analysis of response data for the items. Taken together, the findings highlight the scope and impact of such cognate items in large-scale language assessments. Discussion centers on how test developers can and/or should deal with the inclusion of cognate words in terms of context validity and test fairness.
AllenD. (2019a). Cognate frequency and assessment of second language lexical knowledge. International Journal of Bilingualism, 23(5), 1121–1136. https://doi.org/10.1177/1367006918781063
AllenD. (2019c). The prevalence and frequency of Japanese–English cognates: Recommendations for future research in applied linguistics. International Review of Applied Linguistics in Language Teaching, 57(3), 355–376. https://doi.org/10.1515/iral-2017-0028
AllenD.ConklinK.MiwaK. (2021). Cross-linguistic lexical effects in different-script bilingual reading are modulated by task. International Journal of Bilingualism, 25(1), 168–188. https://doi.org/10.1177/1367006920943974
7.
BaayenR. H. (2008). Analyzing linguistic data: A practical introduction to statistics using R. Cambridge University Press. https://doi.org/10.1017/CBO9780511801686
8.
BachmanL. (1990). Fundamental considerations in language testing. Oxford University Press.
9.
BachmanL.PalmerA. (2010). Language assessment in practice. Oxford University Press.
BatistaR.HorstM. (2016). A new receptive vocabulary size test for French. Canadian Modern Language Review, 72(2), 211–233. https://doi.org/10.3138/cmlr.2820
12.
BeglarD.NationI. S. P. (2014). Chapter 10: Assessing vocabulary. In KunnanA. J. (Ed.), The companion to language assessment (pp. 172–184). Wiley-Blackwell. https://doi.org/10.1002/9781118411360.wbcla053
13.
BrendersP.Van HellJ. G.DijkstraT. (2011). Word recognition in child second language learners: Evidence from cognates and false friends. Journal of Experimental Child Psychology, 109(4), 383–396. https://doi.org/10.1016/j.jecp.2011.03.012
BrysbaertM.NewB. (2009). Moving beyond Kucera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behaviour Research Methods, 41, 977–990. https://doi.org/10.3758/BRM.41.4.977
16.
CarrollJ. B. (1961). Fundamental considerations in testing for English language proficiency of foreign students. Paper presented at the conference on Testing the English Proficiency of Foreign Students, Washington, DC, May11–12, 1961 [Reprinted in AllenH. B.CampbellR. N. (Eds), Teaching English as a second language: A book of readings (pp. 313–321). McGraw Hill].
CostaA.CaramazzaA.Sebástian-GallésN. (2000). The cognate facilitation effect: Implications for models of lexical access. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26(5), 1283–1296. https://doi.org/10.1037/0278-7393.26.5.1283
19.
DijkstraT. (2005). Bilingual visual word recognition and lexical access. In KrollJ. F.de GrootA. M. B. (Eds.), Handbook of bilingualism: Psycholinguistic approaches (pp. 179–201). Oxford University Press.
20.
DijkstraT.van HeuvenW. J. B. (2018). Visual word recognition in multilinguals. In RueshemeyerS.GaskellM. G. (Eds.), The Oxford handbook of psycholinguistics, second edition (pp. 118–143). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780198786825.013.6
21.
DunleaJ. (2015). Validating a set of Japanese EFL proficiency tests: Demonstrating locally designed tests meet international standards [Unpublished doctoral dissertation]. University of Bedfordshire. https://uobrep.openrepository.com/handle/10547/618581
ElgortI. (2013). Effects of L1 definitions and cognate status of test items on the Vocabulary Size Test. Language Testing, 30(2), 253–272. https://doi.org/10.1177/0265532212459028
HoshinoN.KrollJ. (2008). Cognate effects in picture naming: Does cross-linguistic activation survive a change of script?Cognition, 106(1), 501–511. https://doi.org/10.1016/j.cognition.2007.02.001
32.
KaneM. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
33.
KhalifaH.WeirC. (2009). Examining reading: Research and practice in assessing second language reading (Studies in Language Testing 29). Cambridge University Press.
34.
LadoR. (1961). Language testing. Longmans.
35.
LauferB.McLeanS. (2016). Loanwords and vocabulary size test scores: A case of different estimates for different L1 learners. Language Assessment Quarterly, 13(3), 202–217. https://doi.org/10.1080/15434303.2016.1210611
36.
MearaP. M.LightbrownP. M.HalterR. H. (1994). The effect of cognates on the applicability of YES/NO vocabulary tests. Canadian Modern Language Review, 50(2), 296–311. https://doi.org/10.3138/cmlr.50.2.296
37.
MidgleyK.HolcombP. J.GraingerJ. (2011). Effects of cognate status on word comprehension in second language learners: An ERP investigation. Journal of Cognitive Neuroscience, 23(7), 1634–1647. https://doi.org/10.1162/jocn.2010.21463
38.
MiwaK.DijkstraT.BolgerP.BaayenH. (2014). Reading English with Japanese in mind: Effects of frequency, phonology, and meaning in different-script bilinguals. Bilingualism: Language and Cognition, 17(3), 445–463. https://doi.org/10.1017/S1366728913000576
SchmittN.NationP.KremmelB. (2020). Moving the field of vocabulary assessment forward: The need for more rigorous test development and validation. Language Teaching, 53(1), 109–120. https://doi.org/10.1017/S0261444819000326
46.
SchmittN.SchmittD.ClaphamC. (2001). Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test. Language Testing, 18(1), 55–88. https://doi.org/10.1177/026553220101800103
TurnbullB. (2020). Towards new standards in foreign language assessment: Learning from bilingual education. International Journal of Bilingual Education and Bilingualism, 23(4), 488–498. https://doi.org/10.1080/13670050.2017.1375891
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
