Measuring language dominance, broadly defined as the relative strength of each of a bilingual’s two languages, remains a crucial methodological issue in bilingualism research. While various methods have been proposed, the Bilingual Language Profile (BLP) has been one of the most widely used tools for measuring language dominance. While previous studies have begun to establish its validity, the BLP has yet to be systematically evaluated with respect to reliability. Addressing this methodological gap, the current study examines the reliability of the BLP, employing a test–retest methodology with a large (N = 248), varied sample of Spanish–English bilinguals. Analysis focuses on the test–retest reliability of the overall dominance score, the dominant and non-dominant global language scores, and the subcomponent scores. The results demonstrate that the language dominance score produced by the BLP shows “excellent” levels of test–retest reliability. In addition, while some differences were found between the reliability of global language scores for the dominant and non-dominant languages, and for the different subcomponent scores, all components of the BLP display strong reliability. Taken as a whole, this study provides evidence for the reliability of BLP as a measure of bilingual language dominance.
AllenS. E.DenchC. (2015). Calculating mean length of utterance for eastern Canadian Inuktitut. First Language, 35(4–5), 377–406. https://doi.org/10.1177/0142723715596648
2.
AmengualM.ChamorroP. (2015). The effects of language dominance in the perception and production of the Galician mid vowel contrasts. Phonetica, 72(4), 207–236. https://doi.org/10.1159/000439406
BachmanL. F. (1990). Fundamental considerations in language testing. Oxford University Press.
6.
BachmanL. F.PalmerA. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford University Press.
7.
Baker-SmemoeW.DeweyD. P.BownJ.MartinsenR. A. (2014). Does measuring L2 utterance fluency equal measuring overall L2 proficiency? Evidence from five languages. Foreign Language Annals, 47(4), 707–728. https://doi.org/10.1111/flan.12110
8.
BedoreL. M.PeñaE. D.SummersC. L.BoergerK. M.ResendizM. D.GreeneK.BohmanT. M.GillamR. B. (2012). The measure matters: Language dominance profiles across measures in Spanish–English bilingual children. Bilingualism: Language and Cognition, 15(3), 616–629. https://doi.org/10.1017/S1366728912000090
9.
BerinskyA. J.MargolisM. F.SancesM. W. (2013). Separating the shirkers from the workers? Making sure respondents pay attention on self-administered surveys. American Journal of Political Science, 58(3), 739–753. https://doi.org/10.1111/ajps.12081
BirdsongD. (2016). Dominance in bilingualism: Foundations of measurement, with insights from the study of handedness. In Silva-CorvalánC.Treffers-DallerJ. (Eds.), Language dominance in bilinguals: Issues of measurement and operationalization (pp. 85–105). Cambridge University Press. https://doi.org/10.1017/CBO9781107375345.005
12.
BirdsongD.GertkenL. M.AmengualM. (2012, January20). Bilingual Language Profile: An easy-to-use instrument to assess bilingualism. COERLL, University of Texas at Austin. https://sites.la.utexas.edu/bilingual/
13.
BlandJ. M.AltmanD. G. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327(8476), 307–310. https://doi.org/10.1016/S0140-6736(86)90837-8
14.
ByrdP. D.BuckhaltJ. A. (1991). A multitrait-multimethod construct validity study of the Differential Ability Scales. Journal of Psychoeducational Assessment, 9(2), 121–129. https://doi.org/10.1177/07342829910090020
15.
CanaleM.SwainM. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1), 1–47. https://doi.org/10.1093/applin/I.1.1
16.
CantoneK.KupischT.MüllerN.SchmitzK. (2008). Rethinking language dominance in bilingual children. Linguistische Berichte, 2008(215), 307–343.
CarrollJ. (1972). Fundamental considerations in testing for English language proficiency of foreign students. In AllenH. B.CampbellR. N. (Eds.), Teaching English as a second language (pp. 313–320). McGraw-Hill.
19.
ChapelleC. A. (2011). Validity argument for language assessment: The framework is simple. . . . Language Testing, 29(1), 19–27. https://doi.org/10.1177/0265532211417211
20.
ChmielewiskiM.WatsonD. (2009). What is being assessed and why it matters: The impact of error on trait research. Journal of Personality and Social Psychology, 97(1), 186–202. https://doi.org/10.1037/a0015618
21.
Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge University Press. https://rm.coe.int/16802fc1bf
22.
de BruinA. (2019). Not all bilinguals are the same: A call for more detailed assessments and descriptions of bilingual experiences. Behavioral Sciences, 9(33), 1–13. https://doi.org/10.3390/bs9030033
23.
DelgadoP.GuerreroG.GogginJ. P.EllisB. B. (1999). Self-assessment of linguistic skills by bilingual Hispanics. Hispanic Journal of Behavioral Sciences, 21(1), 31–46. https://doi.org/10.1177/0739986399211
24.
DörnyeiZ.TaguchiT. (2009). Questionnaires in second language research: Construction, administration, and processing. Routledge.
25.
DunnA. L.Fox TreeJ. E. (2009). A quick, gradient bilingual dominance scale. Bilingualism: Language and Cognition, 12(3), 273–289. https://doi.org/10.1017/S1366728909990113
26.
Educational Testing Service. (2020). Test of English as a foreign language. www.ets.org/toefl
FifeD. A.MendozaJ. L.TerryR. (2012). The assessment of reliability under range restriction: A comparison of α, ω, and test–retest reliability for dichotomous data. Educational and Psychological Measurement, 72(5), 862–888. https://doi.org/10.1177/0013164411430225
GaillardS.TremblayA. (2016). Linguistic proficiency assessment in second language acquisition research: The elicited imitation task. Language Learning, 66(2), 419–447. https://doi.org/10.1111/lang.12157
31.
GamerM.LemonJ.FellowsI.SinghP. (2019). irr: Various coefficients of interrater reliability (R Package Version 0.84.1). https://CRAN.R-project.org/package=irr
GertkenL. M.AmengualM.BirdsongD. (2014). Assessing language dominance with the bilingual language profile. In LeclercqP.EdmondsA.HiltonH. (Eds.), Measuring L2 proficiency: Perspectives from SLA (pp. 208–225). Multilingual Matters. https://doi.org/10.21832/9781783092291-014
34.
GollanT. H.MontoyaR. I.WernerG. A. (2002). Semantic and letter fluency in Spanish-English bilinguals. Neuropsychology, 16(4), 562–576. https://doi.org/10.1037/0894-4105.16.4.562
35.
GollanT. H.WeissbergerG. H.RunnqvistE.MontoyaR. I.CeraC. M. (2012). Self-ratings of spoken language dominance: A Multilingual Naming Test (MINT) and preliminary norms for young and aging Spanish–English bilinguals. Bilingualism: Language and Cognition, 15(3), 594–615. https://doi.org/10.1017/S1366728911000332
36.
GrosjeanF. (2008). Studying bilinguals. Oxford University Press.
37.
HarrisC. L.GleasonJ. B.AycicegiA. (2006). When is a first language more emotional? Psychophysiological evidence from bilingual speakers. In PavlenkoA. (Ed.), Bilingual minds: Emotional experience, expression, and representation (pp. 257–283). Multilingual Matters.
38.
HauserD. J.SchwarzN. (2016). Attentive turkers: MTurk participants perform better online attention checks than do subject pool participants. Behavioral Research, 48, 400–407. https://doi.org/10.3758/s13428-015-0578-z
39.
HulstijnJ. H. (2011). Language proficiency in native and nonnative speakers: An agenda for research and suggestions for second-language assessment. Language Assessment Quarterly, 8(3), 229–249. https://doi.org/10.1080/15434303.2011.565844
40.
HulstijnJ. H. (2015). Language proficiency in native and non-native speakers: Theory and research. John Benjamins. https://doi.org/10.1075/lllt.41
41.
IssacsT.RoseH. (2021). Redressing the balance in the native speaker debate: Assessment standards, standard language, and exposing double standards. TESOL Quarterly, 56(1), 401–412. https://doi.org/10.1002/tesq.3041
KooT. K.LiM. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
44.
KootstraG. J.DoedensW. J. (2016). How multiple sources of experience influence bilingual syntactic choice: Immediate and cumulative cross-language effects of structural priming, verb bias, and language dominance. Bilingualism: Language and Cognition, 19(4), 710–732. https://doi.org/10.1017/S1366728916000420
LimV. P.LiowS. J. R.LincolnM.ChanY. H.OnslowM. (2008). Determining language dominance in English–Mandarin bilinguals: Development of a self-report classification tool for clinical use. Applied Psycholinguistics, 29(3), 389–412. https://doi.org/10.1017/S0142716408080181
47.
Mallonee GertkenS. E. (2013). Priming of relative clause attachment during comprehension in French as a first and second language [Doctoral dissertation, University of Texas at Austin]. https://repositories.lib.utexas.edu/handle/2152/21778
48.
MarianV.BlumenfeldH. K.KaushanskayaM. (2007). The Language Experience and Proficiency Questionnaire (LEAP-Q): Assessing language profiles in bilinguals and multilinguals. Journal of Speech, Language, and Hearing Research, 50(4), 940–967. https://doi.org/10.1044/1092-4388(2007/067)
49.
MartinJ. M.AltarribaJ.KazanasS. A. (2020). Is it possible to predict which bilingual speakers have switched language dominance? A discriminant analysis. Journal of Multilingual and Multicultural Development, 41(3), 206–218. https://doi.org/10.1080/01434632.2019.1603236
50.
MontrulS. (2011). Introduction: The linguistic competence of heritage speakers. Studies in Second Language Acquisition, 33(2), 155–161. https://doi.org/10.1017/S0272263110000719
51.
MontrulS. (2016). Dominance and proficiency in early and late bilingualism. In Silva-CorvalánC.Treffers-DallerJ. (Eds.), Language dominance in bilinguals: Issues of measurement and operationalization (pp. 15–35). Wiley-Blackwell. https://doi.org/10.1017/CBO9781107375345
52.
NagleC. (2019). Developing and validating a methodology for crowdsourcing L2 speech ratings in Amazon Mechanical Turk. Journal of Second Language Pronunciation, 5(2), 294–323. https://doi.org/10.1075/jslp.18016.nag
OlsonD. J. (2022). The bilingual code-switching profile (BCSP): Assessing the reliability and validity of the BCSP questionnaire. Linguistic Approaches to Bilingualism. Advance online publication. https://doi.org/10.1075/lab.21039.ols
55.
Olson, D.J., Deemer, E., & King, J. (2021). A systematic review of proficiency assessment methods in bilingualism research [Conference Presentation]. Bilingualism Matters Research Symposium, Online.
56.
Ortega-SantosI. (2019). Crowdsourcing for Hispanic linguistics: Amazon’s mechanical turk as a source of Spanish data. Borealis: An International Journal of Hispanic Linguistics, 8(1), 187–215. https://doi.org/10.7557/1.8.1.4670
57.
PeerE.VosgerauJ.AcquistiA. (2014). Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavioral Research Methods, 46(4), 1023–1031. https://doi.org/10.3758/s13428-013-0434-y
58.
PerpiñánS. (2018). On convergence, ongoing language change, and crosslinguistic influence in direct object expression in Catalan–Spanish bilingualism. Languages, 3(2), 14. https://doi.org/10.3390/languages3020014
59.
PoarchG. J.VanhoveJ.BertheleR. (2019). The effect of bidialectalism on executive function. International Journal of Bilingualism, 23(2), 612–628. https://doi.org/10.1177/1367006918763132
60.
RahmanA.YapN. T.DarmiR. (2018). The association between vocabulary size and language dominance of bilingual Malay-English undergraduates. 3L, Language, Linguistics, Literature, 24(4), 85–101. https://doi.org/10.17576/3L-2018-2404-07
61.
R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org
62.
RomaineS. (1999). Bilingualism (2nd ed.). Wiley.
63.
SatrakiM. (2019). Language attitudes: An overview. International Journal of Linguistics, Literature, and Translation, 2(6), 98–113.
64.
SchleefE. (2022). Measuring language attitudes. In GeeslinK. (Ed.), The Routledge handbook of second language acquisition and sociolinguistics (pp. 212–223). Routledge. https://doi.org/10.4324/9781003017325-20
65.
Solís-BarrosoC.StefanichS. (2019). Measuring language dominance in early Spanish/English bilinguals. Languages, 4(3), 62. https://doi.org/10.3390/languages4030062
66.
StockerL.BertheleR. (2020). The roles of language mode and dominance in French–German bilinguals’ motion event descriptions. Bilingualism: Language and Cognition, 23(3), 519–531. https://doi.org/10.1017/S1366728919000294
Tomé LouridoG. (2018). The role of social factors in bilingual speech processing: The case of Galician new speakers [Unpublished doctoral dissertation]. University College London. https://discovery.ucl.ac.uk/id/eprint/10041801/
69.
TomoschukB.FerreiraV. S.GollanT. H. (2019). When a seven is not a seven: Self-ratings of bilingual language proficiency differ between and within language populations. Bilingualism: Language and Cognition, 22(3), 516–536. https://doi.org/10.1017/S1366728918000421
70.
Treffers-DallerJ. (2011). Operationalizing and measuring language dominance. International Journal of Bilingualism, 15(2), 147–163. https://doi.org/10.1177/1367006910381186
71.
Treffers-DallerJ. (2016). Language dominance: The construct, its measure, and operationalization. In Silva-CorvalánC.Treffers-DallerJ. (Eds.), Language dominance in bilinguals: Issues of measurement and operationalization (pp. 235–265). Cambridge University Press. https://doi.org/10.1017/CBO9781107375345.005
UnsworthS. (2016). Amount of exposure as a proxy for dominance in bilingual language acquisition. In Silva-CorvalánC.Treffers-DallerJ. (Eds.), Language dominance in bilinguals: Issues of measurement and operationalization (pp. 156–173). Cambridge University Press. https://doi.org/10.1017/CBO9781107375345.008
74.
UnsworthS.ChondrogianniV.SkarabelaB. (2018). Experiential measures can be used as a proxy for language dominance in bilingual language acquisition research. Frontiers in Psychology, 9, Article 1809. https://doi.org/10.3389/fpsyg.2018.01809
van der BroekE. W.Oolbekkink-MarchandH. W.van KemenadeA. M.MeijerP. C.UnsworthS. (2022). Stimulating language awareness in the foreign language classroom: Exploring EFL teaching practices. The Language Learning Journal, 50(1), 59–73. https://doi.org/10.1080/09571736.2019.1688857
77.
WoodcockR. W.Muñoz-SandovalA. F. (1993). Woodcock-Muñoz Language Survey Comprehensive Manual. Chicago: Riverside.
78.
WeiL. (2000). Dimensions of bilingualism. In WeiL. (Ed.), The bilingualism reader (pp. 3–25). Routledge.
79.
YipV.MatthewsS. (2006). Assessing language dominance in bilingual acquisition: A case for mean length utterance differentials. Language Assessment Quarterly: An International Journal, 3(2), 97–116. https://doi.org/10.1207/s15434311laq0302_2
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
