Measuring bilingual language dominance: An examination of the reliability of the Bilingual Language Profile

Abstract

Spanish

Chinese

Measuring language dominance, broadly defined as the relative strength of each of a bilingual’s two languages, remains a crucial methodological issue in bilingualism research. While various methods have been proposed, the Bilingual Language Profile (BLP) has been one of the most widely used tools for measuring language dominance. While previous studies have begun to establish its validity, the BLP has yet to be systematically evaluated with respect to reliability. Addressing this methodological gap, the current study examines the reliability of the BLP, employing a test–retest methodology with a large (N = 248), varied sample of Spanish–English bilinguals. Analysis focuses on the test–retest reliability of the overall dominance score, the dominant and non-dominant global language scores, and the subcomponent scores. The results demonstrate that the language dominance score produced by the BLP shows “excellent” levels of test–retest reliability. In addition, while some differences were found between the reliability of global language scores for the dominant and non-dominant languages, and for the different subcomponent scores, all components of the BLP display strong reliability. Taken as a whole, this study provides evidence for the reliability of BLP as a measure of bilingual language dominance.

Keywords

Bilingual Language Profile bilingualism language dominance proficiency test–retest reliability

Get full access to this article

View all access options for this article.

References

Allen

S. E.

Dench

(2015). Calculating mean length of utterance for eastern Canadian Inuktitut. First Language, 35(4–5), 377–406. https://doi.org/10.1177/0142723715596648

Amengual

Chamorro

(2015). The effects of language dominance in the perception and production of the Galician mid vowel contrasts. Phonetica, 72(4), 207–236. https://doi.org/10.1159/000439406

American Council of Teachers of Foreign Languages. (2012). ACTFL proficiency guidelines 2012. https://www.actfl.org/resources/actfl-proficiency-guidelines-2012

Association for Language Awareness. (n.d.). About. https://www.languageawareness.org/?page_id=48

Bachman

L. F.

(1990). Fundamental considerations in language testing. Oxford University Press.

Bachman

L. F.

Palmer

A. S.

(1996). Language testing in practice: Designing and developing useful language tests. Oxford University Press.

Baker-Smemoe

Dewey

D. P.

Bown

Martinsen

R. A.

(2014). Does measuring L2 utterance fluency equal measuring overall L2 proficiency? Evidence from five languages. Foreign Language Annals, 47(4), 707–728. https://doi.org/10.1111/flan.12110

Bedore

L. M.

Peña

E. D.

Summers

C. L.

Boerger

K. M.

Resendiz

M. D.

Greene

Bohman

T. M.

Gillam

R. B.

(2012). The measure matters: Language dominance profiles across measures in Spanish–English bilingual children. Bilingualism: Language and Cognition, 15(3), 616–629. https://doi.org/10.1017/S1366728912000090

Berinsky

A. J.

Margolis

M. F.

Sances

M. W.

(2013). Separating the shirkers from the workers? Making sure respondents pay attention on self-administered surveys. American Journal of Political Science, 58(3), 739–753. https://doi.org/10.1111/ajps.12081

10.

Birdsong

(2014). Dominance and age in bilingualism. Applied Linguistics, 35(4), 374–392. https://doi.org/10.1093/applin/amu031

11.

Birdsong

(2016). Dominance in bilingualism: Foundations of measurement, with insights from the study of handedness. In Silva-Corvalán

Treffers-Daller

(Eds.), Language dominance in bilinguals: Issues of measurement and operationalization (pp. 85–105). Cambridge University Press. https://doi.org/10.1017/CBO9781107375345.005

12.

Birdsong

Gertken

L. M.

Amengual

(2012, January 20). Bilingual Language Profile: An easy-to-use instrument to assess bilingualism. COERLL, University of Texas at Austin. https://sites.la.utexas.edu/bilingual/

13.

Bland

J. M.

Altman

D. G.

(1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327(8476), 307–310. https://doi.org/10.1016/S0140-6736(86)90837-8

14.

Byrd

P. D.

Buckhalt

J. A.

(1991). A multitrait-multimethod construct validity study of the Differential Ability Scales. Journal of Psychoeducational Assessment, 9(2), 121–129. https://doi.org/10.1177/07342829910090020

15.

Canale

Swain

(1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1), 1–47. https://doi.org/10.1093/applin/I.1.1

16.

Cantone

Kupisch

Müller

Schmitz

(2008). Rethinking language dominance in bilingual children. Linguistische Berichte, 2008(215), 307–343.

17.

Canty

Ripley

B. D.

(2021). boot: Bootstrap R (S-Plus) functions (R Package Version 1.3-28). https://cran.r-project.org/web/packages/boot

18.

Carroll

(1972). Fundamental considerations in testing for English language proficiency of foreign students. In Allen

H. B.

Campbell

R. N.

(Eds.), Teaching English as a second language (pp. 313–320). McGraw-Hill.

19.

Chapelle

C. A.

(2011). Validity argument for language assessment: The framework is simple. . . . Language Testing, 29(1), 19–27. https://doi.org/10.1177/0265532211417211

20.

Chmielewiski

Watson

(2009). What is being assessed and why it matters: The impact of error on trait research. Journal of Personality and Social Psychology, 97(1), 186–202. https://doi.org/10.1037/a0015618

21.

Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge University Press. https://rm.coe.int/16802fc1bf

22.

de Bruin

. (2019). Not all bilinguals are the same: A call for more detailed assessments and descriptions of bilingual experiences. Behavioral Sciences, 9(33), 1–13. https://doi.org/10.3390/bs9030033

23.

Delgado

Guerrero

Goggin

J. P.

Ellis

B. B.

(1999). Self-assessment of linguistic skills by bilingual Hispanics. Hispanic Journal of Behavioral Sciences, 21(1), 31–46. https://doi.org/10.1177/0739986399211

24.

Dörnyei

Taguchi

(2009). Questionnaires in second language research: Construction, administration, and processing. Routledge.

25.

Dunn

A. L.

Fox Tree

J. E.

(2009). A quick, gradient bilingual dominance scale. Bilingualism: Language and Cognition, 12(3), 273–289. https://doi.org/10.1017/S1366728909990113

26.

Educational Testing Service. (2020). Test of English as a foreign language. www.ets.org/toefl

27.

Farias

(2005). Critical language awareness in foreign language learning. Literatura y Lingüística, 16, 211–222. https://doi.org/10.4067/S0716-58112005000100012

28.

Fife

D. A.

Mendoza

J. L.

Terry

(2012). The assessment of reliability under range restriction: A comparison of α, ω, and test–retest reliability for dichotomous data. Educational and Psychological Measurement, 72(5), 862–888. https://doi.org/10.1177/0013164411430225

29.

Flege

J. E.

MacKay

I. R.

Piske

(2002). Assessing bilingual dominance. Applied Psycholinguistics, 23(4), 567–598. https://doi.org/10.1017/S0142716402004046

30.

Gaillard

Tremblay

(2016). Linguistic proficiency assessment in second language acquisition research: The elicited imitation task. Language Learning, 66(2), 419–447. https://doi.org/10.1111/lang.12157

31.

Gamer

Lemon

Fellows

Singh

(2019). irr: Various coefficients of interrater reliability (R Package Version 0.84.1). https://CRAN.R-project.org/package=irr

32.

Garrett

(2010). Attitudes to language. Cambridge University Press. https://doi.org/10.1017/CBO9780511844713

33.

Gertken

L. M.

Amengual

Birdsong

(2014). Assessing language dominance with the bilingual language profile. In Leclercq

Edmonds

Hilton

(Eds.), Measuring L2 proficiency: Perspectives from SLA (pp. 208–225). Multilingual Matters. https://doi.org/10.21832/9781783092291-014

34.

Gollan

T. H.

Montoya

R. I.

Werner

G. A.

(2002). Semantic and letter fluency in Spanish-English bilinguals. Neuropsychology, 16(4), 562–576. https://doi.org/10.1037/0894-4105.16.4.562

35.

Gollan

T. H.

Weissberger

G. H.

Runnqvist

Montoya

R. I.

Cera

C. M.

(2012). Self-ratings of spoken language dominance: A Multilingual Naming Test (MINT) and preliminary norms for young and aging Spanish–English bilinguals. Bilingualism: Language and Cognition, 15(3), 594–615. https://doi.org/10.1017/S1366728911000332

36.

Grosjean

(2008). Studying bilinguals. Oxford University Press.

37.

Harris

C. L.

Gleason

J. B.

Aycicegi

(2006). When is a first language more emotional? Psychophysiological evidence from bilingual speakers. In Pavlenko

(Ed.), Bilingual minds: Emotional experience, expression, and representation (pp. 257–283). Multilingual Matters.

38.

Hauser

D. J.

Schwarz

(2016). Attentive turkers: MTurk participants perform better online attention checks than do subject pool participants. Behavioral Research, 48, 400–407. https://doi.org/10.3758/s13428-015-0578-z

39.

Hulstijn

J. H.

(2011). Language proficiency in native and nonnative speakers: An agenda for research and suggestions for second-language assessment. Language Assessment Quarterly, 8(3), 229–249. https://doi.org/10.1080/15434303.2011.565844

40.

Hulstijn

J. H.

(2015). Language proficiency in native and non-native speakers: Theory and research. John Benjamins. https://doi.org/10.1075/lllt.41

41.

Issacs

Rose

(2021). Redressing the balance in the native speaker debate: Assessment standards, standard language, and exposing double standards. TESOL Quarterly, 56(1), 401–412. https://doi.org/10.1002/tesq.3041

42.

Kane

M. T.

(2016). Explicating validity. Assessment in Education: Principles, Policy & Practice, 23(2), 198–211. https://doi.org/10.1080/0969594X.2015.1060192

43.

Koo

T. K.

M. Y.

(2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012

44.

Kootstra

G. J.

Doedens

W. J.

(2016). How multiple sources of experience influence bilingual syntactic choice: Immediate and cumulative cross-language effects of structural priming, verb bias, and language dominance. Bilingualism: Language and Cognition, 19(4), 710–732. https://doi.org/10.1017/S1366728916000420

45.

Lehmann

(2007). Linguistic competence: Theory and empiry. Folia Linguistica, 41(3–4), 223–287. https://doi.org/10.1515/flin.41.3-4.223

46.

Lim

V. P.

Liow

S. J. R.

Lincoln

Chan

Y. H.

Onslow

(2008). Determining language dominance in English–Mandarin bilinguals: Development of a self-report classification tool for clinical use. Applied Psycholinguistics, 29(3), 389–412. https://doi.org/10.1017/S0142716408080181

47.

Mallonee Gertken

S. E

. (2013). Priming of relative clause attachment during comprehension in French as a first and second language [Doctoral dissertation, University of Texas at Austin]. https://repositories.lib.utexas.edu/handle/2152/21778

48.

Marian

Blumenfeld

H. K.

Kaushanskaya

(2007). The Language Experience and Proficiency Questionnaire (LEAP-Q): Assessing language profiles in bilinguals and multilinguals. Journal of Speech, Language, and Hearing Research, 50(4), 940–967. https://doi.org/10.1044/1092-4388(2007/067)

49.

Martin

J. M.

Altarriba

Kazanas

S. A.

(2020). Is it possible to predict which bilingual speakers have switched language dominance? A discriminant analysis. Journal of Multilingual and Multicultural Development, 41(3), 206–218. https://doi.org/10.1080/01434632.2019.1603236

50.

Montrul

(2011). Introduction: The linguistic competence of heritage speakers. Studies in Second Language Acquisition, 33(2), 155–161. https://doi.org/10.1017/S0272263110000719

51.

Montrul

(2016). Dominance and proficiency in early and late bilingualism. In Silva-Corvalán

Treffers-Daller

(Eds.), Language dominance in bilinguals: Issues of measurement and operationalization (pp. 15–35). Wiley-Blackwell. https://doi.org/10.1017/CBO9781107375345

52.

Nagle

(2019). Developing and validating a methodology for crowdsourcing L2 speech ratings in Amazon Mechanical Turk. Journal of Second Language Pronunciation, 5(2), 294–323. https://doi.org/10.1075/jslp.18016.nag

53.

Noe-Bustamante

Flores

Shah

(2019, September 16). Facts on Hispanics of Mexican Origin in the United States, 2017. Pew Research Center. https://www.pewresearch.org/hispanic/fact-sheet/u-s-hispanics-facts-on-mexican-origin-latinos/

54.

Olson

D. J.

(2022). The bilingual code-switching profile (BCSP): Assessing the reliability and validity of the BCSP questionnaire. Linguistic Approaches to Bilingualism. Advance online publication. https://doi.org/10.1075/lab.21039.ols

55.

Olson, D.J., Deemer, E., & King, J. (2021). A systematic review of proficiency assessment methods in bilingualism research [Conference Presentation]. Bilingualism Matters Research Symposium, Online.

56.

Ortega-Santos

(2019). Crowdsourcing for Hispanic linguistics: Amazon’s mechanical turk as a source of Spanish data. Borealis: An International Journal of Hispanic Linguistics, 8(1), 187–215. https://doi.org/10.7557/1.8.1.4670

57.

Peer

Vosgerau

Acquisti

(2014). Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavioral Research Methods, 46(4), 1023–1031. https://doi.org/10.3758/s13428-013-0434-y

58.

Perpiñán

(2018). On convergence, ongoing language change, and crosslinguistic influence in direct object expression in Catalan–Spanish bilingualism. Languages, 3(2), 14. https://doi.org/10.3390/languages3020014

59.

Poarch

G. J.

Vanhove

Berthele

(2019). The effect of bidialectalism on executive function. International Journal of Bilingualism, 23(2), 612–628. https://doi.org/10.1177/1367006918763132

60.

Rahman

Yap

N. T.

Darmi

(2018). The association between vocabulary size and language dominance of bilingual Malay-English undergraduates. 3L, Language, Linguistics, Literature, 24(4), 85–101. https://doi.org/10.17576/3L-2018-2404-07

61.

R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org

62.

Romaine

(1999). Bilingualism (2nd ed.). Wiley.

63.

Satraki

(2019). Language attitudes: An overview. International Journal of Linguistics, Literature, and Translation, 2(6), 98–113.

64.

Schleef

(2022). Measuring language attitudes. In Geeslin

(Ed.), The Routledge handbook of second language acquisition and sociolinguistics (pp. 212–223). Routledge. https://doi.org/10.4324/9781003017325-20

65.

Solís-Barroso

Stefanich

(2019). Measuring language dominance in early Spanish/English bilinguals. Languages, 4(3), 62. https://doi.org/10.3390/languages4030062

66.

Stocker

Berthele

(2020). The roles of language mode and dominance in French–German bilinguals’ motion event descriptions. Bilingualism: Language and Cognition, 23(3), 519–531. https://doi.org/10.1017/S1366728919000294

67.

Thomas

(1994). Assessment of L2 proficiency in second language acquisition research. Language Learning, 44(2), 307–307. https://doi.org/10.1111/j.1467-1770.1994.tb01104.x

68.

Tomé Lourido

. (2018). The role of social factors in bilingual speech processing: The case of Galician new speakers [Unpublished doctoral dissertation]. University College London. https://discovery.ucl.ac.uk/id/eprint/10041801/

69.

Tomoschuk

Ferreira

V. S.

Gollan

T. H.

(2019). When a seven is not a seven: Self-ratings of bilingual language proficiency differ between and within language populations. Bilingualism: Language and Cognition, 22(3), 516–536. https://doi.org/10.1017/S1366728918000421

70.

Treffers-Daller

(2011). Operationalizing and measuring language dominance. International Journal of Bilingualism, 15(2), 147–163. https://doi.org/10.1177/1367006910381186

71.

Treffers-Daller

(2016). Language dominance: The construct, its measure, and operationalization. In Silva-Corvalán

Treffers-Daller

(Eds.), Language dominance in bilinguals: Issues of measurement and operationalization (pp. 235–265). Cambridge University Press. https://doi.org/10.1017/CBO9781107375345.005

72.

Treffers-Daller

(2019). What defines language dominance in bilinguals? Annual Review of Linguistics, 5, 375–393. https://doi.org/10.1146/annurev-linguistics-011817-045554

73.

Unsworth

(2016). Amount of exposure as a proxy for dominance in bilingual language acquisition. In Silva-Corvalán

Treffers-Daller

(Eds.), Language dominance in bilinguals: Issues of measurement and operationalization (pp. 156–173). Cambridge University Press. https://doi.org/10.1017/CBO9781107375345.008

74.

Unsworth

Chondrogianni

Skarabela

(2018). Experiential measures can be used as a proxy for language dominance in bilingual language acquisition research. Frontiers in Psychology, 9, Article 1809. https://doi.org/10.3389/fpsyg.2018.01809

75.

U.S. Census. (2020). Dashboard—United States: Hispanic or Latino percent. https://www.census.gov/quickfacts/fact/dashboard/US/RHI725219

76.

van der Broek

E. W.

Oolbekkink-Marchand

H. W.

van Kemenade

A. M.

Meijer

P. C.

Unsworth

. (2022). Stimulating language awareness in the foreign language classroom: Exploring EFL teaching practices. The Language Learning Journal, 50(1), 59–73. https://doi.org/10.1080/09571736.2019.1688857

77.

Woodcock

R. W.

Muñoz-Sandoval

A. F.

(1993). Woodcock-Muñoz Language Survey Comprehensive Manual. Chicago: Riverside.

78.

Wei

(2000). Dimensions of bilingualism. In Wei

(Ed.), The bilingualism reader (pp. 3–25). Routledge.

79.

Yip

Matthews

(2006). Assessing language dominance in bilingual acquisition: A case for mean length utterance differentials. Language Assessment Quarterly: An International Journal, 3(2), 97–116. https://doi.org/10.1207/s15434311laq0302_2

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB