Using the ACTFL OPIc to assess proficiency and monitor progress in a tertiary foreign languages program

Abstract

Language tests are useful for many purposes, including monitoring language learning progress and evaluating language programs. This study investigated the interpretations and usefulness of the American Council on the Teaching of Foreign Languages (ACTFL) Oral Proficiency Interview – Computer (OPIc) scores across four languages at a large US public university, focusing on (1) the influence of learner backgrounds on oral proficiency variation and growth and (2) patterns in growth across languages. Two years of OPIc score data from 814 students studying Chinese (n = 143), French (n = 251), Spanish (n = 374), and Russian (n = 46) were collected, together with survey data on language learning experiences and motivation. Each participant had two to four OPIc scores. A growth curve analysis using mixed-effect modeling revealed that language learning experience and motivation explained approximately 43% of variation in initial proficiency and 23% of variation in rates of change over time. The target language had no unique effects on initial proficiency or change over time. On average, learners gained roughly one-third of an ACTFL sublevel per semester of study (while controlling for differences in their backgrounds). Findings are informative for appropriate interpretations and uses of OPIc scores in tertiary educational settings, but also question individual growth-tracking suitability on a semester-by-semester basis.

Keywords

ACTFL longitudinal analysis OPIc program evaluation sensitivity

Get full access to this article

View all access options for this article.

References

American Council on the Teaching of Foreign Languages. (1986). ACTFL proficiency guidelines. Yonkers, NY: ACTFL.

American Council on the Teaching of Foreign Languages. (2012). ACTFL proficiency guidelines 2012 [Electronic version]. Retrieved October 25, 2016, from www.actfl.org/sites/default/files/pdfs/ACTFLProficiencyGuidelines2012_FINAL.pdf

American Council on the Teaching of Foreign Languages. (2014). ACTFL OPIc familiarization manual [Electronic version]. Retrieved October 25, 2016 from www.languagetesting.com/wp-content/uploads/2012/07/OPIc-Familiarization-Manual.pdf

American Councils for International Education. (2017). The national foreign language enrollment survey report [June 2017]. Washington, DC: American Councils for International Education. Retrieved from https://www.americancouncils.org/sites/default/files/FLE-report-June17.pdf

Bachman

Palmer

(2010). Language assessment in practice. New York: Oxford University Press.

Bates

Maechler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. doi:10.18637/jss.v067.i01

Bernhardt

Molitoris

Romeo

Lin

Valderrama

(2015). Designing and sustaining a foreign language writing proficiency assessment program at the postsecondary level. Foreign Language Annals, 48(3), 329–349.

Carroll

J. B.

(1967). Foreign language proficiency levels attained by language majors near graduation from college. Foreign Language Annals, 1(2), 131–151.

Collentine

(2004). The effects of learning contexts on morphosyntactic and lexical development. Studies in Second Language Acquisition, 26, 227–248.

10.

Council of Europe (2001). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge, UK: Cambridge University Press.

11.

Council of Europe (2009). Manual for relating language examinations to the Common European Framework of Reference for Languages (CEFR). Strasbourg, France: Author.

12.

Cubbellotti

(2015). Examination evaluation of the ACTFL OPIc® in Arabic, English, and Spanish for the ACE Review. White Plains, NY: ACTFL. Retrieved from www.languagetesting.com/pub/media/wysiwyg/research/reports/Examination_Evaluation_of_the_ACTFL_OPIc_in_Arabic_English_and_Spanish_for_the_ACE_Review.pdf

13.

Cunnings

Finlayson

(2015). Mixed effects modeling and longitudinal data analysis. In Plonsky

(Ed.), Advancing quantitative methods in second language research (pp. 159–181). New York: Routledge.

14.

Dandonoli

Henning

(1990). An investigation of the construct validity of the ACTFL Proficiency Guidelines and oral interview procedure. Foreign Language Annals, 23(1), 11–22.

15.

Davidson

D. E.

(2010). Study abroad: When, how long, and with what results? New data from the Russian front. Foreign Language Annals, 43(1), 6–26.

16.

Davin

K. J.

Rempert

T. A.

Hammerand

A. A.

(2014). Converting data to knowledge: One district’s experience using large-scale proficiency assessment. Foreign Language Annals, 47(2), 241–260.

17.

Ekkens

Winke

(2009). Evaluating workplace English language programs. Language Assessment Quarterly, 6(4), 265–287.

18.

European Commission, Directorate-General for Education and Culture. (2015). Study on comparability of language testing in Europe. Luxembourg: Publications Office of the European Union.

19.

Freed

B. F.

Segalowitz

Dewey

D. P.

(2004). Context of learning and second language fluency in French: Comparing regular classroom, study abroad, and intensive domestic immersion programs. Studies in Second Language Acquisition, 26, 275–301.

20.

Goertler

Kramer

Schenker

(2016). Setting evidence-based language goals. Foreign Language Annals, 49(3), 434–454.

21.

Lockwood

Powers

D. E.

(2015). Evaluating the TOEFL Junior® standard test as a measure of progress for young English language learners (Research Report No. RR-15–22). Princeton, NJ: Educational Testing Service. doi:10.1002/ets2.12064

22.

Hamm

(1988). The ACTFL Oral Proficiency Interview in a Canadian context: The French speaking proficiency of two groups of Ontario high-school graduates. Foreign Language Annals, 21(6), 561–567.

23.

Hernandez

T. A.

(2010). Promoting speaking proficiency through motivation and interaction: The study abroad and classroom learning contexts. Foreign Language Annals, 43(4), 650–670.

24.

Huhta

Alanen

Tarnanen

Martin

Hirvela

(2014). Assessing learners’ writing skills in a SLA study: Validating the rating process across tasks, scales and languages. Language Testing, 31(3), 307–328.

25.

Isbell

D. R.

Winke

P. M.

(in press). Test review: ACTFL Oral Proficiency Interview – Computer ® (OPIc). Language Testing.

26.

Kane

(2013). Validating the interpretation and uses of test scores. Journal of Educational Measurement, 50, 1–73.

27.

Kuznetsova

Brockhoff

P. B.

Christensen

R. H. B.

(2016). lmerTest: Tests in linear mixed effects models [Software]. R package version 2.0–33. https://CRAN.R-project.org/package=lmerTest

28.

Language Testing International. (n.d.). How long does it take to become proficient? Retrieved from www.languagetesting.com/how-long-does-it-take

29.

Liskin-Gasparro

J. E.

(1983). Professional development: Oral proficiency testing and rating. Final report of U.S. Department of Education Grant. Princeton, NJ: Educational Testing Service.

30.

Magnan

S. S.

(1986). Assessing speaking proficiency in the undergraduate curriculum: Data from French. Foreign Language Annals, 19(5), 429–438.

31.

Magnan

S. S.

Back

(2007). Social interaction and linguistic gain during study abroad. Foreign Language Annals, 40(1), 43–61.

32.

Mirman

(2014). Growth curve analysis and visualization using R. Boca Raton, LA: CRC Press.

33.

Montrul

(2008). Incomplete acquisition and bilingualism: Re-examining the age factor. Amsterdam, Netherlands: John Benjamins.

34.

Nakagawa

Schielzeth

(2013). A general and simple method for obtaining R² from generalized linear mixed-effects models. Methods in Ecology and Evolution, 4(2), 133–142.

35.

Nieuwenhuis

te Grotenhuis

Pelzer

(2012). influence.ME: Tools for detecting influential data in mixed effects models. R Journal, 4(2), 38–47.

36.

Norris

J. M.

(2009). Understanding and improving language education through program evaluation [Special issue]. Language Teaching Research, 13, 1–13.

37.

Norris

J. M.

(2016). Language program evaluation. The Modern Language Journal, 100(Supplement 2016), 169–189.

38.

Norris

J. M.

Pfeiffer

P.C

. (2003). Exploring the uses and usefulness of ACTFL oral proficiency ratings and standards in college foreign language departments. Foreign Language Annals, 36(4), 572–581.

39.

Polio

(2016). Are some languages really more difficult to learn? Maybe, maybe not. CLEAR News, 20(2), 1–5.

40.

R Core Team. (2017). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. www.R-project.org

41.

Rifkin

(2005). A ceiling effect in traditional classroom foreign language instruction: Data from Russian. The Modern Language Journal, 89, 3–18.

42.

Skehan

(1989). Individual differences in second-language learning. London: Edward Arnold.

43.

Surface

E. A.

Poncheri

R. M.

Bhavsar

K. S.

(2008). Two studies investigating the reliability and validity of the English ACTFL OPIc with Korean test takers. Raleigh, NC: SWA Consulting.

44.

Stevens

P. B.

(2006). Is Spanish really so easy? Is Arabic really so hard?: Perceived difficulty in learning Arabic as a second language. In Wahba

Taha

England

(Eds.), Handbook for Arabic language teaching professionals in the 21st century (pp. 35–63). Mahwah, NJ: Lawrence Erlbaum.

45.

Swender

(2003). Oral proficiency testing in the real world: Answers to frequently asked questions. Foreign Language Annals, 36(4), 520–526.

46.

Thompson

G. L.

Cox

T. L.

Knapp

(2016). Comparing the OPI and the OPIc: The effect of test method on oral proficiency scores and student preference. Foreign Language Annals, 49(1), 75–92.

47.

Tschirner

(2016). Listening and reading proficiency levels of college students. Foreign Language Annals, 49(2), 201–223.

48.

Watson

J. R.

Wolfel

(2016). The intersection of language and culture in study abroad: Assessment and analysis of study abroad outcomes. Frontiers: The Interdisciplinary Journal of Study Abroad, 25, 57–72.

49.

Young

M. J.

Tong

(2015). Vertical scales. In Lane

Raymond

M. R.

Haladyna

T. M.

(Eds.), Handbook of test development (pp. 450–466). New York: Routledge.

50.

Zyzik

Polio

(2008). Incidental focus on form in university Spanish literature courses. The Modern Language Journal, 92(1), 53–70.