This study strengthens the validation of learner speech assessment in the Common European Framework of Reference (CEFR) by analyzing the quantitative variables related to fluency and accuracy across four CEFR levels (A2, B1, B2, and C1). Drawing on a learner corpus approach, we examine 500,000 tokens from the Louvain International Database of Spoken English Interlanguage (LINDSEI) and its extensions, supplemented by post hoc rater evaluations. Three task types—a semi-monologic topic discussion, a dialogic interaction, and a monologic picture description—are used to elicit variation in speech production. The analysis focuses on speech rates, the frequency of filled and unfilled pauses, and error rates to unveil developmental trends in learner speech. The results reveal strong correlations between these fluency and accuracy metrics and CEFR levels, with speech rate emerging as the most reliable indicator of proficiency. The frequency of unfilled pauses decreases as proficiency increases, while filled pauses, although less critical to fluency assessment, offer insights into speech planning mechanisms. Error rates similarly decline with higher proficiency, reflecting greater accuracy in speech production. Exemplary instances for each CEFR level are presented, offering practical metrics for teaching, assessment, and rater training. While the study’s limitations include an overrepresentation of Mandarin Chinese learners and the exclusion of pronunciation errors, these gaps highlight avenues for future research. This study provides empirical, task-sensitive evidence to enrich CEFR can-do descriptors, enhance rater training, and refine speaking assessments, contributing to more effective language teaching, learning, and assessment practices.
ArnoldJ. E.FagnanonM.TanenhausM. K. (2003). Disfluencies signal theee, um, new information. Journal of Psycholinguistic Research, 32(1), 25–36. https://doi.org/10.1023/A:1021980931292
2.
BiberD.FineganE.JohanssonS.ConradS.LeechG. (1999). Longman grammar of spoken and written English. Pearson Education Limited.
3.
BiberD.JohanssonS.LeechG.ConradS.FineganE. (2021). Grammar of spoken and written English. John Benjamins.
4.
BoskerH. R.PingetA.-F.QuenéH.SandersT.De JongN. H. (2013). What makes speech sound fluent? The contributions of pauses, speed and repairs. Language Testing, 30(2), 159–175. https://doi.org/10.1177/0265532212455394
5.
BrezinaV. (2018). Statistics in corpus linguistics: A practical guide. Cambridge University Press.
6.
CalliesM.GötzS. (2015). Learner corpora in language testing and assessment: Prospects and challenges. In CalliesM.GötzS. (Eds.), Learner corpora in language testing and assessment (pp. 1–9). John Benjamins.
7.
CapelA. (2015). The English vocabulary profile. In HarrisonJ.BarkerF. (Eds.), English profile in practice (pp. 9–27). Cambridge University Press.
8.
CarterR.McCarthyM. (2006). Cambridge grammar of English. Cambridge University Press.
9.
ÇebiA.GilanliogluI. (2023). Evaluation of the CEFR framed English language curriculum in primary and secondary schools in Turkey: Teachers’ perspectives. Revista de Educacion, 400(6), 48–93.
10.
CenozJ. (1998). Pauses and communication strategies in second language speech [Research report]. University of the Basque Country.
11.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.
12.
CorleyM.StewartO. W. (2008). Hesitation disfluencies in spontaneous speech: The meaning of um: Hesitation disfluencies in spontaneous speech. Language and Linguistics Compass, 2(4), 589–602. https://doi.org/10.1111/j.1749-818X.2008.00068.x
13.
Council of Europe. (2020). Common European framework of reference for languages: Learning, teaching, assessment companion volume. Council of Europe Publishing.
14.
CucchiariniC.StrikH.BovesL. (2002). Quantitative assessment of second language learners’ fluency: Comparisons between read and spontaneous speech. Journal of Acoustical Society of America, 111(6), 2862–2873. https://doi.org/10.1121/1.1471894
DagneauxE.DennessS.GrangerS.MeumierF. (1996). Error tagging manual (version 1.1). Centre for English Corpus Linguistics, Université catholique de Louvain.
17.
DagneauxE.DennessS.GrangerS.MeumierF.NeffJ.ThewissenJ. (2008). The Louvain error tagging manual (version 1.3). Centre for English Corpus Linguistics, Université catholique de Louvain.
18.
De CockS. (2004). Preferred sequences of words in NS and NNS speech. Belgian Journal of English Language and Literatures, New Series, 2, 225–246. http://hdl.handle.net/2078.1/75157
19.
De JongN.PerfettiC. A. (2011). Fluency training in the ESL classroom: An experimental study of fluency development and proceduralization. Language Learning, 61(2), 533–568. https://doi.org/10.1111/j.1467-9922.2010.00620.x
20.
De JongN. H. (2016). Predicting pauses in L1 and L2 speech: The effects of utterance boundaries and word frequency. International Review of Applied Linguistics in Language Teaching, 54(2), 113–132. https://doi.org/10.1515/iral-2016-9993
21.
De JongN. H.BoskerH. R. (2013). Choosing a threshold for silent pauses to measure second language fluency. In EklundR. (Ed.), Proceedings of disfluency in spontaneous speech (pp. 17–20). Royal Institute of Technology.
22.
Díez-BedmarM. B. (2015). Article use and criterial features in Spanish EFL writing: A pilot study from CEFR A2 to B2 levels. In CalliesM.GötzS. (Eds.), Learner corpora in language testing and assessment (pp. 163–190). John Benjamins.
23.
Díez-BedmarM. B. (2021). Error analysis. In Tracy-VenturaN.PaquotM. (Eds.), The Routledge handbook of second language acquisition and corpora (pp. 90–104). Routledge.
24.
EklundR. (2004). Disfluency in Swedish human–human and human–machine travel booking dialogues [Doctoral dissertation]. Linköping University, Linköping Studies in Science and Technology. https://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-5012
25.
EllisR. (1999). The study of second language acquisition. Oxford University Press.
26.
FreedB. (1995). What makes us think that students who study abroad become fluent? In FreedB. (Ed.), Second language acquisition in a study abroad context (pp. 123–148). John Benjamin.
27.
FreedB. F. (2000). Is fluency, like beauty, in the eyes (and ears) of the beholder? In RiggenbachH. (Ed.), Perspectives on fluency (pp. 243–265). The University of Michigan Press.
28.
GablasovaD.BrezinaV.McEneryT. (2019). The Trinity Lancaster Corpus development, description and application. International Journal of Learner Corpus Research, 5(2), 126–158. https://doi.org/10.1075/ijlcr.19001.gab
GilquinG.De CockS.GrangerS. (Eds.), (2010). LINDSEI Louvain international database of spoken English interlanguage. Handbook and CD-ROM. Presses universitaires de Louvain.
GötzS. (2013). Fluency in native and non-native English speech. John Benjamins.
33.
GötzS. (2015). Tense and aspect errors in spoken learner English: Implications for language testing and assessment. In CalliesM.GötzS. (Eds.), Learner corpora in language testing and assessment (pp. 191–215). John Benjamins.
GráfT. (2017). The story of the learner corpus LINDSEI_CZ. Studie Z Aplikované Lingvistiky / Studies in Applied Linguistics, 8(2), 22–35. http://hdl.handle.net/20.500.11956/97524
36.
GráfT. (2019). Speech rate revisited. In GötzS.MukherjeeJ. (Eds.), Learner corpora and language teaching (pp. 175–189). John Benjamins.
37.
GráfT.HuangL.-F. (2019). Repeats in advanced spoken English. In DegandL.GilquinG.MeurantL.SimonA. C. (Eds.), Fluency and disfluency across languages and language varieties (pp. 219–241). Presses Universitaires de Louvain.
38.
GráfT.HuangL.-F. (2022). Persistent errors in spoken English among Taiwanese and Czech learners at CEFR B2 and C1. In Leńko-SzymańskaA.GötzS. (Eds.), Complexity, accuracy and fluency in learner corpus research (pp. 137–158). John Benjamins.
39.
GrangerS. (1996). From CA to CIA and back: An integrated approach to computerized bilingual and learner corpora. In AijmerK.AltenbergB.JohanssonM. (Eds.), Languages in contrast. Text-based cross-linguistic studies (pp. 37–51). Lund University Press.
40.
GrangerS. (2015). Contrastive interlanguage analysis: A reappraisal. International Journal of Learner Corpus Research, 1(1), 7–24. https://doi.org/10.1075/ijlcr.1.1.01gra
41.
HarrisonJ. (2015). What is English profile?. In HarrisonJ.BarkerF. (Eds.), English profile in practice (Vol. 5, pp. 1–8). Cambridge University Press.
42.
HawkinsJ. A.ButteryP. (2010). Criterial features in learner corpora: Theory and illustrations. English Profile Journal, 1(E5), 1–23. https://doi.org/10.1017/S2041536210000103
43.
HiltonH. (2014). Oral fluency and spoken proficiency: Considerations for research and testing. In LeclercqP.EdmondsA.HiltonH. (Eds.), Measuring L2 proficiency (pp. 27–53). Multilingual Matters.
44.
HuangL.-F. (2014). Constructing the Taiwanese component of the Louvain International Database of Spoken English Interlanguage (LINDSEI). Taiwan Journal of TESOL, 11(1), 31–74.
45.
HuangL.-F.GráfT. (2020). Speech rate and pausing in English: Comparing learners at different levels of proficiency with native speakers. Taiwan Journal of TESOL, 17(1), 57–86. https://doi.org/10.30397/TJTESOL.202004_17(1).0003
46.
HuangL.-F.GráfT. (2021). Expanding LINDSEI to spoken learner English from several L1s across CEFR levels. Corpora, 16(2), 271–285. https://doi.org/10.3366/cor.2021.0220
47.
HuangL.-F.KubelecS.KengN.HsuL.-H. (2018). Evaluating CEFR rater performance through the analysis of spoken learner corpora. Language Testing in Asia, 8(14), 1–17. https://doi.org/10.1186/s40468-018-0069-0
48.
HuenschA. (2021). Fluency. In Tracy-VenturaN.PaquotM. (Eds.), The Routledge handbook of second language acquisition and corpora (pp. 293–304). Routledge.
49.
JenkinsJ. (2006). Current perspectives on teaching World Englishes and English as a Lingua Franca. TESOL Quarterly, 40(1), 157–181. https://doi.org/10.2307/40264515
KormosJ.DénesM. (2004). Exploring measures and perceptions of fluency in the speech of second language learners. System, 32(2), 145–164. https://doi.org/10.1016/j.system.2004.01.001
52.
KosmalaL.CribleL. (2022). The dual status of filled pauses: Evidence from genre, proficiency and co-occurrence. Language and Speech, 65(1), 216–239. https://doi.org/10.1177/00238309211010862
53.
Leńko-SzymańskaA. (2015). The English Vocabulary Profile as a benchmark for assigning levels to learner corpus data. In CalliesM.GötzS. (Eds.), Learner corpora in language testing and assessment (pp. 115–140). John Benjamins.
LevkinaM.GilabertR. (2012). The effects of cognitive task complexity on L2 oral production. In HousenA.KuikenF.VedderI. (Eds.), Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA. John Benjamins.
56.
LomaxR. G.Hahs-VaughnD. L. (2012). Statistical concepts: A second course (4th ed.). Routledge.
57.
LüdelingA.HirschmannH. (2015). Error annotation systems. In GrangerS.GilquinG.MeunierF. (Eds.), The Cambridge handbook of learner corpus research (pp. 135–158). Cambridge University Press.
58.
MauriceK. (1983). The fluency workshop. TESOL Newsletter, 17, Article 130.
59.
MilanovicM. (2009). Cambridge ESOL and the CEFR. University of Cambridge ESOL Examinations Research Notes, 37, 2–5.
60.
MoraJ. C. (2006). Age effect on oral fluency development. In MuñozC. (Ed.), Age and the rate of foreign language learning (pp. 65–88). Multilingual Matters.
NationP. (1999). Fluency and accuracy. In SpolskyB. (Ed.), Concise encyclopedia of educational linguistics (pp. 611–612). Elsevier.
63.
NicholsonH.EberhardK.ScheutzM. (2010, September25–26). “Um. . . I don’t see any”: The function of FPs and repairs [Conference session]. Paper presented at the DISS-LPSS Joint Workshop, Tokyo, Japan.
64.
PallantJ. (2011). SPSS survival manual (4th ed.). Allen & Unwin.
PawleyA.SyderF. H. (1983). Two puzzles for linguistic theory: Native-like selection and native-like fluency. In RichardsJ. C.SchmidtR. W. (Eds.), Language and communication (pp. 191–226). Longman.
67.
PawleyA.SyderF. H. (2000). The one-clause-at-a-time-hypothesis. In RiggenbachH. (Ed.), Perspectives on fluency (pp. 163–199). University of Michigan Press.
RaupachM. (1984). Formulae in second language speech production. In DechertH.-W.MöhleD.RaupachM.DechertH. W. (Eds.), Second language productions (pp. 114–137). Gunter Narr.
70.
RazaliN. M.WahY. B. (2011). Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests. Journal of Statistical Modeling and Analytics, 2(1), 21–33.
RiggenbachH. (1991). Toward an understanding of fluency: A microanalysis of nonnative speaker conversations. Discourse Processes, 14(4), 423–441. https://doi.org/10.1080/01638539109544795
73.
RobinsonP. (2001). Task complexity, task difficulty, and task production: Exploring interactions in a componential framework. Applied Linguistics, 22(1), 27–57. https://doi.org/10.1093/applin/22.1.27
74.
RoseR. L. (1998). The communicative value of filled pauses in spontaneous speech [MA thesis]. University of Birmingham.
75.
SchmidtT.WörnerK. (2009). EXMARaLDA: Creating, analysing and sharing spoken language corpora for pragmatic research. Pragmatics, 19(4), 565–582. https://doi.org/10.1075/prag.19.4.06sch
SegalowitzN. (2010). Cognitive bases of second language fluency. Routledge
78.
SeidlhoferB. (2004). Research perspectives on teaching English as a lingua franca. Annual Review of Applied Linguistics, 24, 209–239. https://doi.org/10.1017/S0267190504000145
79.
ShapiroS. S.WilkM. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52(3/4), 591–611. https://doi.org/10.1093/biomet/52.3-4.591
80.
SheaC.LeonardK. (2019). Evaluating measures of pausing for second language fluency research. The Canadian Modern Language Review, 75(3), 216–235. https://doi.org/10.3138/cmlr.2018-0258
81.
SkehanP. (1996). Second language acquisition research and task-based instruction. In WillisJ.WillisD. (Eds.), Challenge and change in language teaching (pp. 17–30). Heinemann.
82.
SkehanP. (2009). Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied Linguistics, 30(4), 510–532. https://doi.org/10.1093/applin/amp047
83.
SkehanP.FosterP. (2012). Complexity, accuracy, fluency and lexis in task-based performance: A meta-analysis of the Ealing research. In HousenA.KuikenF.VedderI. (Eds.), Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (pp. 199–220). John Benjamins.
84.
StenströmA.-B. (1990). Pauses in monologue and dialogue. In SvartvikJ. (Ed.), The London: Lund corpus of spoken English. Description and research (pp. 211–252). Lund University Press.
85.
SuzukiS.KormosJ.UchiharaT. (2021). The relationship between utterance and perceived fluency: A meta analysis of correlational studies. The Modern Language Journal, 105(2), 435–463. https://doi.org/10.1111/modl.12706
86.
TabachnickB. G.FidellL. S. (2013). Using multivariate statistics (6th ed.). Pearson.
87.
TavakoliP. (2011). Pausing patterns: Differences between L2 learners and native speakers. ELT Journal, 65(1), 71–79. https://doi.org/10.1093/elt/ccq020
88.
TavakoliP. (2016). Fluency in monologic and dialogic task performance: Challenges in defining and measuring L2 fluency. International Review of Applied Linguistics in Language Teaching, 54(2), 133–150. https://doi.org/10.1515/iral-2016-9994
89.
TavakoliP.HunterA.-M. (2018). Is fluency being “neglected” in the classroom? Teacher understanding of fluency and related classroom practices. Language Teaching Research, 22(3), 330–349. https://doi.org/10.1177/1362168817708462
90.
TavakoliP.NakatsuharaF.HunterA.-M. (2020). Aspects of fluency across assessed levels of speaking proficiency. Modern Language Journal, 104(1), 169–191. https://doi.org/10.1111/modl.12620
91.
ThewissenJ. (2015). Accuracy across proficiency levels: A learner corpus approach. Presses universitaires de Louvain.
TonoY. (2007). The roles of oral L2 learner corpora in language teaching: The case of the NICT JLE corpus. In CampoyM. C.LuzónM. J. (Eds.), Spoken corpora in Applied Linguistics (pp. 163–179). Peter Lang.
94.
TottieG.SvalduzS. (2009). Er/uh, erm/um: Planners in varieties and registers of English [Conference session]. Paper presented at the 30th Annual Conference of the International Computer Archive of Modern and Medieval English (ICAME 30), Lancaster, UK.
95.
WisniewskiK. (2017). Empirical learner language and the levels of the Common European Framework of Reference. Language Learning, 67, 232–253. https://doi.org/10.1111/lang.12223
96.
WisniewskiK. (2018). The empirical validity of the Common European Framework of Reference Scales: An exemplary study for the vocabulary and fluency scales in a language testing context. Applied Linguistics, 39(6), 933–959. https://doi.org/10.1093/applin/amw057
97.
Witton-DaviesG. (2012, November 9–11). The variability of fluency in dialogue and monologue. Paper presented at the Twenty-First International Symposium on English Teaching, Taipei, Taiwan.
98.
Wolfe-QuinteroK.InagakiS.KimH.-Y. (1998). Second language development in writing: Measures of fluency, accuracy, & complexity. University of Hawaii Press.
99.
YanX.KimH. R.KimJ. Y. (2021). Dimensionality of speech fluency: Examining the relationships among complexity, accuracy, and fluency (CAF) features of speaking performances on the Aptis test. Language Testing, 38(4), 485–510. https://doi.org/10.1177/0265532220951508
100.
YangY. (2014). The development of speaking fluency: The 4/3/2 technique for the EFL learners in China. International Journal of Research Studies in Language Learning, 3(4), 55–70. https://doi.org/10.5861/ijrsll.2013.624
