Speaking task design and complexity and fluency features of adolescent English learners

Abstract

This paper reports the results of a study designed to explore the relationships between speaking test task variables and linguistic features of spoken responses on a speaking assessment for Grade 7 multilingual English learners (age 12–13) in U.S. public schools. Speaking task responses from 30 high-proficiency test takers were transcribed and coded for 14 complexity measures and 9 fluency measures. Subsequent analyses of simple main effects indicate that many of the complexity and fluency measures significantly differ by task variables. We observed more meaningful differences in the complexity measures, but fluency features also varied in the key area of pause time. The findings reveal that tasks designed to elicit extended spoken responses did indeed prompt spoken language with more complex features than tasks designed to elicit shorter, simpler responses across academic disciplines. Results of the effects of task variables on fluency measures indicate that Grade 7 students pause more in response to more complex tasks. The results provide some preliminary validity evidence for the assessment by indicating expected performance differences based on target task difficulty level and have implications for the design and scoring of speaking assessment tasks for young learners.

Keywords

Complexity fluency speaking technology young learners

Get full access to this article

View all access options for this article.

References

Bailey

A. L.

(2017). Progressions of a new language: Characterizing explanation development for assessment with young language learners. Annual Review of Applied Linguistics, 37, 241–263. https://doi.org/10.1017/s0267190517000113

Bailey

A. L.

Heritage

(2014). The role of language learning progressions in improved instruction and assessment of English language learners. TESOL Quarterly, 48(3), 480–506. https://doi.org/10.1002/tesq.176

Bosker

H. R.

Quené

Sanders

De Jong

N. H.

(2014). The perception of fluency in native and nonnative speech. Language Learning, 64(3), 579–614. https://doi.org/10.1111/lang.12067

Coxhead

(2000). A new academic word list. TESOL Quarterly, 34(2), 213–238. https://doi.org/10.2307/3587951

Crowther

(2020). Rating L2 speaker comprehensibility on monologic vs. interactive tasks: What is the effect of speaking task type? Journal of Second Language Pronunciation, 6(1), 96–121. https://doi.org/10.1075/jslp.19019.cro

De Jong

N. H

. (2018). Fluency in second language testing: Insights from different disciplines. Language Assessment Quarterly, 15(3), 237–254. https://doi.org/10.1080/15434303.2018.1477780

Field

(2013). Discovering statistics using IBM SPSS statistics (4th ed.). Sage.

Foster

(2020). Oral fluency in a second language: A research agenda for the next ten years. Language Teaching, 53(4), 446–461. https://doi.org/10.1017/S026144482000018X

Foster

Tonkyn

Wigglesworth

(2000). Measuring spoken language: A unit for all reasons. Applied Linguistics, 21(3), 354–375. https://doi.org/10.1093/applin/21.3.354

10.

Frost

Clothier

Huisman

Wigglesworth

(2020). Responding to a TOEFL iBT integrated speaking task: Mapping task demands and test takers’ use of stimulus content. Language Testing, 37(1), 133–155. https://doi.org/10.1177/0265532219860750

11.

Fulcher

(2015). Re-examining language testing: A philosophical and social inquiry. Routledge.

12.

Gan

(2012). Complexity measures, task type, and analytic evaluations of speaking proficiency in a school-based assessment context. Language Assessment Quarterly, 9(2), 133–151. https://doi.org/10.1080/15434303.2010.516041

13.

Getman

E. P.

(2024). Speaking test tasks for young language learners: Development and validation. In Davis

Norris

J. M.

(Eds.), Challenges and innovations in speaking assessment. Routledge. https://doi.org/10.4324/9781315201504-7

14.

Ginther

Dimova

Yang

(2010). Conceptual and empirical relationships between temporal measures of fluency and oral English proficiency with implications for automated scoring. Language Testing, 27(3), 379–399. https://doi.org/10.1177/0265532210364407

15.

Hsieh

C.-N.

(2019). Distinguishing features of young English language learners’ oral performance. Language Assessment Quarterly, 16(2), 180–195. https://doi.org/10.1080/15434303.2019.1605518

16.

Halliday

M. A. K.

Hassan

(1976). Cohesion in English. Longman.

17.

Heatley

Nation

I. S. P.

Coxhead

(2002). RANGE and FREQUENCY programs. http://www.victoria.ac.nz/lals/staff/paul-nation.aspx

18.

Housen

Kuiken

Vedder

(2012). Complexity, accuracy and fluency: Definitions, measurement and research. In Housen

Kuiken

Vedder

(Eds.), Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (pp. 1–20). John Benjamins.

19.

Hsieh

C.-N.

Wang

(2019). Speaking proficiency of young language students: A discourse-analytic study. Language Testing, 36(1), 27–50. https://doi.org/10.1177/0265532217734240

20.

Iwashita

Brown

McNamara

O’Hagan

(2008). Assessed levels of second language speaking proficiency: How distinct? Applied Linguistics, 29(1), 24–49. https://doi.org/10.1093/applin/amm017

21.

Iwashita

McNamara

Elder

(2001). Can we predict task difficulty in an oral proficiency test? Exploring the potential of an information-processing approach to task design. Language Learning, 51(3), 401–436. https://doi.org/10.1111/0023-8333.00160

22.

Kang

Yan

(2018). Linguistic features distinguishing examinees’ speaking performances at different proficiency levels. Journal of Language Testing & Assessment, 1(1), 24–39. https://doi.org/10.23977/langta.2018.11003

23.

Koizumi

(2012). Relationships between text length and lexical diversity measures: Can we use short texts of less than 100 tokens. Vocabulary Learning and Instruction, 1(1), 60–69. https://doi.org/10.7820/vli.v01.1.koizumi

24.

Koizumi

In’nami

(2024). Predicting functional adequacy from complexity, accuracy, and fluency of second-language picture-prompted speaking. System, 120, 103208. https://doi.org/10.1016/j.system.2023.103208

25.

Koo

T. K.

M. Y.

(2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012

26.

Kyle

Crossley

Berger

(2018). The tool for the automatic analysis of lexical sophistication (TAALES): Version 2.0. Behavior Research Methods, 50(3), 1030–1046.

27.

Kyle

Crossley

S. A.

(2015). Automatically assessing lexical sophistication: Indices, tools, findings, and application. TESOL Quarterly, 49(4), 757–786. https://doi.org/10.1002/tesq.194

28.

Lee

Winke

(2018). Young learners’ response processes when taking computerized tasks for speaking assessment. Language Testing, 35(2), 239–269. https://doi.org/10.1177/0265532217704009

29.

Lennon

(1990). Investigating fluency in EFL: A quantitative approach. Language Learning, 40(3), 387–417. https://doi.org/10.1111/j.1467-1770.1990.tb00669.x

30.

(2012). The relationship of lexical richness to the quality of ESL learners’ oral narratives. The Modern Language Journal, 96(2), 190–208. https://doi.org/10.1111/j.15404781.2011.01232_1.x

31.

Nakatsuhara

Berry

(2021). Use of innovative technology in oral language assessment. Assessment in Education: Principles, Policy & Practice, 28(4), 343–349. https://doi.org/10.1080/0969594X.2021.2004530

32.

Nikolov

Timpe-Laughlin

(2021). Assessing young learners’ foreign language abilities. Language Teaching, 54(1), 1–37. https://doi.org/10.1017/S0261444820000294

33.

Norris

J. M.

Ortega

(2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30(4), 555–578. https://doi.org/10.1093/applin/amp044

34.

Ockey

G. J.

Neiriz

(2021). Evaluating technology-mediated second language oral communication assessment delivery models. Assessment in Education: Principles, Policy & Practice, 28(4), 350–368. https://doi.org/10.1080/0969594X.2021.1976106

35.

Pallotti

(2015). A simple view of linguistic complexity. Second Language Research, 31(1), 117–134. https://doi.org/10.1177/0267658314536435

36.

Pallotti

(2020). Measuring complexity, accuracy, and fluency (CAF). In Winke

Brunfaut

(Eds.), The Routledge handbook of second language acquisition and language testing (pp. 201–210). Routledge.

37.

Peltonen

(2024). Fluency revisited. ELT Journal, 78, 489–492. https://doi.org/10.1093/elt/ccad047

38.

Plakans

Gebril

Bilki

(2019). Shaping a score: Complexity, accuracy, and fluency in integrated writing performances. Language Testing, 36(2), 161–179. https://doi.org/10.1177/0265532216669537

39.

Robinson

(2001). Individual differences, cognitive abilities, aptitude complexes and learning conditions in second language acquisition. Second Language Research, 17(4), 368–392.

40.

Robinson

(2007). Task complexity, theory of mind, and intentional reasoning: Effects on L2 speech production, interaction, uptake and perceptions of task difficulty. International Review of Applied Linguistics in Language Teaching, 45(3), 193–213. https://doi.org/10.1515/iral.2007.009

41.

Sample

Michel

(2014). An exploratory study into trade-off effects of complexity, accuracy, and fluency on young learners’ oral task repetition. TESL Canada Journal, 31(8), 23–46. https://doi.org/10.18806/tesl.v31i0.1185

42.

Sasayama

(2016). Is a “complex” task really complex? Validating the assumption of cognitive task complexity. The Modern Language Journal, 100(1), 231–254. https://doi-org/10.1111/modl.12313

43.

Sasayama

Norris

(2022). Designing speaking tasks for different assessment goals: The complex relationship between cognitive task complexity, language performance, and task accomplishment. Journal on Task-Based Language Teaching and Learning, 2(2), 184–217. https://doi.org/10.1075/task.21020.sas

44.

Segalowitz

(2010). Cognitive bases of second language fluency. Routledge. https://doi.org/10.4324/9780203851357

45.

Skehan

(1998). A cognitive approach to language learning. Oxford University Press.

46.

Tavakoli

Kendon

Mazhurnaya

Ziomek

(2023). Assessment of fluency in the Test of English for Educational Purposes. Language Testing, 40(3), 607–629. https://doi.org/10.1177/02655322231151384

47.

Tavakoli

Nakatsuhara

Hunter

A. M.

(2020). Aspects of fluency across assessed levels of speaking proficiency. The Modern Language Journal, 104(1), 169–191. https://doi.org/10.1111/modl.12620

48.

Tavakoli

Skehan

(2005). Strategic planning, task structure, and performance testing. In Ellis

(Ed.), Planning and task performance in a second language (pp. 239–276). John Benjamins.

49.

WIDA. (2012). 2012 Amplification of the English language development standards, Kindergarten–Grade 12. Board of Regents of the University of Wisconsin System. https://wida.wisc.edu/sites/default/files/resource/2012-ELD-Standards.pdf

50.

Wolf

M. K.

(2024). Assessment of young language learners. In Kunnan

A. J.

(Ed.), The Concise companion to language assessment (pp. 312–325). Wiley.

51.

Yuan

Ellis

(2003). The effects of pre-task planning and on-line planning on fluency, complexity and accuracy in L2 monologic oral production. Applied Linguistics, 24(1), 1–27.