The Impact of Subordination Type and Finiteness on Second Language Development in Timed Impromptu Writing: An NLP-Based Analysis Using the Subordination Sophistication Analyzer

Abstract

The use of subordination enables language users to achieve syntactic efficiency by allowing them to connect ideas in temporal/logical relation. Although the importance of subordination has been recognized in previous research on second language (L2) writing, it has been typically assessed with global indices that measure overall ratio of subordination. In order to capture more nuanced patterns in the development of L2 writing, this study measures the sophistication of subordination, considering subordination type (adverbialization, complementization, relativization) and finiteness (finite, nonfinite). Our natural language processing analysis of 6,566 timed impromptu essays using the Subordination Sophistication Analyzer 1.0 showed that higher-proficiency L2 learners used more subordination, and, importantly, their patterns of use differed by subordination type and finiteness. Whereas the amount of adverbialization and relativization increased along with proficiency regardless of finiteness, the use of complementization increased only for nonfinite clauses. The broader impacts of this study for education and assessment are discussed.

Keywords

second language development syntactic complexity subordination type finiteness proficiency natural language processing subordination sophistication analyzer

Get full access to this article

View all access options for this article.

References

Barr

D. J.

Levy

Scheepers

Tily

H. J.

(2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68, 255-278. https://doi.org/10.1016/j.jml.2012.11.001

Bates

Maechler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67, 1-48. https://doi.org/10.18637/jss.v067.i01

Biber

(1988). Variation across speech and writing. Cambridge University Press. https://doi.org/10.1017/CBO9780511621024

Biber

Gray

Poonpon

(2011). Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? TESOL Quarterly, 45, 5-35. https://doi.org/10.5054/tq.2011.244483

Bulté

Housen

(2014). Conceptualizing and measuring short-term changes in L2 writing complexity. Journal of Second Language Writing, 26, 42-65. https://doi.org/10.1016/j.jslw.2014.09.005

Chen

Alexopoulou

Tsimpli

(2021). Automatic extraction of subordinate clauses and its application in second language acquisition research. Behavior Research Methods, 53, 803-817. https://doi.org/10.3758/s13428-020-01456-7

Chujo

Oghigian

(2009). How many words do you need to know to understand TOEIC, TOEFL & EIKEN? An examination of text coverage and high frequency vocabulary. The Journal of Asia TEFL, 6, 121-148.

Council of Europe. (2011). Common European framework of reference for languages: Learning, teaching, assessment. Author.

Crossley

S. A.

Weston

J. L.

McLain Sullivan

S. T.

McNamara

D. S.

(2011). The development of writing proficiency as a function of grade level: A linguistic analysis. Written Communication, 28, 282-311. https://doi.org/10.1177/0741088311410188

10.

Deng

Lei

Liu

(2021). Calling for more consistency, refinement, and critical consideration in the use of syntactic complexity measures for writing. Applied Linguistics, 42, 1021-1028. https://doi.org/10.1093/applin/amz069

11.

Ellis

(2003). Task-based language learning and teaching. Oxford University Press. https://doi.org/10.1017/9781108643689

12.

Fang

Gresser

Cao

Zhang

(2022). Genre and register features of sixth-grade students’ factual writing. Written Communication, 39, 426-461. https://doi.org/10.1177/07410883221085993

13.

Grant

Ginther

(2000). Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing, 9, 123-145. https://doi.org/10.1016/S1060-3743(00)00019-9

14.

Gries

S. T.

(2015). The most under-used statistical method in corpus linguistics: Multi-level (and mixed-effects) models. Corpora, 10, 95-125. https://doi.org/10.3366/cor.2015.0068

15.

Halliday

Hasan

(1985). Language, context, and text: Aspects of language in a social-semiotic perspective. Deakin University Press.

16.

Hardt

Recht

(2023). Patterns, predictions, and actions: A story about machine learning. Princeton University Press. https://doi.org/10.48550/arXiv.2102.05242

17.

Hawkins

J. A.

Filipović

(2012). Criterial features in L2 English: Specifying the reference levels of the Common European Framework (Vol. 1). Cambridge University Press.

18.

Hayes

J. R.

Hatch

J. A.

Silk

C. M.

(2000). Does holistic assessment predict writing performance? Estimating the consistency of student performance on holistically scored writing assignments. Written Communication, 17, 3-26. https://doi.org/10.1177/074108830001700100

19.

Honnibal

Montani

(2019). spaCy 2: Natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing. Version 2.0.18. https://spacy.io

20.

Hunt

K. W.

(1966). Recent measures in syntactic development. Elementary English, 43, 732-739.

21.

Hwang

Kim

(2023). Automatic analysis of constructional diversity as a predictor of EFL students’ writing proficiency. Applied Linguistics, 44, 127-147. https://doi.org/10.1093/applin/amac046

22.

Jiang

Liu

(2019). Syntactic complexity development in the writings of EFL learners: Insights from a dependency syntactically-annotated corpus. Journal of Second Language Writing, 46, 100666. https://doi.org/10.1016/j.jslw.2019.100666

23.

C. W.

(2021). Exploring general versus academic English proficiency as predictors of adolescent EFL essay writing. Written Communication, 38, 208-246. https://doi.org/10.1177/0741088320986364

24.

Kyle

(2021). Natural language processing for learner corpus research. International Journal of Learner Corpus Research, 7, 1-16. https://doi.org/10.1075/ijlcr.00019.int

25.

Kyle

Crossley

S. A.

(2017). Assessing syntactic sophistication in L2 writing: A usage-based approach. Language Testing, 34, 513-535. https://doi.org/10.1177/0265532217712554

26.

Lambert

Kormos

(2014). Complexity, accuracy, and fluency in task-based L2 research: Toward more developmentally based measures of second language acquisition. Applied Linguistics, 35, 607-614. https://doi.org/10.1093/applin/amu047

27.

Lei

Wen

Yang

(2023). A large-scale longitudinal study of syntactic complexity development in EFL writing: A mixed-effects model approach. Journal of Second Language Writing, 59, 100962. https://doi.org/10.1016/j.jslw.2022.100962

28.

Lorenzo

Rodríguez

(2014). Onset and expansion of L2 cognitive academic language proficiency in bilingual settings: CALP in CLIL. System, 47, 64-72. https://doi.org/10.1016/j.system.2014.09.016

29.

(2011). A corpus-based evaluation of syntactic complexity measures as indices of college- level ESL writers’ language development. TESOL Quarterly, 45, 36-62. https://doi.org/10.5054/tq.2011.240859

30.

Martin

J. R.

(2001). Language, register and genre. In Christie

(Ed.), Children writing: A reader (pp. 21-29). Deakin University Press.

31.

Messina

C. M.

Jones

C. E.

Poe

(2023). Prompting reflection: Using corpus linguistic methods in the local assessment of reflective writing. Written Communication, 40, 620-650. https://doi.org/10.1177/07410883221149425

32.

Ministry of Education. (2015). The English national curriculum. Author.

33.

Norris

J. M.

Ortega

(2009). Towards an organic approach to investigating CAF in SLA: The case of complexity. Applied Linguistics, 30, 555-578. https://doi.org/10.1093/applin/amp044

34.

R Core Team. (2023). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.r-project.org/

35.

Rhee

S. C.

Jung

C. K.

(2012, March). Yonsei English Learner Corpus (YELC) [Paper presentation]. First Yonsei Corpus Symposium, Seoul, Korea.

36.

Robitzsch

(2020). Why ordinal variables can (almost) always be treated as continuous variables: Clarifying assumptions of robust continuous and ordinal factor analysis estimation methods. Frontiers in Education, 5, 589965. https://doi.org/10.3389/feduc.2020.589965

37.

Sahu

P. K.

(2013). Research methodology: A guide for researchers in agricultural science, social science and other related fields. Springer Science & Business Media.

38.

Sung

M. C.

(2020). Underuse of English verb-particle constructions in an L2 learner corpus: Focus on structural patterns and one-word preference. Corpus Linguistics and Linguistic Theory, 16, 189-214. https://doi.org/10.1515/cllt-2017-0002

39.

Wang

Troia

G. A.

(2023). Writing quality predictive modeling: Integrating register-related factors. Written Communication, 40(4), 1070-1112. https://doi.org/10.1177/074108832311852

40.

Xia

V. Y.

White

Guzzo

N. B.

(2022). Intervention in relative clauses: Effects of relativized minimality on L2 representation and processing. Second Language Research, 38, 347-372. https://doi.org/10.1177/0267658320958742