AI chatbot-assisted vocabulary learning: Relationships with self-regulation,motivation,and performance among Chinese private college students

Abstract

This study empirically investigated the application of a large language-model-driven artificial intelligence (AI) chatbot in self-regulated vocabulary acquisition among Chinese private college students. An 8-week experiment comparing AI-chatbot-assisted and conventional self-regulated vocabulary learning demonstrated that AI chatbot assistance could significantly improve vocabulary performance both across and within groups. Results revealed that the use of AI chatbot not only improved learning outcomes directly through personalized, self-paced learning but also indirectly by enhancing vocabulary learning strategy use and satisfying basic psychological needs. Notably, metacognitive strategy use fully mediated the relationship between perceived autonomy and vocabulary performance. Moreover, perceived autonomy moderated the effect of the chatbot intervention on students’ vocabulary learning outcomes. These findings outline the potential of AI chatbots in facilitating vocabulary acquisition and underscore the critical role of perceived autonomy and metacognitive strategy use in AI-chatbot-assisted language education.

Keywords

Large language model artificial intelligence chatbots self-regulated vocabulary learning self-determination theory private college students

Get full access to this article

View all access options for this article.

References

Annamalai

Bervell

Mireku

D.O.

Andoh

R.P.K.

(2025). Artificial intelligence in higher education: Modelling students’ motivation for continuous use of ChatGPT based on a modified self-determination theory. Computers and Education: Artificial Intelligence, 8, Article 100346. https://doi.org/10.1016/j.caeai.2024.100346

Bulut

Cormier

D.C.

(2018). Validity evidence for progress monitoring with star reading: Slope estimates, administration frequency, and number of data points. Frontiers in Education, 3, 68. https://doi.org/10.3389/feduc.2018.00068

Campbell

(2024, September 5). Liang Rubo: The 100 most influential people in AI 2024. Time. Available at: https://time.com/7012773/liang-rubo/ (accessed 12 January 2025).

Cao

(2014). Quality and quality assurance in Chinese private higher education: A multi-dimensional analysis and a proposed framework. Quality Assurance in Education, 22(1), 65–87. https://doi.org/10.1108/QAE-09-2011-0061

Chow

B.W.-Y.

Chiu

H.T.

Wong

S.W.L.

(2018). Anxiety in reading and listening English as a foreign language in Chinese undergraduate students. Language Teaching Research, 22(6), 719–738. https://doi.org/10.1177/1362168817702159

Chang

W.-L.

Sun

J.C.-Y.

(2024). Evaluating AI’s impact on self-regulated language learning: A systematic review. System, 126, Article 103484. https://doi.org/10.1016/j.system.2024.103484

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.

Dan

Bai

Huang

(2024). Gender differences in the relations between EFL students’ classroom relationships and English language proficiency: The mediating role of self-regulated learning strategy use. System, 123, Article 103311. https://doi.org/10.1016/j.system.2024.103311

Deci

E.L.

Ryan

R.M.

(1985). Intrinsic motivation and self-determination in human behavior. Springer Science & Business Media.

10.

Deci

E.L.

Ryan

R.M.

(2000). The ‘what’ and ‘why’ of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11(4), 227–268. https://doi.org/10.1207/S15327965PLI1104_01

11.

Deci

E.L.

Ryan

R.M.

(2004). Handbook of self-determination research. University Rochester Press.

12.

Doran

G.T.

(1981). There’s a S.M.A.R.T. way to write management’s goals and objectives. Management Review, 70(11), 35–36.

13.

Dörnyei

(2005). The psychology of the language learner: Individual differences in second language acquisition (1st ed.). Routledge.

14.

Foster

M.A.

(2017). The effectiveness of high frequency word list instruction on STAR reading test scores. Doctoral dissertation, Liberty University. https://www.proquest.com/dissertations-theses/effectiveness-high-frequency-word-list/docview/1950961564

15.

P.Y.

(2018). Validation of an online questionnaire of vocabulary learning strategies for ESL learners. Studies in Second Language Learning and Teaching, 8(2), 325–350. https://doi.org/10.14746/ssllt.2018.8.2.7

16.

P.Y.

Johnson

R.K.

(1996). Vocabulary learning strategies and language learning outcomes. Language Learning, 46, 643–679. https://doi.org/10.1111/j.1467-1770.1996.tb01355.x

17.

Gupta

Prashar

(2024). Learners’ psychological needs in online learning environment for executive education: Role of cognitive overload and learning self-efficacy. Behaviour & Information Technology, 44(9), 1942–1963. https://doi.org/10.1080/0144929X.2024.2383777

18.

Hair

J.F.

Black

W.C.

Babin

B.J.

Anderson

R.E.

(2010). Multivariate data analysis (7th ed.). Pearson.

19.

Hayes

A.F.

(2017) Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. Guilford Press.

20.

Heil

C.R.

J.S.

Lee

J.J.

Schmidt

(2020). A review of mobile language learning applications: Trends, challenges, and opportunities. The EuroCALL Review, 28(2), 59–76. https://doi.org/10.4995/eurocall.2016.6402

21.

Hwang

G.-J.

Chang

C.-Y.

(2023). A review of opportunities and challenges of chatbots in education. Interactive Learning Environments, 31(7), 4099–4112. https://doi.org/10.1080/10494820.2021.1952615

22.

Jeon

(2023). Chatbot-assisted dynamic assessment (CA-DA) for L2 vocabulary learning and diagnosis. Computer Assisted Language Learning, 36(7), 1338–1364. https://doi.org/10.1080/09588221.2021.1987272

23.

Jeon

(2024). Exploring AI chatbot affordances in the EFL classroom: Young learners’ experiences and perspectives. Computer Assisted Language Learning, 37(1–2), 1–26. https://doi.org/10.1080/09588221.2021.2021241

24.

Kim

J.-S.

Jang

E.-S.

(2020). Artificial intelligence literacy-based teaching conditions and model for improving primary English language learners’ global digital citizenship. Association of Global Studies Education, 12(3), 169–198.

25.

Labadze

Grigolia

Machaidze

(2023). Role of AI chatbots in education: Systematic literature review. International Journal of Educational Technology in Higher Education, 20(1), 56. https://doi.org/10.1186/s41239-023-00426-1

26.

Lee

J.H.

Ahn

J.J.

Lee

(2022). The role of motivation and vocabulary learning strategies in L2 vocabulary knowledge: A structural equation modeling analysis. Studies in Second Language Learning and Teaching, 12(3), 435–458. https://doi.org/10.14746/ssllt.2022.12.3.5

27.

King

R.B.

Wang

(2022). Adaptation and validation of the vocabulary learning motivation questionnaire for Chinese learners: A construct validation approach. System, 108, Article 102853. https://doi.org/10.1016/j.system.2022.102853

28.

Wang

(2024). Mediating roles of motivational beliefs and vocabulary learning strategies for the relationship between self-regulation and vocabulary proficiency. European Journal of Education, 59(4), Article e12706. https://doi.org/10.1111/ejed.12706

29.

Chiu

M.M.

(2024). Self-regulated and collaborative personalised vocabulary learning approach in MALL. Language Learning & Technology, 28(1), 1–27.

30.

MacIntyre

P.D.

Noels

K.A.

(1996). Using social-psychological variables to predict the use of language learning strategies. Foreign Language Annals, 29, 373−386.

31.

Ministry of Education of the People’s Republic of China. (2024). Overview of the development of national education in 2023. Available at: http://www.moe.gov.cn/fbh/live/2024/55831/sfcl/202403/t20240301_1117517.html (accessed 4 January 2025).

32.

Mizumoto

(2010). Exploring the art of vocabulary learning strategies: A closer look at Japanese EFL university students. Kinseido.

33.

Mizumoto

Takeuchi

(2008). Exploring the driving forces behind TOEIC scores: Focusing on vocabulary learning strategies, motivation, and study time. JACET Journal, 46, 17–32.

34.

Mizumoto

Takeuchi

(2009). Examining the effectiveness of explicit instruction of vocabulary learning strategies with Japanese EFL university students. Language Teaching Research, 13(4), 425–449. https://doi.org/10.1177/1362168809341511

35.

Nation

I.S.P.

(2001). Learning vocabulary in another language. Cambridge University Press.

36.

Oxford

R.L.

(2017). Teaching and researching language learning strategies: Self-regulation in context (2nd ed.). Routledge.

37.

Ryan

R.M.

Deci

E.L.

(2020). Intrinsic and extrinsic motivation from a self-determination theory perspective: Definitions, theory, practices, and future directions. Contemporary Educational Psychology, 61, Article 101860. https://doi.org/10.1016/j.cedpsych.2020.101860

38.

Schmitt

(1997). Vocabulary learning strategies. In Schmitt

D. N.

McCarthy

(Eds.), Vocabulary: Description, acquisition and pedagogy (pp. 199–227). Cambridge University Press.

39.

Schmitt

(2010). Researching vocabulary: A vocabulary research manual. Palgrave Macmillan.

40.

Tabachnick

B.G.

Fidell

L.S.

(2019). Using multivariate statistics (7th ed.). Pearson.

41.

Teng

M.F.

Mizumoto

(2024). Validation of metacognitive knowledge in vocabulary learning and its predictive effects on incidental vocabulary learning from reading. International Review of Applied Linguistics in Language Teaching. Advance online publication. https://doi.org/10.1515/iral-2023-0294

42.

Teng

M.F.

Mizumoto

Takeuchi

(2024). Understanding growth mindset, self-regulated vocabulary learning, and vocabulary knowledge. System, 122, Article 103255. https://doi.org/10.1016/j.system.2024.103255

43.

Teng

M.F.

Zhang

(2024). Task-induced involvement load, vocabulary learning in a foreign language, and their association with metacognition. Language Teaching Research, 28(2), 531–555. https://doi.org/10.1177/13621688211008798

44.

Tseng

W.-T.

Dörnyei

Schmitt

(2006). A new approach to assessing strategic learning: The case of self-regulation in vocabulary acquisition. Applied Linguistics, 27(1), 78–102. https://doi.org/10.1093/applin/ami046

45.

Tseng

W.-T.

Schmitt

(2008). Toward a model of motivated vocabulary learning: A structural equation modeling approach. Language Learning, 58(2), 357–400. https://doi.org/10.1111/j.1467-9922.2008.00444.x

46.

Vonkova

Jones

Moore

Altinkalp

Selcuk

(2021). A review of recent research in EFL motivation: Research trends, emerging methodologies, and diversity of researched populations. System, 103, Article 102622. https://doi.org/10.1016/j.system.2021.102622

47.

Walkington

C.A.

(2013). Using adaptive learning technologies to personalize instruction to student interests: The impact of relevant contexts on performance and learning outcomes. Journal of Educational Psychology, 105(4), 932–945. https://doi.org/10.1037/a0031882

48.

Wang

Cheung

A.C.K.

Neitzel

A.J.

Chai

C.S.

(2024). Does chatting with chatbots improve language learning performance? A meta-analysis of chatbot-assisted language learning. Review of Educational Research, 95(4), 623–660. https://doi.org/10.3102/00346543241255621

49.

Wang

Reynolds

B.L.

(2024). Beyond the books: Exploring factors shaping Chinese English learners’ engagement with large language models for vocabulary learning. Education Sciences, 14(5), 496. https://doi.org/10.3390/educsci14050496

50.

Wollny

Schneider

Di Mitri

Weidlich

Rittberger

Drachsler

(2021). Are we there yet? - A systematic literature review on chatbots in education. Frontiers in Artificial Intelligence, 4, Article 654924. https://doi.org/10.3389/frai.2021.654924

51.

Xia

Chiu

T.K.F.

Chai

C.S.

(2023). The moderating effects of gender and need satisfaction on self-regulated learning through artificial intelligence (AI). Education and Information Technologies, 28(7), 8691–8713. https://doi.org/10.1007/s10639-022-11547-x

52.

Xia

Chiu

T.K.F.

Chai

C.S.

Xie

(2023). The mediating effects of needs satisfaction on the relationships between prior knowledge and self-regulated learning through artificial intelligence chatbot. British Journal of Educational Technology, 54(4), 967–986. https://doi.org/10.1111/bjet.13305

53.

Yang

Dai

(2011). Rote memorization of vocabulary and vocabulary development. English Language Teaching, 4, 61–64. https://doi.org/10.5539/elt.v4n4p61

54.

(2021). Do academic characteristics predict Chinese private university students’ success in English language testing? Doctoral dissertation, Chapman University. https://doi.org/10.36837/chapman.000310

55.

Zadorozhnyy

Lee

J.S.

(2024). Linking EFL students’ psychological needs to engagement in informal digital learning of English: A structural equation modeling analysis. Computer Assisted Language Learning, 1–25. https://doi.org/10.1080/09588221.2024.2387269

56.

Zimmerman

B.J.

(2002). Becoming a self-regulated learner: An overview. Theory Into Practice, 41(2), 64–70. https://doi.org/10.1207/s15430421tip4102_2

57.

Zimmerman

B.J.

(2008). Investigating self-regulation and motivation: Historical background, methodological developments, and future prospects. American Educational Research Journal, 45(1), 166–183. https://doi.org/10.3102/0002831207312909

58.

Zhang

Huang

(2024). The impact of chatbots based on large language models on second language vocabulary acquisition. Heliyon, 10(3), Article e25370. https://doi.org/10.1016/j.heliyon.2024.e25370

59.

Zhang

Shan

Lee

J.S.Y.

Che

Kim

J.H.

(2023). Effect of chatbot-assisted language learning: A meta-analysis. Education and Information Technologies, 28(11), 15223–15243. https://doi.org/10.1007/s10639-023-11805-6

60.

Zhang

(2025). The paradox of self-efficacy and technological dependence: Unraveling generative AI’s impact on university students’ task completion. The Internet and Higher Education, 65, Article 100978. https://doi.org/10.1016/j.iheduc.2024.100978

61.

Zhang

H.M.

Zhuang

(2017). Multi-faceted morphological awareness and vocabulary knowledge in English as a second language learners: A multivariate analysis. Chinese Journal of Applied Linguistics, 40(1), 42–55. https://doi.org/10.1515/cjal-2017-0003