Evaluating the Longitudinal Effects of AI-Enhanced Collaborative Dialogue Modes on Computational Thinking and Language Proficiency in EFL Learners: A Mixed-Methods Approach

Abstract

This mixed-methods study compared the effectiveness of three AI-powered collaborative dialogue approaches—guided feedback (GF), interactive Q&A (IQ), and inquiry-driven scaffolding (IS)—on the computational thinking (CT) and language proficiency of 456 EFL learners. The triple-blind, randomized controlled trial was based on sociocultural theory and modern language learning models. We used advanced statistical growth modeling, analyzed conversation networks, and reviewed AI interaction logs. Participants (CEFR B1–B2; M_age = 25.25) were divided into four groups (GF, IQ, IS, and control), with initial similarities confirmed through careful statistical checks. Results showed that the guided feedback group made the largest and most lasting gains in CT and language, significantly outperforming the control group and retaining skills over time. The IQ and IS groups improved moderately but had more difficulty retaining skills and applying them creatively. Qualitative analysis highlighted that GF helped learners monitor their thinking and correct mistakes, while the other AI methods led to higher mental effort and over-reliance on technology. The findings support AI designs that balance structured feedback with learner control; however, as this study was conducted exclusively with Chinese EFL learners, we strongly recommend cross-cultural replications to validate these findings and ensure fair, effective implementation beyond this specific cultural context.

Keywords

AI-enhanced feedback mixed methods RCT EFL learners collaborative dialogue computational thinking language proficiency

Get full access to this article

View all access options for this article.

References

Alharbi

(2023). AI in the foreign language classroom: A pedagogical overview of automated writing assistance tools. Education Research International, 2023(1), 15. https://doi.org/10.1155/2023/4253331

Alshumaimeri

Y. A.

Alshememry

A. K.

(2023). The extent of AI applications in EFL learning and teaching. IEEE Transactions on Learning Technologies. https://doi.org/10.1109/TLT.2023.3322128

Atkinson

R. C.

Shiffrin

R. M.

(1968). Human memory: A proposed system and its control processes. In Spence

K. W.

Spence

J. T.

(Eds.), The psychology of learning and motivation (pp. 89–195). Academic Press.

Badawy

H. R.

Alkaabi

A. M.

Qablan

A. M.

Badawy

H. R.

Almaazmi

K. M. M. H.

Ndlela

N. P.

(2025). Nurturing future innovators integrating computational thinking, AI, and STEM in early education. In Empowering early education with computational thinking, AI, and STEM (pp. 97–134). IGI Global. https://doi.org/10.4018/979836936210-5.ch004

Bahari

(2019). FonF practice model from theory to practice: CALL via focus on form approach and nonlinear dynamic motivation to develop listening and speaking proficiency. Computers & Education, 130(3), 40–58. https://doi.org/10.1016/j.compedu.2018.11.009

Bandura

(2006). Guide for constructing self-efficacy scales . In Pajares

Urdan

(Eds.), Self-efficacy beliefs of adolescents (pp. 307–337). Information Age Publishing.

Barrios-Beltran

(2024). Enriching the teaching-learning experience by using AI tools in the L2 classroom. In AI in language teaching, learning, and assessment (pp. 61–77). IGI Global. https://doi.org/10.4018/979-8-3693-0872-1.ch004

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Chang

Lee

Wong

S. F.

Jeong

S. P.

(2022). AI-powered learning application use and gratification: An integrative model. Information Technology & People, 35(7), 2115–2139. https://doi.org/10.1108/ITP-09-2020-0632

10.

Cohen

(1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.

11.

Crum

Kou

(2024). Generative artificial intelligence and interactive learning platforms: Second language vocabulary acquisition. In International conference on human-computer interaction (pp. 48–53). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-61953-3_6

12.

Ebadi

Amini

(2024). Examining the roles of social presence and human-likeness on Iranian EFL learners’ motivation using artificial intelligence technology: A case of CSIEC chatbot. Interactive Learning Environments, 32(2), 655–673. https://doi.org/10.1080/10494820.2022.2096638

13.

Fathi

Rahimi

(2024). Utilising artificial intelligence-enhanced writing mediation to develop academic writing skills in EFL learners: A qualitative study. Computer Assisted Language Learning, 1–40. https://doi.org/10.1080/09588221.2024.2374772

14.

Faul

Erdfelder

Lang

A. G.

Buchner

(2007). G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods, 39(2), 175–191. https://doi.org/10.3758/bf03193146

15.

Feng

Zhang

L. J.

(2025). What does AI bring to second language writing? A systematic review (2014-2024). Language, Learning and Technology, 29(1). 1–27. https://doi.org/10.64152/10125/73629

16.

Goh

C. C. M.

Aryadoust

(2025). Developing and assessing second language listening and speaking: Does AI make it better? Annual Review of Applied Linguistics, 45, 179–199. https://doi.org/10.1017/S0267190525100111

17.

Hao

Demir

Eyers

(2024). Exploring collaborative decision-making: A quasi-experimental study of human and generative AI interaction. Technology in Society, 78, Article 102662. https://doi.org/10.1016/j.techsoc.2024.102662

18.

Hsu

T. C.

Chang

Jen

T. H.

(2023). Artificial intelligence image recognition using self-regulation learning strategies: Effects on vocabulary acquisition, learning anxiety, and learning behaviours of English language learners. Interactive Learning Environments, 32(6), 3060–3078. https://doi.org/10.1080/10494820.2023.2165508

19.

Chiu

T. K.

Zhao

(2023). Using a teacher scheme for educational dialogue analysis to investigate student–student interaction patterns for optimal group activities in an artificial intelligence course. Education and Information Technologies, 28(7), 8789–8813. https://doi.org/10.1007/s10639-022-11556-w

20.

Huang

Qiao

(2024). Enhancing computational thinking skills through artificial intelligence education at a STEAM high school. Science & Education, 33(2), 383–403. https://doi.org/10.1007/s11191-022-00392-6

21.

Huang

Zou

Cheng

Chen

Xie

(2023). Trends, research issues and applications of artificial intelligence in language education. Educational Technology & Society, 26(1), 112–131. https://doi.org/10.30191/ETS.202301_26(1).0009

22.

Kim

Dinç

(2025). ESL teachers’ experiences while designing projects with block-based coding: Language instruction and computational thinking. Journal of Educational Technology Systems, 53(3), 170–204. https://doi.org/10.1177/00472395241312652

23.

Kirkpatrick

(2016). Kirkpatrick’s Four Levels of Training Evaluation. ATD Press.

24.

Korkmaz

Ö.

Çakir

Özden

M. Y.

(2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior, 72, 558–569. https://doi.org/10.1016/j.chb.2017.01.005

25.

Lakens

(2013). Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in psychology, 4, 863. https://doi.org/10.3389/fpsyg.2013.00863

26.

Zhao

(2025). Artificial intelligence applications for oral communication skills in EFL contexts: A systematic review. The Asia-Pacific Education Researcher. Advance Online Publication. https://doi.org/10.1007/s40299-025-01023-8

27.

Liao

Zhong

Zhe

Liu

Xie

(2024). Scaffolding computational thinking with ChatGPT. IEEE Transactions on Learning Technologies. https://doi.org/10.1109/TLT.2024.3392896

28.

Manire

Kilag

O. K.

Cordova Jr

Tan

S. J.

Poligrates

Omaña

(2023). Artificial intelligence and English language learning: A systematic review. Excellencia: International Multi-disciplinary Journal of Education, 1(5), 485–497.

29.

Papert

(1980). Mindstorms: Children, computers, and powerful ideas. Basic Books.

30.

Qin

Chuaychoowong

(2025). Artificial intelligence in EFL education in China: A systematic review of trends, gaps, and future directions (2015-2024). Language Teaching Research Quarterly, 49, 59–89. https://doi.org/10.32038/ltrq.2025.49.04

31.

Rao

T. S. S.

Bhagat

K. K.

(2024). Computational thinking for the digital age: A systematic review of tools, pedagogical strategies, and assessment practices. Educational Technology Research & Development, 72(4), 1893–1924. https://doi.org/10.1007/s11423-02410364y

32.

Saldana

(2021). The Coding Manual for Qualitative Researchers. Thousand Oaks, CA: SAGE Publications Limited.

33.

Sarnovska

Rybinska

Mykhailichenko

(2024). Enhancing university remote language learning through innovative applications of artificial intelligence technologies amidst global challenges. Teaching Languages at Higher Educational Establishments at the Present Stage. Inter Subject Relations, (44), 151–165. https://doi.org/10.26565/2073-4379-2024-44-10

34.

Selby

Woollard

(2013). Computational thinking: The developing definition. University of Southampton.

35.

Shafiee Rad

Roohani

(2024). Fostering L2 learners’ pronunciation and motivation via affordances of artificial intelligence. Computers in the Schools, 1–22. https://doi.org/10.1080/07380569.2024.2330427

36.

Shanshan

Sen

(2024). Empowering learners with AI‐generated content for programming learning and computational thinking: The lens of extended effective use theory. Journal of Computer Assisted Learning, 40(4), 1941–1958. https://doi.org/10.1111/jcal.12996

37.

Shin

Jung

Choi

Jung

(2025). The influence of scaffolding for computational thinking on cognitive load and problem-solving skills in collaborative programming. Education and Information Technologies, 30(1), 583–606. https://doi.org/10.1007/s10639-024-13104-0

38.

Swain

(1985). Communicative competence: Some roles of comprehensible input and comprehensible output in its development . In Gass

Madden

(Eds.), Input in second language acquisition (pp. 235–253). Newbury: House.

39.

Tan

S. C.

Chen

Chua

B. L.

(2023). Leveraging generative artificial intelligence based on large language models for collaborative learning. Learning: Research and Practice, 9(2), 125–134. https://doi.org/10.1080/23735082.2023.2258895

40.

Tsai

C. Y.

Lin

C. S.

Chang

Y. S.

Chang

Y. S.

(2021). Development and validation of a computational thinking scale for computer literacy education. Journal of Educational Computing Research, 59(4), 691–714. https://doi.org/10.1177/0735633120974232

41.

Tulasi

Rao

C. S.

(2023). Integration of AI-Technologies into ELT: A brief study. Journal for Research Scholars and Professionals of English Language Teaching, 7(38). https://doi.org/10.54850/jrspelt.7.38.003

42.

Umar

(2024). Advancements in English language teaching: Harnessing the power of artificial intelligence. Foreign Language Instruction Probe, 3(1), 29–42. https://doi.org/10.54213/flip.v3i1.402

43.

Vygotsky

L. S.

(1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.

44.

Weng

Dai

O. L.

(2024). Integrating artificial intelligence and computational thinking in educational contexts: A systematic review of instructional design and student learning outcomes. Journal of Educational Computing Research, 62(6), 1420–1450. https://doi.org/10.1177/07356331241248686

45.

Wing

J. M.

(2006). Computational thinking. Communications of the ACM, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215

46.

T. T.

Asmara

Huang

Y. M.

Permata Hapsari

(2024a). Identification of problem-solving techniques in computational thinking studies: Systematic literature review. Sage Open, 14(2), Article 21582440241249897. https://doi.org/10.1177/21582440241249897

47.

T. T.

Silitonga

L. M.

Murti

A. T.

(2024b). Enhancing English writing and higher-order thinking skills through computational thinking. Computers & Education, 213, Article 105012. https://doi.org/10.1016/j.compedu.2024.105012

48.

Xia

Shin

S. Y.

Kim

J. C.

(2024). Cross-cultural intelligent language learning system (cils): Leveraging ai to facilitate language learning strategies in cross-cultural communication. Applied Sciences, 14(13), 5651. https://doi.org/10.3390/app14135651

49.

Xue

(2024). Towards automated writing evaluation: A comprehensive review with bibliometric, scientometric, and meta-analytic approaches. Education and Information Technologies, 29(15), 19553–19594. https://doi.org/10.1007/s10639-024-12596-0

50.

Yadav

(2025). Empowering educators for the artificial intelligence (AI) era: Strategies and challenges in integrating computational thinking into professional development. In Empowering STEM educators with digital tools (pp. 1–28). IGI Global Scientific Publishing. https://doi.org/10.4018/979-8-3693-9806-7.ch001

51.

Yan

Nakajima

Sawada

(2024). Benefits and challenges of collaboration between students and conversational generative artificial intelligence in programming learning: An empirical case study. Education Sciences, 14(4), 433. https://doi.org/10.3390/educsci14040433

52.

Yuan

(2023). An empirical study of the efficacy of AI chatbots for English as a foreign language learning in primary education. Interactive Learning Environments, 32(10), 6774–6789. https://doi.org/10.1109/EMBC40787.2023.10340280

53.

Zhai

Wibowo

(2023). A systematic review on artificial intelligence dialogue systems for enhancing English as foreign language students’ interactional competence in the university. Computers and Education: Artificial Intelligence, 4, Article 100134. https://doi.org/10.1016/j.caeai.2023.100134

54.

Zhang

Viriyavejakul

Sumettikoon

(2023). Integrating chatbots in educational administration for improved language learning outcomes. Eurasian Journal of Educational Research, 104(104), 142–163. https://doi.org/10.14689/ejer.2023.104.009

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.56 MB