Restricted accessResearch articleFirst published online 2026-4
Evaluating the Longitudinal Effects of AI-Enhanced Collaborative Dialogue Modes on Computational Thinking and Language Proficiency in EFL Learners: A Mixed-Methods Approach
This mixed-methods study compared the effectiveness of three AI-powered collaborative dialogue approaches—guided feedback (GF), interactive Q&A (IQ), and inquiry-driven scaffolding (IS)—on the computational thinking (CT) and language proficiency of 456 EFL learners. The triple-blind, randomized controlled trial was based on sociocultural theory and modern language learning models. We used advanced statistical growth modeling, analyzed conversation networks, and reviewed AI interaction logs. Participants (CEFR B1–B2; Mage = 25.25) were divided into four groups (GF, IQ, IS, and control), with initial similarities confirmed through careful statistical checks. Results showed that the guided feedback group made the largest and most lasting gains in CT and language, significantly outperforming the control group and retaining skills over time. The IQ and IS groups improved moderately but had more difficulty retaining skills and applying them creatively. Qualitative analysis highlighted that GF helped learners monitor their thinking and correct mistakes, while the other AI methods led to higher mental effort and over-reliance on technology. The findings support AI designs that balance structured feedback with learner control; however, as this study was conducted exclusively with Chinese EFL learners, we strongly recommend cross-cultural replications to validate these findings and ensure fair, effective implementation beyond this specific cultural context.
AlharbiW. (2023). AI in the foreign language classroom: A pedagogical overview of automated writing assistance tools. Education Research International, 2023(1), 15. https://doi.org/10.1155/2023/4253331
2.
AlshumaimeriY. A.AlshememryA. K. (2023). The extent of AI applications in EFL learning and teaching. IEEE Transactions on Learning Technologies. https://doi.org/10.1109/TLT.2023.3322128
3.
AtkinsonR. C.ShiffrinR. M. (1968). Human memory: A proposed system and its control processes. In SpenceK. W.SpenceJ. T. (Eds.), The psychology of learning and motivation (pp. 89–195). Academic Press.
4.
BadawyH. R.AlkaabiA. M.QablanA. M.BadawyH. R.AlmaazmiK. M. M. H.NdlelaN. P. (2025). Nurturing future innovators integrating computational thinking, AI, and STEM in early education. In Empowering early education with computational thinking, AI, and STEM (pp. 97–134). IGI Global. https://doi.org/10.4018/979836936210-5.ch004
5.
BahariA. (2019). FonF practice model from theory to practice: CALL via focus on form approach and nonlinear dynamic motivation to develop listening and speaking proficiency. Computers & Education, 130(3), 40–58. https://doi.org/10.1016/j.compedu.2018.11.009
6.
BanduraA. (2006). Guide for constructing self-efficacy scales. In PajaresF.UrdanT. (Eds.), Self-efficacy beliefs of adolescents (pp. 307–337). Information Age Publishing.
7.
Barrios-BeltranD. (2024). Enriching the teaching-learning experience by using AI tools in the L2 classroom. In AI in language teaching, learning, and assessment (pp. 61–77). IGI Global. https://doi.org/10.4018/979-8-3693-0872-1.ch004
ChangY.LeeS.WongS. F.JeongS. P. (2022). AI-powered learning application use and gratification: An integrative model. Information Technology & People, 35(7), 2115–2139. https://doi.org/10.1108/ITP-09-2020-0632
10.
CohenJ. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
11.
CrumS.LiB.KouX. (2024). Generative artificial intelligence and interactive learning platforms: Second language vocabulary acquisition. In International conference on human-computer interaction (pp. 48–53). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-61953-3_6
12.
EbadiS.AminiA. (2024). Examining the roles of social presence and human-likeness on Iranian EFL learners’ motivation using artificial intelligence technology: A case of CSIEC chatbot. Interactive Learning Environments, 32(2), 655–673. https://doi.org/10.1080/10494820.2022.2096638
13.
FathiJ.RahimiM. (2024). Utilising artificial intelligence-enhanced writing mediation to develop academic writing skills in EFL learners: A qualitative study. Computer Assisted Language Learning, 1–40. https://doi.org/10.1080/09588221.2024.2374772
14.
FaulF.ErdfelderE.LangA. G.BuchnerA. (2007). G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods, 39(2), 175–191. https://doi.org/10.3758/bf03193146
15.
FengH.LiK.ZhangL. J. (2025). What does AI bring to second language writing? A systematic review (2014-2024). Language, Learning and Technology, 29(1). 1–27. https://doi.org/10.64152/10125/73629
16.
GohC. C. M.AryadoustV. (2025). Developing and assessing second language listening and speaking: Does AI make it better?Annual Review of Applied Linguistics, 45, 179–199. https://doi.org/10.1017/S0267190525100111
17.
HaoX.DemirE.EyersD. (2024). Exploring collaborative decision-making: A quasi-experimental study of human and generative AI interaction. Technology in Society, 78, Article 102662. https://doi.org/10.1016/j.techsoc.2024.102662
18.
HsuT. C.ChangC.JenT. H. (2023). Artificial intelligence image recognition using self-regulation learning strategies: Effects on vocabulary acquisition, learning anxiety, and learning behaviours of English language learners. Interactive Learning Environments, 32(6), 3060–3078. https://doi.org/10.1080/10494820.2023.2165508
19.
HuX.HeW.ChiuT. K.ZhaoL. (2023). Using a teacher scheme for educational dialogue analysis to investigate student–student interaction patterns for optimal group activities in an artificial intelligence course. Education and Information Technologies, 28(7), 8789–8813. https://doi.org/10.1007/s10639-022-11556-w
20.
HuangX.QiaoC. (2024). Enhancing computational thinking skills through artificial intelligence education at a STEAM high school. Science & Education, 33(2), 383–403. https://doi.org/10.1007/s11191-022-00392-6
21.
HuangX.ZouD.ChengG.ChenX.XieH. (2023). Trends, research issues and applications of artificial intelligence in language education. Educational Technology & Society, 26(1), 112–131. https://doi.org/10.30191/ETS.202301_26(1).0009
22.
KimE.DinçE. (2025). ESL teachers’ experiences while designing projects with block-based coding: Language instruction and computational thinking. Journal of Educational Technology Systems, 53(3), 170–204. https://doi.org/10.1177/00472395241312652
23.
KirkpatrickJ.KirkpatrickW. (2016). Kirkpatrick’s Four Levels of Training Evaluation. ATD Press.
24.
KorkmazÖ.ÇakirR.ÖzdenM. Y. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior, 72, 558–569. https://doi.org/10.1016/j.chb.2017.01.005
25.
LakensD. (2013). Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in psychology, 4, 863. https://doi.org/10.3389/fpsyg.2013.00863
26.
LiD.ZhaoY. (2025). Artificial intelligence applications for oral communication skills in EFL contexts: A systematic review. The Asia-Pacific Education Researcher. Advance Online Publication. https://doi.org/10.1007/s40299-025-01023-8
27.
LiaoJ.ZhongL.ZheL.XuH.LiuM.XieT. (2024). Scaffolding computational thinking with ChatGPT. IEEE Transactions on Learning Technologies. https://doi.org/10.1109/TLT.2024.3392896
28.
ManireE.KilagO. K.Cordova JrN.TanS. J.PoligratesJ.OmañaE. (2023). Artificial intelligence and English language learning: A systematic review. Excellencia: International Multi-disciplinary Journal of Education, 1(5), 485–497.
29.
PapertS. (1980). Mindstorms: Children, computers, and powerful ideas. Basic Books.
30.
QinZ.ChuaychoowongM. (2025). Artificial intelligence in EFL education in China: A systematic review of trends, gaps, and future directions (2015-2024). Language Teaching Research Quarterly, 49, 59–89. https://doi.org/10.32038/ltrq.2025.49.04
31.
RaoT. S. S.BhagatK. K. (2024). Computational thinking for the digital age: A systematic review of tools, pedagogical strategies, and assessment practices. Educational Technology Research & Development, 72(4), 1893–1924. https://doi.org/10.1007/s11423-02410364y
32.
SaldanaJ. (2021). The Coding Manual for Qualitative Researchers. Thousand Oaks, CA: SAGE Publications Limited.
33.
SarnovskaN.RybinskaJ.MykhailichenkoY. (2024). Enhancing university remote language learning through innovative applications of artificial intelligence technologies amidst global challenges. Teaching Languages at Higher Educational Establishments at the Present Stage. Inter Subject Relations, (44), 151–165. https://doi.org/10.26565/2073-4379-2024-44-10
34.
SelbyC.WoollardJ. (2013). Computational thinking: The developing definition. University of Southampton.
35.
Shafiee RadH.RoohaniA. (2024). Fostering L2 learners’ pronunciation and motivation via affordances of artificial intelligence. Computers in the Schools, 1–22. https://doi.org/10.1080/07380569.2024.2330427
36.
ShanshanS.SenG. (2024). Empowering learners with AI‐generated content for programming learning and computational thinking: The lens of extended effective use theory. Journal of Computer Assisted Learning, 40(4), 1941–1958. https://doi.org/10.1111/jcal.12996
37.
ShinY.JungJ.ChoiS.JungB. (2025). The influence of scaffolding for computational thinking on cognitive load and problem-solving skills in collaborative programming. Education and Information Technologies, 30(1), 583–606. https://doi.org/10.1007/s10639-024-13104-0
38.
SwainM. (1985). Communicative competence: Some roles of comprehensible input and comprehensible output in its development. In GassS.MaddenC. (Eds.), Input in second language acquisition (pp. 235–253). Newbury: House.
39.
TanS. C.ChenW.ChuaB. L. (2023). Leveraging generative artificial intelligence based on large language models for collaborative learning. Learning: Research and Practice, 9(2), 125–134. https://doi.org/10.1080/23735082.2023.2258895
40.
TsaiC. Y.LinC. S.ChangY. S.ChangY. S. (2021). Development and validation of a computational thinking scale for computer literacy education. Journal of Educational Computing Research, 59(4), 691–714. https://doi.org/10.1177/0735633120974232
41.
TulasiL.RaoC. S. (2023). Integration of AI-Technologies into ELT: A brief study. Journal for Research Scholars and Professionals of English Language Teaching, 7(38). https://doi.org/10.54850/jrspelt.7.38.003
42.
UmarU. (2024). Advancements in English language teaching: Harnessing the power of artificial intelligence. Foreign Language Instruction Probe, 3(1), 29–42. https://doi.org/10.54213/flip.v3i1.402
43.
VygotskyL. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.
44.
WengX.YeH.DaiY.NgO. L. (2024). Integrating artificial intelligence and computational thinking in educational contexts: A systematic review of instructional design and student learning outcomes. Journal of Educational Computing Research, 62(6), 1420–1450. https://doi.org/10.1177/07356331241248686
WuT. T.AsmaraA.HuangY. M.Permata HapsariI. (2024a). Identification of problem-solving techniques in computational thinking studies: Systematic literature review. Sage Open, 14(2), Article 21582440241249897. https://doi.org/10.1177/21582440241249897
47.
WuT. T.SilitongaL. M.MurtiA. T. (2024b). Enhancing English writing and higher-order thinking skills through computational thinking. Computers & Education, 213, Article 105012. https://doi.org/10.1016/j.compedu.2024.105012
48.
XiaY.ShinS. Y.KimJ. C. (2024). Cross-cultural intelligent language learning system (cils): Leveraging ai to facilitate language learning strategies in cross-cultural communication. Applied Sciences, 14(13), 5651. https://doi.org/10.3390/app14135651
49.
XueY. (2024). Towards automated writing evaluation: A comprehensive review with bibliometric, scientometric, and meta-analytic approaches. Education and Information Technologies, 29(15), 19553–19594. https://doi.org/10.1007/s10639-024-12596-0
50.
YadavS. (2025). Empowering educators for the artificial intelligence (AI) era: Strategies and challenges in integrating computational thinking into professional development. In Empowering STEM educators with digital tools (pp. 1–28). IGI Global Scientific Publishing. https://doi.org/10.4018/979-8-3693-9806-7.ch001
51.
YanW.NakajimaT.SawadaR. (2024). Benefits and challenges of collaboration between students and conversational generative artificial intelligence in programming learning: An empirical case study. Education Sciences, 14(4), 433. https://doi.org/10.3390/educsci14040433
52.
YuanY. (2023). An empirical study of the efficacy of AI chatbots for English as a foreign language learning in primary education. Interactive Learning Environments, 32(10), 6774–6789. https://doi.org/10.1109/EMBC40787.2023.10340280
53.
ZhaiC.WibowoS. (2023). A systematic review on artificial intelligence dialogue systems for enhancing English as foreign language students’ interactional competence in the university. Computers and Education: Artificial Intelligence, 4, Article 100134. https://doi.org/10.1016/j.caeai.2023.100134
54.
ZhangY.ViriyavejakulC.SumettikoonP. (2023). Integrating chatbots in educational administration for improved language learning outcomes. Eurasian Journal of Educational Research, 104(104), 142–163. https://doi.org/10.14689/ejer.2023.104.009
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
