Effects of Generative Artificial Intelligence on K-12 and Higher Education Students’ Learning Outcomes: A Meta-Analysis

Abstract

Generative artificial intelligence (GenAI) has significant potential for educational innovation, although its impact on students’ learning outcomes remains controversial. This study aimed to examine the impact of GenAI on the learning outcomes of K-12 and higher education students, and explore the moderating factors influencing this impact. A meta-analysis of 49 articles showed that the mean effect sizes of GenAI on students’ learning achievement and learning motivation were 0.857 and 0.803, respectively, indicating a positive impact of GenAI on education. However, this effect varied according to moderators, including education level, subject classification, GenAI interface, GenAI development, interaction approaches, and experimentation time, which enhanced the impact of GenAI on education. Specifically, GenAI had a greater impact on the academic performance of higher education students, and students interacted more effectively with GenAI using text than with mixed media, such as images or audio. Although GenAI has a novel effect on students’ learning motivation, the effect size decreases over time. These findings provide empirical support for the beneficial effects of GenAI on education and offer insights for optimizing its use in teaching practices.

Keywords

generative AI learning outcomes learning achievement learning motivation meta-analysis

Get full access to this article

View all access options for this article.

References

Abdul-Kader

S. A.

Woods

J. C.

(2015). Survey on chatbot design techniques in speech conversation systems. International Journal of Advanced Computer Science and Applications, 6(7), 72–80. https://doi.org/10.14569/ijacsa.2015.060712

Alammari

(2024). Evaluating generative AI integration in Saudi Arabian ed-ucation: A mixed-methods study. PeerJ Computer Science, 10, Article e1879. https://doi.org/10.7717/peerj-cs.1879

Alemdag

(2023). The effect of chatbots on learning: A meta-analysis of em-pirical research. Journal of Research on Technology in Education, 57(2), 459–481. https://doi.org/10.1080/15391523.2023.2255698

Alier

García-Peñalvo

F. J.

Camba

J. D.

(2024). Generative artificial intelligence in education: From deceptive to disruptive. International Journal of Interactive Multimedia and Artificial Intelligence, 8(5), 5–14. https://doi.org/10.9781/ijimai.2024.02.011

Amoozadeh

Daniels

Nam

Kumar

Chen

Hilton

Srinivasa Ragavan

Alipour

M. A.

(2024). Trust in generative AI among students: An exploratory study. In Proceedings of the 55th ACM Technical Symposium on Computer Science Education V.1, Association for Computing Machinery, 67–73. https://doi.org/10.1145/3626252.3630842

* Anyanwu

C. C.

Ononiwu

P. N.

Isiozor

G. N.

(2024). Comparative impact of Whatsapp chatbot technology and Glaser’s teaching approaches on the academic performance of education economics students in tertiary institutions in Nigeria. Education and Information Technologies, 29(17), 23801–23816. https://doi.org/10.1007/s10639-024-12780-2

* Baha

T. A.

Hajji

M. E.

Es-Saady

Fadili

(2023). The impact of educational chatbot on student learning experience. Education and Information Technologies, 29(8), 10153–10176. https://doi.org/10.1007/s10639-023-12166-w

Bandura

(1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. https://doi.org/10.1037//0033-295x.84.2.191

Barnes

M. A.

Raghubar

K. P.

Faulkner

Denton

C. A.

(2014). The construction of visual—relation to reading comprehension. Journal of Experimental Child Psychology, 119, 101–111. https://doi.org/10.1016/j.jecp.2013.10.011

10.

Bhullar

P. S.

Joshi

Chugh

(2024). ChatGPT in higher education—a synthesis of the literature and a future research agenda. Education and Information Technologies, 29(16), 21501–21522. https://doi.org/10.1007/s10639-024-12723-x

11.

Bloom

(1956). Taxonomy of educational objectives: The classification of educational goals. Handbook 1, cognitive domain. Longman.

12.

Boscardin

C. K.

Gin

Golde

P. B.

Hauer

K. E.

(2024). ChatGPT and generative artificial intelligence for medical education: Potential impact and opportunity. Academic Medicine, 99(1), 22–27. https://doi.org/10.1097/acm.0000000000005439

13.

* Boudouaia

Mouas

Kouider

(2024). A study on ChatGPT-4 as an innovative approach to enhancing English as a foreign language writing learning. Journal of Educational Computing Research, 62(6), 1509–1537. https://doi.org/10.1177/07356331241247465

14.

Bower

Torrington

Lai

J. W. M.

Petocz

Alfano

(2024). How should we change teaching and assessment in response to increasingly powerful generative artificial intelligence? Outcomes of the ChatGPT teacher survey. Education and Information Technologies, 29, 15403–15439. https://doi.org/10.1007/s10639-023-12405-0

15.

Carabantes

González-Geraldo

J. L.

Jover

(2023). ChatGPT could be the reviewer of your next scientific paper. Evidence on the limits of AI-assisted academic reviews. Profesional de La Información, 32(5). https://doi.org/10.3145/epi.2023.sep.16

16.

Chan

C. K. Y.

W. J.

(2023). Students’ voices on generative AI: Perceptions, benefits, and challenges in higher education. International Journal of Educational Technology in Higher Education, 20(1), 43. https://doi.org/10.1186/s41239-023-00411-8

17.

* Chang

C.-Y.

Hwang

G.-J.

Gau

M.-L.

(2021). Promoting students’ learning achievement and self‐efficacy: A mobile chatbot approach for nursing training. British Journal of Educational Technology, 53(1), 171–188. https://doi.org/10.1111/bjet.13158

18.

* Chang

C.-Y.

Kuo

S.-Y.

Hwang

G.-H.

(2022). Chatbot-facilitated nursing education: Incorporating a knowledge-based chatbot system into a nursing training program. Educational Technology & Society, 25(1), 15–27. https://www.jstor.org/stable/48647027

19.

Chauhan

(2017). A meta-analysis of the impact of technology on learning effectiveness of elementary students. Computers & Education, 105, 14–30. https://doi.org/10.1016/j.compedu.2016.11.005

20.

Chen

J. A.

Tutwiler

M. S.

Metcalf

S. J.

Kamarainen

Grotzer

Dede

(2016). A multi-user virtual environment to support students’ self-efficacy and interest in science: A latent growth model analysis. Learning and Instruction, 41, 11–22. https://doi.org/10.1016/j.learninstruc.2015.09.007

21.

* Cheng

Croteau

Baral

Heffernan

(2024). Facilitating student learning with a chatbot in an online math learning platform. Journal of Educational Computing Research, 62(4), 907–937. https://doi.org/10.1177/07356331241226592

22.

Chiu

T. K. F.

(2023). The impact of generative AI (GenAI) on practices, policies and research direction in education: A case of ChatGPT and midjourney. Interactive Learning Environments, 32(10), 6187–6203. https://doi.org/10.1080/10494820.2023.2253861

23.

Clark

R. A.

(2002). Learning outcomes: The bottom line. Communication Education, 51(4), 396–404. https://doi.org/10.1080/03634520216531

24.

Cohen

(1990). Statistical power analysis for the behavioral sciences. Computers, Environment and Urban Systems, 14(1), 71. https://doi.org/10.1016/0198-9715(90)90050-4

25.

Cooper

(2023). Examining science education in ChatGPT: An exploratory study of generative artificial intelligence. Journal of Science Education and Technology, 32(3), 444–452. https://doi.org/10.1007/s10956-023-10039-y

26.

Deng

Jiang

Liu

(2024). Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies. Computers & Education, 227, 105224. https://doi.org/10.1016/j.compedu.2024.105224

27.

Deng

(2023). A meta-analysis and systematic review of the effect of chatbot technology use in sustainable education. Sustainability, 15(4), 2940. https://doi.org/10.3390/su15042940

28.

Denny

Prather

Becker

B. A.

Finnie-Ansley

Hellas

Leinonen

Luxton-Reilly

Reeves

B. N.

Santos

E. A.

Sarsa

(2024). Computing education in the era of generative AI. Communications of the ACM, 67(2), 56–67. https://doi.org/10.1145/3624720

29.

Duval

Tweedie

(2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463. https://doi.org/10.1111/j.0006-341x.2000.00455.x

30.

Egger

Smith

G. D.

Schneider

Minder

(1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315(7109), 629–634. https://doi.org/10.1136/bmj.315.7109.629

31.

* Escalante

Pack

Barrett

(2023). AI-generated feedback on writing: Insights into efficacy and ENL student preference. International Journal of Educational Technology in Higher Education, 20(1), 57. https://doi.org/10.1186/s41239-023-00425-2

32.

* Essel

H. B.

Vlachopoulos

Tachie-Menson

Johnson

E. E.

Baah

P. K.

(2022). The impact of a virtual teaching assistant (chatbot) on students’ learning in Ghanaian higher education. International Journal of Educational Technology in Higher Education, 19(1), 57. https://doi.org/10.1186/s41239-022-00362-6

33.

* Essien

Bukoye

O. T.

O’Dea

Kremantzis

(2024). The influence of AI text generators on critical thinking skills in UK business schools. Studies in Higher Education, 49(5), 865–882. https://doi.org/10.1080/03075079.2024.2316881

34.

* Fathi

Rahimi

Derakhshan

(2024). Improving EFL learners’ speaking skills and willingness to communicate via artificial intelligence-mediated interactions. System, 121, 103254. https://doi.org/10.1016/j.system.2024.103254

35.

Fernández-Llorca

Gómez

Sánchez

Mazzini

(2024). An interdisciplinary account of the terminological choices by EU policymakers ahead of the final agreement on the AI act: AI system, general purpose AI system, foundation model, and generative AI. Artificial Intelligence and Law. https://doi.org/10.1007/s10506-024-09412-y

36.

Fryer

L. K.

Ainley

Thompson

Gibson

Sherlock

(2017). Stimulating and sustaining interest in a language course: An experimental comparison of chatbot and human task partners. Computers in Human Behavior, 75, 461–468. https://doi.org/10.1016/j.chb.2017.05.045

37.

* Hakiki

Fadli

Samala

A. D.

Fricticarani

Dayurni

Rahmadani

Astiti

A. D.

Sabir

(2023). Exploring the impact of using Chat-GPT on student learning outcomes in technology learning: The comprehensive experiment. Advances in Mobile Learning Educational Research, 3(2), 859–872. https://doi.org/10.25082/AMLER.2023.02.013

38.

* Han

J.-W.

Park

Lee

(2022). Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: A quasi-experimental study. BMC Medical Education, 22(1), 830. https://doi.org/10.1186/s12909-022-03898-3

39.

Hedges

L. V.

Vevea

J. L.

(1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3(4), 486–504. https://doi.org/10.0987/1082-989x.3.4.486

40.

Higgins

J. P. T.

Thompson

S. G.

(2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558. https://doi.org/10.1002/sim.1186

41.

* Hsu

T.-C.

Huang

H.-L.

Hwang

G.-J.

Chen

M.-S.

(2023). Effects of incorporating an expert decision-making mechanism into chatbots on students’ achievement, enjoyment, and anxiety. Educational Technology & Society, 26(1), 218–231. https://doi.org/10.30191/ETS.202301_26(1).0016

42.

Huang

Hew

K. F.

Fryer

L. K.

(2021). Chatbots for language learning—are they really useful? A systematic review of chatbot‐supported language learning. Journal of Computer Assisted Learning, 38(1), 237–257. https://doi.org/10.1111/jcal.12610

43.

Hwang

G.-J.

Chang

C.-Y.

(2021). A review of opportunities and challenges of chatbots in education. Interactive Learning Environments, 31(7), 4099–4112. https://doi.org/10.1080/10494820.2021.1952615

44.

* Hwang

W.-Y.

Guo

B.-C.

Hoang

Chang

C.-C.

N.-T.

(2022). Facilitating authentic contextual EFL speaking and conversation with smart mechanisms and investigating its influence on learning achievements. Computer Assisted Language Learning, 37(7), 1632–1658. https://doi.org/10.1080/09588221.2022.2095406

45.

* Iku-Silan

Hwang

G.-J.

Chen

C.-H.

(2023). Decision-guided chatbots and cognitive styles in interdisciplinary learning. Computers & Education, 201, 104812. https://doi.org/10.1016/j.compedu.2023.130812

46.

Jauhiainen

J. S.

Guerra

A. G.

(2023). Generative AI and ChatGPT in school children’s education: Evidence from a school lesson. Sustainability, 15(18), 14025. https://doi.org/10.3390/su151814025

47.

Jensen

L. X.

Buhl

Sharma

Bearman

(2024). Generative AI and higher education: A review of claims from the first months of ChatGPT. Higher Education. https://doi.org/10.1007/s10734-024-01265-3

48.

* Jeon

(2021). Chatbot-assisted dynamic assessment (CA-DA) for L2 vocabulary learning and diagnosis. Computer Assisted Language Learning, 36(7), 1338–1364. https://doi.org/10.1080/09588221.2021.1987272

49.

Jeon

Lee

Choi

(2023). A systematic review of research on speech-recognition chatbots for language learning: Implications for future directions in the era of large language models. Interactive Learning Environments, 32(8), 4613–4631. https://doi.org/10.1080/10494820.2023.2204343

50.

* Ji

Zou

Zhan

(2023). The effectiveness of ChatGPT on pre-service teachers’ STEM teaching literacy, learning performance, and cognitive load in a teacher training course. In Proceedings of the 2023 6th International Conference on Educational Technology Management (pp. 16–22). Association for Computing Machinery. https://doi.org/10.1145/3637907.3637948

51.

Kasneci

Sessler

Küchemann

Bannert

Dementieva

Fischer

Gasser

Groh

Günnemann

Hüllermeier

Krusche

Kutyniok

Michaeli

Nerdel

Pfeffer

Poquet

Sailer

Schmidt

Seidel

Stadler

(2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274

52.

Khan

Koubaa

Khan

M. K.

Salleh

R. B.

(2024). Global insights and the impact of generative AI-ChatGPT on multidisciplinary: A systematic review and bibliometric analysis. Connection Science, 36(1). https://doi.org/10.1080/09540091.2024.2353630

53.

Kim

Detrick

(2024). Exploring students’ perspectives on generative AI-assisted academic writing. Education and Information Technologies, 30(1), 1265–1300. https://doi.org/10.1007/s10639-024-12878-7

54.

* Kim

Lee

S.-S.

(2022). Are two heads better than one? The effect of student-AI collaboration on students’ learning task performance. TechTrends, 67(2), 365–375. https://doi.org/10.1007/s11528-022-00788-9

55.

* Kim

(2024). How implementing an AI chatbot impacts Korean as a foreign language learners’ willingness to communicate in Korean. System, 122, 103256. https://doi.org/10.1016/j.system.2024.103256

56.

Koh

Zhang

Lee

A. V. Y.

Wang

(2024). Revolutionizing word clouds for teaching and learning with generative artificial intelligence: Cases from China and Singapore. IEEE Transactions on Learning Technologies, 17, 1390–1401. https://doi.org/10.1109/TLT.2024.3385009

57.

Kohnke

Moorhouse

B. L.

Zou

(2023). ChatGPT for language teaching and learning. RELC Journal, 54(2), 537–550. https://doi.org/10.1177/00336882231162868

58.

Kong

S.-C.

Yang

(2024). A human-centred learning and teaching framework using generative artificial intelligence for self-regulated learning development through domain knowledge learning in K-12 settings (pp. 1588–1599). IEEE Transactions on Learning Technologies. https://doi.org/10.1109/TLT.2024.3392830

59.

* Kosar

Ostojić

Liu

Y. D.

Mernik

(2024). Computer science education in ChatGPT era: Experiences from an experiment in a programming course for novice programmers. Mathematics, 12(5), 629. https://doi.org/10.3390/math12050629

60.

* Kuo

Y.-C.

Chen

Y.-A.

(2022). The impact of chatbots using concept maps on correction outcomes—a case study of programming courses. Education and Information Technologies, 28(7), 7899–7925. https://doi.org/10.1007/s10639-022-11506-6

61.

* Lee

Y.-F.

Hwang

G.-J.

Chen

P.-Y.

(2022). Impacts of an AI-based chabot on college students’ after-class review, academic performance, self-efficacy, learning attitude, and motivation. Educational Technology Research and Development, 70(5), 1843–1865. https://doi.org/10.1007/s11423-022-10142-8

62.

* Lee

Jeon

Choe

(2024). Enhancing pre‐service teachers’ global englishes awareness with technology: A focus on AI chatbots in 3D metaverse environments. TESOL Quarterly, 59(1), 49–74. https://doi.org/10.1002/tesq.3300

63.

Levine

Asada

K. J.

Carpenter

(2009). Sample sizes and effect sizes are negatively correlated in meta-analyses: Evidence and implications of a publication bias against nonsignificant findings. Communication Monographs, 76(3), 286–302. https://doi.org/10.1080/03637750903074685

64.

* Li

H.-F.

(2023). Effects of a ChatGPT-based flipped learning guiding approach on learners’ courseware project performances and perceptions. Australasian Journal of Educational Technology, 39(5), 40–58. https://doi.org/10.14742/ajet.8923

65.

Wang

Luo

Huang

(2024). The influence of GenAI on the effectiveness of argumentative writing in higher education: Evidence from a quasi-experimental study in China. Journal of Asian Public Policy, 1–26. https://doi.org/10.1080/17516234.2024.2363128

66.

Lim

W. M.

Gunasekara

Pallant

J. L.

Pallant

J. I.

Pechenkina

(2023). Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. International Journal of Management Education, 21(2), 100790. https://doi.org/10.1016/j.ijme.2023.100790

67.

Lin

Chu

(2017). Quantifying publication bias in meta-analysis. Biometrics, 74(3), 785–794. https://doi.org/10.1111/biom.12817

68.

* Lin

Ginns

Wang

Zhang

(2020). Using a pedagogical agent to deliver conversational style instruction: What benefits can you obtain? Computers & Education, 143, 103658. https://doi.org/10.1016/j.compedu.2019.103658

69.

* Lin

Y. -T.

J. -H.

(2023). Development of an educational chatbot system for enhancing students’ biology learning performance. Journal of Internet Technology, 24(2), 275–281. https://doi.org/10.53106/160792642023032402006

70.

Liu

C.-C.

Liao

M.-G.

Chang

C.-H.

Lin

H.-M.

(2022). An analysis of children’ interaction with an AI chatbot and its impact on their interest in reading. Computers & Education, 189(7), 104576. https://doi.org/10.1016/j.compedu.2022.104576

71.

Liu

Wang

Liu

Gao

Chen

Cheng

(2024). A bibliometric analysis of generative AI in education: Current status and development. Asia Pacific Journal of Education, 44(1), 156–175. https://doi.org/10.3390/cells13020156

72.

* Liu

C.-C.

Wang

H.-J.

Wang

Y.-F.

Hwang

G.-J.

Wang

(2023). An interactive technological solution to foster preservice teachers’ theoretical knowledge and instructional design skills: A chatbot-based 5E learning approach. Interactive Learning Environments, 32(10), 6698–6716. https://doi.org/10.1080/10494820.2023.2277761

73.

Zheng

Gong

(2024). Supporting teachers’ professional development with generative AI: The effects on higher order thinking and self-efficacy. IEEE Transactions on Learning Technologies, 17, 1267–1277. https://doi.org/10.1109/TLT.2024.3369690

74.

* Mageira

Pittou

Papasalouros

Kotis

Zangogianni

Daradoumis

(2022). Educational AI chatbots for content and language integrated learning. Applied Sciences, 12(7), 3239. https://doi.org/10.3390/app12073239

75.

* Mahapatra

(2024). Impact of ChatGPT on ESL students’ academic writing skills: A mixed methods intervention study. Smart Learning Environments, 11(1), 9. https://doi.org/10.1186/s40561-024-00295-9

76.

Minton

P. D.

Cohen

(1971). Statistical power analysis for the behavioral sciences. Journal of the American Statistical Association, 66(334), 428. https://doi.org/10.2307/2283959

77.

* Mirzababaei

Pammer-Schindler

(2022). Learning to give a complete argument with a conversational agent: An experimental study in two domains of argumentation. In Educating for a New Future: Making Sense of Technology-Enhanced Learning Adoption, (pp. 215–228). Springer. https://doi.org/10.1007/978-3-031-16290-9_16

78.

Moon

Ryu

(2020). The effects of social and cognitive cues on learning comprehension, eye-gaze pattern, and cognitive load in video instruction. Journal of Computing in Higher Education, 33(1), 39–63. https://doi.org/10.1007/s12528-020-09255-x

79.

Moore

B. N.

Parker

(2012). Critical thinking. Mcgraw-Hill.

80.

Moorhouse

B. L.

Kohnke

(2024). The effects of generative AI on initial language teacher education: The perceptions of teacher educators. System, 122, 103290. https://doi.org/10.1016/j.system.2024.103290

81.

* Nguyen

(2023). Role design considerations of conversational agents to facilitate discussion and systems thinking. Computers & Education, 192(3), 104661. https://doi.org/10.1016/j.compedu.2022.104661

82.

* Niloy

A. C.

Akter

Sultana

Rahman

S. I. U.

(2023). Is ChatGPT a menace for creative writing ability? An experiment. Journal of Computer Assisted Learning, 40(2), 919–930. https://doi.org/10.1111/jcal.12929

83.

Ogunleye

Zakariyyah

K. I.

Ajao

Olayinka

Sharma

(2024). A systematic review of generative AI for teaching and learning practice. Education Sciences, 14(6), 636. https://doi.org/10.3390/educsci14060636

84.

OpenAI . (2023). ChatGPT: Optimizing language models for dialogue. https://openai.com/index/chatgpt/ (Accessed Auguest 21, 2024).

85.

* Ortega-Ochoa

Pérez

J. Q.

Arguedas

Daradoumis

Puig

J. M. M.

(2024). The effectiveness of empathic chatbot feedback for developing computer competencies, motivation, self-regulation, and metacognitive reasoning in online higher education. Internet of Things, 25, 101101. https://doi.org/10.1016/j.iot.2024.101101

86.

Page

M. J.

McKenzie

J. E.

Bossuyt

P. M.

Boutron

Hoffmann

T. C.

Mulrow

C. D.

Shamseer

Tetzlaff

J. M.

Akl

E. A.

Brennan

S. E.

Chou

Glanville

Grimshaw

J. M.

Hróbjartsson

Lalu

M. M.

Loder

E. W.

Mayo-Wilson

McDonald

Moher

(2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. International Journal of Surgery, 88, 105906. https://doi.org/10.1016/j.ijsu.2021.105906

87.

* Pellas

(2023). The effects of generative AI platforms on undergraduates’ narrative intelligence and writing self-efficacy. Education Sciences, 13(11), 1155. https://doi.org/10.3390/educsci13111155

88.

* Qureshi

(2023). Exploring the use of chatGPT as a tool for learning and assessment in undergraduate computer science curriculum: Opportunities and challenges. arXiv (Cornell University). https://doi.org/10.48550/arXiv.2304.11214

89.

Radkowitsch

Vogel

Fischer

(2020). Good for learning, bad for motivation? A meta-analysis on the effects of computer-supported collaboration scripts. International Journal of Computer-Supported Collaborative Learning, 15(1), 5–47. https://doi.org/10.1007/s11412-020-09316-4

90.

Raman

Pattnaik

Hughes

Nedungadi

(2024). Unveiling the dynamics of AI applications: A review of reviews using scientometrics and BERTopic modeling. Journal of Innovation & Knowledge, 9(3), 100517. https://doi.org/10.1016/j.jik.2024.100517

91.

Reisman

(2024). Practical classroom use of generative artificial intelligence—a case study. Computer, 57(6), 110–114. https://doi.org/10.1109/MC.2024.3382453

92.

Rosenthal

(1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1016/0002-8703(79)90291-6

93.

* Sadek

Mohamed

N. A. G.

(2023). Artificial intelligence as a pedagogical tool for architectural education: What does the empirical evidence tell us? MSA Engineering Journal, 2(2), 133–148. https://doi.org/10.21608/msaeng.2023.291867

94.

Saihi

Ben-Daya

Hariga

(2024). The moderating role of technology proficiency and academic discipline in AI-chatbot adoption within higher education: Insights from a PLS-SEM analysis. Education and Information Technologies. https://doi.org/10.1007/s10639-024-13023-0

95.

Sáiz-Manzanares

M. C.

Marticorena-Sánchez

Martín-Antón

L. J.

González Díez

Almeida

(2023). Perceived satisfaction of university students with the use of chatbots as a tool for self-regulated learning. Heliyon, 9(1). https://doi.org/10.1016/j.heliyon.2023.e12843

96.

Shi

Tsai

(2022). Fostering vocabulary learning: Mind mapping app enhances performances of EFL learners. Computer Assisted Language Learning, 37(4), 634–686. https://doi.org/10.1080/09588221.2022.2052905

97.

* Shi

S. J.

J. W.

Zhang

(2024). A study on the impact of generative artificial intelligence supported situational interactive teaching on students’ ‘flow’ experience and learning effectiveness—a case study of legal education in China. Asia Pacific Journal of Education, 44(1), 112–138. https://doi.org/10.1080/02188791.2024.2305161

98.

* Silitonga

L. M.

Hawanti

Aziez

Furqon

Zain

D. S. M.

Anjarani

T.-T.

(2023). The impact of AI chatbot-based learning on students’ motivation in English writing classroom. In International conference on innovative technologies and learning (pp. 542–549). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-40113-8_53

99.

* Song

Kim

(2021). Effects of self-regulation scaffolding on online participation and learning outcomes. Journal of Research on Technology in Education, 53(3), 1–15. https://doi.org/10.1080/15391523.2020.1767525

100.

* Song

Song

(2023). Enhancing academic writing skills and motivation: Assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students. Frontiers in Psychology, 14, 1260843. https://doi.org/10.3389/fpsyg.2023.1260843

101.

Summers

Haddad

M. E.

Prichard

Clarke

K. A.

Lee

Oprescu

(2024). Navigating challenges and opportunities: Nursing student’s views on generative AI in higher education. Nurse Education in Practice, 79, 104062. https://doi.org/10.1016/j.nepr.2024.104062

102.

Sun

Zhou

(2024). Does generative artificial intelligence improve the academic achievement of college students? A meta-analysis. Journal of Educational Computing Research, 62(7), 1896–1933. https://doi.org/10.1177/07356331241277937

103.

* Sun

Boudouaia

Zhu

(2024). Would ChatGPT-facilitated programming mode impact college students’ programming behaviors, performances, and perceptions? An empirical study. International Journal of Educational Technology in Higher Education, 21(1), 14. https://doi.org/10.1186/s41239-024-00446-5

104.

Sung

Y.-T.

Yang

J.-M.

Lee

H.-Y.

(2017). The effects of mobile-computer-supported collaborative learning: Meta-analysis and critical synthesis. Review of Educational Research, 87(4), 768–805. https://doi.org/10.3102/0034654317704307

105.

* Tai

T.-Y.

(2024). Comparing the effects of intelligent personal assistant-human and human-human interactions on EFL learners’ willingness to communicate beyond the classroom. Computers & Education, 210(3):104965. https://doi.org/10.1016/j.compedu.2023.130965

106.

Tuan

N. T.

Moore

Thanh

D. H. V.

Pham

H. V.

(2024). A generative artificial intelligence using multilingual large language models for ChatGPT applications. Applied Sciences, 14(7), 3036. https://doi.org/10.3390/app14073036

107.

UNESCO . (2015). International standard classification of education: Fields of education and training 2013 (ISCED-F 2013) detailed field descriptions. UNESCO Institute for Statistics. https://doi.org/10.15220/978-92-9189-179-5-en

108.

* Urban

Děchtěrenko

Lukavský

Hrabalová

Svacha

Brom

Urban

(2024). ChatGPT improves creative problem-solving performance in university students: An experimental study. Computers & Education, 215, 105031. https://doi.org/10.1016/j.compedu.2024.105031

109.

* Vázquez-Cano

Mengual-Andrés

López-Meneses

(2021). Chatbot to improve learning punctuation in Spanish and to enhance open and flexible learning environments. International Journal of Educational Technology in Higher Education, 18(1). https://doi.org/10.1186/s41239-021-00269-8

110.

Viechtbauer

(2007). H.R. Rothstein, A.J. Sutton, & M. Borenstein (Eds.) (2005). Publication bias in meta-analysis: Prevention, assessment and adjustments. NewYork: Wiley, xvii+356 p. US$100.00. ISBN: 0-470-87014-1. Psychometrika, 72(2), 269–271. https://doi.org/10.1007/s11336-006-1450-y

111.

Wan

Moorhouse

B. L.

(2024). Using call Annie as a generative artificial intelligence speaking partner for language learners. RELC Journal. https://doi.org/10.1177/00336882231224813

112.

Wang

Y.-S.

(2024). Critical analysis of the technological affordances, challenges and future directions of generative AI in education: A systematic review. Asia Pacific Journal of Education, 44(1), 139–155. https://doi.org/10.1080/02188791.2024.2305156

113.

Williams

Rhodes

R. E.

(2016). The confounded self-efficacy construct: conceptual analysis and recommendations for future research. Health Psychology Review, 10(2), 113–128. https://doi.org/10.1080/17437199.2014.941998

114.

Wilson

S. E.

Nishomoto

(2024). Assessing learning of computer programming skills in the age of generative artificial intelligence. Journal of Biomechanical Engineering, 146(5), 051003. https://doi.org/10.1115/1.4064364

115.

Winkler

Hobert

Salovaara

Söllner

Leimeister

J. M.

(2020). Sara, the lecturer: Improving learning in online education with a scaffolding-based conversational agent. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1–14). Association for Computing Machinery. https://doi.org/10.1145/3313831.3376781

116.

(2023). Do AI chatbots improve students learning outcomes? Evidence from a meta-analysis. British Journal of Educational Technology, 55(1), 10–33. https://doi.org/10.1111/bjet.13334

117.

* Wu

Zhou

Chen

(2024). The impact of intelligent personal assistants on Mandarin second language learners: interaction process, acquisition of listening and speaking ability. Computer Assisted Language Learning, 1–26. https://doi.org/10.1080/09588221.2024.2317849

118.

Xia

Chiu

T. K. F.

Chai

C. S.

Xie

(2023). The mediating effects of needs satisfaction on the relationships between prior knowledge and self‐regulated learning through artificial intelligence chatbot. British Journal of Educational Technology, 54(4), 967–986. https://doi.org/10.1111/bjet.13305

119.

J.-H.

Zhang

Nong

Wang

Yang

(2024). The relationship between inert thinking and ChatGPT dependence: An I-PACE model perspective. Education and Information Technologies, 30(3), 3885–3909. https://doi.org/10.1007/s10639-024-12966-8

120.

* Yilmaz

Yilmaz

F. G. K.

(2023). The effect of generative artificial intelligence (AI)-based tool use on students’ computational thinking skills, programming self-efficacy and motivation. Computers and Education: Artificial Intelligence, 4(2): 100147. https://doi.org/10.1016/j.caeai.2023.100147

121.

* Yin

Goh

T.-T.

Yang

Xiaobin

(2021). Conversation technology with micro-learning: The impact of chatbot-based learning on students’ learning motivation and performance. Journal of Educational Computing Research, 59(1), 154–177. https://doi.org/10.1177/0735633120952067

122.

Zhang

Shan

Lee

J. S. Y.

Che

Kim

J. H.

(2023b). Effect of chatbot-assisted language learning: A meta-analysis. Education and Information Technologies, 28(11), 15223–15243. https://doi.org/10.1007/s10639-023-11805-6

123.

* Zhang

Meng

(2024). Artificial intelligence in EFL speaking: Impact on enjoyment, anxiety, and willingness to communicate. System, 121(3), 103259. https://doi.org/10.1016/j.system.2024.103259

124.

* Zhang

Zou

Cheng

(2023a). Chatbot-based learning of logical fallacies in EFL writing: Perceived effectiveness in improving target knowledge and learner motivation. Interactive Learning Environments, 32(9), 5552–5569. https://doi.org/10.1080/10494820.2023.2220374

125.

Zhao

Song

(2022). Establishing a fusion model of attention mechanism and generative adversarial network to estimate students’ attitudes in English classes. Tehnicki Vjesnik-Technical Gazette, 29(5), 1464–1471. https://doi.org/10.17559/tv-20210922053009

126.

Zheng

Niu

Zhong

Gyasi

J. F.

(2021). The effectiveness of artificial intelligence on learning achievement and learning perception: A meta-analysis. Interactive Learning Environments, 31(9), 5650–5664. https://doi.org/10.1080/10494820.2021.2015693

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.38 MB