Automated written corrective feedback (AWCF) tools play a crucial role in supporting English writing instruction. However, issues such as insufficient accuracy and hallucination have undermined users’ trust in these systems. To address these challenges, this study investigates the potential of Generative Artificial Intelligence (GAI) enhanced by prompting. Specifically, we evaluate the performance of several GAI models, including GPT-4, in providing written corrective feedback compared to commercial AWCF tools and other advanced models. The study adopts a dual-method approach: (1) a comprehensive model evaluation using established English writing datasets to assess error correction performance via various prompting strategies, and (2) an empirical study involving teachers and students to examine the system’s practical efficacy and users’ perspective. Quantitative results indicate that GPT-4 with chain-of-thought prompting significantly outperforms commercial tools, achieving improved consistency and accuracy in error detection and correction. Qualitative feedback from participants further supports the system’s potential to enhance students’ writing quality and confidence, while concurrently reducing teachers’ workload and optimizing instructional efficiency, despite concerns regarding occasional overcorrection. These findings emphasize the benefits of task-specific prompting in GAI-based AWCF systems and provide actionable insights for integrating advanced AI feedback into educational practices.
AlmaneaM. (2024). Instructors’ and learners’ perspectives on using ChatGPT in English as a foreign language courses and its effect on academic integrity. Computer Assisted Language Learning, 1–26. Advance online publication. https://doi.org/10.1080/09588221.2024.2410158
ArroyoD. C.YilmazY. (2018). An open for replication study: The role of feedback timing in synchronous computer-mediated communication. Language Learning, 68(4), 942–972. https://doi.org/10.1111/lang.12300
BarrotJ. S. (2023). Using automated written corrective feedback in the writing classrooms: Effects on L2 writing accuracy. Computer Assisted Language Learning, 36(4), 584–607. https://doi.org/10.1080/09588221.2021.1936071
6.
BrownT.MannB.RyderN.SubbiahM.KaplanJ.DhariwalP.NeelakantanA.ShyamP.SastryG.AskellA.AgarwalS.Herbert-VossA.KruegerG.HenighanT.ChildR.RameshA.ZieglerD. M.WuJ.WinterC.AmodeiD. (2020). Language models are few-shot learners. In Advances in neural information processing systems (pp. 1877–1901). Curran Associates.
7.
BryantC.FeliceM.AndersenØ. E.BriscoeT. (2019). The BEA-2019 shared task on grammatical error correction. In Proceedings of the fourteenth workshop on innovative use of NLP for building educational applications (pp. 52–75). https://doi.org/10.18653/v1/W19-4406
8.
BuiN. M.BarrotJ. S. (2025). ChatGPT as an automated essay scoring tool in the writing classrooms: How it compares with human scoring. Education and Information Technologies, 30(2), 2041–2058. https://doi.org/10.1007/s10639-024-12891-w
9.
CoyneS.SakaguchiK.Galvan-SosaD.ZockM.InuiK. (2023). Analyzing the performance of gpt-3.5 and gpt-4 in grammatical error correction. arXiv preprint arXiv:2303.14342.https://doi.org/10.48550/arXiv.2303.14342
10.
CsikszentmihalyiM. (2000). Beyond boredom and anxiety. Jossey-Bass.
11.
DahlmeierD.NgH. T. (2012). Better evaluation for grammatical error correction. In Proceedings of the 2012 conference of the North American chapter of the association for computational linguistics: Human language technologies (pp. 568–572). https://aclanthology.org/N12-1067
12.
DavisF. D. (1989). Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly.
13.
FuQ. K.ZouD.XieH.ChengG. (2024). A review of AWE feedback: Types, learning outcomes, and implications. Computer Assisted Language Learning, 37(1–2), 179–221. https://doi.org/10.1080/09588221.2022.2033787
14.
HoangG. T. L.KunnanA. J. (2016). Automated essay evaluation for English language learners: A case study of MY access. Language Assessment Quarterly, 13(4), 359–376. https://doi.org/10.1080/15434303.2016.1230121
JeonJ. (2023). Chatbot-assisted dynamic assessment (CA-DA) for L2 vocabulary learning and diagnosis. Computer Assisted Language Learning, 36(7), 1338–1364. https://doi.org/10.1080/09588221.2021.1987272
17.
KatinskaiaA.YangarberR. (2024). GPT-3.5 for grammatical error correction. In Proceedings of the 2024 joint international conference on computational linguistics, language resources and evaluation (LREC-COLING 2024) (pp. 7831–7843). ELRA and ICCL.
18.
KoC. J. (2024). Online individualized corrective feedback on EFL learners’ grammatical error correction. Computer Assisted Language Learning, 37(7), 1449–1477. https://doi.org/10.1080/09588221.2022.2118783
19.
KourtaliN. E.BorgesL. (2023). The effects of feedback timing on L2 development in written SCMC. Computer Assisted Language Learning, 37(8), 2291–2319. https://doi.org/10.1080/09588221.2023.2171066
20.
LeeY. J. (2006). The process-oriented ESL writing assessment: Promises and challenges. Journal of Second Language Writing, 15(4), 307–330. https://doi.org/10.1016/j.jslw.2006.09.003
21.
LiJ.LinkS.HegelheimerV. (2015). Rethinking the role of automated writing evaluation (AWE) feedback in ESL writing instruction. Journal of Second Language Writing, 27, 1–18. https://doi.org/10.1016/j.jslw.2014.10.004
22.
LiJ.WuJ. (2023). Automated essay scoring incorporating multi-level semantic features. In International conference on artificial intelligence in education (pp. 206–211). https://doi.org/10.1007/978-3-031-36336-8_31
23.
LiY.HuangH.MaS.JiangY.LiY.ZhouF.ZhengH.-T.ZhouQ. (2023). On the (in) effectiveness of large language models for Chinese text correction. arXiv preprint arXiv:2307.09007. https://doi.org/10.48550/arXiv.2307.09007
LoC. K.HewK. F.JongM. S. Y. (2024). The influence of ChatGPT on student engagement: A systematic review and future research agenda. Computers & Education, 219, Article 105100. https://doi.org/10.1016/j.compedu.2024.105100
27.
LoemM.KanekoM.TakaseS.OkazakiN. (2023). Exploring effectiveness of GPT-3 in grammatical error correction: A study on performance and controllability in prompt-based methods. In Proceedings of the 18th workshop on innovative use of NLP for building educational applications (BEA 2023) (pp. 205–219). Association for Computational Linguistics.
28.
MahapatraS. (2024). Impact of ChatGPT on ESL students’ academic writing skills: A mixed methods intervention study. Smart Learning Environments, 11(1), 9–18. https://doi.org/10.1186/s40561-024-00295-9
29.
MohamedA. M. (2024). Exploring the potential of an AI-based chatbot (ChatGPT) in enhancing English as a foreign language (EFL) teaching: Perceptions of EFL faculty members. Education and Information Technologies, 29(3), 3195–3217. https://doi.org/10.1007/s10639-023-11917-z
30.
NetlandT.von DzengelevskiO.TeschK.KwasnitschkaD. (2025). Comparing human-made and AI-generated teaching videos: An experimental study on learning effects. Computers & Education, 224, Article 105164. https://doi.org/10.1016/j.compedu.2024.105164
31.
NgH. T.WuS. M.BriscoeT.HadiwinotoC.SusantoR. H.BryantC. (2014). The CoNLL-2014 shared task on grammatical error correction. In Proceedings of the eighteenth conference on computational natural language learning: Shared task (pp. 1–14). Association for Computational Linguistics.
PapineniK.RoukosS.WardT.ZhuW. J. (2002). Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the association for computational linguistics (pp. 311–318). Association for Computational Linguistics.
34.
PavlikJ. V. (2023). Collaborating with ChatGPT: Considering the implications of generative artificial intelligence for journalism and media education. Journalism and Mass Communication Educator, 78(1), 84–93. https://doi.org/10.1177/10776958221149577
35.
RanalliJ. (2018). Automated written corrective feedback: How well can students make use of it?Computer Assisted Language Learning, 31(7), 653–674. https://doi.org/10.1080/09588221.2018.1428994
36.
RoscoeR. D.AllenL. K.JohnsonA. C.McNamaraD. S. (2018). Automated writing instruction and feedback: Instructional mode, attitudes, and revising. In Proceedings of the 62nd annual meeting of the human factors and ergonomics society (pp. 2089–2093). https://doi.org/10.1177/1541931218621471
37.
ShabaniK.KhatibM.EbadiS. (2010). Vygotsky’s zone of proximal development: Instructional implications and teachers' professional development. English language teaching, 3(4), 237–248. https://doi.org/10.5539/elt.v3n4p237
38.
ShadievR.FengY. (2023). Using automated corrective feedback tools in language learning: A review study. Interactive Learning Environments, 1–29. https://doi.org/10.1080/10494820.2022.2153145
39.
ShangHSivaparthipanC. B. (2022). Interactive teaching using human-machine interaction for higher education systems. Computers and Electrical Engineering, 100. https://doi.org/10.1016/j.compeleceng.2022.107811
40.
SuY.LinY.LaiC. (2023). Collaborating with ChatGPT in argumentative writing classrooms. Assessing Writing, 57, Article 100752. https://doi.org/10.1016/j.asw.2023.100752
41.
TaiT. Y.ChenH. H. J. (2024). Improving elementary EFL speaking skills with generative AI chatbots: Exploring individual and paired interactions. Computers & Education, 220, Article 105112. https://doi.org/10.1016/j.compedu.2024.105112
42.
WangE. L.MatsumuraL. C.CorrentiR.LitmanD.ZhangH.HoweE.QuintanaR.MagoodaA. (2020). eRevis (ing): Students’ revision of text evidence use in an automated writing evaluation system. Assessing Writing, 44, Article 100449. https://doi.org/10.1016/j.asw.2020.100449
43.
WangQ.GayedJ. M. (2024). Effectiveness of large language models in automated evaluation of argumentative essays: Finetuning vs. zero-shot prompting. Computer Assisted Language Learning, 1–29. Advance online publication. https://doi.org/10.1080/09588221.2024.2371395
44.
WangS.SunY.XiangY.WuZ.DingS.GongW.FengS.ShangJ.ZhaoY.PangC.LiuJ.ChenX.LuY.LiuW.WangX.BaiY.ChenQ.ZhaoL.LiS.WangH. (2023a). ERNIE 3.0 Titan: Exploring larger-scale knowledge enhanced pre-training for language understanding and generation. arXiv preprint arXiv:2112.12731.
45.
WangX.WeiJ.SchuurmansD.LeQ.ChiE.NarangS.ChowdheryA.ZhouD. (2023b). Self-consistency improves chain of thought reasoning in language models. In Proceedings of 12th international conference on learning representations (ICLR 2023). OpenReview.https://openreview.net/forum?id=1PL1NIMMrw
46.
WeiJ.WangX.SchuurmansD.BosmaM.IchterB.XiaF.ChiE.LeQ.ZhouD. (2023). Chain-of-Thought prompting elicits reasoning in large language models. In Proceedings of 36th conference on neural information processing systems (NeurIPS 2022).
47.
WilsonJ.RoscoeR. D. (2020). Automated writing evaluation and feedback: Multiple metrics of efficacy. Journal of Educational Computing Research, 58(1), 87–125. https://doi.org/10.1177/0735633119830764
48.
WoodworthJ.BarkaouiK. (2020). Perspectives on using automated writing evaluation systems to provide written corrective feedback in the ESL Classroom. TESL Canada Journal, 37(2), 234–247. https://doi.org/10.18806/tesl.v37i2.1340
49.
WuR.YuZ. (2024). Do AI chatbots improve students learning outcomes? Evidence from a meta-analysis. British Journal of Educational Technology, 55(1), 10–33. https://doi.org/10.1111/bjet.13334
50.
XiaoY.ZhiY. (2023). An exploratory study of EFL learners’ use of ChatGPT for language learning tasks: Experience and perceptions. Experience and perceptions. Languages, 8(3), 212. https://doi.org/10.3390/languages8030212
51.
XuZ.JainS.KankanhalliM. S. (2024). Hallucination is inevitable: An innate limitation of large language models. arXiv preprint arXiv:2401.11817. https://doi.org/10.48550/arXiv.2401.11817
52.
YanD.ZhangS. (2024). L2 writer engagement with automated written corrective feedback provided by ChatGPT: A mixed-method multiple case study. Humanities and Social Sciences Communications, 11(1), Article 1086. https://doi.org/10.1057/s41599-024-03543-y
53.
YangH.HuHawkinsM. W. A.MurrayB.SmithS. E.LiS.SikoraA. (2025). Evaluating accuracy and reproducibility of large language model performance on critical care assessments in pharmacy education. Frontiers in Artificial Intelligence, 7, Article 1514896. https://doi.org/10.3389/frai.2024.1514896
54.
YannakoudakisH.BriscoeT.MedlockB. (2011). A new dataset and method for automatically grading ESOL texts. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies (pp. 180–189). https://aclanthology.org/P11-1019
55.
YaoS.YuD.ZhaoJ.ShafranI.GriffithsT. L.CaoY.NarasimhanK. (2023). Tree of thoughts: Deliberate problem solving with large language models. In The 37th conference on neural information processing systems (NeurIPS 2023).
56.
ZengA.LiuX.DuZ.WangZ.LaiH.DingM.YangZ.XuY.ZhengW.XiaX.TamW. L.MaZ.XueY.ZhaiJ.ChenW.ZhangP.TangJ. (2023). GLM-130B: An open bilingual pre-trained. arXiv preprint, arXiv:2210.02414. https://doi.org/10.48550/arXiv.2210.02414
57.
ZhangD.Perez-ParedesP. (2021). Chinese postgraduate EFL learners’ self-directed use of mobile English learning resources. Computer Assisted Language Learning, 34(8), 1128–1153. https://doi.org/10.1080/09588221.2019.1662455
58.
ZhaoR.ZhuangY.XieZ.YuP. L. H. (2024). Facilitating self-directed language learning in real-life scene description tasks with automated evaluation. Computers & Education, 219, Article 105106. https://doi.org/10.1016/j.compedu.2024.105106
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
