Sage Journals: Discover world-class research

Abstract

Purposes

In light of the usefulness theoretical framework, this research seeks to explore the affordances and challenges of integrating a generative artificial intelligence (GenAI) tool named ERNIE Bot into the writing feedback process.

Design/Approach/Method

A qualitative design was adopted for this study. Twelve students were purposively chosen as the focal participants. Data came from multiple sources. Semi-structured interviews were used as the primary source, triangulated with other data sources including writing drafts, reviews from peers, and the chatlogs with ERNIE Bot. The data were inductively and deductively analyzed.

Findings

The study highlighted the technological, educational, and social benefits of GenAI feedback, including its timeliness and personalization (technological), its roles as both an essay editor and a responsive tutor (educational), and its ability to provide a supportive environment and an engaging process (social). Three categories of challenges were revealed, namely lack of AI literacy among students, potential risk of worsening writing ability, and inability to catch emotion and to be perceptive.

Originality/Value

Enlightened by the usefulness theoretical framework, the study discovered the technological, educational, and social affordances as well as the limitations of GenAI feedback, which provides insights for the application of GenAI in L2 writing instruction.

Keywords

Affordances challenges ERNIE bot GenAI feedback L2 writing usefulness theoretical framework

Introduction

In the realm of EFL (English as a Foreign Language) education, peer feedback is hailed as a trustworthy approach for learner development (Hansen & Liu, 2005). However, it might not be easy to find an ideal partner for providing quality feedback in practice (Barron, 2003). In recent years, generative artificial intelligence (GenAI) has garnered considerable attention in L2 writing fields as a potential pedagogical solution (Al-khresheh, 2024). The GenAI tools which mimic human-like conversations have the capacity to respond to questions, admit mistakes, and provide instantaneous linguistic feedback (Baskara, 2023). Recent studies have witnessed the uses of GenAI to support students’ language learning in various ways, such as facilitating language practice (Su et al., 2023), increasing the interest and motivation of language learning (Chiu et al., 2023), and delivering language assessments (Barrot, 2024). Under this background, writing researchers and practitioners began to develop an interest in prompting the powerful GenAI to provide feedback to L2 writing learners. GenAI may help highlight the strengths and weaknesses of students’ draft and offer pertinent suggestions for writing improvement (Barrot, 2023). While the general effectiveness of GenAI in improving students’ language learning has been well documented in the literature, its practical assimilation into feedback process of L2 writing remains unclear. As frontline users, L2 writing learners possess invaluable insights into the comparative advantages and challenges of employing GenAI tools as the feedback providers compared with peer reviewers. Understanding students’ perspectives is essential for the widespread adoption of GenAI in L2 writing feedback (Al-khresheh, 2024). However, relevant studies in L2 writing field are relatively scarce. In light of usefulness theoretical framework (Huang et al., 2021), this research seeks to bridge such a research gap by exploring the affordances and challenges of integrating a kind of GenAI tool named ERNIE Bot into the writing feedback process from the students’ perspective.

Literature review

GenAI in second language learning

In the era of digital technology, computer-assisted language learning (CALL) demonstrates the role of technology in facilitating learners’ language input and interaction, laying the foundation for AI integration (Warschauer, 1996). Among the CALL technologies, AI tools have received considerable attention due to their capacity to engage learners in communication in the target language (Fryer et al., 2019). The subtle capabilities of AI tools, such as natural language processing, enable them to interact with learners in a more human-like manner (Kerly et al., 2007). However, traditional AI tools can only provide pre-defined guidance for L2 learners. They lack the “intelligence” to answer questions that they were not programmed to address (Kohnke et al., 2023). The advent of highly sophisticated generative pretrained transformer (GPT) large language models (LLMs), such as ChatGPT, allows users to interface with LLMs, thereby tremendously changing the L2 learning scenarios (Escalante et al., 2023). ChatGPT, built on LLM technology, features three capabilities (OpenAI, 2023): (a) the ability to retain prior statements made by the user during the conversation; (b) the ability to comprehend subsequent corrections made by the user; (c) the ability to decline any inappropriate requests. These capabilities enable the GenAI to respond to a wider range of user inquiries, suggesting a broader array of educational roles in the language learning (Kasneci et al., 2023). Compared with traditional AI, GenAI tools can harness its full potential for L2 learning when provided with clear and specific prompts (Kohnke et al., 2023). They process extensive input, generate creative responses to diverse prompts, and sustain a continuous chat flow (Su et al., 2023).

The affordances of GenAI for L2 learning have been well documented in existing literature. Yang and Li (2024) contended that ChatGPT held extensive knowledge, providing rich information for language input and practice anytime and anywhere. Kohnke et al. (2023) noted that ChatGPT supported language learning by simulating authentic interactions. They also pointed out that it could adjust the complexity of the dialogue to better suit beginners or advanced learners. Jeon and Lee (2023) highlighted that GenAI promoted learners’ learning, making them active investigators. This was achieved not only by providing learning resources and feedback but also by functioning as a chat partner and evaluator. Vera (2023) examined the pedagogical affordances of ChatGPT for EFL learners’ language proficiency, and the findings indicated that the experimental group, which received ChatGPT-based instruction, outperformed the control group that received traditional classroom instruction, suggesting that ChatGPT could improve learners’ language proficiency more effectively with personalized, interactive, and engaging affordances. Lee et al. (2023) incorporated ChatGPT into augmented reality (AR) glasses to create a contextual learning environment and evaluated its effectiveness and acceptance among EFL learners. The results indicated that participants found the system beneficial in improving their oral language skills through a more engaging and tailored approach. Furthermore, L2 learners could utilize the affordances of GenAI for sustained and coherent conversations without the fear of being judged. Therefore, they might feel less stressed during the learning process (Barrot, 2023; Lee et al., 2023).

Despite the benefits of GenAI, its integration into EFL learning environments has raised concerns about potential challenges (e.g., Hockly, 2023; Kohnke et al., 2023). One major concern was the ethical issues created by its unregulated use by language learners (Hockly, 2023). It could be argued that the responses given by GenAI were not entirely original and have not been appropriately cited (Kohnke et al., 2023), thus prompting concerns about cheating and plagiarism in educational settings. In a recent qualitative study conducted by Yan (2023), EFL learners generally expressed concerns with ChatGPT's threats to academic honesty and educational equity. The popularity of such tools therefore impaired L2 learners’ motivation as they depreciated their efforts. A second debate focused on the reliability of GenAI's responses. The AI research companies cautioned that the generated responses might not always be accurate and provided users with a feedback mechanism to indicate their satisfaction with the responses. Despite the AI research companies’ disclaimer about accuracy, the perceived authority of the responses could potentially mislead the learners (Kohnke et al., 2023). Moreover, concerns were raised regarding the cultural bias present within the source database. As most of the texts in the database were sourced from an English corpus, AI-driven tools like ChatGPT were not culturally neutral, which L2 learners might not be aware (Kohnke et al., 2023). Additionally, student-related issues were also identified as barriers that emerge during the GenAI-supported language learning activity. Students’ deficiency in AI literacy, negative attitudes toward AI, and the lack of higher-order thinking may affect the complete utilization of AI, thereby impeding the learning effectiveness in writing tasks (Kim et al., 2024). The incorporation of AI into the language course may also pose challenges to instructors, including barriers to aligning AI with the curriculum and the learning objectives, selecting and designing the language tasks, and evaluating and assessing the learning progress (Ghafouri et al., 2024). New expertise is therefore required to take advantage of this novel technology; however, most language teacher educators perceived that they lack the confidence and competence to handle these difficulties successfully (Moorhouse et al., 2024).

GenAI feedback in L2 writing

It is widely recognized that quality feedback is crucial for encouraging and consolidating learning in L2 writing (Hyland & Hyland, 2006). However, limited teaching resources make it challenging to provide timely and constructive feedback for a large cohort of students (Dai et al., 2023). While peer feedback is considered beneficial for cultivating audience awareness and learner autonomy, its perceived effectiveness may be influenced by the peers’ knowledge of the topic and their language proficiency (Wang, 2014). The emergence of automated writing evaluation (AWE) systems has provided a potential solution. Early AWE systems such as Grammarly and Pigai assisted writing processes by providing corrective feedback on learner writing. With the recent unleashing of LLM-supported GenAI, AWE may be entering a new era (Escalante et al., 2023).

Compared to their predecessors (e.g., Grammarly), GenAI is capable of suggesting language corrections, generating ideas (Ingley & Pack, 2023), evaluating the content and clarity of audience (Barrot, 2023), providing feedback on a draft's organization (Escalante et al., 2023), and demonstrating a superior mastery of advanced discourse skill (Zhou et al., 2023). The integration of GenAI into feedback mechanisms in L2 writing pedagogy has led to a number of studies aimed at examining their feasibility and reliability. With regards to feasibility, in a recent study that investigated ChatGPT for essay feedback and evaluation, Mizumoto and Eguchi (2023) fed a corpus of 12,100 essays by English learners to ChatGPT and compared the scores to benchmark levels. The results demonstrated that ChatGPT possessed a certain level of accuracy and reliability, making it a valuable tool for supporting human evaluations. Another study by Lu et al. (2024) indicated that ChatGPT enhanced students’ critical evaluation of feedback, especially assisting high- and medium-level students in comprehending teacher feedback. Escalante et al. (2023) compared the feedback generated by the human tutor and by GenAI. The results suggested that while there was no significant difference between the two types of feedback in terms of students’ linguistic gain, GenAI feedback had the potential time-saving benefits. However, the effectiveness of these AI tools is not without controversy. Abunaseer's (2023) study suggested that while GenAI could provide instant and consistent feedback, they might lack the depth and nuance provided by human feedback.

In addition to the feasibility studies, understanding the acceptance of GenAI feedback by L2 writing learners is crucial, as the success of any learning technology hinges on whether users adopt it (Escalante et al., 2023). Several recent studies have used qualitative approaches to investigate students’ reflections on the implementation of ChatGPT in the L2 writing process. For example, Yan (2023) applied ChatGPT in an L2 writing practicum and found that while the undergraduates appreciated the speedy generation of quality texts and flexibility in language styles, they were much perturbed by the potential threats ChatGPT poses to educational equity and academic integrity. Zou and Huang (2023) examined doctoral students’ perceptions of the impact of using ChatGPT on L2 writing. The results revealed that students perceived ChatGPT as a personal tutor and learning partner but also had concerns about learning loss, authorial voice loss, and unintelligent texts. However, the perceptions gathered from these studies pertained to the use of ChatGPT throughout the entire L2 writing process, rather than its limited application for providing feedback. Given the concerns associated with the use of GenAI in the L2 writing, some researchers (e.g., Yan, 2023) have advocated for their limited use. When GenAI is restricted to providing feedback, it remains unclear whether L2 writing learners are willing to accept feedback from them, especially in comparison to peer feedback. An investigation into L2 writing learners’ perceptions of affordances and challenges of GenAI for providing feedback is essential.

Usefulness theoretical framework

To investigate the potential affordances that GenAI applications have or have not in providing feedback, this article adopts a usefulness theoretical perspective (Huang et al., 2021) to analyze the utility and usability of GenAI in the learning contexts of L2 writing. The choice of this framework is driven by its comprehensive approach to evaluating technological tools, focusing on both utility and usability, factors crucial to the successful integration of technology in educational settings. In this study, we aim to explore how GenAI feedback is perceived and used by L2 writing students. The usefulness theoretical framework emphasizes the importance of user perceptions and experiences in determining the effectiveness of technology, which is particularly aligned with our research goals.

Usability is an objective of product design, which pertains to the capability of a system to allow users to fulfill tasks with ease and efficiency (Kirschner et al., 2004). Utility describes the functional aspects of a system—essentially, whether it offers the necessary functions that users require. The collective impact of usability and utility determines the overall usefulness of a system to its users. An object's usability is connected with the tool's technological affordances (Norman, 1988), whereas utility is shaped by the educational and social affordances of the tool. The concept of affordance, as introduced by Gibson (1977), refers to the actionable properties of an object or environment that prompt specific behaviors from people. For example, a chair has the affordance of seating due to its design which allows people to sit on it.

The design of the learning environment is driven by three key types of affordances: technological, educational, and social (Kirschner et al., 2004). We examine these crucial aspects to assess the usefulness of GenAI in writing learning contexts, as shown in Figure 1. Technological affordances are the design features of a technology that facilitate students in easily achieving their set learning objectives (Kirschner et al., 2004). In the present study, technological affordances refer to GenAI's design features that facilitate easy access, handling, and utilization of feedback, thereby promoting students’ writing improvement. Educational affordances address how a tool can be used to enact particular learning behavior and support real learning to take place. In this study, educational affordances refer to the potential of GenAI feedback to facilitate L2 writing improvement. Social affordances, as discussed in Kirschner et al. (2004), refer to the functions of a tool that enables and enhances social interactions. In the context of this study, social affordances are conceptualized as GenAI's capacity to encourage a feeling of social support during interactions with the learners.

Figure 1.

Usefulness conceptual framework. Source. Adapted from Huang et al. (2021, p. 239).

As frontline users, L2 writing learners’ insight into the advantages and challenges of employing GenAI as the feedback provider merits attention. The understanding of student receptiveness and attitudes toward the feedback generated by GenAI is essential for its widespread adoption in L2 writing pedagogy (Al-khresheh, 2024). However, a limited number of investigations have examined this particular issue. In light of usefulness theoretical framework (Huang et al., 2021), this qualitative study intended to explore the affordances and challenges of integrating ERNIE Bot into writing feedback process from the students’ perspective. The following two research questions guided the study:

What are the (technological, educational, and social) affordances, if any, of utilizing GenAI for feedback in L2 writing learning?

What are the challenges, if any, of utilizing GenAI for providing feedback in L2 writing learning?

Method

Context and participants

The study was carried out within the context of a 16-week English course at a mid-sized public university in China. The Advanced English Writing course was compulsory for freshmen majoring in English. The course was intended for improving their comprehensive English language abilities, especially English reading and writing. Approximately 200 students registered in this course each term. One instructor taught five classes, with 40 students in each class. Throughout the term, students were tasked with composing four distinct essays ranging from 300 to 500 words on topics designed to challenge various writing skills: (a) A cause-and-effect essay on why choosing this University, (b) A descriptive essay (“My dear Dad” or “My Study”), (c) A classification essay on “Plagiarism in the Academic Community,” and (d) An evaluation essay on “Square Dancing in Public.” For each essay, students finished the writing individually and then exchanged their work for peer feedback. Peer feedback was integrated into a comparative baseline to elucidate the distinctive contributions and inherent limitations of GenAI feedback on student writing. Subsequently, students sought feedback by submitting their original essays to GenAI, ERNIE Bot, using prompts such as “请问这一段怎么修改呢？” (“Could you please tell me how to revise this paragraph?”) “这句话有语法错误吗？” (“Are there any grammar mistakes in this sentence?”). ERNIE Bot (https://yiyan.baidu.com/), is an advanced generative AI product and a knowledge-enhanced LLM developed by Baidu, Inc. It has been widely used across the teaching sector for its ideal features such as fast, multilingual, and contextually related sentence-level check and feedback (Labadze et al., 2023). ERNIE Bot system was deeply integrated into the course for providing insightful, instant remarks which hopefully strengthen learners’ writing and assessment skills. After receiving feedback from their peers and ERNIE Bot, the students made the necessary revisions to their writing.

Purposive sampling was chosen to investigate the students’ practice and perceptions of using GenAI feedback. Specifically, to achieve maximum variation in the sample (Miles & Huberman, 1994), 12 students were selected from volunteers, ensuring a mix of different genders and varying levels of language proficiency. The profile of the participants is shown in Table 1. All the participants were informed about the purpose and procedures of the study. Written informed consent was received from the participants and the Ethical Committee of the first researcher's university.

Table 1.

Profile of the participants.

Group	Name	Gender	Age	English test score
1	Deng	F	19	82
1	Xiao	F	18	85
2	Kang	M	19	79
2	Chen	F	20	90
3	Luo	F	19	78
3	Ning	F	19	60
4	Tan	M	19	87
4	Wang	F	19	72
5	Yan	F	19	88
5	Lu	F	20	84
6	Zhang	M	19	59
6	Yu	M	19	92

Data collection and analysis

The data for this study came from multiple sources, including the writing drafts of each participant, the reviews from the peers, the chat logs with ERNIE Bot, and the semi-structured interviews. The interview transcripts were used as the primary source of data, triangulating with other data sources.

Each time, after receiving the reviews from the peer and ERNIE Bot, the participant was interviewed for 20–30 min. An interview guide was developed for the study, which included three questions: (a) What advantages do you think GenAI feedback has, compared with peer feedback? (b) What disadvantages does GenAI feedback have? (c) What expectations do you have about GenAI feedback? The interviews were conducted in Chinese, the native language of the participants.

The interview data were analyzed using both inductive and deductive thematic analysis taking the following steps: (a) We immersed ourselves in the transcripts through reading and rereading to achieve a comprehensive understanding of the data. (b) Initial codes were meticulously applied to pertinent excerpts. For example, the excerpt “跟文心一言可以持续地互动，这个过程很有意思” (“interacting continuously with ERNIE Bot is an interesting process”) was coded as “an engaging process.” These initial codes were continuously scrutinized, refined, and compared for salience and consistency. (c) We reviewed the initial codes and systematically organized them into broader categories that corresponded to the dimensions of the usefulness conceptual framework, which included technological affordances, educational affordances, and social affordances. During the process, we remained open to the emergence of new categories that could provide additional insights into the research questions. (d) The research teams met and agreed upon the final themes and sub-themes. The coding process for the challenges follows a similar procedure to that used for the affordances. The coding scheme developed in the process is demonstrated in the Appendix. Throughout the process, the researchers maintained communication: asking for clarification, checking for understanding, and discussing any differences in the interpretation of the data (Creswell & Poth, 2016).

Findings

Affordances of GenAI feedback

Technological affordances

All the participants believed that the integration of GenAI into L2 writing feedback opened new avenues to enhance their learning experience, thus adding more potential for improving learning. The data revealed two technological affordances of GenAI feedback, namely, timeliness and personification.

Timeliness. During the interviews, participants unanimously praised the effectiveness of ERNIE Bot in provision of feedback. Students reported that the tool enabled them to quickly identify errors by providing immediate feedback. As reported by Ning, “ERNIE Bot provides immediate feedback, enabling me to revise my work more efficiently.” She mentioned that ERNIE Bot helped her become more efficient in revising and editing her work, particularly in identifying grammatical and lexical problems. Luo said, “ERNIE Bot provided a very quick response.” Lu also appreciated the “timely polishing” of the feedback provided by ERNIE Bot, which according to her can hardly be achieved by peer feedback, especially when the peer deals with more than one group member's work or the peer handles a large number of other assignments. They also made appreciation of the convenience of receiving feedback instantly as it facilitates iterative improvement in their writing.

Personification. Participants mentioned that GenAI provided personalized and adaptive writing feedback. Despite a common writing topic to discuss, ERNIE Bot responded differently with different inputs in catering to the unique needs of individual student each time. In addition, through the way in which ERNIE Bot interacted with students, ERNIE Bot processed their inputs, acknowledging the areas for improvement and, as a result, offered tailored feedback that was meant to enhance students’ writing. Kang expressed his view, “It can clearly show you what is correct and what is wrong.” As reported by Kang, “ERNIE bot took into account the weaknesses that I showed and suggested methods and techniques for me to improve my article.” As students’ levels improved, adjustments to the feedback were made. Kang said in the interview, “When my writing was free of error, ERNIE Bot provided me with more advanced writing patterns instead, such as enhancing my vocabulary and sentence structures.” Tan explained that ERNIE bot provided him with more detailed and comprehensive feedback covering aspects of language, content, and structure in his essay.

Educational affordances

The utilization of ERNIE Bot for providing feedback was consistently acknowledged by students as beneficial for enhancing the overall quality of their writing. This section reports two educational roles of ERNIE Bot reported by participants when it was employed in L2 writing process: (a) as an essay editor and (b) as a responsive human tutor.

Essay editor. Students particularly reported on the advantage of ERNIE Bot for providing information about grammar and vocabulary. They acknowledged it as “a competent editor,” providing comprehensive and precise suggestions for language modification. During the interview, Xiao stated, “I was amazed by how comprehensively ERNIE Bot detected grammatical errors in my writing.” She further explained, “ERNIE Bot significantly enhanced the essay by correcting grammatical and lexical errors, and refining sentences to increase their specificity, accuracy, and clarity.” Wang expressed a similar view, “ERNIE Bot can identify words and grammar errors more accurately.”

Deng regarded ERNIE Bot as sources of new vocabulary and grammar, expressing that the Bot helped expand her vocabulary usage and significantly enhance her ability to articulate thoughts clearly. Ning made acknowledgment of ERNIE Bot's proficiency in identifying grammatical errors with precision. She stated that, “I was amazed by how comprehensively ERNIE Bot detected grammatical errors in my writing. It even detected a slight logical inconsistency, which had been missed out by peer reviewers.” A comparison of peer feedback and GenAI feedback is demonstrated in Figure 2.

Figure 2.

Peer feedback versus GenAI feedback on Ning's writing.

As can be seen from Figure 2, the peer feedback highlighted only a few grammatical and spelling errors; in contrast, the feedback from ERNIE Bot evaluated each sentence, listing concrete suggestions for improvement.

Responsive human tutor. More than an editor who helps to spot and correct mistakes, GenAI was reported by participants as a responsive human tutor who helped them analyze and learn. Kang stated that, “When I was puzzled and asked follow-up questions, ERNIE Bot provided detailed explanations and examples, which brought me a deeper understanding of the grammar rules.” Yu pointed out that ERNIE Bot made him feel that it paid close attention to his needs. Furthermore, he said “It always responds to my questions very patiently.” Such interaction facilitated the participants to engage in writing initially and obtain efficient review.

Apart from language suggestions, students further pointed out the comprehensive educational function of GenAI feedback. They reported that ERNIE Bot offered more than learning opportunities for grammar and vocabulary. For example, Kang mentioned the multidimensional help that he received from ERNIE Bot, “With ERNIE Bot's assistance, the essay was assessed from multiple perspectives, not only grammatical accuracy and vocabulary but also the overall structure of the essay.” Deng echoed this idea by expressing that ERNIE Bot assisted in idea improvement and structure analysis. She commented, “I feel that it is more like a knowledgeable teacher who can provide comprehensive targeted advice.” According to participants, such immediate guidance was highly beneficial for learning, yet it was often challenging to obtain from peers, who typically possess a comparable level of language proficiency.

Social affordances

Students regarded GenAI as a helpful assistant who developed a supportive and engaging environment for learning. They mentioned that with the timely and constructive feedback generated by ERNIE Bot, the writing process was made comfortable as they passed through the journey with confidence.

Supportive environment. A salient theme in the dataset was the positive impact of ERNIE Bot on the psychological status of the participants. Xiao suggested that being supportive in the writing process, ERNIE Bot made her feel less stressed and psychologically assisted. Yan described the bot as a counselor who assured her of no writing tensions. Luo emphasized the objectivity and consistency of the feedback provided by AI. This non-judgmental feedback mechanism, according to her, reduced negative emotions such as anxiety which was caused by the subjectivity and inconsistency often inevitable in human grading. Therefore, she found her motivation and self-confidence in writing increased.

Engaging process. The students also suggested that the constant interaction and the robot’ s immediate response created an engaging learning environment. As Chen ascertained, “The instant response made me feel more involved in the writing process. It's like I was sitting in front of a live tutor who was there to guide me every step toward learning.” Deng also regarded “Interacting continuously with ERNIE Bot” as “an interesting process.” Kang expressed that he could ask ERNIE Bot any question concerning the essay, and in this process, he felt relaxed. This psychological support increased students’ interest in involving themselves in the writing activity.

Challenges of leveraging ERNIE bot-generated feedback

Despite the technological, educational, and social affordances, substantial existing challenges in using GenAI feedback for L2 writing learning were discovered through analyzing students’ interviews and their discourses with ERNIE Bot. Three categories of challenges were revealed, namely students’ lack of AI literacy, potential risk of worsening writing ability, and inability to catch emotion and to be perceptive. These obstacles, to some extent, deter the users from employing the technological, educational, and social affordances effectively.

Students’ lack of AI literacy

From the chat logs and interviews of students, we can find that not all students had sufficient competence and experience to use GenAI for providing written feedback. While Chen, Xiao, and Kang acknowledged the comprehensive feedback provided by ERNIE Bot, other students simply used the basic function of the chatbot to correct their grammar and vocabulary. Without realizing the insufficiency of their own AI literacy, they expressed expectations on AI improvements. Luo looked forward to AI’ s improvements in coherence and rhetorical skills, “I hope that the chat technology can provide suggestions on coherence and rhetorical skills to improve the quality of our writing.”

Her expectations were echoed by Kang who expressed similar opinions that ERNIE Bot should not be limited to grammar and vocabulary corrections, but provide support in a wider range of writing skills. Chen further pointed out the need for getting personalized feedback from ERNIE Bot, “I hope that ERNIE Bot will be able to provide more personalized feedback in the future, adapting to each student's writing style and needs.” Another student Chen also expressed similar opinion, “It is best used for improving the structure of articles, as well as grammar and vocabulary. However, I believe it cannot be used to revise viewpoints or thought processes, even though I wish it could.” Likewise, Wang sighed, “If only it could be as flexible and intelligent as a human.”

Unaware that generating feedback tailored to diverse personal needs is a key technological affordance of chatbots, more students in this study simply took limited use of the chatbots for providing general feedback on grammar and vocabulary. ERNIE Bot was not fully taken advantage of for giving effective feedback on writing.

Potential risk of worsening writing ability

Despite the multidimensional assistance of AI feedback as the educational affordance, the data collected through interviews with participants revealed their concerns on integrating GenAI into the learning process. They suggested that ERINE Bot's effective writing assistance might lead students to become overly reliant on the AI, thereby potentially hindering their opportunities to further develop their writing skills through independent practice. During the interview, Ning voiced her concerns over the utilization of ERNIE Bot. She remarked, “ERNIE Bot’s writing assistance is so efficient that it may make us overly dependent on it and lose the opportunity to exercise our writing skills.”

This issue is not unwarranted because students may become less motivated to practice and hone their writing skills if they rely excessively on the feedback of GenAI to accomplish writing assignments or assignments. A similar viewpoint was voiced by Luo regarding this matter, expressing concern that an overreliance on the writing suggestions provided by ERNIE Bot may lessen the motivation of students to learn. During the interview, Luo said that, “A decrease in our writing ability and a decline in our ability to learn could result from placing an excessive amount of reliance on GenAI for writing.”

In addition, Kang and Chen expressed their worries on the potential for dependence that could be caused by ERNIE Bot. Students emphasized the importance of keeping the ability to think freely and the habit of writing creatively, even though they found ERNIE Bot helpful in providing feedback on grammar and structure. They believed that students needed to practice writing creatively instead of turning to AI for help without thinking whenever they encounter difficulties. Despite possessing other educational affordances, chatbots had the potential risk of worsening autonomous writing abilities of L2 writing learners.

Inability to catch emotion

Although GenAI has social affordances as being considered to be helpful assistants, students reported its shortcomings in terms of emotion and sensibility compared with traditional peer feedback. Deng commented, “ERNIE Bot is fast and accurate, but it is still not as good as human revision in terms of sensibility.” Kang concurred, noting, “ERNIE Bot is inferior to peer feedback in terms of catching emotions expressed in the writing.” Zhang, Yan, Xiao, and Luo also recognized these differences in their interviews and pointed out that traditional feedback is better able to identify the personality and emotions of the author, the capability that GenAI is currently unable to fully replicate. Figure 3 shows the feedback given to Xiao by his peer Deng.

Figure 3.

Deng's feedback for Xiao.

In the feedback to Xiao, Deng commented, “You have expressed your deep love toward the lake; it seems that you will show gratitude for the good things that emerge in life. …” Deng gave full recognition in response to the emotion conveyed in Xiao's essay, as if he truly felt it. This kind of emotional expression might be limited to human-to-human, and ERNIE Bot, as a GenAI tool, was hard to experience the emotional experience that human beings got from the lake. Therefore, participants including Deng and Kang, noted that ERNIE Bot had gaps with human feedback in capturing the deep feeling and emotions of human expressions in writing. Traditional methods are better at identifying the author's personality and emotional nuances, elements that ERNIE Bot may currently struggle to replicate. The empathic effect, an aspect of human interaction, can hardly appear in GenAI feedback. GenAI tools, such as ERNIE Bot, while excelling in many aspects, lacked the ability to engage in human emotional experiences.

Inability to be perceptive

ERNIE Bot was reported to offer too many suggestions at a time, as it could not sense participants’ tones or feelings through typed words. This could lead to misunderstanding or information processing problems. Zhang identified this issue, saying, “When it gave me too many suggestions, I feel annoyed. But it failed to perceive my impatience from the tone in the interaction and provide more concise feedback.” Wang also pointed out, “My classmates may understand my tone during interaction if I was impatient or not confident in writing.” In such specific cases, participants’ feelings were constrained to a certain extent when interacting with GenAI. It offers prospective insights for further enhancing GenAI's functions in interaction with humans.

Discussion

The study has investigated the usefulness of GenAI employed in generating feedback for students’ L2 writing assignment, to be specific, the technological, educational, and social affordances of ERNIE Bot-generated feedback as well as the challenges that might deter it from facilitating L2 writing learning. Figure 4 illustrates the identified affordances and challenges.

Figure 4.

Identified technological, educational, and social affordances and challenges of GenAI feedback in L2 writing.

In terms of the usability, the study revealed two technological affordances of the GenAI: timeliness and personification. ERNIE Bot provided immediate feedback on students’ writing, which is crucial in language learning, since prompt correction allows for deeper understanding and better retention (Schmulian & Coetzee, 2019). This is especially the case in large-class contexts where the teacher instruction in the classroom often deals with time constraints. In addition, ERNIE Bot gave responses based on the specific needs of the students, adjusting its language complexity in accordance with the input. The AI-aided personalization provided a learner-centric approach in the field of L2 learning. However, technological challenges simultaneously ensued. As the interactive processes with the GenAI were mostly students-initiated and guided in their individual time and space, the information elicited varies to a large extent on the AI literacy of the students. Once their AI literacy—the ability to fully use and apply AI (Ng et al., 2021) in this study—was not sufficient, the quality of the feedback would be unavoidably affected, further limiting the learning effects of the students.

As for the utility, the educational and social aspects were examined. The study identified the educational affordances of ERNIE Bot as an essay editor and a responsive tutor. Holding a vast amount of knowledge, ERNIE Bot serves as a rich source of information for language, providing a wide range of expressions that human language partners may lack (Huang et al., 2021). Through giving feedback on language like an essay editor, GenAI helped students notice the particular linguistic features or vocabulary that they had not mastered. This important noticing might be the premise of the subsequent acquisition of L2 knowledge (Schmidt, 1990). Additionally, instead of being stored in a traditional database, the corpus of GenAI was saved and processed as a collection of statistical patterns and associations, being constantly updated and expanded (Barrot, 2023). Therefore, GenAI could facilitate an authentic, interactive learning environment (Chiu et al., 2023). Through answering a wide range of questions like a responsive tutor, ERNIE Bot provided customized comprehensive learning for students (Kuhail et al., 2023). This may give them an advantage over the peers of the students, who basically share similar levels of language proficiency and writing competence.

However, students also expressed concerns about the overreliance on GenAI. Similar concerns were raised in existing literature (e.g., Barrot, 2023); that is, reliance on AI due to its convenience might undermine some important pedagogical goals in the writing course, such as the cultivation of creative thinking. When the bot was taken advantage of simply for completing the editing task, the educational role of the GenAI feedback would fail to be taken.

In terms of social affordance, GenAI was reported to help students feel less stressed about their performance compared with human reviewers. As reported in traditional peer feedback literature, students often fear making mistakes and appearing less than capable when working with human peers (Fryer & Carpenter, 2006). In contrast, when practicing with chatbots, students can develop and organize their ideas at their own pace in a more comfortable environment (Guo et al., 2024), which may lessen their anxiety and enhance their enjoyment and motivation during the learning process. Research has found that this will encourage students to practice more frequently and take risks in language use, which is essential for language learning (Barrot, 2023). However, some challenges facing the chatbot feedback were also mentioned by participants, specifically a lack of emotional nuances in feedback. The loss of empathy in the GenAI feedback may affect the students’ interest in writing for expressing.

ERNIE Bot's feedback represents a synthesis of three technological, educational, and social dimensions, where the benefits of each dimension intersect and reinforce each other. For example, the technological dimension's immediate and personalized feedback enables the educational dimension to function effectively as a responsive tutor, allowing the chatbot to provide tailored instructions and comprehensive learning support to students based on their needs. The educational dimension's role as a responsive tutor and essay editor, in turn, creates a supportive and engaging environment, where students feel supported and motivated to write and learn. The social dimension's supportive environment and engaging process further reinforce the benefits of the technological and educational dimensions. The supportive and engaging environment encourages students to actively engage with the feedback, make revisions accordingly, and seek further guidance. GenAI's ability of providing information in a deep and personal manner raises this capability as a powerful educational tool in a supportive language learning environment. However, challenges in different dimensions exist, which potentially deter the learning effects of GenAI feedback from maximizing its learning impact. The interplay between the affordances and challenges observed in the ERNIEBot-generated feedback could be explained by the metacognitive strategies employed by students in their GenAI-supported English writing (Yao et al., 2024). As students exhibited varying metacognitive strategies, they may experience the different affordances or the challenges across technological, educational, and social dimensions when dealing with ERNIE Bot feedback.

Conclusion

This study explored the GenAI feedback utilization in L2 writing process. Grounded in the usefulness theoretical framework, this study has identified the main affordances in technical, educational, and social aspects. However, the integration of these technologies in L2 writing learning is not without challenges. The study revealed issues such as the limited AI skills of end users, the risk of dependency that could hinder self-learning, and GenAI's limitations in understanding human emotions.

In reaction to the findings above, some pedagogical suggestions about the employment of GenAI feedback in L2 writing courses are provided. First, AI education should be integrated into L2 writing curriculum to raise students’ AI literacy. Teachers are suggested to provide workshops or training sessions on the effective use of AI tools in learning. More importantly, teachers can encourage students to actively explore and experiment with AI tools, such as ERNIE Bot, to foster a better understanding of their capabilities and the art of crafting effective prompts. Second, the value of the writing process should be valued (Barrot, 2023). Students can document their writing steps (e.g., topic selection, outlining, drafting, and revising) that lead to their final product. Students can employ the GenAI feedback in the revising and editing stages but use their own critical thinking skills to evaluate and refine the feedback. Third, students should be encouraged to develop their distinctive writing style and voice. Teachers provide guidance on how to effectively convey emotions and express true feelings through writing. Finally, peer review can still be used alongside GenAI feedback. As a valuable resource for providing feedback on the emotional aspects of writing, peer review should be kept for providing feedback on emotional expression.

The emergence of GenAI offers both difficulties and prospects for L2 writing teachers, urging them to reevaluate their teaching methods in the writing class. Instead of prohibiting GenAI completely, teachers can seek methods to collaborate with these AI-driven tools and harness their benefits. The study focused on the initial experience of integrating ERNIE Bot into the writing feedback process. However, a long-term evaluation of its impact on students’ writing skills, motivation, and engagement could provide more comprehensive insights. Future studies could examine these aspects over an extended period of time. Despite the limitation, the insights gained in this study may contribute to the broader discussion in the field of educational technology, providing researchers, practitioners, and policymakers with valuable perspectives aimed at leveraging the capabilities of GenAI to enrich L2 writing outcomes while maintaining educational equity and empathy.

Footnotes

Contributorship

Yudan Mi contributed to the acquisition of data, the analysis and interpretation of data, and was instrumental in drafting and editing the manuscript. Mi Rong participated in the analysis and interpretation of data and was involved in revising and editing the manuscript. Xuchuan (Winnie) Chen was responsible for the conceptualization of the study, contributed to the analysis and interpretation of data, and played a pivotal role in revising the manuscript for intellectual content.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical statement

The authors received written informed consent from the participants and the Ethical Committee of the School of Foreign Languages of Huaihua University.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study is funded by the Hunan Provincial Social Science Project, “Peer Interaction among Chinese College Students in EFL Contexts” (Grant No. 23YBA040); the Hunan Provincial Education Science “Thirteenth Five-Year” Planning Project, “The Impact of the Scaffolding Application of AWE on L2 Writing Revisions of Chinese College Students” (Grant No. XJK20CGD002); and the Ministry of Education Industry-Academia Collaborative Education Program, “iWrite-Based Smart Teaching and Research for College English Writing” (Grant No. 231101339100548).

ORCID iD

Xuchuan (Winnie) Chen

Appendix. Coding scheme.

Themes	Categories	Codes	Examples
Technological affordances	Timeliness	Immediate	“文心一言的反馈非常及时，让我更好地修改作文。”(ERNIE Bot provides immediate feedback, enabling me to revise my work more efficiently.)
	Timeliness	Fast	“它反馈很快。” (ERNIE Bot provided a very quick response.)
	Personification	Adaptive guidance	“它能直观地告诉你，哪里对，哪里错了。” (It can clearly show you what is correct and what is wrong.)
	Personification	Progressive guidance	“当我的作文里面的错误没了，文心一言就会提供更高级的写作模式，比如让词汇和句型变得更高级。” (When my writing was free of error, ERNIE Bot provided me with more advanced writing patterns instead, such as enhancing my vocabulary and sentence structures.)
Challenges in leveraging technological affordances	Students’ lack of AI literacy	Fail to grasp AI's intelligence	“要是文心一言也能像人一样灵活聪明就好了。” (If only it could be as flexible and intelligent as a human.)
Challenges in leveraging technological affordances	Students’ lack of AI literacy	Fail to understand AI's humanization	“我认为最好是用它来改文章的结构，还有语法和词。但是它不能够用来去改观点或者思维上的东西，虽然我想让它这么做。”(I believe it is best used for improving the structure of articles, as well as grammar and vocabulary. However, it cannot be used to revise viewpoints or thought processes, even though I wish it could.)
Educational affordances	Essay editor	Local editing	“文心一言对我作文里面的语法错误做了全面的检测，我感到很惊讶。” (I was amazed by how comprehensively ERNIE Bot detected grammatical errors in my writing.)
	Essay editor	Global editing	“它还发现了个不起眼的逻辑问题，但是我的同学并没有指出这个错误。” (It even detected a slight logical inconsistency, which had been missed out by peer reviewers.)
	Responsive tutor	Attentive	“当我感到困惑并接着提出问题时，文心一言提供了详细的解释和例子······我感觉它很关注我的需求。”(When I was puzzled and asked follow-up questions, ERNIE Bot provided detailed explanations and examples. … This makes me feel that it pays close attention to my needs.)
		Patient	“它总是很耐心地回应我的提问。” (It always responds to my questions very patiently.)
		Knowledgeable	“我感觉它更像是那种知识渊博的老师，能给出有针对性而且很全面的建议。” (I feel that it is more like a knowledgeable teacher who can provide comprehensive targeted advice.)
Challenges in leveraging technological affordances	Potential risk of worsening writing skill	Declining writing ability	“过度依赖生成式人工智能进行写作可能导致我们的写作能力和学习能力下降。”(A decrease in our writing ability and a decline in our ability to learn could result from placing an excessive amount of reliance on GenAI for writing.)
Challenges in leveraging technological affordances	Potential risk of worsening writing skill	Waning creative writing ability	“感觉用了文心一言，我就被它带着走了，就没有更多的想法了。” (I feel like using ERNIE Bot directs my thoughts and it makes harder for me to come up with my own ideas.)
Social affordances	Supportive environment	Relieved	“它就像一个能帮助我的老师一样，让我写作文时不觉得有很多压力。” (It is like a counselor who assured me of little anxiety in the writing process.)
	Supportive environment	Encouraging	“它还夸了我一下，我挺开心的。” (It also complimented me, which made me very happy.)。
	Engaging process	Interesting	“跟文心一言可以持续地互动，这个过程很有意思。” (Interacting continuously with ERNIE Bot is an interesting process.)
	Engaging process	Relaxing	“我随便问它什么问题都可以，我觉得很放松。” (I can ask it any question I want, and I feel very relaxed.)
Challenges in leveraging social affordances	Inability to catch emotion	Emotionless	“和同学的反馈相比，它的反馈不那么有感情。” (Compared to peer feedback, its feedback is less emotional.)
Challenges in leveraging social affordances	Inability to catch emotion	Unperceptive	“它给了我太多的建议，我感到挺烦的，但它又不能根据我跟它互动的语气察觉到我的不耐烦，并提供更加简洁的反馈。” (When It Gave Me Too Many Suggestions, I Feel Annoyed. But It Failed To Perceive My Impatience From The Tone In The Interaction And Provide More Concise Feedback.)

References

Abunaseer

(2023). The use of generative AI in education: Applications, and impact . 2023. Technology and the Curriculum: Summer. https://pressbooks.pub/techcurr2023/chapter/the-use-of-generative-ai-in-education-applications-and-impact/

Al-khresheh

M. H.

(2024). Bridging technology and pedagogy from a global lens: Teachers’ perspectives on integrating ChatGPT in English language teaching. Computers and Education: Artificial Intelligence, 6, 100218. https://doi.org/10.1016/j.caeai.2024.100218

Amin

M. Y. M.

(2023). AI And ChatGPT in language teaching: Enhancing EFL classroom support and transforming assessment techniques. International Journal of Higher Education Pedagogies, 4(4), 1–15. https://doi.org/10.33422/ijhep.v4i4.554

Barron

(2003). When smart groups fail. The Journal of the Learning Sciences, 12(3), 307–359. https://doi.org/10.1207/S15327809JLS1203_1

Barrot

J. S.

(2023). Using ChatGPT for second language writing: Pitfalls and potentials. Assessing Writing, 57, 100745. https://doi.org/10.1016/j.asw.2023.100745

Barrot

J. S.

(2024). ChatGPT as a language learning tool: An emerging technology report. Technology, Knowledge and Learning, 29(2), 1151–1156. https://doi.org/10.1007/s10758-023-09711-4

Baskara

(2023). Exploring the implications of ChatGPT for language learning in higher education. Indonesian Journal of English Language Teaching and Applied Linguistics, 7(2), 343–358. https://doi.org/10.21093/ijeltal.v7i2.1387

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Chiu

T. K.

Moorhouse

B. L.

Chai

C. S.

Ismailov

(2023). Teacher support and student motivation to learn with artificial intelligence (AI) based chatbot. Interactive Learning Environments, 32(7), 3240–3256. https://doi.org/10.1080/10494820.2023.2172044

10.

Coniam

(2014). The linguistic accuracy of chatbots: Usability from an ESL perspective. Text & Talk, 34(5), 545–567. https://doi.org/10.1515/text-2014-0018.

11.

Cotton

D. R. E.

Cotton

P. A.

Shipway

J. R.

(2024). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International, 61(2), 228–239. https://doi.org/10.1080/14703297.2023.2190148

12.

Creswell

J. W.

Poth

C. N.

(2016). Qualitative inquiry and research design: Choosing among five approaches. Sage Publications.

13.

Dai

Lin

Jin

Tsai

Y. S.

Gašević

Chen

(2023). Can large language models provide feedback to students? A case study on ChatGPT. In 2023 IEEE International Conference on Advanced Learning Technologies (ICALT), Orem, UT, USA, July 10–13 2023 (pp. 323–325). IEEE.

14.

Dempere

J. M.

Modugu

K. P.

Hesham

Ramasamy

L. K.

(2023). The impact of ChatGPT on higher education. Frontiers in Education, 8(8), 1–13. https://doi.org/10.3389/feduc.2023.1206936

15.

Dwivedi

Y. K.

Nir

Laurie

Emma

L. S.

Anand

Arpan

K. K.

Abdullah

M. B.

Alex

Vishnupriya

Manju

(2023). "So what if ChatGPT wrote it?' Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642

16.

Escalante

Pack

Barrett

(2023). AI-generated feedback on writing: Insights into efficacy and ENL student preference. International Journal of Educational Technology in Higher Education, 20(1), 57. https://doi.org/10.1186/s41239-023-00425-2

17.

Fryer

L. K.

Ainley

Thompson

Gibson

Sherlock

(2017). Stimulating and sustaining interest in a language course: An experimental comparison of Chatbot and Human task partners. Computers in Human Behavior, 75, 461–468. https://doi.org/10.1016/j.chb.2017.05.045

18.

Fryer

Carpenter

(2006). Bots as language learning tools. Language Learning & Technology, 10(3), 8–14. https://eric.ed.gov/?id=EJ815729

19.

Fryer

L. K.

Nakao

Thompson

(2019). Chatbot learning partners: Connecting learning experiences, interest and competence. Computers in Human Behavior, 93, 279–289. https://doi.org/10.1016/j.chb.2018.12.023

20.

Ghafouri

Hassaskhah

Mahdavi-Zafarghandi

(2024). From virtual assistant to writing mentor: Exploring the impact of a ChatGPT-based writing instruction protocol on EFL teachers’ self-efficacy and learners’ writing skill. Language Teaching Research, 13621688241239764. https://doi.org/10.1177/13621688241239764

21.

Gibson

J. J.

(1977). The theory of affordances. In Shaw

Bransford

(Eds.), Perceiving, acting, and knowing: Toward an ecological psychology (pp. 67–82). Lawrence Erlbaum Associates.

22.

Guo

Chu

(2024). Understanding EFL students’ chatbot-assisted argumentative writing: An activity theory perspective. Education Information Technology, 29, 1–20. https://doi.org/10.1007/s10639-023-12230-5

23.

Guo

Wang

Chu

S. K. W.

(2022). Using chatbots to scaffold EFL students’ argumentative writing. Assessing Writing, 54, 100666. https://doi.org/10.1016/j.asw.2022.100666

24.

Hansen

Liu

(2005). Guiding principles for effective peer response. ELT Journal, 59(1), 31–38. https://doi.org/10.1093/elt/cci004

25.

Hockly

(2023). Artificial intelligence in English language teaching: The good, the bad and the ugly. RELC Journal, 54, 445–451. https://doi.org/10.1177/00336882231168504

26.

Hong

W. C. H.

(2023). The impact of ChatGPT on foreign language teaching and learning: Opportunities in education and research. Journal of Educational Technology and Innovation, 5(1), 37–45.

27.

Huang

Hew

K. F.

Fryer

L. K.

(2021). Chatbots for language learning—Are they really useful? A systematic review of chatbot-supported language learning. Journal of Computer Assisted Learning, 38(1), 237–257. https://doi.org/10.1111/jcal.12610

28.

Hyland

(2006). Feedback on second language students’ writing. Language Teaching, 39(2), 83–101. https://doi.org/10.1017/S0261444806003399

29.

Ingley

S. J.

Pack

(2023). Leveraging AI tools to develop the writer rather than the writing. Trends in Ecology Evolution, 38(9), 785–787. https://doi.org/10.1016/j.tree.2023.05.007

30.

Jeon

Lee

(2023). Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT. Education and Information Technologies, 28(12), 15873–15892. https://doi.org/10.1007/s10639-023-11834-1

31.

Jia

Chen

Ding

Ruan

(2012). Effects of a vocabulary acquisition and assessment system on students' performance in a blended learning class for English subject. Computers & Education, 58(1), 63–76. https://doi.org/10.1016/j.compedu.2011.08.002

32.

Jia

Ruan

(2008, June). Use chatbot CSIEC to facilitate the individual learning in English instruction: A case study. In B. P. Woolf, E. Aïmeur, R. Nkambou, & S. Lajoie (Eds.), International conference on intelligent tutoring systems (pp. 706–708). Springer Berlin Heidelberg.

33.

Johnson

W. L.

Valente

(2009). Tactical language and culture training systems: Using AI to teach foreign languages and cultures. AI Magazine, 30(2), 72–83. https://doi.org/10.1609/aimag.v30i2.2240

34.

Jones

Hafner

(2022). Understanding Digital literacies: A practical introduction. Routledge.

35.

Kasneci

Seßler

Küchemann

Bannert

Dementieva

Fischer

Kasneci

(2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274

36.

Kerly

Hall

Bull

(2007). Bringing chatbots into education: Towards natural language negotiation of open learner models. Knowledge-Based Systems, 20(2), 177–185. https://doi.org/10.1016/j.knosys.2006.11.014

37.

Kim

Detrick

(2024). Exploring students’ perspectives on Generative AI-assisted academic writing. Education and Information Technologies, 29(1), 1–36. https://doi.org/10.1007/s10639-024-12878-7

38.

Kirschner

Strijbos

J.-W.

Kreijns

Beers

P. J.

(2004). Designing electronic collaborative learning environments. Educational Technology Research and Development, 52(3), 47–66. https://doi.org/10.1007/BF02504675

39.

Kohnke

Moorhouse

B. L.

Zou

(2023). ChatGPT for language teaching and learning. RELC Journal, 54(2), 537–550. https://doi.org/10.1177/00336882231162868

40.

Kuhail

M. A.

Alturki

Alramlawi

Alhejori

(2023). Interacting with educational chatbots: A systematic review. Education and Information Technologies, 28, 973–1018. https://doi.org/10.1007/s10639-022-11177-3

41.

Labadze

Grigolia

Machaidze

(2023). Role of AI chatbots in education: Systematic literature review. International Journal of Educational Technology in Higher Education, 20(1), 56. https://doi.org/10.1186/s41239-023-00426-1

42.

Lee

Hsia

C. C.

Tsoy

Choi

Hou

(2023). VisionARy: Exploratory research on contextual language learning using AR glasses with ChatGPT. In Gena

De Russis

Spano

Lanzilotti

Di Mascio

Prandi

Andolina

(Eds.), Proceedings of the 15th biannual conference of the Italian SIGCHI Chapter (pp. 1–6). Association for Computing Machinery.

43.

Liu

(2023). Measuring EFL learners' use of ChatGPT in informal digital learning of English based on the technology acceptance model. Innovation in Language Learning and Teaching, 18(2), 1–14. https://doi.org/10.1080/17501229.2023.2240316

44.

Yao

Xiao

Yuan

Wang

Zhu

(2024). Can ChatGPT effectively complement teacher assessment of undergraduate students' academic writing? Assessment & Evaluation in Higher Education, 49(2), 1–18. https://doi.org/10.1080/02602938.2024.2301722

45.

Mageira

Pittou

Papasalourous

Kotis

Zangogianni

Daradoumis

(2022). Educational AI chatbot for content and language integrated learning. Applied Sciences, 12(7), 3239. https://doi.org/10.3390/app12073239

46.

Marzuki, Widiati

Rusdin

Darwin Indrawati

(2023). The impact of AI writing tools on the content and organization of students’ writing: EFL teachers’ perspective. Cogent Education, 10, 1–17. https://doi.org/10.1080/2331186X.2023.2236469

47.

Miles

M. B.

Huberman

A. M.

(1994). Qualitative data analysis: An expanded sourcebook. Sage Publications.

48.

Mizumoto

Eguchi

(2023). Exploring the potential of using an AI language model for automated essay scoring. Research Methods in Applied Linguistics, 2(2), 100050. https://doi.org/10.1016/j.rmal.2023.100050

49.

Moorhouse

B. L.

Kohnke

(2024). The effects of generative AI on initial language teacher education: The perceptions of teacher educators. System, 122, 103290. https://doi.org/10.1016/j.system.2024.103290

50.

Moorhouse

Wan

Lin

(2024). Generative AI-assisted, evidence-informed use of L1 in L2 classrooms. ELT Journal, 78(4), 453–465. https://doi.org/10.1093/elt/ccae033

51.

D. T. K.

Leung

J. K. L.

Chu

S. K. W.

Qiao

M. S.

(2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, 100041. https://doi.org/10.1016/j.caeai.2021.100041

52.

Nguyen

Ngo

H. N.

Hong

Dang

Nguyen

B.-P. T.

(2022). Ethical principles for artificial intelligence in education. Education and Information Technologies, 28(4), 4221–4241. https://link.springer.com/article/10.1007/s10639-022-11316-w

53.

Norman

(1988). The psychology of everyday things. Basic Books.

54.

OpenAI. (2023). ChatGPT. OpenAI. https://chat.openai.com/chat

55.

Reed

D. K.

Petscher

Truckenmiller

A. J.

(2016). The contribution of general reading ability to science achievement. Reading Research Quarterly, 52(2), 253–266. https://doi.org/10.1002/rrq.158

56.

Rudolph

J. W.

Simon

Dufresne

R. L.

Raemer

D. B.

(2006). There's no such thing as “nonjudgmental” debriefing: A theory and method for debriefing with good judgment. Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare, 1(1), 49–55. https://doi.org/10.1097/01266021-200600110-00006

57.

Schmidt-Fajlik

(2023). ChatGPT as a grammar checker for Japanese English language learners: A comparison with Grammarly and ProWriting Aid. Asia CALL Online Journal, 14(1), 105–119. https://doi.org/10.54855/acoj.231417

58.

Schmidt

(1990). The role of consciousness in second language learning. Applied Linguistics, 11, 129–158. https://doi.org/10.1093/applin/11.2.129

59.

Schmulian

Coetzee

S. A.

(2019). The development of messenger bots for teaching and learning and accounting students' experience of the use thereof. British Journal of Educational Technology, 50(5), 2751–2777. https://doi.org/10.1111/bjet.12723

60.

Lin

Lai

(2023). Collaborating with ChatGPT in argumentative writing classrooms. Assessing Writing, 57, 1–11. https://doi.org/10.1016/j.asw.2023.100752

61.

Vera

(2023). Enhancing English language learning in undergraduate students using ChatGPT: A quasi-experimental study. Congreso International de Aprendizaje Activo, 2023, 18.

62.

Wang

(2014). Students' perceptions of rubric-referenced peer feedback on EFL writing: A longitudinal inquiry. Assessing Writing, 19, 80–96. https://doi.org/10.1016/j.asw.2013.11.008

63.

Wang

E. L.

Matsumura

L. C.

Correnti

Litman

Zhang

Howe

Quintana

(2020). eRevis(ing): Students' revision of text evidence use in an automated writing evaluation system. Assessing Writing, 44, 100449. https://doi.org/10.1016/j.asw.2020.100449

64.

Wardat

Tashtoush

M. A.

AlAli

Jarrah

A. M.

(2023). ChatGPT: A revolutionary tool for teaching and learning mathematics. Eurasia Journal of Mathematics, Science and Technology Education, 19(7), em2286. https://doi.org/10.29333/ejmste/13272

65.

Warschauer

(1996). Computer-assisted language learning: An introduction. In Fotos

(Ed.), Multimedia language teaching (pp. 3–20). Logos International.

66.

Warschauer

Tseng

Yim

Webster

Jacob

Tate

(2023). The affordances and contradictions of AI-generated text for writers of English as a second or foreign language. Journal of Second Language Writing, 62, 101071https://doi.org/10.1016/j.jslw.2023.101071

67.

Zhang

(2024). Understanding learners' perceptions of ChatGPT: A thematic analysis of peer interviews among undergraduates and postgraduates in China. Heliyon, 10(4). https://doi.org/10.1016/j.heliyon.2024.e26239

68.

Yan

(2023). Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation. Education and Information Technologies, 28(11), 13943–13967. https://doi.org/10.1007/s10639-023-11742-4

69.

Yang

(2024). ChatGPT for L2 learning: Current status and implications. System, 124, 103351. https://doi.org/10.1016/j.system.2024.103351

70.

Yao

Sun

Zhu

(2024). A qualitative inquiry into metacognitive strategies of postgraduate students in employing ChatGPT for English academic writing. European Journal of Education, e12824. https://doi.org/10.1111/ejed.12824

71.

Zhou

Cao

Zhou

Zhang

(2023). Chinese intermediate English learners outdid ChatGPT in deep cohesion: Evidence from English narrative writing. System, 118, 103141. https://doi.org/10.1016/j.system.2023.103141

72.

Zou

Huang

(2023). To use or not to use? Understanding doctoral students' acceptance of ChatGPT in writing through technology acceptance model. Frontiers in Psychology, 14, 1259531. https://doi.org/10.3389/fpsyg.2023.1259531

Exploring the Affordances and Challenges of GenAI Feedback in L2 Writing Instruction: A Comparative Analysis With Peer Feedback

Abstract

Purposes

Design/Approach/Method

Findings

Originality/Value

Keywords

Introduction

Literature review

GenAI in second language learning

GenAI feedback in L2 writing

Usefulness theoretical framework

Method

Context and participants

Data collection and analysis

Findings

Affordances of GenAI feedback

Technological affordances

Educational affordances

Social affordances

Challenges of leveraging ERNIE bot-generated feedback

Students’ lack of AI literacy

Potential risk of worsening writing ability

Inability to catch emotion

Inability to be perceptive

Discussion

Conclusion

Footnotes

Contributorship

Declaration of conflicting interests

Ethical statement

Funding

ORCID iD

Appendix. Coding scheme.

References