Role-play activities are considered a useful instructional design in enhancing the speaking performance of foreign language learners. However, in the traditional classroom context, learners may not readily have access to interlocutors for role-play activities. In this study, we proposed a designed method that integrated the GenAI agent into role-play activities and conducted a quasi-experiment at a Chinese university. A total of 53 Chinese students in an English course were assigned to an experimental group (n = 29) that engaged in role-playing with a GenAI agent and a control group (n = 24) that engaged in role-playing with their class peers. The results showed role-play activities with GenAI could facilitate students’ speaking performance, and one-way ANCOVA showed there was no significant difference in speaking performance improvement when compared with traditional role-play activities with peers. Moreover, the paired-sample Wilcoxon test showed that the treatment students had significant intrinsic motivation and self-efficacy improvement, which were higher than the control group. The findings provide important implications for the design and implementation of GenAI-integrated role-play activities for foreign language speaking learning.
AlshumaimeriY. A.AlshememryA. K. (2024). The extent of AI applications in EFL learning and teaching. IEEE Transactions on Learning Technologies, 17, 653–663. https://doi.org/10.1109/TLT.2023.3322128
2.
ArifahK. F.SantosaR.NgadisoN. (2018). Role playing as technique for teaching speaking to help developing the students’ character. Research and Innovation in Language Learning, 1(2), 67–75. https://doi.org/10.33603/rill.v1i2.1068
BibauwS.FrançoisT.DesmetP. (2018). Discussing with a computer to practice a foreign language: From a conceptual framework to a research agenda for dialogue-based CALL. Computer Assisted Language Learning, 32(8), 827–877. https://doi.org/10.1080/09588221.2018.1535508
5.
ChiuT. K. F.MoorhouseB. L.ChaiC. S.IsmailovM. (2023). Teacher support and student motivation to learn with Artificial Intelligence (AI) based chatbot. Interactive Learning Environments, 32(7), 3240–3256. https://doi.org/10.1080/10494820.2023.2172044
6.
CreelyE. (2024). Exploring the role of generative AI in enhancing language learning: Opportunities and challenges. International Journal of Changes in Education, 1(3), 158–167. https://doi.org/10.47852/bonviewIJCE42022495
7.
CromptonH.BurkeD. (2024). The educational affordances and challenges of ChatGPT: State of the field. TechTrends, 68(2), 380–392. https://doi.org/10.1007/s11528-024-00939-0
8.
EbadiS.AzizimajdH. (2023). Exploring the impact of role-playing in clubhouse on the oral fluency and willingness to communicate of EFL learners. Education and Information Technologies, 29(7), 7927–7968. https://doi.org/10.1007/s10639-023-12053-4
9.
EloualiF. Z. (2023). Incorporating role-play as an innovative approach to fostering English Foreign Language students’ creativity and critical thinking. Global Journal of Foreign Language Teaching, 13(2), 142–155. https://doi.org/10.18844/gjflt.v13i2.8718
10.
EndahatiN. (2014). Improving speaking ability on public speaking class by using role-playing technique of English education department students. Journal of English Language Teaching and English Linguistics, 1(1), 85–95. https://doi.org/10.31316/eltics.v1i1.384
11.
FallonT. L.AylettR.MinnisH.RajendranG. (2018). Investigating social vulnerability in children using computer mediated role-play. Computers & Education, 125, 458–464. https://doi.org/10.1016/j.compedu.2018.06.027
12.
GaoF.NohJ. J.KoehlerM. J. (2009). Comparing role-playing activities in second life and face-to-face environments. Journal of Interactive Learning Research, 20(4), 423–443.
13.
GopalanM.RosingerK.AhnJ. B. (2020). Use of quasi-experimental research designs in education research: Growth, promise, and challenges. Review of Research in Education, 44(1), 218–243. https://doi.org/10.3102/0091732X20903302
14.
HamzahM. Z.De VegaN.RahayuS.Sumihatul UmmahM. S.PattiasinaP. J. (2023). Role-playing method for language development in elementary school. Journal of Childhood Development, 3(2), 36–47. https://doi.org/10.25217/jcd.v3i2.3799
15.
HenisahR.MarganaM.PutriR. Y.KhanH. S. (2023). Role play technique to improve students’ speaking skills. International Journal of Contemporary Studies in Education, 2(3), 176–182. https://doi.org/10.56855/ijcse.v2i3.618
16.
HenryA.CliffordsonC. (2015). The impact of out-of-school factors on motivation to learn English: Self-discrepancies, beliefs, and experiences of self-authenticity. Applied Linguistics, 38(5), 713–736. https://doi.org/10.1093/applin/amv060
17.
HuB.ZhengL.ZhuJ.DingL.WangY.GuX. (2024). Teaching plan generation and evaluation with GPT-4: Unleashing the potential of LLM in instructional design. IEEE Transactions on Learning Technologies, 17, 1471–1485. https://doi.org/10.1109/TLT.2024.3384765
18.
IdhamS. Y.SubramaniamI.KhanA.MugairS. K. (2022). The effect of role-playing techniques on the speaking skills of students at university. Theory and Practice in Language Studies, 12(8), 1622–1629. https://doi.org/10.17507/tpls.1208.19
19.
IshakS.Abdul AzizA. (2022). Role play to improve ESL learners’ communication skills: A systematic review. International Journal of Academic Research in Business and Social Sciences, 12(10), 884. https://doi.org/10.6007/IJARBSS/v12-i10/14851
20.
JangY.LeeJ.KimK.-E. (2021). GPT-critic: Offline reinforcement learning for end-to-end task-oriented dialogue systems. In International conference on learning representations.
21.
KaoG. Y.-M.ChiangX.-Z.FoulshamT. (2019). Reading behavior and the effect of embedded selfies in role-playing picture e-books: An eye-tracking investigation. Computers & Education, 136(4), 99–112. https://doi.org/10.1016/j.compedu.2019.03.010
22.
KarwowskiM.SoszynskiM. (2008). How to develop creative imagination?: Assumptions, aims and effectiveness of role play training in creativity (RPTC). Thinking Skills and Creativity, 3(2), 163–171. https://doi.org/10.1016/j.tsc.2008.07.001
23.
KasneciE.SeßlerK.KüchemannS.BannertM.DementievaD.FischerF.GasserU.GrohG.GünnemannS.HüllermeierE.KruscheS.KutyniokG.MichaeliT.NerdelC.PfefferJ.PoquetO.SailerM.SchmidtA.SeidelT.KasneciG. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274
24.
Kukulska-HulmeA.ShieldL. (2007). An overview of mobile assisted language learning: Can mobile devices support collaborative practice in speaking and listening. ReCALL, 20(3), 1–20. https://doi.org/10.1017/S0958344008000335
25.
KuśnierekA. (2015). Developing students’ speaking skills through role-play. World Scientific News, 7(1), 73–111.
26.
LaiC.-L.TuY.-F. (2024). Roles, strategies, and research issues of generative AI in the mobile learning era. International Journal of Mobile Learning and Organisation, 18(4), 516–537. https://doi.org/10.1504/ijmlo.2024.141836
27.
LawL. (2024). Application of generative artificial intelligence (GenAI) in language teaching and learning: A scoping literature review. Computers and Education Open, 6, 100174. https://doi.org/10.1016/j.caeo.2024.100174
28.
LeeU.JungH.JeonY.SohnY.HwangW.MoonJ.KimH. (2023). Few-shot is enough: Exploring ChatGPT prompt engineering method for automatic question generation in English education. Education and Information Technologies, 29(9), 11483–11515. https://doi.org/10.1007/s10639-023-12249-8
29.
LiB.BonkC. J.WangC.KouX. (2024). Reconceptualizing self-directed learning in the era of generative AI: An exploratory analysis of language learning. IEEE Transactions on Learning Technologies.
30.
LiaoH.XiaoH.HuB. (2023). Revolutionizing ESL teaching with generative artificial intelligence—Take ChatGPT as an example. International Journal of New Developments in Education, 5(20), 39–46. https://doi.org/10.25236/IJNDE.2023.052008
31.
LiaoX.ZhangX.WangZ.LuoH. (2024). Design and implementation of an AI-enabled visual report tool as formative assessment to promote learning achievement and self-regulated learning: An experimental study. British Journal of Educational Technology, 55(3), 1253–1276. https://doi.org/10.1111/bjet.13424
32.
LiuX. (2010). Arousing the college students’ motivation in speaking English through role-play. International Education Studies, 3(1), 136–144. https://doi.org/10.5539/ies.v3n1p136
33.
MaarofN.RahmanN. A. (2018). The effect of role-play and simulation approach on enhancing ESL oral communication skills. International Journal of Research in English Education, 3(3), 63–71. https://doi.org/10.29252/ijree.3.3.63
34.
OckeyG. J.Chukharev-HudilainenE. (2021). Human versus computer partner in the paired oral discussion test. Applied Linguistics, 42(5), 924–944. https://doi.org/10.1093/applin/amaa067
35.
OckeyG. J.Chukharev-HudilainenE.HirchR. R. (2023). Assessing interactional competence: ICE versus a human partner. Language Assessment Quarterly, 20(4–5), 377–398. https://doi.org/10.1080/15434303.2023.2237486
36.
OckeyG. J.GuL.KeehnerM. (2017). Web-based virtual environments for facilitating assessment of L2 oral communication ability. Language Assessment Quarterly, 14(4), 346–359. https://doi.org/10.1080/15434303.2017.1400036
37.
PintrichP. R.De GrootE. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33–40. https://doi.org/10.1037/0022-0663.82.1.33
RobinsonC. I.HarveyT. M.TsengM. (2016). Perceptions of Taiwanese nursing students’ English-language progression following interactive scenario development and role play. Contemporary Nurse, 52(5), 590–601. https://doi.org/10.1080/10376178.2016.1183463
40.
RoeJ.RenandyaW. A.JacobsG. M. (2023). A review of AI-powered writing tools and their implications for academic integrity in the language classroom. Journal of English and Applied Linguistics, 2(1), 1–10. https://doi.org/10.59588/2961-3094.1035
41.
SariE. R.HamzahI. (2023). The use of role play to improve students’ speaking skill. Journal of English Language Teaching and Learning, 4(1), 58–63. https://doi.org/10.22460/project.v2i3.p416-420
42.
SariN. D.MulyadiM.TherianaA. (2020). English teacher’s perception towards the use of role-play in teaching speaking. English Empower: Journal of Linguistics and Literature, 5(2), 62–65. https://doi.org/10.51517/eejll.v5i2.246
43.
Schmidt-FajlikR. (2023). Chatgpt as a grammar checker for Japanese English language learners: A comparison with grammarly and prowritingaid. AsiaCALL Online Journal, 14(1), 105–119. https://doi.org/10.54855/acoj.231417
44.
ShadievR.LiuJ.ChengP.-Y. (2023). The Impact of mobile-assisted social language learning activities on speaking skills and self-efficacy development. IEEE Transactions on Learning Technologies, 16(5), 664–679. https://doi.org/10.1109/tlt.2023.3243721
45.
SmithS. A.FosterM. E.Baffoe-DjanJ. B.LiZ.YuS. (2020). Unifying the current self, ideal self, attributions, self-authenticity, and intended effort: A partial replication study among Chinese university English learners. System, 95, 102377. https://doi.org/10.1016/j.system.2020.102377
46.
SongY.ZhuQ.WangH.ZhengQ. (2024). Automated essay scoring and revising based on open-source large language models. IEEE Transactions on Learning Technologies, 17, 1920–1930. https://doi.org/10.1109/TLT.2024.3396873
47.
SuY.ZhengC.LiangJ.-C.TsaiC.-C. (2018). Examining the relationship between English language learners’ online self-regulation and their self-efficacy. Australasian Journal of Educational Technology, 34(3), 105–121. https://doi.org/10.14742/ajet.3548
48.
SydorenkoT. (2015). The use of computer-delivered structured tasks in pragmatic instruction: An exploratory study. Intercultural Pragmatics, 12(3), 333–362. https://doi.org/10.1515/ip-2015-0017
49.
SydorenkoT.DaurioP.L ThorneS. (2018). Refining pragmatically-appropriate oral communication via computer-simulated conversations. Computer Assisted Language Learning, 31(1–2), 157–180. https://doi.org/10.1080/09588221.2017.1394326
50.
SydorenkoT.JonesZ.DaurioP.ThorneS. (2020). Beyond the curriculum: Extended discourse practice through self-access pragmatics simulations. Language, Learning and Technology, 24(2), 48–69. https://doi.org/10.125/44725
51.
Timpe-LaughlinV.ParkI. (2019). “Are you into beer pong?” Exploring question-answer sequences in an L2 oral performance assessment. Language Assessment Quarterly, 16(1), 21–38. https://doi.org/10.1080/15434303.2019.1609964
52.
Timpe-LaughlinV.SydorenkoT.DaurioP. (2022). Using spoken dialogue technology for L2 speaking practice: What do teachers think?Computer Assisted Language Learning, 35(5–6), 1194–1217. https://doi.org/10.1080/09588221.2020.1774904
53.
Timpe-LaughlinV.SydorenkoT.DombiJ. (2024). Human versus machine: Investigating L2 learner output in face-to-face versus fully automated role-plays. Computer Assisted Language Learning, 37(1–2), 149–178. https://doi.org/10.1080/09588221.2022.2032184
54.
TossellC. C.TenhundfeldN. L.MomenA.CooleyK.de VisserE. J. (2024). Student perceptions of ChatGPT use in a college essay assignment: Implications for learning, grading, and trust in artificial intelligence. IEEE Transactions on Learning Technologies, 17, 1069–1081. https://doi.org/10.1109/tlt.2024.3355015
55.
TuY.-F. (2024). Roles and functionalities of ChatGPT for students with different growth mindsets: Findings of drawing analysis. Educational Technology & Society, 27(1), 198–214. https://doi.org/10.30191/ETS.202401_27(1).TP01
56.
UshiodaE. (2011). Why autonomy? Insights from motivation theory and research. Innovation in Language Learning and Teaching, 5(2), 221–232. https://doi.org/10.1080/17501229.2011.577536
57.
Van DoremalenJ.BovesL.ColpaertJ.CucchiariniC.StrikH. (2016). Evaluating automatic speech recognition-based language learning systems: A case study. Computer Assisted Language Learning, 29(4), 833–851. https://doi.org/10.1080/09588221.2016.1167090
58.
WangL.WangC.YaoX. (2024). Befriended to polarise? The impact of friend identity on review polarisation—A quasi-experiment. Information Systems Journal, 34(3), 680–710. https://doi.org/10.1111/isj.12425
59.
YanD. (2023). Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation. Education and Information Technologies, 28(11), 13943–13967. https://doi.org/10.1007/s10639-023-11742-4
60.
YenY.-C.HouH.-T.ChangK. E. (2015). Applying role-playing strategy to enhance learners’ writing and speaking skills in EFL courses using Facebook and Skype as learning tools: A case study in Taiwan. Computer Assisted Language Learning, 28(5), 383–406. https://doi.org/10.1080/09588221.2013.839568
61.
YoungJ. C.ShishidoM. (2023). Investigating OpenAI’s ChatGPT potentials in generating chatbot’s dialogue for English as a foreign language learning. International Journal of Advanced Computer Science and Applications, 14(6), 1–9. https://doi.org/10.14569/ijacsa.2023.0140607
62.
ZastudilC.RogalskaM.KappC.VaughnJ.MacNeilS. (2023). Generative ai in computing education: Perspectives of students and instructors. In 2023 IEEE Frontiers in Education Conference (FIE), College Station, TX, USA, 18–21 October 2023, 1–9.
63.
ZhaiC.WibowoS. (2023). A systematic review on artificial intelligence dialogue systems for enhancing English as foreign language students’ interactional competence in the university. Computers and Education: Artificial Intelligence, 4, 100134. https://doi.org/10.1016/j.caeai.2023.100134
64.
ZhangZ.CrawfordJ. (2023). EFL learners’ motivation in a gamified formative assessment: The case of Quizizz. Education and Information Technologies, 29(5), 6217–6239. https://doi.org/10.1007/s10639-023-12034-7
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
