When ChatGPT gets it wrong: investigating third position repair by L2 learners at varying proficiency levels

Abstract

This study explores how 30 Korean English as a foreign language (EFL) learners at three proficiency levels (CEFR A2, B1, and B2) manage third position repair during voice-based role-play tasks with ChatGPT-4. Over 12 hours and 30 minutes of recorded interactions were analyzed using conversation analysis, complemented by retrospective verbal reports (RVRs) to capture both interactional patterns and learner perceptions. The findings show clear proficiency-related differences. A2 learners (n = 10) consistently abandoned repair, either by switching to their first language or abruptly terminating the conversation, and none of their TPR sequences resulted in successful repair. B1 learners (n = 10) managed repair partially through simple repetition or confirmation checks but often prioritized progressivity over intersubjectivity; approximately 50% of their TPR sequences were successful. B2 learners (n = 10) consistently produced TPR and maintained intersubjectivity through a wider range of strategies, achieving an 80% success rate. RVRs show that limited repair knowledge and ChatGPT’s technical constraints contributed to frustration and demotivation, particularly among lower-proficiency learners. Overall, the findings suggest that ChatGPT’s technical features can function as both constraints and learning opportunities, underscoring the enduring value of EFL teachers in mediating learner experiences and optimizing the pedagogical potential of AI chatbots.

Keywords

Third position repair AI chatbot conversation analysis ChatGPT repair human–AI interaction AI-mediated language learning

Get full access to this article

View all access options for this article.

References

Albert

de Ruiter

J.P.

(2018). Repair: The interface between interaction and cognition. Topics in Cognitive Science, 10(2), 279–313. https://doi.org/10.1111/tops.12339

Balaman

(2016). A conversation analytic study on the development of interactional competence in English in an online task-oriented environment. Unpublished doctoral dissertation, Hacettepe University.

Beneteau

Richars

Zhang

Kientz

Yip

Hiniker

(2019). Communication breakdowns between families and Alexa. In Proceedings of the 2019 conference on human factors in computing systems (pp. 1–13). Association for Computing Machinery.

Bibauw

Van den Noortgate

François

Desmet

(2022). Dialogue systems for language learning: A meta-analysis. Language Learning & Technology, 26(1), 1–24.

Cambridge University Press & Assessment. (n.d.). Test your English.

Choi

M.G.

S.Y.

(2026). Developing L2 turn-taking with ChatGPT: A longitudinal conversation analytic study. System, 138, 103959. https://doi.org/10.1016/j.system.2025.103959

Collister

L.B.

(2011). *-repair in online discourse. Journal of Pragmatics, 43(3), 918–921. https://doi.org/10.1016/j.pragma.2010.09.025

Derwing

T.M.

Munro

M.J.

(2009). Putting accent in its place: Rethinking obstacles to communication. Language Teaching, 42(4), 476–490. https://doi.org/10.1017/S026144480800551X

Dippold

(2023). ‘Can I have the scan on Tuesday?’ User repair in interaction with a task-oriented chatbot and the question of communication skills for AI. Journal of Pragmatics, 204, 21–32.

10.

Daniel

B.K.

(2024). Transforming language education: A systematic review of AI-powered chatbots for English as a foreign language speaking practice. Computers & Education: Artificial Intelligence, 6, Article 100230. https://doi.org/10.1016/j.caeai.2024.100230

11.

Ebn-Abbasi

Fattahi

Azari Noughabi

Botes

(2024). The strength of self and L2 willingness to communicate: The role of L2 grit, ideal L2 self and language mindset. System, 123, 103334. https://doi.org/10.1016/j.system.2024.103334

12.

Ericsson

K.A.

Simon

H.A.

(1980). Verbal reports as data. Psychological Review, 87(3), 215–251.

13.

Firth

(1996). The discursive accomplishment of normality: On ‘lingua franca’ English and conversation analysis. Journal of Pragmatics, 26(2), 237–259. https://doi.org/10.1016/0378-2166(96)00014-8

14.

Fryer

L.K.

Carpenter

(2006). Bots as language learning tools. Language Learning & Technology, 10(3), 8–14.

15.

Fryer

L.K.

Nakao

Thompson

(2019). Chatbot learning partners: Connecting learning experiences, interest and competence. Computers in Human Behavior, 93, 279–289. https://doi.org/10.1016/j.chb.2018.12.023

16.

Gallacher

Thompson

Howarth

(2018). “My robot is an idiot!” – Students’ perceptions of AI in the L2 classroom. In Taalas

Jalkanen

Bradley

Thouësny

(Eds.), Future-proof CALL: Language learning as exploration and encounters – Short papers from EUROCALL 2018 (pp. 70–76). Research-publishing.net.

17.

Ghafouri

(2024). ChatGPT: The catalyst for teacher-student rapport and grit development in L2 class. System, 120, 103209. https://doi.org/10.1016/j.system.2023.103209.

18.

Goodwin

M.H.

(1980). Processes of mutual monitoring implicated in the production of description sequences. Sociological Inquiry, 50(3–4), 303–317. https://doi.org/10.1111/j.1475-682X.1980.tb00024.x

19.

Goodwin

M.H.

Goodwin

(1986). Gesture and coparticipation in the activity of searching for a word. Semiotica, 62(1–2), 51–75. https://doi.org/10.1515/semi.1986.62.1-2.51

20.

Hall

J.K.

Hellermann

Pekarek Doehler

(Eds.). (2011). L2 interactional competence and development. Multilingual Matters.

21.

A.W.

Young

(1998). Talking and testing. Springer.

22.

Hellermann

(2009). Looking for evidence of language learning in practices for repair: A case study of self-initiated self-repair by an adult learner of English. Scandinavian Journal of Educational Research, 53(2), 113–132.

23.

Hellermann

(2011). Members’ methods, members’ competencies: Looking for evidence of language learning in longitudinal investigations of other-initiated repair. In Hall

Hellermann

Doehler

(Eds.), L2 interactional competence and development (pp. 147–172). Multilingual Matters.

24.

Hosoda

(2006). Repair and relevance of differential language expertise in second language conversations. Applied Linguistics, 27(1), 25–50.

25.

Hwang

Lee

K.-H.

Lee

H.-K.

(2025). A word to the wise: Crafting impactful prompts for ChatGPT. System, 133, Article 103756. https://doi.org/10.1016/j.system.2025.103756

26.

Jefferson

(1987). On exposed and embedded correction in conversation. In Button

Lee

J.R.E.

(Eds.), Talk and social organisation (pp. 86–100). Multilingual Matters.

27.

Jefferson

(2004). Glossary of transcript symbols with an introduction. In Lerner

G.H.

(Ed.), Conversation analysis: Studies from the first generation (pp. 13–23). John Benjamins.

28.

Jeon

(2024). Exploring AI chatbot affordances in the EFL classroom: Young learners’ experiences and perspectives. Computer Assisted Language Learning, 37(1–2), 1–26. https://doi.org/10.1080/09588221.2021.2021241

29.

Kaneyasu

(2025). “Huh?” “Good!”: Teaching conversational repair in introductory L2 classrooms. Classroom Discourse. Advance online publication. https://doi.org/10.1080/19463014.2025.2555006

30.

Kasper

Kim

(2007). Handling sequentially inapposite responses. In Hua

Seedhouse

Wei

Cook

(Eds.), Language learning and teaching as social interaction (pp. 22–41). Palgrave Macmillan.

31.

Khodadady

Alifathabady

(2014). Repair in EFL talk: A case of Iranian intermediate and advanced EFL learners. Theory and Practice in Language Studies, 4(10), 2129–2137. https://doi.org/10.4304/tpls.4.10.2129-2137

32.

Kim

D.R.

(2020). Emergence of proactive self-initiated self-repair as an indicator of L2 IC development. Applied Linguistics, 41(6), 901–921.

33.

Kormos

(1998). Verbal reports in L2 speech production research. TESOL Quarterly, 32(2), 353–358.

34.

Kramsch

(1986). From language proficiency to interactional competence. The Modern Language Journal, 70(4), 366–372. https://doi.org/10.1111/j.1540-4781.1986.tb05291.x

35.

Kwon

S.K.

Shin

Lee

(2023). The application of chatbot as an L2 writing practice tool. Language Learning & Technology, 27(1), 1–19. https://doi.org/10125/73541

36.

Lee

J.S.

(2022). The role of grit and classroom enjoyment in EFL learners’ willingness to communicate. Journal of Multilingual and Multicultural Development, 43(5), 452–468.

37.

Lerner

G.H.

(1996). On the “semi-permeable” character of grammatical units in conversation: Conditional entry into the turn space of another speaker. In Ochs

Schegloff

E.A.

Thompson

S.A.

(Eds.), Interaction and grammar (pp. 238–276). Cambridge University Press.

38.

Meredith

Stokoe

(2014). Repair: Comparing Facebook ‘chat’ with spoken interaction. Discourse & Communication, 8(2), 181–207. https://doi.org/10.1177/1750481313510815

39.

Ministry of Education. (2022). English curriculum, Ministry of Education Notice No. 2022-33 [Appendix 14].

40.

Myers

Furqan

Nebolsy

Caro

Zhu

(2018). Patterns for how users overcome obstacles in voice user interfaces. In Proceedings of the 2018 CHI conference on human factors in computing systems (pp. 1–7). Association for Computing Machinery.

41.

Park

S.-Y.

(2019). Korean EFL learners’ refusals to requests and their perceptions. English Teaching, 74(4), 75–103. https://doi.org/10.15858/engtea.74.4.201912.75

42.

Pekarek Doehler

. (2019). On the nature and the development of L2 interactional competence: State of the art and implications for praxis. In Salaberry

Kunitz

(Eds.), Teaching and testing L2 interactional competence: Bridging theory and practice (pp. 25–59). Routledge.

43.

Pekarek Doehler

Pochon-Berger

. (2011). Developing “methods” for interaction: A cross-sectional study of disagreement sequences in French L2. In Hall

J.K.

Hellermann

Pekarek Doehler

(Eds.), Interactional competence and development (pp. 173–205). Multilingual Matters.

44.

Pekarek Doehler

Pochon-Berger

. (2015). The development of L2 interactional competence: Evidence from turn-taking organization, sequence organization, repair organization and preference organization. In Cadierno

Eskildsen

S.W.

(Eds.), Usage-based perspectives on second language learning (pp. 233–268). De Gruyter.

45.

Pekarek Doehler

Pochon-Berger

. (2018). L2 interactional competence as increased ability for context-sensitive conduct: A longitudinal study of story-openings. Applied Linguistics, 39(4), 555–578. https://doi.org/10.1093/applin/amw021

46.

Randall

(2019). A survey of robot-assisted language learning (RALL). ACM Transactions on Human-Robot Interaction, 9(1), 1–36. https://doi.org/10.1145/3345506

47.

Sacks

(1995). Lectures on conversation. Wiley–Blackwell.

48.

Schegloff

E.A.

(1992). Repair after next turn: The last structurally provided defense of intersubjectivity in conversation. American Journal of Sociology, 98, 1295–1345.

49.

Schegloff

E.A.

(2007). Sequence organization in interaction: A primer in conversation analysis. Cambridge University Press.

50.

Schegloff

E.A.

Jefferson

Sacks

(1977). The preference for self-correction in the organization of repair in conversation. Language, 53(2), 361–382.

51.

Tai

T.Y.

Chen

H.H.J.

(2022). The impact of intelligent personal assistants on adolescent EFL learners’ speaking proficiency. Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2022.2040536

52.

Teimouri

Plonsky

Tabandeh

(2022). L2 grit: Passion and perseverance for second-language learning. Language Teaching Research, 26(5), 893–918. https://doi.org/10.1177/1362168820921895

53.

ten Have

. (2004). Understanding qualitative research and ethnomethodology. Sage Publications Ltd.

54.

ten Have

. (2007). Doing conversation analysis. Sage Publications.

55.

Thompson

Gallacher

Howarth

(2018). Stimulating task interest: Human partners or chatbots? In Taalas

Jalkanen

Bradley

Thouësny

(Eds.), Future-proof CALL: Language learning as exploration and encounters – Short papers from EUROCALL 2018 (pp. 302–306). Research-publishing.net.

56.

Timpe-Laughlin

Sydorenko

Daurio

(2020). Using spoken dialogue technology for L2 speaking practice: What do teachers think? Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2020.1774904

57.

Veivo

Mutta

(2025). Dialogue breakdowns in robot-assisted L2 learning. Computer Assisted Language Learning, 38(1–2), 30–51. https://doi.org/10.1080/09588221.2022.2158203

58.

Wong

Waring

H.Z.

(2020). Conversation analysis and second language pedagogy: A guide for ESL/EFL teachers (2nd ed.). Routledge.

59.

Xin

Derakhshan

(2025). From excitement to anxiety: Exploring English as a foreign language learners’ emotional experiences in the artificial intelligence-powered classrooms. European Journal of Education, 60(1), e12845.

60.

Yang

Zhao

(2024). AI-induced emotions in L2 education: Exploring EFL students’ perceived emotions and regulation strategies. Computers in Human Behavior, 159, 108337. https://doi.org/10.1016/j.chb.2024.108337

61.

Yasui

(2011). Repair and language proficiency: Differences of advanced and beginning language learners in an English–Japanese conversation group. Texas Papers in Foreign Language Education, 14(1), 41–57.

62.

Zhang

Shan

Lee

J.S.Y.

Che

Kim

J.H.

(2023). Effect of chatbot-assisted language learning: A meta-analysis. Education and Information Technologies. Advance online publication. https://doi.org/10.1007/s10639-023-11805-6