From Immediate Answers to Structured Inquiry: A Socratic Method-Enhanced ChatGPT System for Blended Learning Discussions

Abstract

Artificial Intelligence integration with mobile instant messaging (MIM) offers promising support for blended learning discussions, yet current implementations often favor immediacy over pedagogical structure. Despite separate studies on Socratic methods, ChatGPT applications, and MIM platforms, their systematic integration remains unexplored. This study introduces SCMIM (Socratic-based ChatGPT Mobile Instant Messaging) and compares its impact with standard ChatGPT-MIM and traditional MIM through a 12-week randomized trial with 72 master’s students. Using mixed-methods analysis of engagement questionnaires, discussion logs, and interviews, we examined behavioral, cognitive, and emotional engagement alongside technology acceptance perceptions. Results showed ChatGPT-MIM enhanced behavioral participation and emotional comfort through immediate support, while SCMIM more effectively fostered cognitive engagement, particularly analytical thinking and knowledge construction. Despite initial adaptation challenges, SCMIM users developed appreciation for systematic questioning, reporting high perceived usefulness and comparable intention to use. These findings demonstrate that effective AI integration requires balancing immediate support with structured pedagogical guidance to promote both engagement and deeper learning outcomes in educational discussions.

Keywords

blended learning ChatGPT mobile instant messaging socratic method student engagement student perceptions

Get full access to this article

View all access options for this article.

References

Allison

(2025). RAISE the standard: A framework for transparent reporting of artificial intelligence studies in education. Journal of Educational Computing Research, 64(1). https://doi.org/10.1177/07356331251377430

Annamalai

(2019). Using WhatsApp to extend learning in a blended classroom environment. Teaching English with Technology, 19(1), 3–20.

Ardèvol-Abreu

Patricia

Helena

Rodríguez-Wangüemert

(2023). Mobile instant messaging uses and technostress: A qualitative approach. International Journal of Human-Computer Interaction, 39(15), 3003–3015. https://doi.org/10.1080/10447318.2022.2090525

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Chen

Aklikokou

A. K.

(2020). Determinants of E-government adoption: Testing the mediating effects of perceived usefulness and perceived ease of use. International Journal of Public Administration, 43(10), 850–865. https://doi.org/10.1080/01900692.2019.1660989

Cohen

(2013). Statistical power analysis for the behavioral sciences. Routledge.

Cuban

Kirkpatrick

Peck

(2001). High access and low use of technologies in high School classrooms: Explaining an apparent paradox. American Educational Research Journal, 38(4), 813–834. https://doi.org/10.3102/00028312038004813

Davis

F. D.

(1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340. https://doi.org/10.2307/249008

Elder

Paul

(2010). Critical thinking: Competency standards essential for the cultivation of intellectual skills, Part 1. Journal of Developmental Education, 34(2), 38–39. https://www.proquest.com/scholarly-journals/critical-thinking-competency-standards-essential/docview/1285469423/se-2

10.

Entwistle

(2017). Teaching for understanding at university: Deep approaches and distinctive ways of thinking. Bloomsbury Publishing.

11.

Etori

N. A.

Gini

(2024). WisCompanion: Integrating the Socratic Method with ChatGPT-Based AI for Enhanced Explainability in Emotional Support for Older Adults. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2024. Lecture Notes in Computer Science (vol 14734). Springer, Cham. https://doi.org/10.1007/978-3-031-60606-9_11

12.

Faranda

W. T.

Clarke

T. B.

ClarkeIii

(2020). Marketing student perceptions of academic program quality and relationships to surface, deep, and strategic learning approaches. Journal of Marketing Education, 43(1), 9–24. https://doi.org/10.1177/0273475320939261

13.

Faul

Erdfelder

Lang

A.-G.

Buchner

(2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146

14.

Fisher

Perényi

Á.

Birdthistle

(2018). The positive relationship between flipped and blended learning and student engagement, performance and satisfaction. Active Learning in Higher Education, 22(2), 97–113. https://doi.org/10.1177/1469787418801702

15.

Fredricks

J. A.

Blumenfeld

P. C.

Paris

A. H.

(2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109. https://doi.org/10.3102/00346543074001059

16.

Garrison

D. R.

Cleveland-Innes

(2005). Facilitating cognitive presence in online learning: Interaction is not enough. American Journal of Distance Education, 19(3), 133–148. https://doi.org/10.1207/s15389286ajde1903_2

17.

Huang

Y.-M.

Chen

P.-H.

Lee

H.-Y.

Sandnes

F. E.

T.-T.

(2025). ChatGPT-enhanced mobile instant messaging in online learning: Effects on student outcomes and perceptions. Computers in Human Behavior, 168, Article 108659. https://doi.org/10.1016/j.chb.2025.108659

18.

Huberty

C. J.

Morris

J. D.

(1992). Multivariate analysis versus multiple univariate analyses. In Methodological issues & strategies in clinical research (pp. 351–365). American Psychological Association. https://doi.org/10.1037/10109-030

19.

Labadze

Grigolia

Machaidze

(2023). Role of AI chatbots in education: Systematic literature review. International Journal of Educational Technology in Higher Education, 20(1), 56. https://doi.org/10.1186/s41239-023-00426-1

20.

Lee

H.-Y.

Chen

P.-H.

Wang

W.-S.

Huang

Y.-M.

T.-T.

(2024). Empowering ChatGPT with guidance mechanism in blended learning: Effect of self-regulated learning, higher-order thinking skills, and knowledge construction. International Journal of Educational Technology in Higher Education, 21(1), 16. https://doi.org/10.1186/s41239-024-00447-4

21.

Lee

H.-Y.

Huang

Y.-M.

(2024). Bridging STEM Education and Ubiquitous Learning: A Case Study on Developing a LINE Chatbot with Google's Gemini for Virtual Peer Collaboration. In: Cheng, YP., Pedaste, M., Bardone, E., Huang, YM. (eds) Innovative Technologies and Learning. ICITL 2024. Lecture Notes in Computer Science, vol 14786. Springer, Cham. https://doi.org/10.1007/978-3-031-65884-6_25

22.

Lee

H.-Y.

Huang

Y.-M.

T.-T.

(2025). ChatGPT in education: A systematic review of current landscape, limitations and future directions through general system theory lens. European Journal of Education, 60(4), Article e70262. https://doi.org/10.1111/ejed.70262

23.

Lee

H.-Y.

Lin

C.-J.

Wang

W.-S.

Chang

W.-C.

Huang

Y.-M.

(2023). Precision education via timely intervention in K-12 computer programming course to enhance programming skill and affective-domain learning objectives. International Journal of STEM Education, 10(1), 52. https://doi.org/10.1186/s40594-023-00444-5

24.

Lee

H.-Y.

T.-T.

(2025). Enhancing blended learning discussions with a Scaffolded Knowledge Integration–Based ChatGPT mobile instant messaging system. Computers & Education, 237, Article 105375. https://doi.org/10.1016/j.compedu.2025.105375

25.

Zhang

(2023). Why is instant messaging not instant? Understanding users’ negative use behavior of instant messaging software. Computers in Human Behavior, 142, Article 107655. https://doi.org/10.1016/j.chb.2023.107655

26.

Lodge

Kennedy

Lockyer

Arguel

Pachman

(2018). Understanding difficulties and resulting confusion in learning: An integrative review. Frontiers in Education, 3, 49. https://doi.org/10.3389/feduc.2018.00049

27.

Mali

Lim

(2021). How do students perceive face-to-face/blended learning as a result of the Covid-19 pandemic? International Journal of Management in Education, 19(3), Article 100552. https://doi.org/10.1016/j.ijme.2021.100552

28.

Oyler

D. R.

Romanelli

(2014). The fact of ignorance revisiting the socratic method as a tool for teaching critical thinking. American Journal of Pharmaceutical Education, 78(7), 144. https://doi.org/10.5688/ajpe787144

29.

Patel

N. S.

(2022). Development of criticality in thought: A conceptual framework for online student discussion forums in higher education. https://doi.org/10.46451/ijts.2021.09.02

30.

Paul

Elder

(2007). Critical thinking: The art of socratic questioning. Journal of Developmental Education, 31(1), 36–37.

31.

Ryan

R. M.

Deci

E. L.

(2020). Intrinsic and extrinsic motivation from a self-determination theory perspective: Definitions, theory, practices, and future directions. Contemporary Educational Psychology, 61, Article 101860. https://doi.org/10.1016/j.cedpsych.2020.101860

32.

Shang

H.-F.

(2021). The impact of blended discussions on learning motivation and EFL reading performance. Educational Studies, 57(6), 629–649. https://doi.org/10.1080/00131946.2021.1969932

33.

Steenhof

Woods

N. N.

Mylopoulos

(2020). Exploring why we learn from productive failure: Insights from the cognitive and learning sciences. Advances in Health Sciences Education, 25(5), 1099–1106. https://doi.org/10.1007/s10459-020-10013-y

34.

Steinberg

(2020). LINE as super app: Platformization in East Asia. Social Media + Society, 6(2), Article 2056305120933285. https://doi.org/10.1177/2056305120933285

35.

Sun

J. C.-Y.

Rueda

(2012). Situational interest, computer self-efficacy and self-regulation: Their impact on student engagement in distance education. British Journal of Educational Technology, 43(2), 191–204. https://doi.org/10.1111/j.1467-8535.2010.01157.x

36.

Tang

Hew

K. F.

(2017). Is mobile instant messaging (MIM) useful in education? Examining its technological, pedagogical, and social affordances. Educational Research Review, 21, 85–104. https://doi.org/10.1016/j.edurev.2017.05.001

37.

Tang

Hew

K. F.

(2022). Effects of using mobile instant messaging on student behavioral, emotional, and cognitive engagement: A quasi-experimental study. International Journal of Educational Technology in Higher Education, 19(1), 3. https://doi.org/10.1186/s41239-021-00306-6

38.

Tomei

L. A.

Maine

Moussa

Holler

M. B.

Hobbs

Austin

(2024). The top 12 technologies for teaching and learning in the post-pandemic era. In Exploring technology-infused education in the post-pandemic era (pp. 1–95). IGI Global. https://doi.org/10.4018/979-8-3693-2885-9.ch001

39.

Vygotsky

L. S.

(1978). Mind in society: The development of higher psychological processes (86). Harvard University Press.

40.

T.-T.

Lee

H.-Y.

Chen

P.-H.

Lin

C.-J.

Huang

Y.-M.

(2025). Integrating peer assessment cycle into ChatGPT for STEM education: A randomised controlled trial on knowledge, skills, and attitudes enhancement. Journal of Computer Assisted Learning, 41(1), Article e13085. https://doi.org/10.1111/jcal.13085

41.

T.-T.

Lee

H.-Y.

P.-H.

Huang

C.-N.

Huang

Y.-M.

(2023). Promoting self-regulation progress and knowledge construction in blended learning via ChatGPT-based learning aid. Journal of Educational Computing Research, 61(8), 3–31. https://doi.org/10.1177/07356331231191125

42.

Yen

S.-C.

Lee

Enriquez

(2018). Learning online, offline, and in-between: Comparing student academic outcomes and course satisfaction in face-to-face, online, and blended teaching modalities. Education and Information Technologies, 23(5), 2141–2153. https://doi.org/10.1007/s10639-018-9707-5

43.

Zimba

Z. F.

Khosa

Pillay

(2021). Using blended learning in South African social work education to facilitate student engagement. Social Work Education, 40(2), 263–278. https://doi.org/10.1080/02615479.2020.1746261