Sage Journals: Discover world-class research

Abstract

One-on-one tutoring is highly effective but remains difficult to scale. Large Language Models (LLMs) offer a potential solution for providing scalable, personalized instruction. Commercial publishers are already integrating AI-driven tutoring into courseware, yet open-source textbooks lack the funding to provide such interactive support. This study compared two LLM-based tutor designs intended to provide AI functionality for open-source textbooks: a structured textbook-integrated tutor and a flexible standalone tutor. Participants (N = 79) were randomly assigned to one of three conditions: textbook-integrated tutor, standalone tutor, or textbook-only control, completing a learning task followed by a posttest and user experience surveys. No significant differences emerged in quiz scores, engagement, or satisfaction. However, participants using the standalone tutor or textbook-only condition reported higher effort than those using the integrated tutor. While no short-term learning advantage was found, these findings emphasize design trade-offs, suggesting future research should explore extended interventions, embedded delivery, and within-subject comparisons.

Keywords

artificial intelligence open-educational resources large language models human-computer interaction

Get full access to this article

View all access options for this article.

References

American Psychological Association. (2018, April 19). Distractor. APA Dictionary of Psychology. https://dictionary.apa.org/distractor

Bloom

B. S.

(1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13(6), 4–16.

Brice

(2025, January 4). Textbook giant McGraw Hill sees potential for AI in the classroom—and the risks. Fortune. https://fortune.com/2025/01/04/textbook-giant-mcgraw-hill-potential-risks-ai-classroom-education/

Chen

Jensen

Albert

L. J.

Gupta

Lee

(2023). Artificial intelligence (AI) student assistants in the classroom: Designing chatbots to support student success. Information Systems Frontiers, 25, 161–182. https://doi.org/10.1007/s10796-022-10291-4

Deng

Jiang

Liu

(2025). Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies. Computers & Education, 227, 105224. https://doi.org/10.1016/j.compedu.2024.105224

Graesser

A. C.

Jackson

G. T.

Mitchell

H. H.

Ventura

Olney

Louwerse

M. M.

(2004). AutoTutor: A tutor with dialogue in natural language. Behavior Research Methods Instruments & Computers, 36, 180–192. https://doi.org/10.3758/bf03195563

Adesope

O. O.

Nesbit

J. C.

Liu

(2014). Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of Education & Psychology, 106(4), 901–918. https://doi.org/10.1037/a0037123

O’Brien

H. L.

Cairns

Hall

(2018). A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. International Journal of Human-Computer Studies, 112, 28–39. https://doi.org/10.1016/j.ijhcs.2018.01.004

Spielman

R. M.

Jenkins

W. J.

Lovett

M. D.

(2024). Psychology 2e. OpenStax.

10.

Thong

J. Y. L.

Hong

S.-J.

Tam

K. Y.

(2006). The effects of post-adoption beliefs on the expectation-confirmation model for information technology continuance. International Journal of Human-Computer Studies, 64(9), 799–810. https://doi.org/10.1016/j.ijhcs.2006.05.001

11.

VanLehn

(2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221. https://doi.org/10.1080/00461520.2011.611369

12.

(2024). Do AI chatbots improve students learning outcomes? Evidence from a meta-analysis. British Journal of Educational Technology, 55(1), 10–33. https://doi.org/10.1111/bjet.13334

AI Tutors and Open-Source Textbooks: Bridging the Gap in Digital Learning

Abstract

Keywords

Get full access to this article

References