Supporting the Instructional Videos With Chatbot and Peer Feedback Mechanisms in Online Learning: The Effects on Learning Performance and Intrinsic Motivation

Abstract

This study investigated the effects of artificial intelligence (AI)-based chatbot and peer feedback mechanisms integrated into the instructional videos (IVs) as a feedback tool on learning performance and intrinsic motivation of pre-service teachers (PTs) in online learning. The participants were 144 PTs from a university in Turkey. A pretest–posttest quasi-experimental design was adopted in this study. Two experimental (EG-1: Immediately elaborated feedback with a chatbot for IVs; EG-2: Delayed peer feedback with comments for IVs) groups and a control group (teaching with IVs) were selected. To collect qualitative data, a survey consisting of open-ended questions was conducted in the experimental groups. The results showed that the learning performance and intrinsic motivation scores of chatbot-based and peer feedback groups were higher than the scores in the traditional learning group. The implications for AI-powered feedback mechanisms and directions for future studies were discussed in this study.

Keywords

video-based learning chatbot feedback teacher education intrinsic motivation artificial intelligence peer feedback online learning

Get full access to this article

View all access options for this article.

References

Bailey

(2019). Chatbots as conversational agents in the context of language learning. Proceedings of the fourth industrial revolution and education (pp. 32–41). Dajeon, South Korea, 27–29 August, 2019.

Bergmann

Sams

(2012). Flip your classroom: Reach every student in every class every day. International Society for Technology in Education.

Borup

West

R. E.

Thomas

(2015). The impact of text versus video communication on instructor feedback in blended courses. Educational Technology Research and Development, 63(2), 161–184. https://doi.org/10.1007/s11423-015-9367-8

Chen

C.-M.

M.-C.

Chang

W.-C.

Chen

X.-X.

(2021). Developing a topic analysis instant feedback system to facilitate asynchronous online discussion effectiveness. Computers & Education, 163(3), 104095. https://doi.org/10.1016/j.compedu.2020.104095

Chen

J. C.

Whittinghill

D. C.

Kadlowec

J. A.

(2010). Classes that click: Fast, rich feedback to enhance student learning and satisfaction. Journal of Engineering Education, 99(2), 159–168. https://doi.org/10.1002/j.2168-9830.2010.tb01052.x

Chen

H.-L.

Widarso

G. V

Sutrisno

(2020). A chatbot for learning Chinese: Learning achievement and technology acceptance. Journal of Educational Computing Research, 58(6), 1161–1189. https://doi.org/10.1177/0735633120929622

Chorianopoulos

(2018). A taxonomy of asynchronous instructional video styles. The International Review of Research in Open and Distributed Learning, 19(1), 294–311. https://doi.org/10.19173/irrodl.v19i1.2920

Day

I. N. Z.

Saab

Admiraal

(2021). Online peer feedback on video presentations: Type of feedback and improvement of presentation skills. Assessment & Evaluation in Higher Education, 47(2), 183–197. https://doi.org/10.1080/02602938.2021.1904826

Debbag

Fidan

(2021). Vlogs for enhancing trainee teachers’ motivational beliefs about school experience and teaching practice: They are on the factory floor for professional development. Journal of Research on Technology in Education. Advance online publication. https://doi.org/10.1080/15391523.2021.1911721

10.

Deci

E. L.

Ryan

R. M.

(1985). Intrinsic motivation and self-determination in human behavior. Plenum.

11.

Deveci Topal

Dilek Eren

Kolburan Geçer

(2021). Chatbot application in a 5th grade science course. Education and Information Technology, 26(5), 6241–6265. https://doi.org/10.1007/s10639-021-10627-8

12.

El-Ariss

Zaneldin

Ahmed

(2021). Using videos in blended e-learning for a structural steel design course. Education Science, 11(6), 290. https://doi.org/10.3390/educsci11060290

13.

Fadhil

Villafiorita

(2017). An adaptive learning with gamification & conversational UIs: The rise of CiboPoliBot. UMAP '17: Adjunct publication of the 25th conference on user modeling, adaptation and personalization (pp. 408–417). Bratislava Slovakia, 9–12 July, 2017. https://doi.org/10.1145/3099023.3099112

14.

Finn

Thomas

Rawson

K. A.

(2018). Elaborating feedback with examples enhances concept learning. Learning and Instruction, 54, 104–113. https://doi.org/10.1016/j.learninstruc.2017.08.007

15.

Fryer

L. K.

Nakao

Thompson

(2019). Chatbot learning partners: Connecting learning experiences, interest and competence. Computers in Human Behavior, 93, 279–289. https://doi.org/10.1016/j.chb.2018.12.023

16.

George

Mallery

(2010). SPSS for windows a step by step: A simple guide and reference. 17.0 update (10th ed.). Pearson.

17.

Gikandi

J. W.

Morrow

(2016). Designing and implementing peer formative feedback within online learning environments. Technology, Pedagogy and Education, 25(2), 153–170. http://dx.doi.org/10.1080/1475939X.2015.1058853

18.

Grossman

Lin

Sheng

Wei

J. T.-Z.

Williams

J. J.

Goel

(2019). MathBot: Transforming online resources for learning math into conversational interactions. http://logical.ai/story/papers/mathbot.pdf

19.

Hattie

Timperley

(2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487

20.

Hsia

L.-H.

Huang

I. I.

Hwang

G.-J.

(2016). Effects of different online peer-feedback approaches on students’ performance skills, motivation and self-efficacy in a dance course. Computers & Education, 96, 55–71. https://doi.org/10.1016/j.compedu.2016.02.004

21.

Huang

Hew

K. F.

Fryer

L. K.

(2021). Chatbots for language learning-are they really useful? A systematic review of chatbot-supported language learning. Journal of Computer Assisted Learning, 38(1), 237–257. https://doi.org/10.1111/jcal.12610HUANGETAL.21

22.

Istenic

(2021). Online learning under COVID-19: Re-examining the prominence of video-based and text-based feedback. Educational Technology Research and Development, 69, 117–121. https://doi.org/10.1007/s11423-021-09955-w

23.

Iterbeke

De Witte

Schelfhout

(2021). The effects of computer-assisted adaptive instruction and elaborated feedback on learning outcomes. A randomized control trial. Computers in Human Behavior, 120(1), 106666. https://doi.org/10.1016/j.chb.2020.106666

24.

Jang

Kim

J.-J.

Kim

S.-J.

Hong

Kim

(2021). Mobile app-based chatbot to deliver cognitive behavioral therapy and psychoeducation for adults with attention deficit: A development and feasibility/usability study. International Journal of Medical Informatics, 150, 104440. https://doi.org/10.1016/j.ijmedinf.2021.104440

25.

Jensen

L. X.

Bearman

Boud

(2021). Understanding feedback in online learning-A critical review and metaphor analysis. Computers & Education, 173, 104271. https://doi.org/10.1016/j.compedu.2021.104271

26.

Kim

N.-Y.

(2017). Chatbots and Korean EFL students’ English vocabulary learning. Journal of Digital Convergence, 16(2), 1–7. https://doi.org/10.14400/JDC.2018.16.2.001

27.

Kulhavy

R. W.

Stock

W. A.

(1989). Feedback in written instruction: The place of response certitude. Educational Psychology Review, 1(4), 279–308. https://doi.org/10.1007/BF01320096

28.

Laurillard

(2013). Rethinking university teaching: A conversational framework for the effective use of learning technologies. Routledge.

29.

Leite

Blanco

S. A.

(2020). Effects of human vs. automatic feedback on students’ understanding of AI concepts and programming style. SIGCSE ‘20 Proceedings of the 51st ACM technical symposium on computer science education (pp. 44–50), Portland, OR, 11–14 March, 2020. https://doi.org/10.1145/3328778.3366921

30.

Lim

L.-A.

Dawson

Gašević

Joksimović

Fudge

Pardo

Gentili

(2020). Students’ sense-making of personalised feedback based on learning analytics. Australasian Journal of Educational Technology, 36(6), 15–33. https://doi.org/10.14742/ajet.6370

31.

Lim

L.-A.

Gentili

Pardo

Kovanović

Whitelock-Wainwright

Gašević

Dawson

(2021). What changes, and for whom? A study of the impact of learning analytics-based process feedback in a large course. Learning and Instruction, 72(3), 101202. https://doi.org/10.1016/j.learninstruc.2019.04.003

32.

Liu

Huang

Zhu

(2020). CBET: Design and evaluation of a domain-specific chatbot for mobile learning. Universal Access in the Information Society, 19(3), 655–673. https://doi.org/10.1007/s10209-019-00666-x

33.

Wong

S. C.

Yang

Bell

(2020). Using feedback to promote student participation in online learning programs: Evidence from a quasi-experimental study. Educational Technology Research and Development, 68(1), 485–510. https://doi.org/10.1007/s11423-019-09709-9

34.

Lowenthal

P. R.

(2021). Video feedback: Is it worth the effort? A response to Borup et al. Educational Technology Research and Development, 69(1), 127–131. https://doi.org/10.1007/s11423-020-09872-4

35.

Mayer

R. E.

(2009). Multimedia learning (2nd ed.). Cambridge University Press.

36.

McAuley

Duncan

Tammen

V. V.

(1989). Psychometric properties of the intrinsic motivation inventory in a competitive sport setting: A confirmatory factor analysis. Research Quarterly for Exercise and Sport, 60(1), 48–58. https://doi.org/10.1080/02701367.1989.10607413

37.

Narciss

(2008). Feedback strategies for interactive learning tasks. In Handbook of research on educational communications and technology (4th ed., pp. 125–144). Springer.

38.

Nicol

(2012). Resituating feedback from the reactive to the proactive. In Boud

Molloy

(Eds.), Feedback in higher and professional education: Understanding it and doing it well (pp. 34–49). Routledge.

39.

Okonkwo

C. W.

Ade-Ibijola

(2021). Chatbots applications in education: A systematic review. Computers and Education: Artificial Intelligence, 2, 100033. https://doi.org/10.1016/j.caeai.2021.100033

40.

Ouerhani

Maalel

Ghezela

H. B.

Chouri

(2020). Smart ubiquitous chatbot for COVID-19 assistance with deep learning sentiment analysis model during and after quarantine. Research Square. Advance online publication. https://doi.org/10.21203/rs.3.rs-33343/v1

41.

Quiroga-Perez

Daradoumis

Puig

J. M. M.

(2020). Rediscovering the use of chatbots in education: A systematic literature review. Computer Applications in Engineering Education, 28(6)1549–1565. https://doi.org/10.1002/cae.22326

42.

Robertson

Flowers

M. J.

(2020). Determining the impact of lecture videos on student outcomes. Learning and Teaching, 13(2), 25–40. https://doi.org/10.3167/latiss.2020.130203

43.

Ryan

R. M.

Deci

E. L.

(2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67. https://doi.org/10.1006/ceps.1999.1020

44.

Sablic

Mirosavljevic

Škugor

(2020). Video-based learning (VBL)—past, present and future: an overview of the research published from 2008 to 2019. Technology, Knowledge and Learning, 26(6), 1061–1077. https://doi.org/10.1007/s10758-020-09455-5

45.

Salmon

(2019). May the fourth be with you: Creating education 4.0. Journal of Learning for Development, 6(2), 95–115.

46.

Scagnoli

N. I.

Choo

Tian

(2017). Students’ insights on the use of video lectures in online classes. British Journal of Educational Technology, 50(1), 339–414. https://doi.org/10.1111/bjet.12572

47.

Shute

V. J.

(2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795

48.

Smutny

Schreiberova

(2020). Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Computers & Education, 151, 103862. https://doi.org/10.1016/j.compedu.2020.103862

49.

Sweller

(1994). Cognitive load theory, learning difficulty and instructional design. Learning and Instruction, 4(4), 295–312.

50.

Tsay

C. H.-H.

Kofinas

A. K.

Trivedi

S. K.

Yang

(2020). Overcoming the novelty effect in online gamified learning systems: an empirical evaluation of student engagement and performance. Journal of Computer Assisted Learning, 36(2), 128–146. https://doi.org/10.1111/jcal.12385

51.

Tsigilis

Theodosiou

(2003). Temporal stability of the intrinsic motivation inventory. Perceptual and Motor Skills, 97(1), 271–280. https://doi.org/10.2466/pms.2003.97.1.271

52.

van der Kleij

F. M.

Feskens

R. C. W.

Eggen

T. J. H. M.

(2015). Effects of feedback in a computer-based learning environment on students’ learning outcomes: A meta-analysis. Review of Educational Research, 85(4), 475–511. https://doi.org/10.3102/0034654314564881

53.

van der Meij

Dunkel

(2020). Effects of a review video and practice in video-based statistics training. Computers & Education, 143, 103665. https://doi.org/10.1016/j.compedu.2019.103665

54.

van Popta

Kral

Camp

Martens

R. L.

Simons

P. R.-J.

(2017). Exploring the value of peer feedback in online learning for the provider. Educational Research Review, 20(4), 24–34. https://doi.org/10.1016/j.edurev.2016.10.003

55.

Vazquez-Cano

Mengual-Andrés

López-Meneses

(2021). Chatbot to improve learning punctuation in Spanish and to enhance open and flexible learning environments. International Journal of Educational Technology Higher Education, 18(1), 33. https://doi.org/10.1186/s41239-021-00269-8

56.

Vygotsky

(1962). Thought and language. Cambridge University Press.

57.

Walsh

J. N.

O’Brien

M. P.

Costin

(2021). Investigating student engagement with intentional content: An exploratory study of instructional videos. The International Journal of Management Education, 19(2), 100505. https://doi.org/10.1016/j.ijme.2021.100505

58.

Wang

Gong

S.-Y.

X.-E.

(2019). Elaborated feedback and learning: examining cognitive and motivational influences. Computers & Education, 136, 130–140. https://doi.org/10.1016/j.compedu.2019.04.003

59.

Wang

Lehman

J. D.

(2021). Using achievement goal-based personalized motivational feedback to enhance online learning. Educational Technology Research Development, 69(2), 553–581. https://doi.org/10.1007/s11423-021-09940-3

60.

Weizenbaum

(1966). ELIZA—a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36–45. https://doi.org/10.1145/365153.365168

61.

Winstone

N. E.

Nash

R. A.

Parker

Rowntree

(2017). Supporting learners’ agentic engagement with feedback: A systematic review and a taxonomy of recipience processes. Educational Psychologist, 52(1), 17–37. https://doi.org/10.1080/00461520.2016.1207538

62.

Wisniewski

Zierer

Hattie

(2019). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology, 10, 3087. https://doi.org/10.3389/fpsyg.2019.03087

63.

E. H.-K.

Lin

C.-H.

Y.-Y.

Liu

C.-Z.

Wang

W.-K.

Chao

C.-Y.

(2020). Advantages and constraints of a hybrid model K-12 e-learning assistant chatbot. IEEE, 8, 77788–77801. https://doi.org/10.1109/ACCESS.20202988252

64.

Yin

Goh

T.-T.

Yang

Xiaobin

(2021). Conversation technology with micro-learning: The impact of chatbot-based learning on students’ learning motivation and performance. Journal of Educational Computing Research, 59(1), 154–177. https://doi.org/10.1177/0735633120952067