Beyond Words: A Mediation Analysis of How Instructors’ Nonverbal Emotional Expressions Facilitate Students’ Video Learning

Abstract

Instructors’ emotional expressions play a pivotal role in improving the quality of educational videos, while previous studies have yielded inconsistent conclusions regarding the benefits of instructors’ nonverbal expressions on video learning. This study investigated the impact of instructors’ nonverbal emotional expressions (neutral vs. positive facial expressions vs. positive facial expressions and positive tone vs. positive facial expressions and gestures vs. positive facial expressions, positive tone and gestures) on students’ video learning, while exploring the underlying mechanisms involved. A total of 148 college students viewed educational videos while their eye movements were tracked. The primary results revealed significant differences in students’ self-reported emotions, attention, cognitive load, learning experience, self-efficacy, and learning outcome. Students who viewed the educational video with an instructor displaying happy facial expressions and with a cheerful tone performed the best. Additionally, cognitive load and attention partially mediated the immediate learning outcome. These findings not only underscore the significance of instructors’ nonverbal behavior in educational videos, but also increase our understanding of the role of nonverbal interactions in video learning, providing practical implications for students, educators, and researchers.

Keywords

nonverbal emotional expressions educational videos eye tracking mediation analysis cognitive load

Get full access to this article

View all access options for this article.

References

Asalıoğlu

E. N.

Göksun

(2023). The role of hand gestures in communication: Do type and size of gestures matter? Psychological Research, 87, 1880–1898. https://doi.org/10.1007/s00426-022-01774-9

Bandura

(1986). The explanatory and predictive scope of self-efficacy theory. Journal of Social Clical Psychology, 4(3), 359–373. https://doi.org/10.1521/jscp.1986.4.3.359

Beege

Schneider

(2023). Emotional design of pedagogical agents: The influence of enthusiasm and model-observer similarity. Educational Technology Research and Development, 71, 859–880. https://doi.org/10.1007/s11423-023-10213-4

Beege

Schneider

Nebel

Rey

G. D.

(2020). Does the effect of enthusiasm in a pedagogical agent’s voice depend on mental load in the learner’s working memory? Computers in Human Behavior, 112, Article 106483. https://doi.org/10.1016/j.chb.2020.106483

Brom

Stárková

D’Mello

S. K.

(2018). How effective is emotional design? A meta-analysis on facial anthropomorphisms and happy colors during multimedia learning. Educational Research Review, 25, 100–119. https://doi.org/10.1016/j.edurev.2018.09.004

Cohen

(1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Routledge. https://doi.org/10.4324/9780203771587

Crivelli

Fridlund

A. J.

(2018). Facial displays are tools for social influence. Trends in Cognitive Sciences, 22(5), 388–399. https://doi.org/10.1016/j.tics.2018.02.006

Cronbach

L. J.

(1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555

Dael

Goudbeek

Scherer

K. R.

(2013). Perceived gesture dynamics in nonverbal expression of emotion. Perception, 42(6), 642–657. https://doi.org/10.1068/p7364

10.

Ekman

(1999). Handbook of cognition and emotion. Wiley. https://doi.org/10.1002/0470013494

11.

Fountoukidou

Matzat

Ham

Midden

(2022). The effect of an artificial agent’s vocal expressiveness on immediacy and learning. Journal of Computer Assisted Learning, 38, 500–512. https://doi.org/10.1111/jcal.12632

12.

Peng

Nastase

S. A.

Mayer

R. E.

(2024). Onscreen presence of instructors in video lectures affects learners’ neural synchrony and visual attention during multimedia learning. Proceedings of the National Academy of Sciences of the USA, 121(12), e2309054121. https://doi.org/10.1073/pnas.2309054121

13.

Hareli

Hess

(2012). The social signal value of emotions. Cognition and Emotion, 26(3), 385–389. https://doi.org/10.1080/02699931.2012.665029

14.

Hatfield

Cacioppo

J. T.

Rapson

R. L.

(1993). Emotional contagion. Current Directions in Psychological Science, 2(3), 96–99. https://doi.org/10.1111/1467-8721.ep10770953

15.

Heidig

Beege

Rey

G. D.

Schneider

(2024). Instructor presence in instructional videos in higher education: Three field experiments in university courses. Educational Technology Research and Development, 72, 3047–3074. https://doi.org/10.1007/s11423-024-10391-9

16.

Heidig

Müller

Reichelt

(2015). Emotional design in multimedia learning: Differentiation on relevant design features and their effects on emotions and learning. Computers in Human Behavior, 44, 81–95. https://doi.org/10.1016/j.chb.2014.11.009

17.

Horovitz

Mayer

R. E.

(2021). Learning with human and virtual instructors who display happy or bored emotions in video lectures. Computers in Human Behavior, 119, Article 106724. https://doi.org/10.1016/j.chb.2021.106724

18.

Keren

(1993). Between- or within-subjects design: A methodological dilemma. In Keren

Lewis

(Eds.), A handbook for data analysis in the behavioral sciences: Methodological issues (pp. 257–272). Lawrence Erlbaum Associates, Inc.

19.

Kirkham

A. J.

Hayes

A. E.

Pawling

Tipper

S. P.

(2015) Facial mimicry and emotion consistency: Influences of memory and context. PLoS ONE, 10(12), e0145731. https://doi.org/10.1371/journal.pone.0145731

20.

Lawson

A. P.

Mayer

R. E.

(2022). Does the emotional stance of human and virtual instructors in instructional videos affect learning processes and outcomes? Contemporary Educational Psychology, 70, Article 102080. https://doi.org/10.1016/j.cedpsych.2022.102080

21.

Leppink

Paas

Van der Vleuten

C. P. M.

Van Gog

Van Merriënboer

J. J. G.

(2013). Development of an instrument for measuring different types of cognitive load. Behavior Research Methods, 45, 1058–1072. https://doi.org/10.3758/s13428-013-0334-1

22.

Liew

T. W.

Tan

S. M.

Pang

W. M.

Khan

M. T. I.

Kew

S. N.

(2023). I am Alexa, your virtual tutor! The effects of Amazon Alexa’s text‑to‑speech voice enthusiasm in a multimedia learning environment. Education and Information Technologies, 28, 1455–1489. https://doi.org/10.1007/s10639-022-11255-6

23.

Liew

T. W.

Zin

N. A. M.

Sahari

Tan

S. M.

(2016). The effects of a pedagogical agent’s smiling expression on the learner’s emotions and motivation in a virtual learning environment. International Review of Research in Open and Distributed Learning, 17, 5. https://doi.org/10.19173/irrodl.v17i5.2350

24.

Mayer

R. E.

(2020). Searching for the role of emotions in e-learning. Learning & Instruction, 70, Article 101213. https://doi.org/10.1016/j.learninstruc.2019.05.010

25.

Moreno

Mayer

(2007). Interactive multimodal learning environments: Special issue on interactive learning environments: Contemporary issues and trends. Educational Psychology Review, 19(3), 309–326. https://doi.org/10.1007/s10648-007-9047-2

26.

Pekrun

(2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4), 315–341. https://doi.org/10.1007/s10648-006-9029-9

27.

Pekrun

Goetz

Titz

Perry

R. P.

(2002). Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist, 37(2), 91–105. https://doi.org/10.1207/S15326985EP3702_4

28.

Pekrun

Goetz

Frenzel

A. C.

Perry

R. P.

(2011). Achievement Emotions Questionnaire (AEQ) [Database record]. APA PsycTests. https://doi.org/10.1037/t21196-000

29.

Chen

Zhu

Yang

(2020). Modulation of instructor’s eye gaze by facial expression in video lectures. Innovations in Education and Teaching International, 59(1), 15–23. https://doi.org/10.1080/14703297.2020.1788410

30.

Liu

Ling

Zhang

(2023). Does an instructor’s facial expressions override their body gestures in video lectures? Computers & Education, 193, Article 104679. https://doi.org/10.1016/j.compedu.2022.104679

31.

Zhu

Zhang

Chen

Yang

(2022). Complexity of visual learning material moderates the effects of instructor’s beat gestures and head nods in video lectures. Learning and Instruction, 77, Article 101520. https://doi.org/10.1016/j.learninstruc.2021.101520

32.

Pintrich

P. R.

De Groot

E. V.

(1990). Motivated Strategies for Learning Questionnaire (MSLQ) [Database record]. APA PsycTests. https://doi.org/10.1037/t09161-000

33.

Qian

Tang

Wang

Liu

Chen

(2022). How teacher enthusiasm affects students’ learning of chemistry declarative knowledge in video lectures. Chemistry Education Research and Practice, 23, 898. https://doi.org/10.1039/d2rp00095d

34.

Rossi

Pourtois

(2012). Transient state-dependent fluctuations in anxiety measured using STAI, POMS, PANAS or VAS: A comparative review. Anxiety, Stress & Coping: An International Journal, 25(6), 603–645. https://doi.org/10.1080/10615806.2011.582948

35.

Russell

J. A.

(1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178. https://doi.org/10.1037/h0077714

36.

Schneider

Krieglstein

Beege

Rey

G. D.

(2022). The impact of video lecturers’ nonverbal communication on learning: An experiment on gestures and facial expressions of pedagogical agents. Computers & Education, 176, Article 104350. https://doi.org/10.1016/j.compedu.2021.104350

37.

Streiner

D. L.

(2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80(1), 99–103. https://doi.org/10.1207/S15327752JPA8001_18

38.

Stull

A. T.

Fiorella

Gainer

M. J.

Mayer

R. E.

(2018). Using transparent whiteboards to boost learning from online STEM lectures. Computers & Education, 120, 146–159. https://doi.org/10.1016/j.compedu.2018.02.005

39.

Sweller

(2011). Cognitive load theory. In Mestre

J. P.

Ross

B. H.

(Eds.), The psychology of learning and motivation: Cognition in education (pp. 37–76). Elsevier Academic Press. https://doi.org/10.1016/B978-0-12-387691-1.00002-8

40.

van Doorn

E. A.

van Kleef

G. A.

van der Pligt

(2015). Deriving meaning from others’ emotions: Attribution, appraisal, and the use of emotions as social information. Frontiers in Psychology, 6, Article 1077. https://doi.org/10.3389/fpsyg.2015.01077

41.

van Gompel

R. P. G.

Fischer

M. H.

Murray

W. S.

Hill

R. L.

(2007). Eye movements: A window on mind and brain. Elsevier Science. http://www.sciencedirect.com/science/book/9780080449807

42.

Van Kleef

G. A.

(2009). How emotions regulate social life: The emotions as social information (EASI) model. Current Directions in Psychological Science, 18(3), 184–188. https://doi.org/10.1111/j.1467-8721.2009.01633.x

43.

Van Kleef

G. A.

Cote

(2023). The social effects of emotions. Annual Review of Psychology, 73, 629–658. https://doi.org/10.1146/annurev-psych-020821-010855

44.

Wang

Mayer

R. E.

Han

Zhang

(2023). Two emotional design features are more effective than one in multimedia learning. Journal of Educational Computing Research, 60(8), 1991–2014. https://doi.org/10.1177/07356331221090845

45.

Wang

Feng

Guo

Gong

Wang

(2022). Benefits of affective pedagogical agents in multimedia instruction. Frontiers in Psychology, 12, Article 797236. https://doi.org/10.3389/fpsyg.2021.797236

46.

Wang

Liu

Chen

Wang

Stein

(2019). Effects of instructor’s facial expressions on students’ learning with video lectures. British Journal of Educational Technology, 50(3), 1381–1395. https://doi.org/10.1111/biet.12633

47.

Watson

(1988). Intraindividual and interindividual analyses of positive and negative affect: Their relation to health complaints, perceived stress, and daily activities. Journal of Personality and Social Psychology, 54(6), 1020–1030. https://doi.org/10.1037/0022-3514.54.6.1020

48.

Wong

R. M.

Adesope

O. O.

(2021). Meta-analysis of emotional designs in multimedia learning: A replication and extension study. Educational Psychology Review, 33, 357–385. https://doi.org/10.1007/s10648-020-09545-x

49.

Zahra

S. A.

(2021). Learning through screens: How gestures, facial expressions, and motivational statements influence foreign language learning experience online. University of Twente.

50.

Zhang

Cheng

(2023). Influence of instructor’s facial expressions in video lectures on motor learning in children with autism spectrum disorder. Education and Information Technologies, 28, 11867–11880. https://doi.org/10.1007/s10639-023-11676-x

51.

Zhao

Mayer

R. E.

(2023). Role of emotional tone and gender of computer-generated voices in multimedia lessons. Educational Technology Research and Development, 71, 1449–1469. https://doi.org/10.1007/s11423-023-10228-x

52.

Zhu

Pan

Yang

(2024). How does virtual peer presence relate to learning from video lectures and subsequent explanation generation? The moderated mediation roles of motivation and explanation characteristics. Active Learning in Higher Education. Advance Online. https://doi.org/10.1177/14697874241256375

53.

Zhu

Yang

(2022). The interaction effects of an instructor’s emotions in instructional videos and students’ emotional intelligence on L2 vocabulary learning. Educational Technology Research and Development, 70, 1695–1718. https://doi.org/10.1007/s11423-022-10148-2