Multimedia is ubiquitous in 21st-century education. Cognitive load theory and the cognitive theory of multimedia learning both postulate that the quality of multimedia design heavily influences learning. We sought to identify how to best design multimedia and review how well those learning theories held up to meta-analyses. We conducted an overview of systematic reviews that tested the effects of multimedia design on learning or cognitive load. We found 29 reviews including 1,189 studies and 78,177 participants. We found 11 design principles that demonstrated significant, positive, meta-analytic effects on learning and five that significantly improved management of cognitive load. The largest benefits were for captioning second-language videos, temporal/spatial contiguity, and signaling. We also found robust evidence for modality, animation, coherence/removing seductive details, anthropomorphics, segmentation, personalization, pedagogical agents, and verbal redundancy effects. Good design was more important for more complex materials, and in system-paced environments (e.g., lectures) than self-paced ones (e.g., websites). Results supported many tenets of both theories. We highlight a range of evidence-based strategies that could be implemented by educators.
*AdesopeO. O.NesbitJ. C. (2012). Verbal redundancy in multimedia learning environments: A meta-analysis. Journal of Educational Psychology, 104(1), 250–263. https://doi.org/10.1037/a0026147
2.
*AlemdagE.CagiltayK. (2018). A systematic review of eye tracking research on multimedia learning. Computers & Education, 125, 413–428. https://doi.org/10.1016/j.compedu.2018.06.023
3.
AlexanderP. A. (2020). Methodological guidance paper: The art and science of quality systematic reviews. Review of Educational Research, 90(1), 6–23. https://doi.org/10.3102/0034654319854352
4.
*AlpizarD.AdesopeO. O.WongR. M. (2020). A meta-analysis of signaling principle in multimedia learning environments. Educational Technology Research and Development: ETR&D, 68(5), 2095–2119. https://doi.org/10.1007/s11423-020-09748-7
5.
*AmakaI. H.GoemanK. (2017). Selecting media for effective learning in online and blended courses: A review study. Journal of Educational Multimedia and Hypermedia, 26(1), 29–59. https://www.learntechlib.org/primary/p/173341/
APA Publications and Communications Board Working Group on Journal Article Reporting Standards. (2008). Reporting standards for research in psychology: Why do we need them? What might they be?The American Psychologist, 63(9), 839–851. https://doi.org/10.1037/0003-066X.63.9.839
8.
BakerJ. P.GoodboyA. K.BowmanN. D.WrightA. A. (2018). Does teaching with PowerPoint increase students’ learning? A meta-analysis. Computers & Education, 126, 376–387. https://doi.org/10.1016/j.compedu.2018.08.003
*BromC.StárkováT.D’MelloS. K. (2018). How effective is emotional design? A meta-analysis on facial anthropomorphisms and pleasant colors during multimedia learning. Educational Research Review, 25, 100–119. https://doi.org/10.1016/j.edurev.2018.09.004
11.
CafriG.KromreyJ. D.BrannickM. T. (2010). A meta-meta-analysis: Empirical review of statistical power, Type I error rates, effect sizes, and model selection of meta-analyses published in psychology. Multivariate Behavioral Research, 45, 239–270. https://doi.org/10.1080/00273171003680187
12.
*Castro-AlonsoJ. C.WongR. M.AdesopeO. O.PaasF. (2021). Effectiveness of multimedia pedagogical agents predicted by diverse theories: A meta-analysis. Educational Psychology Review. Advance online publication. https://doi.org/10.1007/s10648-020-09587-1
13.
ChandanS.AroraS.MohanB. P.KhanS. R.ChandanO. C.KassabL. L.MuraliA. R. (2020). Multimedia based education on bowel preparation improves adenoma detection rate: Systematic review & meta-analysis of randomized controlled trials. Digestive Endoscopy, 33(5), 730–740. https://doi.org/10.1111/den.13809
14.
CheungM. W.-L. (2014a). metaSEM: an R package for meta-analysis using structural equation modeling. Frontiers in Psychology, 5, 1521. https://doi.org/10.3389/fpsyg.2014.01521
15.
CheungM. W.-L. (2014b). Modeling dependent effect sizes with three-level meta-analyses: A structural equation modeling approach. Psychological Methods, 19(2), 211–229. https://doi.org/10.1037/a0032968
16.
ChiM. T. H.KangS.YaghmourianD. L. (2017). Why students learn more from dialogue- than monologue-videos: Analyses of peer interactions. Journal of the Learning Sciences, 26(1), 10–50. https://doi.org/10.1080/10508406.2016.1204546
17.
ConigraveJ. (2019). msemtools: Routines, tables, and figures for metaSEM analyses (Version 0.9.8) [Computer software]. Github. https://github.com/JConigrave/msemtools
18.
*CromleyJ. G.LawrenceL. E. M. (2018). Multimedia simulations that foster transfer: Findings from a review of the literature. In ZhengR. Z. (Ed.), Strategies for deep learning with digital technology: Theories and practices in education (pp. 93–110). Nova Science.
19.
*DavisR. O. (2018). The impact of pedagogical agent gesturing in multimedia learning environments: A meta-analysis. Educational Research Review, 24, 193–209. https://doi.org/10.1016/j.edurev.2018.05.002
20.
DeeksJ. J. (2019). Chapter 10: Analysing data and undertaking meta-analyses In: HigginsJ. P. T.ThomasJ.ChandlerJ.CumpstonM.LiT.PageM. J.WelchV. A. (Eds.), Cochrane handbook for systematic reviews of interventions, Version 6. https://training.cochrane.org/handbook/archive/v6/chapter-10
EriksenM. B.FrandsenT. F. (2018). The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: A systematic review. Journal of the Medical Library Association: JMLA, 106(4), 420–431. https://doi.org/10.5195/jmla.2018.345
23.
FunderD. C.OzerD. J. (2019). Evaluating effect size in psychological research: Sense and nonsense. Advances in Methods and Practices in Psychological Science, 2(2), 156–168. https://doi.org/10.1177/2515245919847202
*GinnsP. (2006). Integrating information: A meta-analysis of the spatial contiguity and temporal contiguity effects. Learning and Instruction, 16(6), 511–525. https://doi.org/10.1016/j.learninstruc.2006.10.001
26.
*GinnsP.MartinA. J.MarshH. W. (2013). Designing instructional text in a conversational style: A meta-analysis. Educational Psychology Review, 25(4), 445–472. https://doi.org/10.1007/s10648-013-9228-0
27.
GrantM. J.BoothA. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
HedgesL. V. (1981). Distribution theory for glass’s estimator of effect size and related estimators. Journal of Educational and Behavioral Statistics: A Quarterly Publication Sponsored by the American Educational Research Association and the American Statistical Association, 6(2), 107–128. https://doi.org/10.3102/10769986006002107
31.
HedgesL. V.VeveaJ. L. (1996). Estimating effect size under publication bias: Small sample properties and robustness of a random effects selection model. Journal of Educational and Behavioral Statistics, 21(4), 299–332. https://doi.org/10.3102/10769986021004299
32.
HennessyE. A.JohnsonB. T. (2020). Examining overlap of included studies in meta-reviews: Guidance for using the corrected covered area index. Research Synthesis Methods, 11(1), 134–145. https://doi.org/10.1002/jrsm.1390
33.
HennessyE. A.JohnsonB. T.KeenanC. (2019). Best practice guidelines and essential methodological steps to conduct rigorous and systematic meta-Reviews. Applied Psychology. Health and Well-Being, 11(3), 353–381. https://doi.org/10.1111/aphw.12169
34.
HigginsJ. P. T.AltmanD. G.SterneJ. A. C., & Cochrane Statistical Methods Group and the Cochrane Bias Methods Group. (2011). Assessing risk of bias in included studies. In HigginsJ. P. T.GreenS. (Eds.), Cochrane handbook for systematic reviews of interventions (Vol. 5.1.1). The Cochrane Collaboration. https://handbook-5-1.cochrane.org/chapter_8/
HirscheyR.BryantA. L.WalkerJ. S.NolanT. S. (2020). Systematic review of video education in underrepresented minority cancer survivors. Cancer Nursing, 43(4), 259–268. https://doi.org/10.1097/NCC.0000000000000829
37.
*HöfflerT. N. (2010). Spatial ability: Its influence on learning with visualizations—A meta-analytic review. Educational Psychology Review, 22, 245–269. https://doi.org/10.1007/s10648-010-9126-7
KalyugaS. (2007). Expertise reversal effect and its implications for learner-tailored instruction. Educational Psychology Review, 19(4), 509–539. https://doi.org/10.1007/s10648-007-9054-3
40.
KalyugaS. (2011). Cognitive load theory: How many types of load does it really need?Educational Psychology Review, 23(1), 1–19. https://doi.org/10.1007/s10648-010-9150-7
41.
*KangT. (2019). The effectiveness of multiple media tools in L2 listening: A meta-analysis. In Information Resources Management Association (Ed.), Computer-assisted language learning: Concepts, methodologies, tools, and applications (pp. 585–616). IGI Global. https://doi.org/10.4018/978-1-5225-7663-1.ch028
42.
*KaushalR. K.PandaS. N. (2019). A meta analysis on effective conditions to offer animation based teaching style. Malaysian Journal of Learning and Instruction, 16(1), 129–153. https://eric.ed.gov/?id=EJ1219792
43.
*KimS. (2005). The effect of multimedia design elements on learning outcomes in pedagogical agent research: A meta-analysis. Artificial Intelligence in Education: Supporting Learning through Intelligent and Socially Informed Technology, 963. https://dl.acm.org/doi/proceedings/10.5555/1562524
44.
LawsonM. A.LawsonH. A. (2013). New conceptual frameworks for student engagement research, policy, and practice. Review of Educational Research, 83(3), 432–479. https://doi.org/10.3102/0034654313480891
45.
LazowskiR. A.HullemanC. S. (2016). Motivation interventions in education: A meta-analytic review. Review of Educational Research, 86(2), 602–640. https://doi.org/10.3102/0034654315617832
46.
MayerR. E. (2008). Applying the science of learning: Evidence-based principles for the design of multimedia instruction. The American Psychologist, 63(8), 760–769. https://doi.org/10.1037/0003-066X.63.8.760
MayerR. E.FiorellaL. (2014). Principles for reducing extraneous processing in multimedia learning: Coherence, signaling, redundancy, spatial contiguity, and temporal contiguity principles. In MayerR. E. (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 279–315). New York: Cambridge University Press.
51.
MayerR. E.HeiserJ.LonnS. (2001). Cognitive constraints on multimedia learning: When presenting more material results in less understanding. Journal of Educational Psychology, 93(1), 187–198. https://doi.org/10.1037/0022-0663.93.1.187
52.
MayerR. E.MorenoR. (2010). Techniques that reduce extraneous cognitive load and manage intrinsic cognitive load during multimedia learning. In PlassJ. L.MorenoR.BrünkenR. (Eds.), Cognitive load theory (pp. 131–152). Cambridge University Press. https://doi.org/10.1017/CBO9780511844744.009
53.
MoeyaertM.UgilleM.Natasha BeretvasS.FerronJ.BunuanR.Van den NoortgateW. (2016). Methods for dealing with multiple outcomes in meta-analysis: A comparison between averaging effect sizes, robust variance estimation and multilevel meta-analysis. International Journal of Social Research Methodology, 20(6), 559–572. https://doi.org/10.1080/13645579.2016.1252189
54.
*Montero PerezM.Van Den NoortgateW.DesmetP. (2013). Captioned video for L2 listening and vocabulary learning: A meta-analysis. System, 41(3), 720–739. https://doi.org/10.1016/j.system.2013.07.013
55.
MorenoR.MayerR. E. (2002). Verbal redundancy in multimedia learning: When reading helps listening. Journal of Educational Psychology, 94(1), 156–163. https://doi.org/10.1037/0022-0663.94.1.156
NesbitJ. C.AdesopeO. O. (2006). Learning with concept and knowledge maps: A meta-analysis. Review of Educational Research, 76(3), 413–448. https://doi.org/10.3102/00346543076003413
58.
NoetelM.GriffithS.DelaneyO.SandersT.ParkerP. D.del Pozo CruzB.LonsdaleC. (2021). Video improves learning in higher education: A systematic review. Review of Educational Research, 91(2), 204–236. https://doi.org/10.3102/0034654321990713
59.
PageM. J.McKenzieJ. E.BossuytP. M.BoutronI.HoffmannT. C.MulrowC. D.ShamseerL.TetzlaffJ. M.MoherD. (2021). Updating guidance for reporting systematic reviews: Development of the PRISMA 2020 statement. Journal of Clinical Epidemiology, 134, 103–112. https://doi.org/10.1016/j.jclinepi.2021.02.003
60.
PigottT. D.PolaninJ. R. (2020). Methodological guidance paper: High-quality meta-analysis in a systematic review. Review of Educational Research, 90(1), 24–46. https://doi.org/10.3102/0034654319877153
61.
PollockM.FernandesR. M.BeckerL. A.PieperD.HartlingL.Others. (2018). Chapter V: Overviews of reviews. Cochrane Handbook for systematic reviews of interventions, Version 6. https://training.cochrane.org/handbook/current/chapter-v
62.
PustejovskyJ. E.RodgersM. A. (2019). Testing for funnel plot asymmetry of standardized mean differences. Research Synthesis Methods, 10(1), 57–71. https://doi.org/10.1002/jrsm.1332
63.
R Core Team. (2020). R: A language and environment for statistical computing (Version 3.6.3) [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org/
64.
*RahimR. A.NoorN. M.ZaidN. M. (2015). Meta-analysis on element of cognitive conflict strategies with a focus on multimedia learning material development. International Education Studies, 8(13), 73–78. https://doi.org/10.5539/ies.v8n13p73
65.
RathboneJ.CarterM.HoffmannT.GlasziouP. (2015). Better duplicate detection for systematic reviewers: Evaluation of Systematic Review Assistant-Deduplication Module. Systematic Reviews, 4, 6. https://doi.org/10.1186/2046-4053-4-6
66.
*ReinweinJ. (2012). Does the modality effect exist? And if so, which modality effect?Journal of Psycholinguistic Research, 41(1), 1–32. https://doi.org/10.1007/s10936-011-9180-4
67.
*ReyG. D. (2012). A review of research and a meta-analysis of the seductive detail effect. Educational Research Review, 7(3), 216–237. https://doi.org/10.1016/j.edurev.2012.05.003
68.
*ReyG. D.BeegeM.NebelS.WirzbergerM.SchmittT. H.SchneiderS. (2019). A meta-analysis of the segmenting effect. Educational Psychology Review, 31(2), 389–419. https://doi.org/10.1007/s10648-018-9456-4
69.
*RichterJ.ScheiterK.EitelA. (2016). Signaling text-picture relations in multimedia learning: A comprehensive meta-analysis. Educational Research Review, 17, 19–36. https://doi.org/10.1016/j.edurev.2015.12.003
70.
RolfeV. E.GrayD. T. (2011). Are multimedia resources effective in life science education? A meta-analysis. Bioscience Education, 18(1), 1–14. https://doi.org/10.3108/beej.18.5
71.
Ruiz-RocaJ. A.Martínez-IzquierdoA.Mengual-PujanteD.LópezE. P. F.López-JornetP. (2020). Is YouTube a useful tool for oral care in patients with Parkinson’s disease?Special Care in Dentistry, 40(5), 464–469. https://doi.org/10.1111/scd.12489
72.
SchmidtF. L.OhI.-S. (2013). Methods for second order meta-analysis and illustrative applications. Organizational Behavior and Human Decision Processes, 121(2), 204–218. https://doi.org/10.1016/j.obhdp.2013.03.002
73.
SchneiderM.PreckelF. (2017). Variables associated with achievement in higher education: A systematic review of meta-analyses. Psychological Bulletin, 143(6), 565–600. https://doi.org/10.1037/bul0000098
74.
*SchneiderS.BeegeM.NebelS.ReyG. D. (2018). A meta-analysis of how signaling affects learning with media. Educational Research Review, 23, 1–24. https://doi.org/10.1016/j.edurev.2017.11.001
*SchroederN. L.AdesopeO. O.GilbertR. B. (2013). How effective are pedagogical agents for learning? A meta-analytic review. Journal of Educational Computing Research, 29(1), 1–39. https://doi.org/10.2190/EC.49.1.a
77.
*SchroederN. L.CenkciA. T. (2018). Spatial contiguity and spatial split-attention effects in multimedia learning environments: A meta-analysis. Educational Psychology Review, 30(3), 679–701. https://doi.org/10.1007/s10648-018-9435-9
78.
*SchroederN. L.CenkciA. T. (2020). Do measures of cognitive load explain the spatial split-attention principle in multimedia learning environments? A systematic review. Journal of Educational Psychology, 112(2), 254–270. https://doi.org/10.1037/edu0000372
79.
SheaB. J.ReevesB. C.WellsG.ThukuM.HamelC.MoranJ.MoherD.TugwellP.WelchV.KristjanssonE.HenryD. A. (2017). AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 358, j4008. https://doi.org/10.1136/bmj.j4008
80.
SilberzahnR.UhlmannE. L.MartinD. P.AnselmiP.AustF.AwtreyE.BahníkŠ.BaiF.BannardC.BonnierE.CarlssonR.CheungF.ChristensenG.ClayR.CraigM. A.Dalla RosaA.DamL.EvansM. H.Flores CervantesI.. . .NosekB. A. (2018). Many analysts, one data set: Making transparent how variations in analytic choices affect results. Advances in Methods and Practices in Psychological Science, 1(3), 337–356. https://doi.org/10.1177/2515245917747646
81.
SmithV.DevaneD.BegleyC. M.ClarkeM. (2011). Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Medical Research Methodology, 11(1), 15. https://doi.org/10.1186/1471-2288-11-15
82.
SnookI.O’NeillJ.ClarkJ.O’NeillA.-M.OpenshawR., & Others. (2009). Invisible learnings? A commentary on John Hattie’s book—“Visible learning: A synthesis of Over 800 meta-analyses relating to achievement”. New Zealand Journal of Educational Studies, 44(1), 93. https://doi.org/10.3316/INFORMIT.467818990993648
83.
SpoorenP.BrockxB.MortelmansD. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598–642. https://doi.org/10.3102/0034654313496870
84.
SterneJ. A. C.SavovićJ.PageM. J.ElbersR. G.BlencoweN. S.BoutronI.CatesC. J.ChengH.-Y.CorbettM. S.EldridgeS. M.EmbersonJ. R.HernánM. A.HopewellS.HróbjartssonA.JunqueiraD. R.JüniP.KirkhamJ. J.LassersonT.LiT.. . . HigginsJ. P. T. (2019). RoB 2: A revised tool for assessing risk of bias in randomised trials [Review of RoB 2: A revised tool for assessing risk of bias in randomised trials]. BMJ, 366, l4898. https://doi.org/10.1136/bmj.l4898
85.
*SundararajanN.AdesopeO. (2020). Keep it coherent: A meta-analysis of the seductive details effect. Educational Psychology Review, 32, 707–734. https://doi.org/10.1007/s10648-020-09522-4
86.
SwellerJ. (2010). Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review, 22(2), 123–138. https://doi.org/10.1007/s10648-010-9128-5
87.
SwellerJ.van MerriënboerJ. J. G.PaasF. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31(2), 261–292. https://doi.org/10.1007/s10648-019-09465-5
88.
SwellerJ.van MerrienboerJ. J. G.PaasF. G. W. C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251–296. https://doi.org/10.1023/A:1022193728205
89.
TaylorK. S.MahtaniK. R.AronsonJ. K. (2021). Summarising good practice guidelines for data extraction for systematic reviews and meta-analysis. BMJ Evidence-Based Medicine. https://doi.org/10.1136/bmjebm-2020-111651
90.
Van GogT.EricssonK. A.RikersR. M.PaasF. (2005). Instructional design for advanced learners: Establishing connections between the theoretical frameworks of cognitive load and deliberate practice. Educational Technology Research and Development: ETR&D, 53(3), 73–81. https://doi.org/10.1007/BF02504799
91.
van MerriënboerJ. J. G.SwellerJ. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17(2), 147–177. https://doi.org/10.1007/s10648-005-3951-0
92.
WallaceB. C.SmallK.BrodleyC. E.LauJ.TrikalinosT. A. (2012). Deploying an interactive machine learning system in an evidence-based practice center: Abstrackr. Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium, 819–824. https://doi.org/10.1145/2110363.2110464
93.
*XieH.WangF.ZhouZ.WuP. (2016). Cueing effect in multimedia learning: A meta-analysis. Xin Li Xue Bao. Acta Psychologica Sinica, 48(5), 540. https://doi.org/10.3724/sp.j.1041.2016.00540
94.
*XieH.WangF.HaoY.ChenJ.AnJ.WangY.LiuH. (2017). The more total cognitive load is reduced by cues, the better retention and transfer of multimedia learning: A meta-analysis and two meta-regression analyses. PloS One, 12(8), e0183884. https://doi.org/10.1371/journal.pone.0183884
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
