Although flipped classroom pedagogies have been widely touted for their ability to foster diverse 21st-century learning objectives, previous syntheses of flipped learning have focused almost exclusively on outcomes related to academic achievement. Using data from 317 studies, our research addresses this deficit by providing a comprehensive meta-analysis of the effects of flipped versus lecture-based learning on academic, intra-/interpersonal, and satisfaction-related outcomes in higher education. Overall, flipped classroom interventions produced positive gains across all three learning domains, and we found significant advantages of flipped over lecture-based instruction for seven out of eight outcomes (gs = 0.20–0.53). At the same time, there was substantial heterogeneity in flipped learning effects, and we identified several variables that influenced the relative efficacy of flipped versus traditional courses. Of the three types of moderators examined (contextual, design-based, and methodological), educational context (e.g., discipline, location) accounted for the most variability in flipped learning outcomes.
AbeysekeraL.DawsonP. (2015). Motivation and cognitive load in the flipped classroom: Definition, rationale, and a call for research. Higher Education Research & Development, 34(1), 1–14. https://doi.org/10.1080/07294360.2014.934336
2.
AkçayırG.AkçayırM. (2018). The flipped classroom: A review of its advantages and challenges. Computers & Education, 126(November), 334–345. https://doi.org/10.1016/j.compedu.2018.07.021
3.
Al-ShabibiT. S.Al-AyasraM. A. (2019). Effectiveness of the flipped classroom strategy in learning outcomes (bibliometric study). International Journal of Learning, Teaching and Educational Research, 18(3), 96–127. https://doi.org/10.26803/ijlter.18.3.6
4.
AltemuellerL.LindquistC. (2017). Flipped classroom instructions for inclusive learning. British Journal of Special Education, 44(3), 341–358. https://doi.org/10.1111/1467-8578.12177
5.
BeranT. N.ViolatoC.KlineD. W. (2007). What’s the “use” of student ratings of instruction for administrators? One university’s experience. Canadian Journal of Higher Education, 37(1), 27–43. https://doi.org/10.47678/cjhe.v37i1.183545
BrewerR.MovahedazarhoulighS. (2018). Successful stories and conflicts: A literature review on the effectiveness of flipped learning in higher education. Journal of Computer Assisted Learning, 34(4), 409–416. https://doi.org/10.1111/jcal.12250
10.
ChenF.LuiA. M.MartinelliS. M. (2017). A systematic review of the effectiveness of flipped classrooms in medical education. Medical Education, 51(6), 585–597. https://doi.org/10.1111/medu.13272
11.
ChenK.-S.MonrouxeL.LuY.-H.JenqC.-C.ChangY.-J.ChangY.-C.ChaiP. Y.-C. (2018). Academic outcomes of flipped classroom learning: A meta-analysis. Medical Education in Review, 52(9), 910–924. https://doi.org/10.1111/medu.13616
12.
ChengL.RitzhauptA. D.AntonenkoP. (2019). Effects of the flipped classroom instructional strategy on students’ learning outcomes: A meta-analysis. Educational Technology Research and Development, 67(4), 793–824. https://doi.org/10.1007/s11423-018-9633-7
13.
CheungA. C.SlavinR. E. (2016). How methodological features affect effect sizes in education. Educational Researcher, 45(5), 283–292. https://doi.org/10.3102/0013189X16656615
14.
CohenJ. (1988). Statistical power analysis for the behavioral sciences. Routledge Academic.
15.
CooperH. (2010). Research synthesis and meta-analysis: A step-by-step approach. Sage.
DoveA.DoveE. (2017). How flipping much? Consecutive flipped mathematics courses and their influence on students’ anxieties and perceptions of learning. Journal of Computers in Mathematics and Science Teaching, 36(2), 129–141. https://www.learntechlib.org/primary/p/178271/
20.
DunlapW. P.CortinaJ. M.VaslowJ. B.BurkeM. J. (1996). Meta-analysis of experiments with matched groups or repeated measures designs. Psychological Methods, 1(2), 170–177. https://doi.org/10.1037/1082-989X.1.2.170
21.
DuvalS.TweedieR. (2000). Trim and fill: A simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463. https://doi.org/10.1111/j.0006-341X.2000.00455.x
22.
EddyS. L.HoganK. A. (2014). Getting under the hood: How and for whom does increasing course structure work?CBE Life Sciences Education, 13(3), 453–468. https://doi.org/10.1187/cbe.14-03-0050
23.
EvansL.Vanden BoschM. L.HarringtonS.SchoofsN.CoviakC. (2019). Flipping the classroom in health care higher education: A systematic review. Nurse Educator, 44(2), 74–78. https://doi.org/10.1097/NNE.0000000000000554
24.
Faculty Focus. (2015). Flipped classroom trends: A survey of college faculty. Magna.
25.
FerrareJ. J.HoraM. T. (2014). Cultural models of teaching and learning in math and science: Exploring the intersections of culture, cognition, and pedagogical situations. Journal of Higher Education, 85(6), 792–825. https://doi.org/10/1353/jhe.2014.0030
26.
FinkL. D. (2013). Creating significant learning experiences: An integrated approach to designing college courses, revised and updated. Jossey-Bass.
FongC. J.DavisC. W.KimY.KimY. W.MarriottLKimS. (2017). Psychosocial factors and community college student success: A meta-analytic investigation. Review of Educational Research, 87(2), 388–424. https://doi.org/10.3102/00346653479
29.
FrambachJ. M.DriessenE. W.ChanL.-C.van der VleutenC. P. M. (2012). Rethinking the globalisation of problem-based learning: How culture challenges self-directed learning. Medical Education, 46(8), 738–747. https://doi.org/10.1111/j.1365-2923.2012.04290.x
30.
FreemanS.EddyS. L.McDonoughM.SmithM. K.OkoroaforN.JordtH.WenderothM. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United States of America, 111(23), 8410–8415. https://doi.org/10.1073/pnas.1319030111
31.
FuR.GartlehnerG.GrantM.ShamliyanT.SedrakyanA.WiltT. J.GriffithL.OremusM.RainaP.IsmailaA.SantaguidaP.LauJ.TrikalinosT. A. (2011). Conducting quantitative synthesis when comparing medical interventions: AHRQ and the Effective Health Care Program. Journal of Clinical Epidemiology, 64(11), 1187–1197. https://doi.org/10.1016/j.jclinepi.2010.08.010
32.
FurtakE. M.SeidelT.IversonH.BriggsD. C. (2012). Experimental and quasi-experimental studies of inquiry-based science teaching: A meta-analysis. Review of Educational Research, 82(3), 300–329. https://doi.org/10.3102/0034654312457206
33.
Galindo-DomínguezH.BezanillaM. J. (2019). A systematic review of flipped classroom methodology at university level in Spain. International Journal of Technology and Educational Innovation, 5(1), 81–90. https://doi.org/10.24310/innoeduca.2019.v5i1.4470
34.
Gianoni-CapenakasS.LagravereM.Pachêco-PereiraC.YacyshynJ. (2019). Effectiveness and perceptions of flipped learning model in dental education: A systematic review. Journal of Dental Education, 83(8), 935–945. https://doi.org/10.21815/JDE.019.109
35.
GilletteC.RudolphM.KimbleC.Rockich-WinstonN.SmithL.Broedel-ZauggK. (2018). A meta-analysis of outcomes comparing flipped classroom and lecture. American Journal of Pharmaceutical Education, 82(5), Article 6898. https://doi.org/10.5688/ajpe6898
36.
GleserL. J.OlkinI. (2009). Stochastically dependent effect sizes. In CooperH.HedgesL. V.ValentineJ. C. (Eds.), Handbook of research synthesis and meta-analysis (2nd ed., pp. 357–376). Russell Sage Foundation.
37.
Gorres-MartensB. K.SegoviaA. R.PfeferM. T. (2016). Positive outcomes increase over time with the implementation of a semiflipped teaching model. Advances in Physiological Education, 40(1), 32–37. https://doi.org/10.1152/advan.00034.2015
HedgesL. V. (1981). Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128. https://doi.org/10.3102/10769986006002107
42.
HedgesL. V.OlkinI. (1985). Statistical methods for meta-analysis. Academic Press.
HempelS.MilesJ. N.BoothM. J.WangZ.MortonS. C.ShekelleP. G. (2013). Risk of bias: A simulation study of power to detect study-level moderator effects in meta-analysis. Systematic Reviews, 2(1), Article 107. https://doi.org/10.1186/2046-4053-2-107
45.
HewK. F.LoC. K. (2018). Flipped classroom improves student learning in health professions education: A meta-analysis. BMC Medical Education, 18(1), Article 38. https://doi.org/10.1186/s12909-018-1144-z
46.
HigginsJ. P. T.GreenS. (Eds.). (2011). Cochrane handbook for systematic reviews of interventions (Version 5.1.0). http://handbook.cochrane.org
47.
HigginsJ. P. T.ThompsonS. G.DeeksJ. J.AltmanD. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327(7414), 557–560. https://doi.org/10.1136/bmj.327.7414.557
48.
HofstedeG. H. (2001). Culture’s consequences: Comparing values, behaviours, institutions, and organizations across nations (2nd ed.). Sage.
49.
HoraM. T.FerrareJ. J. (2012). Instructional systems of practice: A multidimensional analysis of math and science undergraduate course planning and classroom teaching. Journal of the Learning Sciences, 22(2), 212–257. https://doi.org/10.1080/10508406.2012.729767
50.
HoweK. R.BervJ. (2000). Constructing constructivism, epistemological and pedagogical. In PhillipsD. C. (Ed.), Constructivism in education: Opinions and second opinions on controversial issues (pp. 19–40). University of Chicago Press.
51.
HuR.GaoH.YeY.NiZ.JiangN.JiangX. (2018). Effectiveness of flipped classrooms in Chinese baccalaureate nursing education: A meta-analysis of randomized controlled trials. International Journal of Nursing Studies, 79(March), 94–103. https://doi.org/10.1016/j.ijnurstu.2017.11.012
52.
HughesY.LyonsN. (2017). Does the flipped classroom improve exam performance in medical education? A systematic review. MedEdPublish, 6(2), Article 38. https://doi.org/10.15694/mep.2017.000100
53.
IvesJ.Castillo-MontoyaM. (2020). First-generation college students as academic learners: A systematic review. Review of Educational Research, 90(2), 139–178. https://doi.org/10.3102/0034654319899707
54.
JensenJ. L.HoltE. A.SowardsJ. B.OgdenT. H.WestR. E. (2018). Investigating strategies for pre-class content learning in a flipped classroom. Journal of Science Education and Technology, 27(6), 523–535. https://doi.org/10.1007/s10956-018-9740-6
55.
JensenS. A. (2011). In-class versus online video lectures: Similar learning outcomes, but a preference for in-class. Teaching of Psychology, 38(4), 298–302. https://doi.org/10.1177/0098628311421336
56.
Karabulut-IlguA.CherrezN. J.JahrenC. T. (2018). A systematic review of research on the flipped learning method in engineering education. British Journal of Educational Technology, 49(3), 398–411. https://doi.org/10.1111/bjet.12548
57.
KaragölI.EsenE. (2019). The effect of flipped learning approach on academic achievement: A meta-analysis study. Hacettepe University Journal of Education, 34(3), 708–727. https://doi.org/10.16986/HUJE.2018046755
58.
KicinskiM.SpringateD. A.KontopantelisE. (2015). Publication bias in meta-analyses from the Cochrane Database of Systematic Reviews. Statistics in Medicine, 34(20), 2781–2793. https://doi.org/10.1002/sim.6525
59.
KimM. K.KimS. M.KheraO.GetmanJ. (2014). The experience of three flipped classrooms in an urban university: An exploration of design principles. Internet and Higher Education, 22(July), 37–50. https://doi.org/10.1016/j.iheduc.2014.04.003
60.
KivunjaC. (2014). Innovative pedagogies in higher education to become effective teachers of 21st century skills: Unpacking the learning and innovations skills domain of the new learning paradigm. International Journal of Higher Education, 3(4), 37–48. https://doi.org/10.5430/ijhe.v3n4p37
61.
KozikoğluI. (2019). Analysis of the studies concerning flipped learning model: A comparative meta-synthesis study. International Journal of Instruction, 12(1), 851–868. https://doi.org/10.29333/iji.2019.12155a
62.
LågT.SæleR. G. (2019). Does the flipped classroom improve student learning and satisfaction? A systematic review and meta-analysis. AERA Open, 5(3), 1–17. https://doi.org/10.1177/2332858419870489
63.
LageM. J.PlattG. J.TregliaM. (2000). Inverting the classroom: A gateway to creating an inclusive learning environment. Journal of Economic Education, 31(1), 30–43. https://doi.org/10.2307/1183338
64.
LazonderA. W.HarmsenR. (2016). Meta-analysis of inquiry-based learning: Effects of guidance. Review of Educational Research, 86(3), 681–718. https://doi.org/10.3102/0034654315627366
65.
LimD. H.MorrisM. L. (2009). Learner and instructional factors influencing learning outcomes with online learning environment. Educational Technology & Society, 12(4), 282–293. https://www.jstor.org/stable/jeductechsoci.12.4.282
66.
LipseyM. W.WilsonD. B. (2001). Practical meta-analysis (2nd ed.). Sage.
67.
LiuY.-Q.LiY.-F.LeiM.-J.LiuP.-X.TheobaldJ.MengL.-N.LiuT.-T.ZhangC.-M.JinC.-D. (2018). Effectiveness of the flipped classroom on the development of self-directed learning in nursing education: A meta-analysis. Frontiers of Nursing, 5(4), 317–329. https://doi.org/10.1515/FON-2018-0032
68.
LoC. K.HewK. F.ChenG. W. (2017). Toward a set of design principles for mathematics flipped classrooms: A synthesis of research in mathematics education. Educational Research Review, 22(November), 50–73. https://doi.org/10.1016/j.edurev.2017.08.002
69.
LyB. H.BrewC. (2010). Philosophical and pedagogical patterns of beliefs among Vietnamese and Australian mathematics preservice teachers: A comparative study. Australian Journal of Teacher Education, 35(2), 67–86. https://doi.org/10.14221/ajte.2010v35n2.5
70.
MartinA. J.DowsonM. (2009). Interpersonal relationships, motivation, engagement, and achievement: Yields for theory, current issues, and educational practice. Review of Educational Research, 79(1), 327–365. https://doi.org/10.3102/0034654308325583
McLaughlinJ. E.RothM. T.GlattD. M.GharkholonareheN.DavidsonC. A.GriffinL. M.EssermanD. A.MumperR. J. (2014). The flipped classroom: A course redesign to foster learning and engagement in a health professions school. Academic Medicine, 89(2), 236–243. https://doi.org/10.1097/ACM.0000000000000086
73.
McLaughlinJ. E.WhiteP. J.KhanovaJ.YurievE. (2016). Flipped classroom implementation: A case report of two higher education institutions in the United States and Australia. Computers in the Schools, 33(1), 24–37. https://doi.org/10.1080/07380569.2016.1137734
74.
MorenoR.MayerR. E. (2002). Verbal redundancy in multimedia learning: When reading helps listening. Journal of Education & Psychology, 94(1), 156–163. https://doi.org/10.1037//0022-0663.94.1.156
75.
MorrisonA.PolisenaJ.HusereauD.MoultonK.ClarkM.FianderM.Mierzwinski-UrbanM.CliffordT.HuttonB.RabbD. (2012). The effect of English language restriction on systematic review-based meta-analyses: A systematic review of empirical studies. International Journal of Technology Assessment in Health Care, 28(2), 138–144. https://doi.org/10.1017/S0266462312000086
76.
National Academies of Sciences, Engineering, and Medicine. (2017). Supporting students’ college success: The role of assessment of intrapersonal and interpersonal competencies. National Academies Press. https://doi.org/10.17226/24697
77.
Nussbaumer-StreitB.KleringsI.DobrescuA. I.PersadE.StevensA.GarrittyC.KamelC.AffengruberL.KingV. J.GartlehnerG. (2020). Excluding non-English publications from evidence-syntheses did not change conclusions: A meta-epidemiological study. Journal of Clinical Epidemiology, 118, 42–54. https://doi.org/10.1016/j.jclinepi.2019.10.011
78.
O’FlahertyJ.PhillipsC. (2015). The use of flipped classrooms in higher education: A scoping review. Internet and Higher Education, 25(April), 95. https://doi.org/10.1016/j.iheduc.2015.02.002
79.
OrhanA. (2019). The effect of flipped learning on students’ academic achievement: A meta-analysis study. Cukurova University Faculty of Education Journal, 48(1), 368–396. https://doi.org/10.14812/cufej.400919
80.
OrwinR. G. (1983). A fail-safeN for effect size in meta-analysis. Journal of Educational Statistics, 8(2), 157–159. https://doi.org/10.2307/1164923
81.
OvermyerG. R. (2014). The flipped classroom model for college algebra: Effects on student achievement [Doctoral dissertation, Colorado State University]. CSU Theses and Dissertations. http://hdl.handle.net/10217/83800
82.
Phuong-MaiN.TerlouwC.PilotA. (2005). Cooperative learning vs Confucian heritage culture’s collectivism: Confrontation to reveal some cultural conflicts and mismatch. Asia Europe Journal, 3(3), 403–419. https://doi.org/10.1007/s10308-005-0008-4
83.
PolaninJ. R.PigottT. D. (2015). The use of meta-analytic statistical significance testing. Research Synthesis Methods, 6(1), 63–73. https://doi.org/10.1002/jrsm.1124
84.
PolaninJ. R.Tanner-SmithE. E.HennessyE. A. (2016). Estimating the difference between published and unpublished effect sizes: A meta-review. Review of Educational Research, 86(1), 207–236. https://doi.org/10.3102/0034654315582067
85.
PrestiC. R. (2016). The flipped learning approach in nursing education: A literature review. Journal of Nursing Education, 55(5), 252–257. https://doi.org/10.3928/01484834-20160414-03
SanchezI. M. (2000). Motivating and maximizing learning in minority classrooms. New Directions for Community Colleges, 2000(112), 35–44. https://doi.org/10.1002/cc.11203
90.
SchneiderM.PreckelF. (2017). Variables associated with achievement in higher education: A systematic review of meta-analyses. Psychological Bulletin, 143(6), 565–600. https://doi.org/10.1037/bul0000098
91.
ShcheglovaI. A. (2018). A cross-cultural comparison of the academic engagement of students. Russian Education & Society, 60(8–9), 665–681. https://doi.org/10.1080/10609393.2018.1598163
92.
ShiY.MaY.MacLeodJ.YangH. H. (2020). College students’ cognitive learning outcomes in flipped classroom instruction: A meta-analysis of the empirical literature. Journal of Computers in Education, 7(1), 79–103. https://doi.org/10.1007/s40692-019-00142-8
93.
SimmonsM.ColvilleD.BullockS.WillemsJ.MachadoM.McArdleA.TareM.KellyJ.Ali TaherM.Sallyann MiddletonS.ShuttleworthM.ReserD. (2020). Introducing the flip: A mixed method approach to gauge student and staff perceptions on the introduction of flipped pedagogy in pre-clinical medical education. Australasian Journal of Educational Technology, 36(3), 163–175. https://doi.org/10.14742/ajet.5600
94.
SmallhornM. (2017). The flipped classroom: A learning model to increase student engagement not academic achievement. Student Success, 8(2), 43–53. https://doi.org/10.5204/ssj.v8i2.381
95.
SwellerJ. (2020). Cognitive load theory and educational technology. Educational Technology Research and Development, 68(1), 1–16. https://doi.org/10.1007/s11423-019-09701-3
96.
TabachnickB. G.FidellL. S. (2013). Using multivariate statistics (6th ed.). Pearson.
97.
TanC.YueW.-G.FuY. (2017). Effectiveness of flipped classrooms in nursing education: Systematic review and meta-analysis. Chinese Nursing Research, 4(4), 192–200. https://doi.org/10.1016/j.cnre.2017.10.006
98.
TheobaldE. J.HillM. J.TranE.AgrawalS.ArroyoE. N.BehlingS.ChambweN.CintrónD. L.CooperJ. D.DunsterG.GrummerJ. A.HennesseyK.HsiaoJ.IranonN.JonesL.IIJordtH.KellerM.LaceyM. E.LittlefieldC. E.… FreemanS. (2020). Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proceedings of the National Academy of Sciences of the United States of America, 117(12), 6476–6483. https://doi.org/10.1073/pnas.1916903117
99.
TobinK.TippinsD. (1993). Constructivism as a referent for teaching and learning. In TobinK. (Ed.), The practice of constructivism in science education (pp. 1–20). Lawrence Erlbaum.
100.
TuranZ.Akdag-CimenB. (2020). Flipped classroom in English language teaching: A systematic review. Computer Assisted Language Learning, 33(5–6), 590–606. https://doi.org/10.1080/09588221.2019.1584117
101.
UttlB.WhiteC. A.GonzalezD. W. (2017). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54(September), 22–42. https://doi.org/10.1016/j.stueduc.2016.08.007
102.
van AltenD. C. D.PhielixC.JanssenJ.KesterL. (2019). Effects of flipping the classroom on learning outcomes and satisfaction: A meta-analysis. Educational Research Review, 28(November), Article 100281. https://doi.org/10.1016/j.edurev.2019.05.003
103.
Van MerriënboerJ. G.SwellerJ. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17(2), 147–177. https://doi.org/10.1007/s10648-005-3951-0
104.
ViechtbauerW.CheungM. W. L. (2010). Outlier and influence diagnostics for meta-analysis. Research Synthesis Methods, 1(2), 112–125. https://doi.org/10.1002/jrsm.11
105.
WankatP. C. (2002). The effective, efficient professor: Teaching, scholarship and service. Allyn & Bacon.
106.
WigfieldA.KlaudaS. L.CambriaJ. (2011). Influences on the development of academic self-regulatory processes. In BarryD. H. S.ZimmermanJ. (Eds.), Handbook of self-regulation of learning and performance (pp. 33–48). Routledge.
107.
WindschitlM. (2002). Framing constructivism in practice as the negotiation of dilemmas: An analysis of the conceptual, pedagogical, cultural, and political challenges facing teachers. Review of Educational Research, 72(2), 131–175. https://doi.org/10.3102/00346543072002131
108.
WrightS. L.Jenkins-GuarnieriM. A. (2012). Student evaluations of teaching: Combining the meta-analyses and demonstrating further evidence for effective use. Assessment & Evaluation in Higher Education, 37(6), 683–699. https://doi.org/10.1080/02602938.2011.563279
109.
XuP.ChenY.NieW.WangY.SongT.LiH.LiJ.YiJ.ZhaoL. (2019). The effectiveness of a flipped classroom on the development of Chinese nursing students’ skill competence: A systematic review and meta-analysis. Nurse Education Today, 80(September), 67–77. https://doi.org/10.1016/j.nedt.2019.06.005
110.
YangL.SunT.LiuY. (2017). A bibliometric investigation of flipped classroom research during 2000–2015. International Journal of Emerging Technologies, 12(6), 178–186. https://doi.org/10.3991/ijet.v12i06.7095
111.
ZainuddinZ.HaliliS. H. (2016). Flipped classroom research and trends from different fields of study. International Review of Research in Open and Distributed Learning, 17(3), 313–340. https://doi.org/10.19173/irrodl.v17i3.2274
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
