College Students’ Cognitive Learning Outcomes in Technology-Enabled Active Learning Environments: A Meta-Analysis of the Empirical Literature

Abstract

Technology-enabled active learning environments (TE-ALEs) have attracted considerable research interest, particularly in higher education. However, research shows inconsistent results describing the influence of TE-ALEs toward students’ cognitive learning outcomes. This study was designed to identify high-quality empirical research examining college students’ cognitive learning outcomes and to utilize meta-analysis to determine the overall effectiveness of TE-ALEs. A systematic literature search identified 31 high-quality peer-reviewed journal articles that met the inclusion criteria. Meta-analysis showed that the calculated effect size of TE-ALEs more positively influenced students’ cognitive learning than traditional lecture-based environments. Moderator variable analysis suggested that social context, study design, and sample size were significant factors that influence the effectiveness of TE-ALE. TE-ALEs were found more effective when instructors employed individualized learning contexts as well as when bias was reduced in randomized controlled trials. TE-ALEs were also found to be more effective in small courses rather than in large courses.

Keywords

technology-enabled active learning active learning environment cognitive learning outcomes meta-analyses effect size

Get full access to this article

View all access options for this article.

References

References marked with an asterisk indicate studies included in the meta-analysis.

*Alcalde

Nagel

(2019) Why does peer instruction improve student satisfaction more than student performance? A randomized experiment. International Review of Economics Education 30: 100149 . doi:10.1016/j.iree.2018.10.001.

Andrews

T. M.

Leonard

M. J.

Colgrove

C. A.

Kalinowski

S. T.

(2011) Active learning not associated with student learning in a random sample of college biology courses. CBE-Life Sciences Education 10: 394–405. doi:10.1187/cbe.11-07-0061.

*Aşıksoy

Özdamli

(2016) Flipped classroom adapted to the arcs model of motivation and applied to a physics course. Eurasia Journal of Mathematics, Science & Technology Education 12(6): 1589–1603. doi:10.12973/eurasia.2016.1251a.

*Baepler

Walker

J. D.

Driessen

(2014) It’s not about seat time: Blending, flipping, and efficiency in active learning classrooms. Computers & Education 78: 227–236. doi:10.1016/j.compedu.2014.06.006.

*Baranowski

M. K.

(2006) Single session simulations: The effectiveness of short congressional simulations in introductory American government classes. Journal of Political Science Education 2(1): 33–49. doi:10.1080/15512160500484135.

Baranowski

M. K.

Weir

K. A.

(2015) Political simulations: What we know, what we think we know, and what we still need to know. Journal of Political Science Education 11(4): 391–403. doi:10.1080/15512169.2015.1065748.

*Baytiyeh

Naja

M. K.

(2017) Students’ perceptions of the flipped classroom model in an engineering course: A case study. European Journal of Engineering Education 42(6): 1048–1061. doi:10.1080/03043797.2016.1252905.

Beichner

R. J.

(2014) History and evolution of active learning spaces. In: Baepler

Brooks

D. C.

Walker

J. D.

(eds) New directions for teaching and learning, special issue: Active Learning Spaces, San Francisco, CA: Jossey-Bass, pp. 9–16.

10.

Beichner, R. J., Saul, J., Abbott, D., Morse, J. J., Deardorff, D. L., Allain, R., … Risley, J. S. (2007). The student-centered activities for large enrollment undergraduate programs (SCALE-UP) project. Retrieved from https://www.compadre.org/Repository/document/ServeFile.cfm?ID=4517&DocID=183.

11.

Bernard

R. M.

Abrami

P. C.

Lou

Borokhovski

Wade

Wozney

Huang

(2004) How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research 74(3): 379–439. doi:10.3102/00346543074003379.

12.

*Blázquez

B. O.

Masluk

Gascon

Díaz

R. F.

Aguilar-Latorre

Magallón

I. A.

Botaya

R. M.

(2019) The use of flipped classroom as an active learning approach improves academic performance in social work: A randomized trial in a university. PLoS One 14(4): e0214623 . doi:10.1371/journal.pone.0214623.

13.

Bonwell, C. C., & Eison, J. A. (1991). Active learning: Creating excitement in the classroom (ASHEERIC Higher Education Report No. 1). Washington, DC: The George Washington University.

14.

Bouroumi

Fajr

(2014) Collaborative and cooperative e-learning in higher education in Morocco: A case study. International Journal of Emerging Technologies in Learning 9(1): 66–72.

15.

*Brooks

D. C.

(2011) Space matters: The impact of formal learning environments on student learning. British Journal of Educational Technology 42(5): 719–726. doi:10.1111/j.1467-8535.2010.01098.x.

16.

Brooks, D. C. (2017). Active learning classrooms: The topic strategic technology for 2017 (EDUCAUSE Center for Analysis and Research Report). Retrieved from https://er.educause.edu/blogs/2017/3/active-learning-classrooms-the-top-strategic-technology-for-2017.

17.

*Chiu

P. H. P.

Cheng

S. H.

(2017) Effects of active learning classrooms on student learning: A two-year empirical investigation on student perceptions and academic performance. Higher Education Research & Development 36: 269–279. doi:10.1080/07294360.2016.1196475.

18.

Cochrane Collaboration. (2014). Review manager (Version 5.3) [Computer software]. Retrieved from https://community.cochrane.org/help/tools-and-software/revman-5/revman-5-download.

19.

*Cotner

Loper

Walker

J. D.

Brooks

(2013) “It’s not you, it’s the room”—Are the high-tech, active learning classrooms worth it? Journal of College Science Teaching 42(6): 82–88. doi:10.2505/4/jcst13_042_06_82.

20.

*Cotta

K. I.

Shah

Almgren

M. M.

Macias-Moriarity

L. Z.

Mody

(2016) Effectiveness of flipped classroom instructional model in teaching pharmaceutical calculations. Currents in Pharmacy Teaching and Learning 8: 646–653. doi:10.1016/j.cptl.2016.06.011.

21.

*Crimmins

M. T.

Midkiff

(2017) High structure active learning pedagogy for the teaching of organic chemistry: Assessing the impact on academic outcomes. Journal of Chemical Education 94(4): 429–438. doi:10.1021/acs.jchemed.6b00663.

22.

Cruz

da Silva

F. Q. B.

Capretz

L. F.

(2015) Forty years of research on personality in software engineering: A mapping study. Computers in Human Behavior 46: 94–113. doi:10.1016/j.chb.2014.12.008.

23.

Deslauriers

Schelew

Wieman

(2011) Improved learning in a large-enrollment physics class. Science 332(6031): 862–864. doi:10.1126/science.1202043.

24.

Dori

Y. J.

Belcher

(2005) How does technology-enabled active learning affect undergraduate students’ understanding of electromagnetism concepts? The Journal of the Learning Science 14(2): 243–279. doi:10.1207/s15327809jls1402_3.

25.

Dori

Y. J.

Belcher

Bessette

Danziger

McKinney

Hult

(2003) Technology for active learning. Materials Today 6(12): 44–49. doi:10.1016/S1369-7021(03)01225-2.

26.

*Dori

Y. J.

Hult

Breslow

Belcher

J. W.

(2007) How much have they retained? Making unseen concepts seen in a freshman electromagnetism course at MIT. Journal of Science Education and Technology 16(4): 299–323. doi:10.1007/s10956-007-9051-9.

27.

Efstathiou

Bailey

(2012) Promoting active learning using Audience Response System in large bioscience classes. Nurse Education Today 32(1): 91–95. doi:10.1016/j.nedt.2011.01.017.

28.

*Entezari

Javdan

(2016) Active learning and flipped classroom, hand in hand approach to improve students learning in human anatomy and physiology. International Journal of Higher Education 5(4): 222–231. doi:10.5430/ijhe.v5n4p222.

29.

Freeman

Eddy

S. L.

McDonough

Smith

M. K.

Okoroafor

Jordt

Wenderoth

M. P.

(2014) Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United State of America 111(23): 8410–8415. doi:10.1073/pnas.1319030111.

30.

*Ghorbani

Ghazvini

(2016) Using paper presentation breaks during didactic lectures improves learning of physiology in undergraduate students. Advances in Physiology Education 40(1): 93–97. doi:10.1152/advan.00137.2015.

31.

Gilies

R. M.

Ashman

A. F.

Terwel

(2008) The teacher’s role in implementing cooperative learning in the classroom, New York, NY: Springer.

32.

Glass

G. V.

(1976) Primary, secondary, and meta-analysis. Educational Researcher 5: 3–8. doi:10.3102/0013189X005010003.

33.

*Green

A. J.

Repetti

(2015) Clickers: A strategy for active learning in a hospitality classroom. Journal of Hospitality & Tourism Education 27(4): 171–179. doi:10.1080/10963758.2015.1089513.

34.

Hake

(1998) Interactive engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics 66(1): 64–74. doi:10.1119/1.18809.

35.

*He

Holton

Farkas

Warschauer

(2016) The effects of flipped instruction on out-of-class study time, exam performance, and student perceptions. Learning & Instruction 45: 61–71. doi:10.1016/j.learninstruc.2016.07.001.

36.

Hedges

L. V.

Olkin

(1985) Statistical methods for meta-analysis, Orlando, FL: Academic Press.

37.

Herreid

C. F.

(2006) “Clicker” cases: Introducing case study teaching into large classrooms. Journal of College Science Teaching 36(2): 43–47.

38.

*Heuett

W. J.

(2017) Flipping the math classroom for non-math majors to enrich their learning experience. Problems, Resources, and Issues in Mathematics Undergraduate Studies (Primus) 27(10): 889–907. doi:10.1080/10511970.2016.1256925.

39.

Hew

K. F.

C. K.

(2018) Flipped classroom improves student learning in health professions education: A meta-analysis. BMC Medical Education 18(1): 38 . doi:10.1186/s12909-018-1144-z.

40.

Higgins

J. P.

Thompson

S. G.

Deeks

J. J.

Altman

D. G.

(2003) Measuring inconsistency in meta-analyses. British Medical Journal 327(7414): 557–560.

41.

Hoeve

Stams

G. J.

van der Put

C. E.

Dubas

J. S.

van der Laan

Gerris

J. R. M.

(2012) A meta-analysis of attachment to parents and delinquency. Journal of Abnormal Child Psychology 40(5): 771–785. doi:10.1007/s10802-011-9608-1.

42.

*Hung

H. T.

(2015) Flipping the classroom for English language learners to foster active learning. Computer Assisted Language Learning 28(1): 81–96. doi:10.1080/09588221.2014.967701.

43.

Jackson

C. K.

Bruegmann

(2009) Teaching students and teaching each other: The importance of peer learning for teachers. American Economic Journal: Applied Economics 1(4): 85–108. doi:10.1257/app.1.4.85.

44.

Kalaian, S. A., & Kasim, R. M. (2014). A meta-analytic review of studies of the effectiveness of small-group learning methods on statistics achievement. Journal of Statistics Education, 22(1). doi:10.1080/10691898.2014.11889691.

45.

Kovalchick

Kawson

(2004) Education and technology: An encyclopedia (Vol. 1), Santa Barbara, CA: ABC-CLIO.

46.

*Krain

Lantis

(2006) Building knowledge? Evaluating the effectiveness of the global problems summit simulation. International Studies Perspectives 7(4): 395–407. doi:10.1111/j.1528-3585.2006.00261.x.

47.

*Lewis

C. E.

Chen

D. C.

Relan

(2018) Implementation of a ﬂipped classroom approach to promote active learning in the third-year surgery clerkship. The American Journal of Surgery 215: 298–303. doi:10.1016/j.amjsurg.2017.08.050.

48.

*Lin

Y.-T.

(2019) Impacts of a flipped classroom with a smart learning diagnosis system on students’ learning performance, perception, and problem solving ability in a software engineering course. Computers in Human Behavior 95: 187–196. doi:10.1016/j.chb.2018.11.036.

49.

Lyon

D. C.

Lagowski

J. J.

(2008) Effectiveness of facilitating small-group learning in large lecture classes. Journal of Chemical Education 85(11): 1571–1576. doi:10.1021/ed085p1571.

50.

MacLeod

Yang

H. H.

Zhu

(2018) Understanding students’ preferences toward the smart classroom learning environment: Development and validation of an instrument. Computers & Education 122: 80–91. doi:10.1016/j.compedu.2018.03.015.

51.

Mazur

(2004) Interactive teaching: Promoting better learning using peer instruction and just-in-time teaching, Upper Saddle River, NJ: Prentice Hall.

52.

Mazur

(2009) Farewell, lecture? Science 323(5910): 50–51. doi:10.1126/science.1168927.

53.

*McCarroll

M. L.

Pohle-Krauza

R. J.

Martin

J. L.

(2009) Active learning in the classroom: A muscle identiﬁcation game in a kinesiology course. Advances in Physiology Education 33(4): 319–322. doi:10.1152/advan.00013.2009.

54.

Means

Toyama

Murphy

Baki

(2013) The effectiveness of online and blended learning: A meta-analysis of the empirical literature. Teachers College Record 115(3): 1–47. doi:10.1080/10888438.2011.624566.

55.

*Mendini

Peter

P. C.

(2019) Research note: The role of smart versus traditional classrooms on students’ engagement. Marketing Education Review 29(1): 17–23. doi:10.1080/10528008.2018.1532301.

56.

Michaelowa

(2010) Primary education quality in francophone Sub-Saharan Africa: Determinants of learning achievement and efficiency considerations. World Development 29(10): 1699–1716. doi:10.1016/S0305-750X(01)00061-4.

57.

Mueller

A. L.

Knobloch

N. A.

Orvis

K. S.

(2015) Exploring the effects of active learning on high school students’ outcomes and teachers’ perceptions of biotechnology and genetics instruction. Journal of Agricultural Education 56(2): 138–152. doi:10.5032/jae.2015.02138.

58.

Naing

Wai

V. N.

Durham

Whittaker

M. A.

Win

N. N.

Aung

Mak

J. W.

(2015) A systematic review and meta-analysis of medical students’ perspectives on the engagement in research. Medicine 94(28): e1089 . doi:10.1097/MD.0000000000001089.

59.

Parishan

Jafari

E. M.

Nosrat

(2011) The effect of technology enabled active learning (TEAL) method in biology on the academic achievements of students. Procedia—Social and Behavioral Sciences 28: 542–546. doi:10.1016/j.sbspro.2011.11.104.

60.

Partnership for 21st Century Learning. (2007). Framework for 21st century learning. Retrieved from http://www.battelleforkids.org/networks/p21/frameworks-resources.

61.

Patterson

Kilpatrick

Woebkenberg

(2010) Evidence for teaching practice: The impact of clickers in a large classroom environment. Nurse Education Today 30: 603–607.

62.

Peplow, M. (2017). Social Sciences suffer from severe publication bias. Retrieved from https://www.nature.com/news/social-sciences-suffer-from-severe-publication-bias-1.15787.

63.

*Poirier

C. R.

Feldman

R. S.

(2007) Promoting active learning using individual response technology in large introductory psychology classes. Teaching of Psychology 34(3): 194–196. doi:10.1080/00986280701498665.

64.

Prince

(2004) Does active learning work? A review of the research. Journal of Engineering Education 93(3): 223–231. doi:10.1002/j.2168-9830.2004. tb00809.x.

65.

*Rau

M. A.

Kennedy

Oxtoby

Bollom

Moore

J. W.

(2017) Unpacking “active learning”: A combination of flipped classroom and collaboration support is more effective but collaboration support alone is not. Journal of Chemical Education 94(10): 1406–1414. doi:10.1021/acs.jchemed.7b00240.

66.

Riley

Ward

(2017) Active learning, cooperative active learning, and passive learning methods in an accounting information systems course. Issues in Accounting Education 32(2): 1–16. doi:10.2308/iace-51366.

67.

Rosenthal

(1979) The file-drawer problem and tolerance of null result. Psychological Bulletin 86(3): 638–641. doi:10.1037/0033-2909.86.3.638.

68.

Rosenthal

(1991) Meta-analytic procedures for social research (Vol. 86), Newbury Park, CA: SAGE.

69.

Ruiz-Primo

M. A.

Briggs

Iverson

Talbot

Shepard

L. A.

(2011) Impact of undergraduate science course innovations on learning. Science 331(6022): 1269–1270. doi:10.1126/science.1198976.

70.

Ryan

M. D.

Reid

S. A.

(2016) Impact of the flipped classroom on student performance and retention: A parallel controlled study in general chemistry. Journal of Chemical Education 93(1): 13–23. doi:10.1021/acs.jchemed.5b00717.

71.

Seery

M. K.

(2015) Flipped learning in higher education chemistry: Emerging trends and potential directions. Chemistry Education Research and Practice 16(4): 758–768. doi:10.1039/C5RP00136F.

72.

*Sevindik

(2010) Future’s learning environments in health education: The effects of smart classrooms on the academic achievements of the students at Health College. Telematics and Informatics 27(3): 314–322. doi:10.1016/j.tele. 2009.08.001.

73.

Shi, Y. H, Ma, Y. Q, MacLeod, J., & Yang, H. H. (2019). College students’ cognitive learning outcomes in flipped classroom instruction: A meta-analysis of the empirical literature. Journal of Computers in Education. Advance online publication. doi:10.1007/s40692-019-00142-8.

74.

Shi, Y. H, Peng, C. L., Wang, S. M., & Yang, H. H. (2018). The effects of smart classroom-based instruction on college students’ learning engagement and internet self-efficacy. In S. Cheung, L. Kwok, K. Kubota, L. K. Lee, & J. Tokito (Eds.), 2018 International conference on blended learning (pp. 263–274). New York, NY: Springer. doi:10.1007/978-3-319-94505-7_21.

75.

Shieh

R. S.

(2012) The impact of technology-enabled active learning (TEAL) implementation on student learning and teachers’ teaching in a high school context. Computers & Education 59: 206–214. doi:10.1016/j.compedu.2012.01.016.

76.

*Shieh

R. S.

Chang

Liu

E. Z.-F.

(2011) Technology enabled active learning (TEAL) in introductory physics: Impact on genders and achievement levels. Australasian Journal of Educational Technology 27(7): 1082–1099. doi:10.14742/ajet.905.

77.

*Shieh

R. S.

Chang

Tang

(2010) The impact of implementing technology-enabled active learning (TEAL) in university physics in Taiwan. The Asia Pacific Education Researcher 19(3): 401–415. doi:10.3860/taper.v19i3.1850.

78.

Springer

Stanne

M. E.

Donovan

S. S.

(1999) Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: A meta-analysis. Review of Educational Research 69(1): 21–51. doi:10.3102/00346543069001021.

79.

Stains

Harshman

Barker

M. K.

Chasteen

S. V.

Cole

DeChenne-Peters

S. E.

Levis-Fitzgerald

(2018) Anatomy of STEM teaching in North American universities. Science 359(6383): 1468–1470. doi:10.1126/science.aap8892.

80.

*Tune

J. D.

Sturek

Basile

D. P.

(2013) Flipped classroom model improves graduate student performance in cardiovascular, respiratory, and renal physiology. Advances in Physiology Education 37(4): 316–320. doi:10.1152/advan.00091.2013.

81.

*Welch

(2013) Effectiveness of classroom response systems within an active learning environment. Journal of Nursing Education 52(11): 653–656. doi:10.3928/01484834-20131014-01.

82.

*Wilson

L. V. E.

(2012) An assessment of learning outcomes based on a comparison of active learning and traditional lecture pedagogical styles in a legal environment classroom. Southern Journal of Business & Ethics 4: 101–110.