What to Blend? Exploring the Relationship Between Student Engagement and Academic Achievement via a Blended Learning Approach

Abstract

The present study investigated whether student engagement with different online blended learning activities predicts academic performance as measured via a multiple-choice online exam for an undergraduate cognitive psychology course. Higher completion rates of weekly online quizzes predicted final exam performance. Findings are discussed in relation to using online learning resources to enhance student engagement and performance and learning analytics to identify students in need of further support. Since findings only revealed one significant predictor, more research is required to identify additional factors influencing academic achievement in an online blended learning approach.

Keywords

Student engagement blended learning academic achievement psychology education

Introduction

With the arrival of COVID-19 in 2020, higher education pedagogies shifted towards blended learning (i.e., a combination between online/on-site synchronous and online asynchronous activities via virtual learning platforms; Kintu et al., 2017) or fully online formats. Even prior to the pandemic, the global online education market (including investments in virtual tutoring, video conferencing tools, integrated learning systems and online teaching/learning software) had been growing and would reach over $350 billion by 2025 (Koksal, 2020). Regardless of the driver of change (e.g., social distancing; accommodating part-time and distant learners), the consensus from the academic world (Yeigh & Lynch, 2020) and regulatory bodies (Barber et al., 2021) is that many teaching and learning activities will remain online, and that blended learning “is here to stay” (Abi-Raad & Odhabi, 2021). A recent systematic review (Raes et al., 2020) on blended learning revealed that the flexibility offered improves self-regulated learning, but synchronous activities lack in interaction, are not well attended, and maintaining comparable learning standards for everyone is difficult. Consequently, the implementation of blended learning should be informed by evidence for the benefits and challenges it carries in relation to learning outcomes, teaching quality and efficiency. In this paper, we investigated whether student engagement with different activities in a blended learning approach predicts academic achievement.

A positive relationship between measurements of student engagement and academic achievement is well-documented in various programmes and instructional formats. In most studies, academic achievement is determined by grades. Student engagement is defined as the “…student psychological investment in an effort directed toward learning, understanding, or mastering the knowledge, skills, or crafts that academic work intends to promote” (Newmann et al., 1992, p. 12). This definition displays that student engagement is multi-faceted and includes behavioural (i.e., student participation in learning activities), emotional (i.e., reaction to learning and other agents) and cognitive components (i.e., strategies used for learning). A meta-analysis of 69 independent studies demonstrated a moderately strong and positive correlation between student engagement and academic achievement (Lei et al., 2018). However, it also revealed that studies report contradicting findings (Chen et al., 2013; Shernoff & Schmidt, 2008), raised the importance of accounting for individual differences in how students master skills (e.g., students who perform well, master their learning skills, and need less time to study), and highlighted that the different components of engagement might have a differential association with academic achievement (Lei et al., 2018). Crucial to this study, behavioural engagement is consistently considered to be more strongly and positively associated with academic achievement, compared to emotional and cognitive engagement (Furrer & Skinner, 2003).

In a blended learning approach, student participation in learning activities includes the three traditional types of interaction (i.e., learner-content, learner-instructor, and learner-learner interaction; Moore, 1989), which occur asynchronously and synchronously. The learner-content interaction is achieved via asynchronous activities that students complete online before or after a synchronous session (e.g., students read a textbook chapter, watch a pre-recorded video, complete a quiz). Furthermore, when designing learner-content interactions, instructors create their own online resources (e.g., via the use of different virtual learning environments and recording software, instructors create videos about a topic and embed self-paced questions for the students). They can also use online learning platforms (i.e., integrated learning systems) provided by educational publishers which include individual access to e-books and online learning activities such as flashcards, animated videos, and quizzes that students can complete asynchronously (Nevid & Gordon, 2018). The learner-instructor and learner-learner interactions mainly occur in synchronous sessions online and/or on-site (e.g., live lectures, break-out groups discussions). When designing synchronous sessions, instructors are expected to use the appropriate technology to make content lively and engaging (Martin & Bolliger, 2018) and improve students’ sense of connection (Yamagata-Lynch, 2014). Despite the rapid development of blended learning, there is limited evidence about its effectiveness in relation to learning outcomes. For example, Kintu et al. (2017) explored the relationship between learner characteristics (e.g., self-regulation), design features (e.g., technology quality) and learning, and demonstrated that no variable predicts learning performance in blended learning. Furthermore, when adopting a blended learning approach instructors encounter challenges such as deciding what activities should be completed synchronously and which asynchronously (Kenney & Newcombe, 2011). Knowledge of which learning activities predict academic achievement and whether the students complete them would be useful when designing and implementing blended learning.

There is literature about the powers and pitfalls of some learning activities used in online learning, but the evidence for their associated learning benefits is mixed and dependent on the metrics of engagement and whether the activity was incentivised or not. For example, numerous studies explore the impact of asynchronous quizzes on learning. Some studies show that students’ scores in practice tests positively correlate with final course grades (Van Camp & Baugh, 2014), whilst others show that using publisher-provided quizzes (Bell et al., 2015) and web-quizzes (Daniel & Broida, 2004) does not improve class performance. In addition, some studies indicate that online quizzes benefit performance only when they are incentivised with course credit (Beard, 2017; Nevid & Gordon, 2018; Sotola & Crede, 2021) and only for the high performing students who are already engaged (Nevid et al., 2020). The benefit of incentivised learning activities, especially via adding assessment weighting, on student engagement and performance has been well-documented (Beard, 2017; Chevalier et al., 2018; Nevid et al., 2021). However, other studies discuss the detriments of extrinsic rewards on student motivation (Deci et al., 2001) and reveal that incentives have no impact on effort and achievement (Gneezy et al., 2011). Moreover, incentivising all activities via assessment weighting would require substantial administrative support (Yamagata-Lynch, 2014). In addition, most studies focus on quiz performance and how it predicts final learning outcomes via retrieval practice (Roediger & Butler, 2011; Sotola & Crede, 2021), but there is no study focussing on quiz engagement per se and its impact on learning. Finally, other learning activities have gained less research attention. For example, digital textbook usage (e.g., reading time and highlighting) can predict course outcomes and can be an early predictor of student failure or success (Junco & Clem, 2015). However, there is need for more evidence about how metrics of engagement through learning analytics (e.g., time spent reading, logins, mouse click counts, video-watching frequency, page turning) can be used to support teaching and learning (Caspari-Sadeghi, 2022; Rets et al., 2021).

In addition to the mixed evidence mentioned above, when implementing online blended learning approaches, institutions, instructors, and students encounter practical challenges. The integration of online resources has a significant purchasing cost for institutions as reported in a recent increase in prices (Lowe et al., 2021). In addition, instructors must invest time to integrate them in their course, and students must rely on other relevant resources (e.g., good internet connection) to ensure access, raising challenges for the embedment of inclusivity when designing teaching and assessment. Furthermore, there is evidence for increased levels of stress in remote professional settings which rely on telecommunication, including education (Mheidly et al., 2020). Consequently, more research is needed to demonstrate the learning benefits when students engage with blended learning activities synchronously and asynchronously, and in turn, identify the best practices for encouraging student engagement with them.

The present study aimed to measure levels of student behavioural engagement with different learning activities (incentivised and non-incentivised) and explore its association with academic achievement in a real-world online blended learning approach. To this end, data were collected from a second-year Cognitive Psychology module taught in an online blended format at Queen Mary University of London in Fall 2020. We captured levels of behavioural engagement in terms of how many online quizzes and experimental tasks students completed, how many asynchronous videos they watched, how many times they logged in to the online integrated system (i.e., Cengage's MindTap), and how many online synchronous sessions they attended live. We measured academic achievement via students’ performance in the final exam. It was hypothesised that student behavioural engagement with the different activities is associated with final exam performance. The relative impact of each of the student engagement measurements was determined, via correlational and multiple regression analysis.

Method

Participants

The participants consisted of 123 second-year undergraduate Psychology students from Queen Mary University of London. All students were enrolled on the Cognitive Psychology module and there were no exclusion criteria. Ethical approval was obtained by the Queen Mary University of London Ethics Research Committee (QMERC20.058) and data collected were considered minimal risk and for normal educational practices. All students received information about the purposes of this study, and any student who did not want their data to be used could opt-out. None of them did. Anonymity was granted to each student by assigning them a random number (e.g., PT01).

Procedure, Module Description and Measures

The module was taught via a blended online learning format. It was designed online using Queen Mary's Virtual Learning Environment (i.e., Moodle) and integrated Cengage MindTap for Cognitive Psychology. Every week, the instructor outlined the weekly learning outcomes and all learning activities (asynchronous-synchronous) that students should complete to meet the learning outcomes. The activities were mapped on online tasks (see Figure 1). Students were asked to attend the live synchronous session online according to their timetable (duration of 2 h). Before the live session, students would watch asynchronous videos. The videos were either pre-recorded by the instructor or YouTube videos providing a brief introduction of the weekly topic (e.g., if the weekly content was on decision making, the asynchronous video would introduce students on how people make decisions). The average duration of the asynchronous videos was 11 min and would not exceed 20 min each. Most importantly, and in line with the discussion on attention span and interactivity while watching online video content (Bradbury, 2016; Lagerstrom et al., 2015), each asynchronous video would include a reflective question, which students were encouraged to explore and discuss during the live synchronous session. In addition, students would complete an online cognitive experiment using Cengage MindTap CogLab, which was relevant to the weekly topic (e.g., if the weekly content were on decision making, students would complete the CogLab on risky decisions). CogLab tasks were available for completion at any-time of the week, and they would be discussed during the live synchronous sessions. Moreover, reading associated with the weekly content was assigned and students could access the e-book via their individual Cengage MindTap account. Finally, students would complete online retrieval quizzes with multiple choice questions (MCQs), created by the instructor to assess knowledge of the weekly content. Quizzes would become available after the live synchronous session, students had one chance and a week to complete them and received feedback once the deadline elapsed. Participation in all tasks was regularly encouraged and highlighted as important to meet the intended learning outcomes and facilitate cognitive effort when completing interdependent activities (e.g., students would need to complete CogLab experiments to be able to actively participate in the synchronous discussions). Only the participation in the quizzes was incentivised via assessment weighting at low stakes (i.e., student performance on each weekly quiz contributed 0.56% to the overall module score).

Figure 1.

An example of the weekly learning activities for the module “Cognitive Psychology” as outlined on Queen Mary's Virtual Learning Environment.

Student behavioural engagement with the blended learning activities was captured throughout the semester. Measures included: (a) asynchronous videos view rate (i.e., number of weekly videos watched before the live session, per twelve in total), (b) live online synchronous session attendance rate (i.e., number of synchronous sessions attended on the scheduled time, per eight in total), (c) online experiment completion rate (i.e., number of weekly CogLab experimental tasks completed, per eight in total), (d) number of logins to online resources during term and revision time (i.e., how many times students logged in to the Cengage online platform to access the e-book and/or other resources), and (e) online quiz completion rate (i.e., how many weekly MCQs quizzes completed, per nine in total). For all behavioural engagement measures, high numbers indicated increased levels of student engagement with the online blended learning activities.

Academic achievement for this module was measured via student performance in the final exam (i.e., a grade from 0–100%). The final exam consisted of 50 MCQs which were completed online at the students’ preferred space and within a time window of three hours. All information was gathered from Learning Analytics Dashboards (i.e., Moodle; MindTap).

Design and Data Analysis

The current study used a cross-sectional design. The independent variables and predictors relative to student engagement with online blended learning activities were determined via correlational and multiple regression analysis, and was operationalised through six different measures: view rate of asynchronous videos watched before the live sessions, the live attendance of the synchronous sessions, completion rate for MindTap CogLab activities, number of logins to MindTap during term-time and revision period, and finally, completion rate for online MCQs quizzes. The dependent variable was the final exam grade. SPSS version 25 was used to analyse the data.

Results

Data from 123 participants was analysed using descriptive statistical tests, to extract frequency information regarding student engagement and their academic achievement. Mean scores for each of the engagement measures and the exam grade are presented in Table 1.

Table 1.

Descriptive statistics of student engagement and academic achievement.

Measure	N	M	SD
Final exam grade	118	77.780	9.32
Online experiment completion rate	123	.523	.353
Online quiz completion rate	123	.823	.245
Live lecture attendance rate	123	.608	.350
Asynchronous video view rate	123	.419	.353
Online content logins	118	34.890	24.5

Note. M and SD represent mean and standard deviation, respectively. Some students did not sit the final exam and did not use the online content via MindTap.

A multiple linear regression was carried out to investigate whether engagement with different online blended learning activities could significantly predict students’ final exam scores. Multiple regression assumptions were checked and met. One hundred and sixteen participants were deemed suitable for multiple regression analysis with up to five predictors (Green, 1991). There were four cases with a standardised residual greater than 2 but none of them greater than 3, as such minimum cases were excluded as outliers (Cook & Weisberg, 1982). Visual inspection of residuals scatterplots between the outcome variable and errors of predictors indicated that the assumptions of normality, linearity, and homoscedasticity were met. The Durbin–Watson test value was close to 2 (i.e., 2.18) and showed that the assumption of independent errors was tenable. Finally, all VIF (Variance Inflation Factor) values for each predictor were well below 10 (Field, 2009).

The regression model was significant F (5, 110) = 2.58, p = .030, and explained 10.5% of the variance in final exam scores (R²). The unique contribution of each predictor is shown in Table 2. Standardised regression coefficients indicated that completion rate of the weekly online quizzes was significantly positively related with the final exam grade and had the greatest influence than all student engagement measures. More specifically, by completing all nine online quizzes students could increase their final exam grade by 16 grade points.

Table 2.

Forced-entry multiple regression with measures of student engagement as predictors and final exam score as the outcome variable.

Variable	r	B	95% CI	t	p
Constant	-	64.99	[55.95–74.00]	14.29	<.001
Online experiment completion rate	.09	−6.09	[−13.21–1.04]	−1.69	.093
Online quiz completion rate	.26**	16.42	[4.78–28.06]	2.80	.006
Live lecture attendance rate	.12	−.82	[−7.36-5.72]	−.25	.249
Asynchronous video view rate	.20*	5.51	[−2.2–13.04]	1.45	.150
Online content logins	.10	.01	[−07–.09]	.11	.912

Note. *p < .05, **p < .001.

Discussion

We explored whether student behavioural engagement with online blended learning activities predicts academic performance, and if so, to what degree. Findings demonstrated that the completion rate of online quizzes was the only indicator of behavioural engagement that predicted performance on the final exam. To our knowledge, this study is unique for two reasons. It assesses the impact of quiz completion rates, rather than quiz performance, on academic achievement and measures behavioural engagement with multiple learning activities, thus suggesting a comparative benefit of completing online quizzes in a real-world online blended learning format.

The learning benefit of engaging with online quizzes is in line with and extends existing literature (e.g., Van Camp & Baugh, 2014) about the positive association between quizzing and final course performance. Differing from previous studies, we did not focus on quiz performance and its association with final academic achievement. Rather, we captured quiz completion rate as a metric of behavioural engagement, and we demonstrated the predictive power of quiz completion rate as opposed to an association. As such, this study overcame shortcomings in correlational studies and can eliminate the alternative explanation, which posited that better performing students tend to perform well in all assessments and make greater use of online learning resources. More specifically, we demonstrated that greater used of online quizzes predicts performance in the final exam, and by completing all weekly quizzes one could increase their final exam grade by 16 grade points and change grade classification (e.g., from 2.1 to a First).

Why completing online quizzes matter? In line with theoretical and empirical evidence for context-dependent memory (Godden & Baddeley, 1980; Grant et al., 1998), it is plausible that by completing many multiple practical quizzes as formative assessment helped students encode information in a context that was similar to the retrieval context of the final exam. Moreover, students completed the quizzes and the final exam in the exact same online format (e.g., MCQs presented in the same online virtual environment) thus facilitating students’ performance via increased familiarity with the environmental context. In addition, practical online quizzes in this study were spaced out evenly per week during the semester, therefore highlighting the importance of the spacing effect in learning (Cepeda et al., 2006) and suggesting that spacing out retrieval practice contributes to academic performance. Finally, students would complete quizzes for all assessed content throughout the semester and in turn increase their awareness of their understanding to guide their final revision efforts accordingly (Sotola & Crede, 2021).

One may argue that the effect of quizzes relies solely on their incentivised nature via increasing the likelihood that students will engage with them. Indeed, findings are in line with the literature about the benefits of incentivised learning (Chevalier et al., 2018; Nevid et al., 2021; Sotola & Crede, 2021) and extend it by showing benefits in an online blended learning approach and by focussing on completion rates of incentivised tasks. However, we do not think that the finding is solely due to quizzes carrying assessment weighting as an incentive. Firstly, completion of all activities was constantly encouraged by highlighting their significance to the learning outcomes. We believe that this encouragement acted as an educational nudge based on goal setting which could lead to positive outcomes (Damgaard & Nielsen, 2018). Indeed, based on supplementary analysis (see Supplemental material), the odds ratios of performing above average in the final exam after completing all as opposed to some asynchronous activities was above 1 for all metrics of engagement, further suggesting the association between participation in the activities and performance (Szumilas, 2010). Secondly, watching asynchronous videos did not carry assessment weighting, yet viewing rates positively correlated with final exam grades. Finally, we used the final exam grade rather than overall module grade, therefore the actual grade from the quizzes (i.e., the incentive per se) did not contribute to our measurement of academic achievement. As such, we think that the predictive power of the quiz completion rates on academic achievement is not contingent to them being incentivised only. However, future research should further compare the predictive power of quiz completion rates when they are incentivised and when not and consider adding appropriate low-stakes incentives to other blended learning activities.

Contrary to our predictions, we found no evidence that the other metrics of behavioural engagement with online learning activities predicted final academic performance. We can speculate why in relation to the actions taken and the learning strategies developed during those learning activities. For example, it is surprising that attendance rate in the live synchronous sessions did not significantly predict performance in the final exam. However, Büchele (2021) suggested that it is not a matter of whether students attend the lecture or not, but how they attend (e.g., actively by taking notes) that predicts academic achievement. Such monitoring was impossible in the current study due to online synchronous teaching, however future research should include more nuanced aspects of behavioural engagement during synchronous sessions (e.g., questions asked by the students, contribution to group-discussions, note taking) and assess their benefit on academic performance. Furthermore, this discussion also relates to the importance of the use of cameras during online sessions for encouraging active learning (Castelli & Sarvary, 2021), which, however, was not part of the rationale of this study. Finally, future research should compare the impact of the synchronous sessions on learning outcomes when teaching is blended online only and in a hybrid mode (e.g., students attending the synchronous session online and in-person).

Findings have implications for course design and add to the timely discussion about the use of learning analytics to inform course design and improve teaching efficiency, student satisfaction and performance (Brown et al., 2020; Caspari-Sadeghi, 2022; Foster & Siddle, 2020). They provide an insight into the design and provision of asynchronous online learning activities which can improve student performance at exams (e.g., at least nine weekly quizzes per semester, spaced out, carrying few credits, in similar formats with the final exam). In addition, it informs pedagogical practice in relation to using quiz engagement analytics as a pre-diagnostic tool for identifying students in need of further support. For example, generating a “no-engagement with quiz” alert could be employed as a time-efficient and student-focused strategy for enhancing student engagement, and in turn academic achievement.

Nevertheless, some caveats must be mentioned. Firstly, we do not know whether our results are solely relevant to academic performance measured via MCQs and to the specific course content. Perhaps behavioural engagement with online activities differentially predicts academic performance measured via essay writing and performance in other courses. More research assessing performance in distinct types of assessment and subjects is essential (e.g., whether engagement with synchronous sessions and discussions predicts performance in essay-based exams). In addition, even though the regression model was significant, there was variance in academic performance that remained unexplained. For example, future research should explore whether the relationship between students’ behavioural engagement and academic performance is moderated by other factors relevant to students’ cognitive styles, their familiarity with online learning resources and the perceived effectiveness and popularity of each online learning activity.

To conclude, the present study found that student behavioural engagement with online quizzes predicts student performance at a final exam in an online blended learning approach. With the significant rise in online and blended learning approaches, more research on the factors influencing the link between student engagement and academic achievement will help educators in the effective development and usage of online learning resources.

Supplemental Material

sj-docx-1-plj-10.1177_14757257221091512 - Supplemental material for What to Blend? Exploring the Relationship Between Student Engagement and Academic Achievement via a Blended Learning Approach

Supplemental material, sj-docx-1-plj-10.1177_14757257221091512 for What to Blend? Exploring the Relationship Between Student Engagement and Academic Achievement via a Blended Learning Approach by Paraskevi Argyriou, Kenza Benamar and Milena Nikolajeva in Psychology Learning & Teaching

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Paraskevi Argyriou

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Dr Paraskevi Argyriou is a Lecturer in Psychology at Queen Mary University of London. She received her PhD in Psychology at the University of Birmingham. Her research expertise is the psychology of language and communication (including non-verbal communication), and she conducts experimental studies with implications in educational and therapeutic settings (e.g., the role of gestures in online, asynchronous learning; the link between gestures and empathy). She currently teaches Emotion, Cognitive Psychology, Business Psychology and Essential Skills for Psychologists. As an educator, she is dedicated to teaching excellence and is particularly interested in enhancing student experience and employability. She is involved in a project evaluating the effectiveness of a school-based, peer-to-peer mentoring intervention which aims to enhance adolescents’ well-being and young adults’ transferable skills. She has disseminated the outcomes of her research projects at conferences and via publications (e.g., Journal of Experimental Psychology, Learning, Memory, and Cognition) and has presented her scholarship activities at HEA Conferences, with a focus on improving assessment and feedback, and diversifying the curriculum. Dr Argyriou is a member of the Experimental Psychology Society and a Fellow of the Higher Education Academy.

Miss Kenza Benamar is currently an undergraduate student in Psychology at Queen Mary University of London. She has sought out opportunities to gain research experience via assisting two projects. One project investigates the relationship between academic achievement and student engagement when psychology is taught online/in a blended format. Her interest stemmed from wanting to better understand how students can best be helped with their academic achievements, especially those struggling from earlier on. The second project explores the role of gesturing/non-verbal communication when academic content is delivered online and asynchronously. This project is still in progress but looks very promising. Although her interests vary, she plans to continue gaining as much experience while contributing to research within multiple fields in psychology.

Miss Milena Nikolajeva is a Queen Mary University of London Psychology alumna and currently a Research Assistant at the Youth Resilience Unit at Queen Mary University of London. During her undergraduate studies, she investigated the impact of COVID-19 on mental wellbeing dependent on sociodemographic status. Additionally, she assisted with research exploring emotion recognition in forced and voluntary migrants. Specifically, the project looked at how emotion recognition and associated cognitive biases are affected by exposure to trauma and change over time, and how these changes correspond with changes in well-being and mental health constructs. Presently, she is part of a team investigating the relationship between student engagement and academic achievement when teaching psychology in an online blended format. The team is also exploring the role of non-verbal communication (particularly the importance of gestures) when delivering learning content asynchronously in an online setting. Finally, she is currently assisting with a systematic literature review to collate the current evidence base on trauma-informed approaches in psychologically distressed adult populations. While she strives to solidify her research interests, she aims to obtain a variety of research experience in different settings to reach her aspirations of pursuing a research career.

References

Abi-Raad

M. E.

Odhabi

(2021). Hybrid learning here to stay!. Frontiers in Education Technology, 4(2), 121–131. https://doi.org/10.22158/fet.v4n2p121

Barber

Bird

Fleming

(2021). Gravity assist: Propelling higher education towards a brighter future: Report of the digital teaching and learning review . https://www.officeforstudents.org.uk/publications/gravity-assist-propelling-higher-education-towards-a-brighter-future/

Beard

L. H.

(2017). ‘Incentivised reading’-using an online VLE to measure engagement and attainment in student learning. International Journal for Innovation Education and Research, 5(11), 74–86. https://doi.org/10.31686/ijier.vol5.iss11.854

Bell

M. C.

Simone

P. M.

Whitfield

L. C.

(2015). Failure of online quizzing to improve performance in introductory psychology courses. Scholarship of Teaching and Learning in Psychology, 1, 163–171. https://doi.org/10.1037/stl0000020

Bradbury

N. A.

(2016). Attention span during lectures: 8 s, 10 min, or more? Advances in Physiology Education, 40(4), 509–513. https://doi.org/10.1152/advan.00109.2016

Brown

Lawrence

Basson

Redmond

(2020). A conceptual framework to enhance student online learning and engagement in higher education. Higher Education Research & Development, 1–16. https://doi.org/10.1080/07294360.2020.1860912

Büchele

(2021). Evaluating the link between attendance and performance in higher education: The role of classroom engagement dimensions. Assessment & Evaluation in Higher Education, 46(1), 132–150. https://doi.org/10.1080/02602938.2020.1754330

Caspari-Sadeghi

(2022). Applying learning analytics in online environments: Measuring Learners’ engagement unobtrusively. Frontiers in Education, 7, 1. https://doi.org/10.3389/feduc.2022.840947

Castelli

F. R.

Sarvary

M. A.

(2021). Why students do not turn on their video cameras during online classes and an equitable and inclusive plan to encourage them to do so. Ecology and Evolution, 11(8), 3565–3576. https://doi.org/10.1002/ece3.7123

10.

Cepeda

N. J.

Pashler

Vul

Wixted

J. T.

Rohrer

(2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132(3), 354–380. https://doi.org/10.1037/0033-2909.132.3.354

11.

Chen

Yang

Bear

Zhen

(2013, February). School engagement as mediator between school climate and achievement [Paper presentation]. The annual convention of the National Association of School Psychologists, Seattle, WA, USA.

12.

Chevalier

Dolton

Lührmann

(2018). ‘Making it count’: Incentives, student effort and performance. Journal of the Royal Statistical Society: Series A (Statistics in Society, 181(2), 323–349. https://doi.org/10.1111/rssa.12278

13.

Cook

R. D.

Weisberg

(1982). Residuals and influence in regression. Chapman and Hall.

14.

Damgaard

M. T.

Nielsen

H. S.

(2018). Nudging in education. Economics of Education Review, 64, 313–342. https://doi.org/10.1016/j.econedurev.2018.03.008

15.

Daniel

D. B.

Broida

(2004). Using web-based quizzing to improve exam performance: Lessons learned. Teaching of Psychology, 31(3), 207–208. https://doi.org/10.1207/s15328023top3103_6

16.

Deci

E. L.

Koestner

Ryan

R. M.

(2001). Extrinsic rewards and intrinsic motivation in education: Reconsidered once again. Review of Educational Research, 71(1), 1–27. https://doi.org/10.3102/00346543071001001

17.

Field

(2009). Discovering statistics using SPSS (3rd ed.). Sage Publications.

18.

Foster

Siddle

(2020). The effectiveness of learning analytics for identifying at-risk students in higher education. Assessment & Evaluation in Higher Education, 45(6), 842–854. https://doi.org/10.1080/02602938.2019.1682118

19.

Furrer

Skinner

(2003). Sense of relatedness as a factor in children’s academic engagement and performance. Journal of Educational Psychology, 95(1), 148–162. https://doi.org/10.1037/0022-0663.95.1.148

20.

Gneezy

Meier

Rey-Biel

(2011). When and why incentives (don’t) work to modify behavior. Journal of Economic Perspectives, 25(4), 191–210. https://doi.org/10.1257/jep.25.4.191

21.

Godden

Baddeley

(1980). When does context influence recognition memory? British Journal of Psychology, 71(1), 99–104. https://doi.org/10.1111/j.2044-8295.1980.tb02735.x

22.

Grant

H. M.

Bredahl

L. C.

Clay

Ferrie

Groves

J. E.

McDorman

T. A.

Dark

V. J.

(1998). Context-dependent memory for meaningful material: Information for students. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, 12(6), 617–623. https://doi.org/10.1002/(SICI)1099-0720(1998120)12:6<617::AID-ACP542>3.0.CO;2-5.

23.

Green

S. B.

(1991). How many subjects does it take to do a regression analysis. Multivariate Behavioral Research, 26(3), 499–510. https://doi.org/10.1207/s15327906mbr2603_7

24.

Junco

Clem

(2015). Predicting course outcomes with digital textbook usage data. The Internet and Higher Education, 27, 54–63. https://doi.org/10.1016/j.iheduc.2015.06.001

25.

Kenney

Newcombe

(2011). Adopting a blended learning approach: Challenges encountered and lessons learned in an action research study. Journal of Asynchronous Learning Networks, 15(1), 45–57. http://doi.org/10.24059/olj.v15i1.182

26.

Kintu

M. J.

Zhu

Kagambe

(2017). Blended learning effectiveness: The relationship between student characteristics, design features and outcomes. International Journal of Educational Technology in Higher Education, 14(1), 1–20. https://doi.org/10.1186/s41239-017-0043-4

27.

Koksal

(2020). The rise of online learning. Forbes . Retrieved March 10, 2022, from https://images.bizbuysell.com/shared/listings/179/1799854/2b054e56-d93d-42d2-9aa8-24d1c6d38ab9.pdf

28.

Lagerstrom

Johanes

Ponsukcharoen

M. U.

(2015, June 14–17). The myth of the six-minute rule: Student engagement with online videos [Paper presentation]. The American Society for Engineering Education, Seattle, WA. Retrieved Mach 10, 2022, from https://www.asee.org/public/conferences/56/papers/13527/download

29.

Lei

Cui

Zhou

(2018). Relationships between student engagement and academic achievement: A meta-analysis. Social Behavior and Personality: An International Journal, 46, 517–528. 10.2224/sbp.7054

30.

Lowe

R. A.

Chirombo

Coogan

J. F.

Dodd

Hutchinson

Nagata

(2021). Electronic resources management in the time of COVID-19: Challenges and opportunities experienced by six academic libraries. Journal of Electronic Resources Librarianship, 33(3), 215–223. https://doi.org/10.1080/1941126X.2021.1949162

31.

Martin

Bolliger

D. U.

(2018). Engagement matters: Student perceptions on the importance of engagement strategies in the online learning environment. Online Learning, 22(1), 205–222. https://doi.org/10.24059/olj.v22i1.1092

32.

Mheidly

Fares

M. Y.

Fares

(2020). Coping with stress and burnout associated with telecommunication and online learning. Frontiers in Public Health, 672. https://doi.org/10.3389/fpubh.2020.574969

33.

Moore

(1989). Three types of interaction. American Journal of Distance Education, 3, 1–7. https://doi.org/10.1080/08923648909526659

34.

Nevid

J. S.

Gordon

A. J.

(2018). Integrated learning systems: Is there a learning benefit? Teaching of Psychology, 45(4), 340–345. https://doi.org/10.1177/0098628318796920

35.

Nevid

J. S.

Gordon

A. J.

Terjesen

M. D.

Hicks

(2020). Classroom evaluation of online quizzing and concept building exercises embedded in an integrated learning system. Psychology Learning & Teaching, 19(2), 184–193. https://doi.org/10.1177/1475725719861868

36.

Nevid

J. S.

Keating

L. H.

Lieblich

. (2021). Effects of in-class engagement activities in online synchronous classes. Scholarship of Teaching and Learning in Psychology. Advance online publication. http://dx.https://doi.org/10.1037/stl0000312

37.

Newmann

F. M.

Wehlage

G. G.

Lamborn

S. D.

(1992). The significance and sources of student engagement. In Newmann

(Ed.), Student engagement and achievement in American secondary schools (pp. 11–39). Teachers College Press.

38.

Raes

Detienne

Windey

Depaepe

(2020). A systematic literature review on synchronous hybrid learning: Gaps identified. Learning Environments Research, 23(3), 269–290. https://doi.org/10.1007/s10984-019-09303-z

39.

Rets

Herodotou

Bayer

Hlosta

Rienties

(2021). Exploring critical factors of the perceived usefulness of a learning analytics dashboard for distance university students. International Journal of Educational Technology in Higher Education, 18(1), 1–23. https://doi.org/10.1186/s41239-021-00284-9

40.

Roediger

H. L.

III Butler

A. C.

(2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15(1), 20–27. https://doi.org/10.1016/j.tics.2010.09.003

41.

Shernoff

D. J.

Schmidt

J. A.

(2008). Further evidence of an engagement-achievement paradox among U.S. high school students. Journal of Youth and Adolescence, 37, 564–580. https://doi.org/10.1007/s10964-007-9241-z

42.

Sotola

L. K.

Crede

(2021). Regarding class quizzes: A meta-analytic synthesis of studies on the relationship between frequent low-stakes testing and class performance. Educational Psychology Review, 33(2), 407–426. https://doi.org/10.1007/s10648-020-09563-9

43.

Szumilas

(2010). Explaining odds ratios. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 19(3), 227–229.

44.

Van Camp

Baugh

(2014). You can lead a horse to water: Efficacy of and students’ perceptions of an online textbook support site. Teaching of Psychology, 41, 228–232. https://doi.org/10.1177/0098628314537987

45.

Yamagata-Lynch

L. C.

(2014). Blending online asynchronous and synchronous learning. International Review of Research in Open and Distributed Learning, 15(2), 189–212. https://doi.org/10.19173/irrodl.v15i2.1778

46.

Yeigh

Lynch

. (2020). Is online teaching and learning here to stay? Academia Letters, Article 24. https://doi.org/10.20935/AL24.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB