Sage Journals: Discover world-class research

Abstract

Lack of student engagement in online learning is reported as the major challenge contributing to poor academic performance and completion rates. When transforming an in-person undergraduate remote sensing course to online, this study implemented interactive storytelling lecture trailers (ISLTs) as a tool to effect changes in the realms of behavioral, cognitive, emotional, and student-instructor engagement. We collected survey data to examine students’ own perception of how ISLTs impacted their online learning, and analyzed students’ course participation and performance on tests. Results indicated that ISLTs enhanced some aspects of students’ behavioral engagement such as page views, effectively engaged students’ emotions when viewing ISLTs, and improved student-instructor engagement. Regarding cognitive engagement, ISLTs were able to improve short-term learning skills like remembering and applying levels of thinking. A majority of students recognized that ISLTs enhanced their learning experience and made learning more accessible, while a few considered them burdensome and overwhelming. However, there was no clear evidence indicating that ISLTs enhanced participation or promoted students’ emotional engagement in the follow-up lectures. Further, the improvement of student-instructor engagement we observed through quantitative data analysis lacked representative qualitative support. In summary, this study demonstrates the utility of ISLTs as an online learning engagement tool for stimulating students’ interest and improving their performance in lower levels of cognitive thinking. Further work is required to explore ways to further enhance students’ participation and emotional engagement throughout the semester and confirm the usefulness of ISLTs for student-instructor engagement.

Keywords

higher education online learning storytelling trailer student engagement

Introduction

Higher education has increasingly offered online courses (Wei and Chou, 2020), and the number of students enrolled in online courses is increasing faster than those taking in-person courses (Stone, 2019). Many studies have reported the advantages of online learning, including improving accessibility for students who cannot attend in-person classes (Bawa, 2016) or saving hours of daily commuting time (Khalil et al., 2020); however, challenges exist. For example, online students are found to be more likely to withdraw from their studies than face-to-face students (Greenland and Moore, 2014; Xu and Jaggars, 2011). Low engagement and poor self-regulation in an online learning environment are key factors contributing to students’ higher attrition (Soffer and Cohen, 2019).

Various pedagogical practices, such as online discussions (Clinton and Kelly, 2020; Williams and Lahman, 2011) and collaborative online learning (Curtis and Lawson, 2001; Macdonald, 2003; Zheng et al., 2015), have been implemented to engage students, aiming at improving their online learning experience and academic performance (Brown et al., 2022; Chen et al., 2010; Redmond et al., 2018). Recently, interactive digital storytelling videos (Shelton et al., 2016) and Hollywood-style movie trailers (Goudsouzian, 2018) have emerged as innovative practices for motivating students’ learning. In specific, Shelton et al. (2016) created short video vignettes that incorporate imagery, data, narrative stories, and embedded interactive quiz questions into the videos to enhance student engagement. Digital storytelling has been reported to be effective in provoking students’ multiple senses that lead to increased levels of engagement and a better understanding of academic content (Lisenbee and Ford, 2018; Ryan and Aasetre, 2020). Alternatively, Goudsouzian (2018) made Hollywood-style movie trailers, which were 60–90 seconds, comprised of introductory materials, and these movie trailers were found to significantly increase student interest (i.e. emotional engagement). Such entertainment techniques have been proven to motivate students’ learning and positively impact their academic performance (Choi, 2018).

While the storytelling videos hold great potential to improve student engagement, a comprehensive evaluation of these methods has not been performed from the perspectives of engagement framework—social, cognitive, behavioral, collaborative, and emotional engagement (Redmond et al., 2018). Using this engagement framework, we examined our pedagogical intervention of packaging the storytelling and entertainment techniques into producing interactive storytelling lecture trailers (ISLTs). According to the engagement framework, social engagement means establishing purposeful relationships with others, creating a sense of belonging, developing relationships, and establishing trust (Redmond et al., 2018). Cognitive engagement, also referred to as intellectual engagement (Pittaway and Moss, 2014), enables students to “comprehend complex ideas and master difficult skills” (Fredricks et al., 2004). The indicators of cognitive engagement include thinking critically, activating metacognition, integrating ideas, justifying decisions, developing deep discipline understandings, and distributing expertise (Redmond et al., 2018). Behavioral engagement includes active participation in the learning tasks and upholding online learning norms such as adhering to rules, asking questions, and contributing to discussions (Fredricks et al., 2004; Pittaway and Moss, 2014; Redmond et al., 2018; Young, 2010). Collaborative engagement includes collaborations with peers, instructors, industry, and the educational institution (Redmond et al., 2018). Emotional engagement relates to students’ feelings toward peers or teachers and attitudes toward learning (Redmond et al., 2018).

Following the engagement framework (Redmond et al., 2018), we examined the effectiveness of ISLTs in student engagement and learning by addressing the following research questions: (1) To what extent do ISLTs describe behavioral engagement (i.e. active participation in the learning tasks and upholding online learning norms such as accessing learning materials and contributing to discussions) in online teaching? (2) How do ISLTs describe cognitive engagement at the remembering, applying, and creating levels? (3) In what ways do ISLTs describe emotional engagement? (4) Do ISLTs describe student-instructor engagement? Although student-instructor engagement is not a major category in Redmond et al.’s (2018) engagement framework, it can potentially fit into social or emotional engagement categories by making students feel less isolated and positively enhancing their academic achievements (Arbaugh and Benbunan-Finch, 2006; Brinkerhoff and Koroghlanian, 2007; Griffiths and Graham, 2009).

Method

This study was conducted in the context of moving an in-peron Environmental Remote Sensing course to online due to COVID-19 pandemic. As outlined above, we used ISLTs as a tool for engaging students in the course. A total of 130 students enrolled in the course. The course was offered for 12 weeks, and in each week, the course instructor taught a 2-hour synchronous lecture through Zoom, followed by a 1-hour lab tutorial. All course materials, including syllabus, lecture slides, ISLTs, and assignments, were uploaded to a learning management system (LMS) that was only available to the students in the class. A discussion board was enabled in the LMS for the course instructor and students to start contributing to course-related topics. The following sections describe how the ISLTs were made and implemented, and how data were collected and analyzed.

Making interactive storytelling lecture trailers (ISLTs)

The course instructor made a series of 2-minute videos as the storytelling lecture trailers (Supplemental Information), each of which revealed the corresponding lecture content in a story, thus serving as a preview for course content. The course instructor was one of the main characters in the story, complementing the teaching presence later in the class. The instructor first drafted a storyboard (available in Supplemental Information); laying out the characters, visuals, and audio or script for each scene, before shooting and editing the videos. The instructor’s actions were then shot by a camera in the field, or with a planned background or green screen. The other character “camera” was held by a Lego character for stop motion shots. The trailers were edited, which included filtering, transitions, sound effects, and magnification, using Premiere Pro CC 2020 (Adobe). We embedded questions in the trailers to encourage active learning and solicit students’ feedback; however, these data were not used in the analysis as a result of low response rate. A summary of student responses is provided in Supplemental Information.

Implementing interactive storytelling lecture trailers (ISLTs)

The trailers were implemented in 3 out of 12 weeks. In specific, the lectures in 9 weeks (from week 2 to week 11; excluding week 10, which was a guest lecture) were clustered into three modules, with each lasting for 3 weeks (Table 1). The pre-made ISLTs were only provided in one of three lectures in each module. To enable comparison, a week without ISLTs was selected in each module that also fulfilled the criteria of having no major assignments or exams scheduled for that week. Each trailer was posted at the beginning of that week, so that students could optionally watch it prior to the synchronous lecture. The instructor also played the trailer at the beginning of the synchronous lecture.

Table 1.

Course lecture schedules to show lecture content and when the quizzes and ISLTs were implemented.

Weeks	Surveys and quizzes	ISLTs implementation
Week 1: Introduction	Pre-survey
Week 2: Regular lecture	Module 1: Remembering-related questions	ISLTs used
Week 3: Regular lecture	Module 1: Remembering-related questions	No ISLTs
Week 4: Student presentations	Module 1: No quiz
Week 5: Regular lecture	Module 2: Applying-related questions	No ISLTs
Week 6: Regular lecture	Module 2: Applying-related questions	ISLTs used
Week 7: Student presentations	Module 2: No quiz
Week 8: Regular lecture	Module 3: Creating-related questions	ISLTs used
Week 9: Regular lecture	Module 3: Creating-related questions	No ISLTs
Week 10: Guest lecture	No quiz
Week 11: Student presentations	Module 3: No quiz
Week 12: Wrap up	Post-survey

Data collection and analysis

The students’ behavioral engagement was analyzed by comparing observation data collected from weeks with ISLTs to data from weeks without ISLTs (Table 1). Students’ cognitive engagement was assessed using both observation and survey data. Emotional engagement of students was evaluated through surveys and semi-structured interviews. Student-instructor interaction was examined utilizing post-surveys and semi-structured interviews. Detailed data collection and analysis steps are described in the following subsections.

Observation data

Students’ participation in quizzes, assignments, the discussion board, page views of trailers, and other course material in the LMS were automatically documented. Student participants in the lectures were documented by Zoom through polls. We used these data as proxy measures for student behavioral engagement. These data were collected from all participants (n = 130) in the 3 weeks with ISLTs and another 3 weeks without ISLTs.

Students’ participation during the synchronous time was observed by their attendance at the Zoom lectures and their responses to the Zoom polls. These data were collected from participants (n = 128) who chose synchronous learning mode. Two students out of 130, who opted for asynchronous learning due to timezone issues, were not included.

We designed the weekly lecture quizzes and analyzed the impact of ISLTs on student’s academic performance by comparing the scores of quizzes in weeks with ISLTs to the scores in weeks without ISLTs. The weekly lecture quizzes in module one involved remembering questions, in module two involved applying questions, and in module three involved creating questions that students were asked to create graphs (Table 1). This design of quizzes was adopted from Bloom’s taxonomy of cognitive levels, discussed by Forehand (2010). The weekly quizzes were made with consistent structure and difficulty levels within each three-week module. Each quiz included a mixture of multiple-choice and short-answer questions, taking students roughly 10-minute to complete. The quizzes were administered during the synchronous lectures after the lecture content was introduced. We considered all students in the class participants, but if a student missed a quiz, the scores for that student were not considered in the analysis. As a result, we collected a complete set of 9-week scores from a total of 84 students.

Survey data

Two web-based surveys were conducted: a pre-survey in week 1 and a post-survey in week 12. The surveys were adapted from Shelton et al. (2016) and modified according to the online engagement framework (Redmond et al., 2018) by a research assistant, supervised by the course instructor and following the principle of question construction (Creswell, 2012). Besides demographic questions, the pre-survey (Supplemental Table S1) included five Likert items exploring students’ perceptions about the online courses and the trailer. A 5-point Likert Scale includes “1” very negative (e.g. very unenjoyable), “2” negative (e.g. unenjoyable), “3” neural (e.g. neutral), “4” positive (e.g. enjoyable), to “5” very positive (e.g. very enjoyable). To further explore students’ attitude toward online learning, the pre-survey included four open-ended questions (Supplemental Table S1). The post-survey (Supplemental Table S2) included demographic questions, nine Likert items that explored students’ perceptions regarding the contribution of ISLTs to their learning experience in lectures and the course, and three open-ended questions regarding the advantages and disadvantages of using ISLTs and opinions on how to improve the use of ISLTs in enhancing student learning and motivation.

Interview data

Four students voluntarily signed up for interviews. A research assistant designed the interview questions under the supervision of the course instructor. Another research assistant conducted the interviews to minimize the anxiety that students may have felt when discussing their experience with the course instructor. Interviews were conducted via Zoom. An interview guide and script was used by the research assistant, who asked questions in a semi-structured questionnaire format. This style creates flexibility while maintaining a core set of questions standardizing the interviews without extinguishing interesting tangents to be explored. The use of semi-structured interviews, which allows for probing and clarifying word choice, acknowledges that not every interviewee uses the same vocabulary, and individual words themselves have different meanings to different individuals (Denzin, 2017). Identifiable information from the interviews was edited out to maintain interviewee anonymity.

Data analysis

A paired t-test was conducted to investigate if the difference in the mean score of quizzes was significant in weeks with ISLTs and weeks without ISLTs. The median scores of students’ perception of the emotional and cognitive engagement in the post-survey were also compared between lectures with and without ISLTs, and a Wilcoxon Signed Rank test was conducted to investigate the statistical significance of these paired samples. The median scores of students’ perception of the emotional, cognitive, and student-instructor engagement in other online courses reported in the pre-survey were compared with this course in the post-survey using Mann-Whitney U-test for non-paired samples. Nonparametric statistical tests were used for Likert scale results since they should be considered ordinal rather than interval scale (Creswell, 2012). Linear and polynomial regressions were carried out to investigate the relationship between students’ grades and their behavioral engagement.

Interview data and student responses to open-ended questions in surveys were open coded using NVivo, drawing on a constructivist design of a grounded theory approach (Charmaz, 2014; Creswell, 2012). The first round of open coding identified recurring themes and subthemes, followed by a discussion that led to a consensus of coding nodes. A finalized set of themes and subthemes were used for all participants’ responses. A single participant’s response could be coded for multiple themes.

Results

Participants’ demographic information

A total of 73 participants (56% response rate) submitted the anonymous pre-survey. Two of the participants who submitted the survey left most of the questions unanswered, and therefore, were omitted from the analysis. A total of 44 participants (34% response rate) submitted post-surveys. Four of the responses left most questions unanswered and were omitted from analysis. Participants’ demographic information and their prior experience with online courses and trailers are reported in Supplemental Table S3.

Students’ attitudes toward online learning

The top four rated advantages of online learning in the pre-survey (Supplemental Table S4) were (1) flexible schedule, location, routine, and pace (53%); (2) saving commuting time and money (49%); (3) reviewing recorded materials (28%); (4) equity, diversity, and inclusion (26%). The top four rated disadvantages (Supplemental Table S5) were (1) less engaging (51%); (2) distraction (27%); (3) technical issues (15%); (4) lack of motivation (14%). Overall, students listed more advantages than disadvantages, which explains a high score (Mdn = 4) regarding how comfortable students were with online learning (Supplemental Table S3). Only two students held extremely negative attitudes toward online learning, stating that there is no advantage. Students suggested making online courses more engaging (Supplemental Table S7) by (1) adopting engaging tools/activities/questions (36%); (2) increasing peer interactions such as group work (16%); (3) creating engaging content (14%); (4) adjusting assessment design (10%).

Behavioral engagement

Students’ behavioral engagement in the asynchronous time was observed through students’ page views and participation in LMS. The total page views in the weeks with ISLTs were significantly higher than for the weeks without ISLTs in all three modules (χ²(1, N = 130) = 436.86, p < 0.001 for module one, χ²(1, N = 130) = 20.01, p < 0.001 for module two, and χ²(1, N = 130) = 402.42, p < 0.001 for module three). Nevertheless, the total student participation in LMS was only significantly higher in the week with ISLTs than in the week without (χ²(1, N = 130) = 67.04, p < 0.001) in module one. In summary, students showed a consistent higher page views in the weeks with ISLTs than those without but not for participation, during their asynchronous time.

Students’ behavioral engagement in the synchronous time was observed by their participation in the Zoom lectures and the Zoom polls. Attendance was similar for weeks with ISLTs (M = 117.7 (92%)) and those without ISLTs (M = 118.0 (92%)). Of the students who attended the synchronous lectures, about 86% of students actively participated (i.e. responded to poll questions), whether in weeks with or without ISLTs. No difference was observed in behavioral engagement in the synchronous time between the weeks with and without ISLTs.

Students’ participation in LMS significantly predicted their grades (Supplemental Figure S1) with an R² of 0.59. The students’ page view significantly predicted their grades as well (Supplemental Figure S2) with an R² of 0.66.

Cognitive engagement

The mean quiz scores (Table 2) for lectures with ISLTs (M = 9.25, SD = 0.78) were significantly higher than those for lectures without ISLTs (M = 8.33, SD = 1.05) in module one with remembering type questions, t(84) = 6.73, p < 0.001, and in module two with applying questions (with ISLTs: M = 8.82, SD = 0.85, and without ISLTs: M = 8.02, SD = 1.27, respectively), t(84) = 5.97, p < 0.001. The effect size for the difference in mean quiz score for remembering type questions is large (d = 0.999), and for applying type questions is medium (d = 0.721). There was no significant difference between the quiz scores in module three with the creating type questions. The results suggest that the use of ISLTs enhanced students’ cognitive performance in remembering and applying type questions, but not the creating type questions.

Table 2.

Number of participants, median, mean, and standard deviation of participants’ average quiz score.

	With ISLTs			Without ISLTs
	N *	Median	M (SD)	N *	Median	M (SD)
Module 2: Recall	84	9.4	9.25 (0.78)	84	8.4	8.33 (1.05)
Module 3: Apply	84	9	8.82 (0.85)	84	8	8.02 (1.27)
Module 4: Create	84	7.5	7.49 (1.05)	84	7.5	7.35 (1.23)

A total of 130 participants’ scores were collected. Participants whose score was 0 for any lecture quiz were deleted from all entries. The full mark for all quizzes is 10.

Students’ perception of cognitive engagement with the lectures was collected in four closed questions in the pre- and post-survey. According to median ranks (Table 3), students perceived that the lectures with ISLTs were more intellectually engaging (Mdn = 3.5) than those without ISLTs (Mdn = 3) or other online courses in general (Mdn = 3). However, such difference is not statistically significant according to Wilcoxon Signed Rank tests (T(40) = 85, p > 0.05, r = 0.215) or Mann-Whitney U-test (U(n₁ = 71, n₂ = 40) = 1174, p > 0.05, r = 0.153). There was no significant difference in students’ perceived challenges in lectures with (Mdn = 3) and without ISLTs (Mdn = 3). In other words, ISLTs neither significantly increased intellectual engagement in lectures, nor altered the challenges for students to learn the lecture content. Further, students reported no differences in their confidence in achieving desired marks in other online courses (Mdn = 3) and this course (Mdn = 3). However, their self-assessed ability to learn course material enhanced by the trailers was higher in other online courses (N = 28, Mdn = 4) than in this course (N = 40, Mdn = 3.5), but the difference was not statistically significant (U(n₁ = 28, n₂ = 40) = 602, p > 0.05, r = 0.066). In other words, no significant difference was observed in cognitive engagement between the lectures with and without ISLTs or between other online courses and this course, according to students’ perception.

Table 3.

Frequency, median, mean, and standard deviation of participants ratings from survey responses for the paired questions.

	Online course in general			This course without ISLTs			This course with ISLTs
	N	Median	M (SD)	N	Median	M (SD)	N	Median	M (SD)
Emotional engagement
Enjoyable	71	3	3.41 (1.05)	40	3	3.35 (0.86)	40	3.5	3.53 (0.96)
Cognitive engagement
Intellectually engaging	71	3	3.16 (0.86)	40	3	3.18 (0.93)	40	3.5	3.48 (0.93)
Challenging	NA			40	3	40	3	3.10 (0.74)	40
				This course in general
				N		Median		M (SD)
Confidence in achieving desired marks	71	3	3.28 (1.02)	40		3		3.15 (0.95)
Ability to learn course material	27	4	3.57 (1.07)	40		3.5		3.45 (1.06)
Student-Instructor Engagement
Instructor-student engagement	71	4	3.58 (0.92)	40		4.5		4.2 (0.97)

In the post-survey, the open-ended question on advantages of using ISLTs (Table 4) shows that cognitive engagement was the top ranked theme (n = 20, 71%), including subthemes of bridging the content (n = 14, 50%) and accessible content (n = 9, 32%). The responses, categorized into the subtheme of bridging the content, indicated that trailers provided a summary or preview of concepts for lectures and helped contextualize terms by connecting them to real-world applications. The high frequency of bridging the content explained why quiz scores were higher in lectures with ISLTs than without ISLTs for remembering and applying questions. The responses, classified into the subtheme of accessible content, pointed out that since trailers were short and focused, it was easy for students to find time and engage with the ISLTs.

Table 4.

Students’ responses to the open-ended question “Reflecting on your experiences in the GGR337H5F virtual classroom, what do you think are some of the advantages of using interactive storytelling lecture trailers in online courses?”

Theme	Frequency*	%	Definition
Cognitive engagement	20	71	Responses indicated that the trailers bridged the content and helped them to familiarize with terms and conceptualize concepts.Responses indicated that the trailers made the lecture content more accessible since they were short and focused, making it easier to comprehend.
Bridging content (summarize, preview, contextualize)	14	50
Accessible content (focused, easy, remember more)	9	32
Emotional engagement	16	57	Responses indicated that the student find the trailer interesting and fun to watch.Responses indicated that they paid more attention and were motivated to learn.
Interesting, fun, engaging, enjoyable, exciting	15	54
Facilitated attention, enhanced motivation	2	7
Engagement with instructor	1	4	Response indicated that it made the instructor approachable.

A total of 34 students provided responses to this question, but six entries were not valid. One response may be coded for more than one theme. Percentages were calculated based on n = 28 respondents.

In the post-survey, the open-ended question on students’ perceived disadvantages of using ISLTs (Table 5) shows that besides the highest frequency theme “no perceived disadvantages” (n = 15, 58%), cognitive engagement was ranked the highest (n = 8, 31%) among other themes. Responses commented on the unnecessary workload, as it was not associated with grades or did not help to learn (n = 6, 23%), and the creation of challenging or confusing content (n = 2, 8%).

Table 5.

Students’ responses to the open-ended question “Reflecting on your experiences in the GGR337H5F virtual classroom, what do you think are some of the disadvantages of using interactive storytelling lecture trailers in online courses?”

Theme	Frequency*	%	Definition
No perceived disadvantages	15	58	Responses indicated no perceived learning disadvantages or used this section to indicate an advantage of the trailer.
Cognitive engagement	8	31	Responses indicated the trailers were extra or unnecessary workload as it was not associated with grades or did not help with learning.Responses indicated that some students may be confused that the trailers were all they need to study and ignored the lectures or that the content in the trailers were challenging.
Unnecessary workload	6	23
Confusion or challenging content	2	8
Wasted resources	6	23	Responses indicated that the creation of trailers was extra work for instructor and the wasted time would otherwise be used in developing other course resources.
Quality issues	5	19	Responses indicated quality issues such as the sound/music quality, the trailer did not summarize enough of the lecture content, or was not mature for the audience.
Sound/music quality	2	8
Shifted content	2	8
Not mature for the audience	1	4
Technical challenges	2	8	Responses indicated the instability of the internet and platform used for the trailers.

A total of 33 students provided responses to this question, but seven entries were not valid. One response may be coded for more than one theme. Percentages were calculated based on n = 26 respondents.

Emotional engagement

Students’ perception of emotional engagement with the lectures was surveyed (Table 3). According to median ranks, students perceived lectures with ISLTs to be more enjoyable (Mdn = 3.5) than lectures without ISLTs (Mdn = 3) or other online courses (Mdn = 3). However, this difference was not statistically significant according to Wilcoxon Signed Rank tests (T(40) = 96, p > 0.05, r = 0.109) or Mann-Whitney U-test (U(n₁ = 71, n₂ = 40) = 1333, p > 0.05, r = 0.054), respectively.

In the open-ended questions that surveyed students’ perceived advantages of using ISLTs, the second highest ranked theme was emotional engagement (n = 16, 57%). Among them, 54% of the responses mentioned that the trailer was interesting or fun to watch, and 7% mentioned that the trailers facilitated their attention and enhanced their motivation to learn. There was no indication of disadvantages of emotional engagement mentioned in open-ended questions.

Interviewees specifically noted that the trailers were fun and engaging to watch as opposed to traditional readings or lectures. Interviewee #3 discussed this: “it can be kind of videos with the creative spin on it like she showed. They can be fun too, not just factual.” Interviewee #1 explicitly stated how the trailers emotionally encouraged students: “I’m just gonna say that they were very, very nicely well done. I think if they were very bland no one would watch them.” The emotional attachment to the trailer motivated students to devote more effort to the course-related activities. Interviewee # 4 expressed that “it gets you excited for the courses. . . bit more motivated to attend class,” “What the videos did, for me, was deﬁnitely I felt more invited into the course,” and “love that because I think just seeing course instructors put in an effort to try new things itself is very engaging and it’s very motivational for the students to also put in that effort to be the ones that try this stuff out with them.”

Student-instructor engagement

Students reported (Table 3) that the learner-instructor engagement in this course (Mdn = 4.5) is significantly higher than in other online courses (Mdn = 4), according to Mann-Whitney U-test (U(n₁ = 71, n₂ = 40) = 894, p < 0.001, r = 0.321).

In the open-ended question that surveyed students’ perceived advantages of using ISLTs, only one (4%) commented that it made the instructor approachable. No disadvantage was mentioned regarding student-instructor interaction.

All interviewees noted that due to the light-hearted trailers, they felt more at ease and open with the instructor. Interviewee #1 explicitly mentioned the effect the trailers had on connection to the instructor: “Yeah, so it just makes her seem more human, I guess, rather than just a professor, you know.” By moving away from, and beyond, the classroom, the interviews also suggest that ISLTs added depth to the instructor through the ability to showcase personal interests and other information that students could connect to, transcending traditional pedagogical activities.

Discussion

This study created and implemented ISLTs to improve student engagement in the third-year remote sensing online course, and the results indicated that ISLTs enhanced students’ online engagement to some extent but with limitations. For behavioral engagement, ISLTs impacted students’ viewing habits but not participation. Students stated emotional engagement with ISLTs, but not in the follow-up lectures. Students rated the student-instructor engagement much higher than in other online courses; nevertheless, our data do not support a strong conclusion considering a lack of representative data in the open-ended responses. The impact on students’ cognitive engagement is multidimensional and convoluted. The framework of this study is summarized in Table 6, along with recommendations for others who are interested in carrying out similar work.

Table 6.

The framework of the assessment measures and the techniques of interactive storytelling trailers in this study, along with recommendations for others.

What we did	Recommendations
We used learning analytics to track student behavioral engagement such as the number of page views and participations. Results show that ISLTs significantly enhanced students’ behavioral engagement through page views but not participation in their asynchronous time.	It would be helpful to add survey questions to measure students’ authentic engagement, and compare students’ behavioral engagement between the asynchronous and synchronous activities.
We used surveys and interviews to investigate the student-instructor engagement. Results suggest that ISLTs significantly enhanced the connections between students and instructors. However, due to the survey design, we are not able to explain why.	We think that including the course instructor as the main character in the trailer is an effective strategy.We suggest adding specific survey questions to investigate the factors that impact student-instructor engagement.
We used surveys and interviews to investigate students’ emotional engagement. Students commented that ISLTs were interesting and motivated them to learn. However, the results did not show significant enhancement of their emotional engagement with the follow-up full lectures.	We suggest embedding similar visuals, characters, or stories in lectures to extend the emotional engagement beyond the ISLTs.
Our ISLTs provide previews of lecture content. We used quizzes to investigate students’ cognitive ability in different levels of thinking, according to Bloom’s taxonomy, in short term learning. The ISLTs significantly improved students’ cognitive ability in lower levels of thinking, including remembering and applying, but not in higher levels of thinking, such as creating.	Future work should consider creating interactive storytelling videos that explain challenging topics more in-depth, investigate different levels of thinking in both short and long terms, and create questions that test other cognitive processes, such as evaluation.
We invited students to comment on the disadvantages of ISLTs, and the major drawback is cognitive overload.	We recommend adjusting the pace and decreasing the amount and frequency of technical terms.
We applied the following techniques to video production: created characters and plots, generated conflicts between characters, and introduced a sense of drama.	The quality of storytelling trailers can be further improved by incorporating personal perspectives and enhancing authenticity.

Our study is the first that reports the effectiveness of interactive storytelling trailers in enhancing students’ behavioral engagement through page views in online learning. Behavioral engagement allows instructors to get timely feedback and determine the effectiveness of teaching practices, although it only provides superficial data that may not capture authentic engagement. Behaviorally-engaged students find interest in learning, seek help when required, and help others (Redmond et al., 2018). As a result, students develop academic skills, such as reading, writing, listening, planning, time management, and goal setting, that contribute to their success (Pittaway and Moss, 2014; Redmond et al., 2018). Nevertheless, the ISLTs did not signiticantly increase students’ participation in the course. Further, we found that ISLTs significantly enhanced students’ behavioral engagement in their asynchronous time, while not in the synchronous time. These differences call for further investigation.

Students reported significantly higher student-instructor engagement in this course than in other online courses. Due to a lack of evidence in open-ended question responses in the post-survey and the low number of interviewees, we don’t have a firm explanation. A possible reason, according to the interviews, is that the instructor’s effort in making the ISLTs and her presence in the videos made the instructor seem more approachable. Interviewee #3 noted this: “I think that it really allowed me to see a different side of the professor, a more creative side. And that got me thinking hey, so she’s also into, like, gets video production and then she has a more creative side. . . She does stop-motion animation and photography and video production. And I thought, hey I’m into that too. That’s pretty cool. . . so it makes her seem more approachable.” Because of such perceived approachability of the instructor, students were willing to seek help when they encountered challenges. Interviewee #3 shared the experience that “I remember we had to do these individual presentation things for the lectures and my group members, kind of left the group when I try to get in contact with them and so I normally. . . I would have just, I don’t know what I would have probably just to whatever leave also but I talked to her. I went to her office hours and she is really nice about it and extended her help and I thought, if I didn’t, you know, see her as being approachable like that then I wouldn’t have gone to her for help like that and it turned out really great because of it.” As the ISLTs provided students with a new means of connecting to instructors, they may inspire students to take more courses with the instructor or in the department. This is important to note as large universities could potentially have trouble supporting student connections with instructors due to the large sizes of classes. Interviewee #2 noted that the ISLTs were a factor in their decision to take another course with the instructor: “She displays her face in trailers I know if you’re familiar with them. Yeah, they’re kind of more engaging more personal, you could say. . .I mean, yeah, compared to other online classes, of course, and then this is why I’m also taking a second class with Professor X in the second half that would not have taken otherwise.” This result is consistent with the other study, showcasing that instructor presence in online courses can help bridge the distance and make students feel less isolated (Arbaugh and Benbunan-Finch, 2006). The instructor’s presence and connection can further contribute to students’ academic achievements (Brinkerhoff and Koroghlanian, 2007; Griffiths and Graham, 2009). Among different types of interactions, such as learner-to-learner, learner-to-content, and learner-to-instructor interactions, the latter was the most valued by students (Martin and Bolliger, 2018). As such, we highly recommend including course instructors as the main characters in trailer production.

Students clearly indicated that the ISLTs were interesting, increased their attention, and motivated them to learn. This was also reported by Goudsouzian (2018) which showed that the Hollywood style trailer increased students’ interest in the topic. Although students reported that the ISLTs made them pay more attention to the concepts that occurred in their learning, the ISLTs did not significantly enhance students’ emotional engagement with the follow-up full lectures, meaning that there is a gap between the emotional engagement with the ISLTs and with lectures themselves. A potential way to bridge this gap could be to embed similar visuals, characters, or stories in lectures to extend emotional engagement beyond the ISLTs.

The ISLTs significantly improved students’ cognitive ability in lower levels of thinking including remembering and applying, but not in the higher levels of thinking such as creating. A possible explanation is that all tests were administered right after the lectures; therefore, the results indicate short-term learning gains. Creation is a longer process that requires enhanced metacognition in which knowledge is transformed into new forms, which are then made into shareable products (Armbruster, 1989). In the future, tests focused on the cognitive process of evaluation may be included as well as tests focused on the creative levels of cognition. Furthermore, tests on creating cognitive level can be administered after students have digested the learning materials (e.g. 1 week after the material was first introduced). Another explanation is that ISLTs only provided a preview rather than an explanation of the content, so we did not find their effectiveness in deep learning. As no existing studies investigated the effect of ISLTs on students’ learning, we compared our results with studies on interactive and storytelling videos, which also reported that interactive lecture videos improved short-term learning gains (Delen et al., 2014; Merkt and Schwan, 2014; Merkt et al., 2011; Shelton et al., 2016). Similarly, digital storytelling techniques were found to have positively affected students’ learning outcomes, especially in recalling content (Choi, 2018). Different from our study, digital storytelling techniques promoted deep learning (Ryan and Aasetre, 2020). Such a difference is expected as our ISLTs were designed to be a preview rather than a means to foster deep learning. To enhance deep learning, interactive storytelling videos can be considered for challenging topics.

According to students’ perceptions, the majority of students reported that ISLTs bridged the content and made course materials more accessible, while a few students believed ISLTs were burdensome and sometimes overwhelming. Interviewees specifically noted that the trailers were very helpful in introducing new topics and delivering information, which could be easily accessed and reviewed later to assist in studying. Interviewee #3 noted this: “the trailer is very useful because it’s short format, and it kind of gives us a rundown of everything we’re expected to learn. Kind of an engaging way like through video, rather than a list of things.” Repeatedly, interviewees explicitly noted that the trailers provided a novel pedagogical device that engaged students through priming them for new material, as opposed to status quo pedagogical strategies such as traditional readings or slideshows. Another interviewee (Interviewee #4) noted how the trailers provided a unique preparation opportunity: “Well the trailer does. . . it’s not really helpful in terms of content knowledge, it’s more of a, you know, a preparation component, but I think it’s really important” and “it would allow everyone to get an idea of what the learning outcomes for that lecture would be or at least after seeing the trailer.” Two out of the four interviewees stated that there were no drawbacks related to ISLTs, while the drawbacks mentioned by the other two interviewees were all from cognitive aspects. For example, two interviewees commented that the ISLTs were burdensome. The interviewee #1 noted, “they were not a requirement. And so when you have so many assignments due, you’re going to tend to forget one.” Similarly, Interviewee #4 commented that “Um I guess when students see it on the list of things that they have to do on top of their readings and the assignments, they think oh it’s like an extra thing that I have to do.” Interviewee #1 further noted that the pace and the amount of information in trailers were overwhelming, “There’s a lot of information, flashing lights, and it’s very chaotic, I think that it’s pretty cool though. That will be a drawback though sometimes it. . . It’s kind of bit overwhelming.” Suggestions for future trailer production are to adjust the pace and decrease the amount and frequency of technical terms.

Stories, using various characters, plots, and conflicts, can easily establish personal connections, and make lecture videos more emotionally appealing (Pregitzer and Clements, 2013; Singhal and Rogers, 2002). However, creating such content may take an enormous amount of effort and time. Making short ISLTs can be a viable alternative. McDonald (2009) interviewed eight filmmakers who had produced many films and concluded using storytelling principles in instructional design, such as conflict, authenticity, and entertainment, can capture students’ interest and attention. Choi (2018) suggested that by creating more authentic content, instructors seem to be more humanized than a virtual figure in a video. Other strategies include providing personal perspective, showing emotions, and giving a sense of drama can enhance the effectiveness of digital storytelling (Lambert and Hessler, 2018). We tried to adopt some of the strategies, such as using various characters, plots, and providing a sense of drama, but quality can be further improved by incorporating personal perspectives and enhancing authenticity.

There are a few limitations in this research. First, the participants in this study were from one course at one university. More similar studies are needed to generalize the results. Second, the research was carried out during COVID-19 pandemic; as a result, the participants’ attitudes toward the online courses may be different. In specific, students were forced to take online courses instead of choosing to, as a result of public health lockdowns, and they needed to take multiple online courses in the same semester, which may result in online learning fatigue. As a result, student’ responded to the survey may be the more engaged students, which could introduce biase to our conclusion. Third, it is crucial to take into account the complexity of the instructional context as multiple course activities and assignments are implemented in a course (Tsai et al., 2021). Fourth, data on behavioral engagement were collected from LMS as page views and participation; however, students may spend varying time on each page or activity, of which, data are not available. In addition, students’ authentic engagement or habits were unobservable. Fifth, this study highly depends on the survey and interview data, while participants’ self-reported beliefs are not necessarily reflective of their actions or observable experience. In addition, nearly half of the students did not respond to pre-survey, two-thirds did not participate in the post-survey, and one-third of the students did not complete all quizzes, the missing data decreased the sample size which may cause bias. Specifically, by removing students who missed one or more quizzes, the samples tend to represent stronger students. The post-survey might not be representative as only a third of the students responded to the post-survey, with a likelihood of excluding students who did not watch or did not like the trailers. Similarly, students who signed up for interviewees may not be representative for the entire population. Furthermore, 45% of the students had not taken an online course that were designed for online delivery prior to the pandemic, making it hard to compare with other online courses under normal conditions. Finally, this study assumed that all students watched the ISLTs, while this was not the case. Due to technological constraints, we were not able to trace who had watched the ISLTs. This can potentially impact the analysis, including students’ perception of the engagement created by ISLTs and their effect on quiz scores. Quantitative measures in surveying student engagement may be problematic; therefore, Dyment et al., (2020) recommend using qualitative measures to more authentically capture engagement.

Although students and faculty were forced to engage with the courses online during the pandemic, their experience shapes perspectives of the delivery modality for future courses. According to the Online Learning Consortium and Bay View Analytics (Seaman et al., 2021), faculty are cautiously optimistic about the future of online education. According to the report, the most common barrier is the insufficient motivation in students to succeed in online courses. Nevertheless, opportunities arise to reassess how education is currently delivered (Seaman et al., 2021). This study provides one strategy to motivate students through the online engagement with students, facilitated by interactive storytelling trailers.

Supplemental Material

sj-docx-1-alh-10.1177_14697874221107574 – Supplemental material for Use of interactive storytelling trailers to engage students in an online learning environment

Supplemental material, sj-docx-1-alh-10.1177_14697874221107574 for Use of interactive storytelling trailers to engage students in an online learning environment by Forrest Hisey, Tingting Zhu and Yuhong He in Active Learning in Higher Education

Footnotes

Acknowledgements

The authors appreciate the feedback provided by the UTM TDI committee for the proposal of the study and the help offered by Library & Instructional Technologies. We thank the Educational Developer Ann Gagné for consultations in the project and survey design, and research assistants, Xiang Zhuang, for contributing to the editing of the lecture trailers, and Symon James-Wilson, for designing survey questions. We thank Professor Nicole Laliberte and Andrew Peterson for proof-reading and providing feedback to our manuscript draft. Finally, we would like to thank the anonymous reviewers for their valuable feedback.

Author contribution

Conceptualization and methodology: Yuhong He and Tingting Zhu; Formal analysis and investigation: Forrest Hisey and Tingting Zhu; Writing—original draft preparation: Forrest Hisey and Tingting Zhu; Writing—review and editing: Yuhong He, Forrest Hisey, and Tingting Zhu. Funding acquisition: Yuhong He and Tingting Zhu.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Teaching Development and Innovation Grant (TDI) at the University of Toronto Mississauga (UTM).

Ethics approval

Approval was obtained from the Office of Research Ethics at University of Toronto Mississauga. The Protocol number is 40139.

Informed consent

Informed consent was provided and collected along with the surveys in the learning management system. For participants who took the interviews, their informed consent was sent and collected through email prior to the interview.

ORCID iD

Tingting Zhu

Supplemental material

Supplemental material for this article is available online.

Author biographies

Forrest Hisey is a Ph.D. student in the human geography program at the University of Toronto Mississauga. His research interests focus upon human-environment interactions, biodiversity conservation and socio-ecologic relations.

Tingting Zhu is an assistant professor cross appointed in the geographic information system and computer science programs at the University of Toronto Mississauga. She teaches environment remote sensing, spatial data science, geographic information system, GIS capstone project, introduction to computer science, and data structures. Her research interests are in student engagement, experiential learning, and educational technology.

Yuhong He is a Professor in the Department of Geography, Geomatics and Environment at the University of Toronto Mississauga. She has mobilized multidisciplinary researchers and collaborated with cross-sectoral practitioners for the common goals of understanding how human activities and global changes affect environments and discovering how to make our build/natural environments healthier, more resilient, and adaptable. She received the Early Researcher Award of Ontario in 2015 and was awarded the Early Career Scholar Award from Remote Sensing Specialty Group at the AAG 2011 annual meeting. Dr. Yuhong He has published more than 70 peer-reviewed journal articles and presented over 80 papers at national and international conferences.

References

Arbaugh

Benbunan-Finch

(2006) An investigation of epistemological and social dimensions of teaching in online learning environments. Academy of Management Learning and Education 5(4): 435–47.

Armbruster

(1989) Metacognition in creativity. In: Glover

Ronning

Reynolds

(eds) Handbook of Creativity. Boston: Springer. pp.177–82.

Bawa

(2016) Retention in online courses: Exploring issues and solutions: A literature review. Sage Open 6(1): 2158244015621777.

Brinkerhoff

Koroghlanian

(2007) Online students’ expectations: Enhancing the fit between online students and course design. Journal of Educational Computing Research 36(4): 383–93.

Brown

Lawrence

Basson

, et al. (2022) A conceptual framework to enhance student online learning and engagement in higher education. Higher Education Research & Development 41: 284–99.

Charmaz

(2014) Constructing Grounded Theory. Los Angeles: SAGE.

Chen

P-SD

Lambert

Guidry

(2010) Engaging online learners: The impact of Web-based learning technology on college student engagement. Computers & Education 54(4): 1222–32.

Choi

(2018) Learning through digital storytelling: Exploring entertainment techniques in lecture video. Educational Media International 55(1): 49–63.

Clinton

Kelly

(2020) Student attitudes toward group discussions. Active Learning in Higher Education 21(2): 154–64.

10.

Creswell

(2012) Educational Research: Planning, Conducting, and Evaluating Quantitative. Educational Research. Boston: Pearson.

11.

Curtis

Lawson

(2001) Exploring collaborative online learning. 5(1): 21–34.

12.

Dyment

Stone

Milthorpe

(2020) Beyond busy work: Rethinking the measurement of online student engagement. Higher Education Research & Development 39(7): 1440–1453.

13.

Liew

Willson

(2014) Effects of interactivity and instructional scaffolding on learning: Self-regulation in online video-based environments. Computers & Education 78: 312–20.

14.

Denzin

(2017) The Research Act: A Theoretical Introduction to Sociological Methods, 1st edn. New York: Routledge. Available at: https://www.taylorfrancis.com/books/9781351475273 (accessed 29 May 2021).

15.

Forehand

(2010) Bloom’s taxonomy. Emerging Perspectives on Learning, Teaching, and Technology 41(4): 47–56.

16.

Fredricks

Blumenfeld

Paris

(2004) School engagement: Potential of the concept, state of the evidence. Review of Educational Research 74(1): 59–109.

17.

Goudsouzian

(2018) Hollywood-style movie trailers increase student interest. Journal of Microbiology and Biology Education 19(1). https://journals.asm.org/doi/10.1128/jmbe.v19i1.1541

18.

Greenland

Moore

(2014) Patterns of student enrolment and attrition in Australian open access online education: A preliminary case study. Open Praxis 6(1): 45–54.

19.

Griffiths

Graham

(2009) The potential of asynchronous video in online education. Distance Learning 6(2): 13.

20.

Khalil

Mansour

Fadda

, et al. (2020) The sudden transition to synchronized online learning during the COVID-19 pandemic in Saudi Arabia: A qualitative study exploring medical students’ perspectives. BMC Medical Education 20(1): 285.

21.

Lambert

Hessler

(2018) Digital Storytelling: Capturing Lives, Creating Community. New York: Routledge.

22.

Lisenbee

Ford

(2018) Engaging students in traditional and digital storytelling to make connections between pedagogy and children’s experiences. Early Childhood Education Journal 46(1): 129–39.

23.

Macdonald

(2003) Assessing online collaborative learning: Process and product. Computers & Education 40(4): 377–91.

24.

Martin

Bolliger

(2018) Engagement matters: Student perceptions on the importance of engagement strategies in the online learning environment. Online Learning 22(1): 205–22.

25.

McDonald

(2009) Imaginative instruction: What master storytellers can teach instructional designers. Educational Media International 46(2): 111–22.

26.

Merkt

Schwan

(2014) Training the use of interactive videos: Effects on mastering different tasks. Instructional Science 42(3): 421–41.

27.

Merkt

Weigand

Heier

, et al. (2011) Learning with videos vs. learning with print: The role of interactive features. Learning and Instruction 21(6): 687–704.

28.

Pittaway

Moss

(2014) “Initially, we were just names on a computer screen”: Designing engagement in online teacher education. Australian Journal of Teacher Education 39(7): 140–56.

29.

Pregitzer

Clements

(2013) Bored with the core: Stimulating student interest in online general education. Educational Media International 50(3): 162–76.

30.

Redmond

Heffernan

Abawi

, et al. (2018) An online engagement framework for higher education. Online Learning 22(1): 183–204.

31.

Ryan

Aasetre

(2020) Digital storytelling, student engagement and deep learning in Geography. Journal of Geography in Higher Education 45(3): 1–17.

32.

Seaman

Allen

Ralph

(2021) Teaching online: STEM education in the time of COVID (p. 15). Bay View Analytics. Available at: https://www.bayviewanalytics.com/reports/stem_education_in_the_time_of_covid.pdf (accessed 19 May 2022).

33.

Shelton

Warren

Archambault

(2016) Exploring the use of interactive digital storytelling video: Promoting student engagement and learning in a university hybrid course. TechTrends 60(5): 465–74.

34.

Singhal

Rogers

(2002) A theoretical agenda for entertainment–education. Communication Theory 12(2): 117–35.

35.

Soffer

Cohen

(2019) Students’ engagement characteristics predict success and completion of online courses. Journal of Computer Assisted Learning 35(3): 378–89.

36.

Stone

(2019) Online learning in Australian higher education: Opportunities, challenges and transformations. Student Success 10(2): 1–11.

37.

Tsai

C-L

H-Y

Campbell

(2021) Impacts of course activities on student perceptions of engagement and learning online. Distance Education 42(1): 106–25.

38.

Wei

H-C

Chou

(2020) Online learning performance and satisfaction: Do perceptions and readiness matter? Distance Education 41(1): 48–69.

39.

Williams

Lahman

(2011) Online discussion, student engagement, and critical thinking. Journal of Political Science Education 7(2): 143–62.

40.

Jaggars

(2011) Online and hybrid course enrollment and performance in Washington state community and technical colleges. CCRC Working Paper No. 31. Available at: https://files.eric.ed.gov/fulltext/ED517746.pdf (accessed 15 May 2021).

41.

Young

(2010) The art and science of fostering engaged learning. Academy of Educational Leadership Journal 14: 1.

42.

Zheng

Niiya

Warschauer

(2015) Wikis and collaborative learning in higher education. Technology Pedagogy and Education 24(3): 357–74.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

6.43 MB