The Efficiency of Online “Inked” Videos Versus Recorded PowerPoint Lectures on Teaching Pathophysiology to Medical Students in Pre-Clerkship Years: A Pilot Study

Abstract

Background:

As online learning develops an increasingly important role in medical education, new online teaching modalities are arising all the time. One such teaching modality that is gaining popularity among medical students is the “inked” video, a type of animated video that utilizes a virtual blackboard. Student reviews suggest that the dynamic style of the inked video allows it to teach more efficiently than traditional teaching modalities, but currently there is no quantitative evidence to support or guide the use of this teaching modality.

Hypothesis:

When compared to the traditional recorded PowerPoint lecture, online inked videos teach the same pathophysiology concepts to pre-clerkship medical students in a shorter amount of time.

Method:

A randomized, crossover-design study was conducted with second-year medical students at the David Geffen School of Medicine at University of California, Los Angeles (n = 22). Students were randomized to 2 groups. Each group received 2 lessons on 2 different pathophysiology topics: one via a shorter inked video and the other via a longer recorded PowerPoint lecture. Two sets of dependent variables were used to test modality effects: immediate post-tests scores and delayed retention post-test scores. A perceptions survey was also administered to assess student preferences between the 2 teaching modalities.

Results:

Students performed similarly on immediate and delayed post-tests for the shorter inked videos and the longer recorded PowerPoint lectures (P > .05). Students reported greater engagement (P < .05) and greater satisfaction with learning (P < .05) with the inked videos than the recorded PowerPoint lectures.

Conclusions:

This study, although limited by its small-scale and single-institution design, provides preliminary evidence that online inked videos may be a more efficient and non-inferior alternative to recorded PowerPoint lectures.

Introduction

As the health care system has evolved over the last 3 decades, medical education has been challenged to adapt.^1-3 Ozuah cited 2 major obstacles facing medical education: (1) limited time for teaching among the clinical faculty and (2) emergent disciplines that need to be incorporated into an already-saturated curriculum. ¹ To meet these needs, didactic approaches in medical education have begun to (1) shift from teacher-oriented to learner-oriented^3,4 and (2) recognize greater need for learning efficiency. ⁵ Institutions have found e-learning or computer-based learning to be a useful tool in these efforts to adapt.^3,6

Features of e-learning make it particularly well-equipped to meet the emerging needs of medical education. It allows instructors to easily reach a wide student population because of its broad resource-sharing capacity and cost-effective scalability. ⁷ E-learning is also especially well-suited for student-directed independent learning. ⁷ Its ease of use and accessibility allow students to use it at a place and time of their choosing. ⁷ Studies have shown that students widely use online learning modalities outside of their curriculums, suggesting that students appreciate and take advantage of these features of online learning.^8-10

The role of e-learning in medical education is well-established across institutions. Many recognize a place for e-learning in medical education, often favoring a hybrid learning approach in which schools incorporate both classroom-based and online teaching methods.^11-13 It has become commonplace for medical schools in the United States to offer online alternatives to in-person classroom-based lectures, such as recorded or live-streamed lectures.^14,15 Within the broad umbrella of e-learning, there are numerous different types of didactic models, with recorded lectures as one such framework.^16,17 While studies have described the efficacy of e-learning as a whole, few studies have compared and studied the nuances between the different types of e-learning methods.^6,8

A newer online learning modality is the “inked” video, a type of animated video that utilizes a virtual blackboard. ¹⁸ The writing appears on the screen as if it is being written in real time, whereas a voiceover narrates content in a synchronized manner. The instructor is not seen on-screen, allowing learners to focus solely on the text and drawings that appear as the lesson unfolds. Well-known platforms for these inked videos include Osmosis and Khan Academy, with one study revealing that more than 90 000 students from 500 medical schools used Osmosis in one year. ¹⁰ Despite the abundance of online inked video resources, it is still not well understood how effective they are at helping medical students learn compared with traditional didactics. Support for the use of inked videos specifically to educate medical students relies mainly on evidence of learner satisfaction. ¹⁸ Qualitative data suggest that the dynamic quality of inked videos may have the ability to convey information more efficiently than a standard, recorded PowerPoint lecture.

In the setting of medical education where educators constantly operate under the restrictions of insufficient time, efficiency (learning per unit time) is a key feature in evaluating different teaching resources. ¹⁹ In his review of the benefits of online learning, Cook argues that e-learning is not inherently more efficient than classroom-based methods, but rather that efficiency depends on how well the instructional design incorporates proven principles of effective learning. ²⁰ Mayer described 12 principles of multimedia learning that shape the production of effective online teaching modules.^21,22 We believe inked videos fulfill more of these essential principles than the widely utilized recorded PowerPoint lecture, especially regarding the redundancy principle, signaling principle, and personalization principle.

In this pilot study, we aim to quantitatively compare inked videos with recorded PowerPoint lectures for the acquisition of knowledge among pre-clerkship medical students, while also measuring the efficiency of both teaching methods in covering concepts in cardiac pathophysiology.

Methods

The study was exempted from institutional review board (IRB) review by the UCLA IRB (IRB#18-000725). A prospective, single-institution study was conducted at the David Geffen School of Medicine (DGSOM).

Study Participants

The study population consisted of second-year medical students at DGSOM at UCLA. Students were recruited via email and postings on social media. Participation in the study was voluntary. The study was conducted independently of the established medical school curriculum. To be eligible for the study, students needed to have completed the first-year cardiac physiology course but not the second-year cardiac pathophysiology course at DGSOM.

Lesson Materials

Paroxysmal supraventricular tachycardia (PSVT) and diastolic murmurs were chosen as the lesson topics of this study. The first lesson on PSVT was created in 2 formats: an inked video (8:05 minutes) and a recorded PowerPoint video (10:13 minutes); the second lesson on diastolic murmurs: inked video (6:48 minutes) and recorded PowerPoint video (9:48 minutes). The online inked videos and the recorded PowerPoint lectures were produced by a single individual and reviewed by a cardiac pathophysiology course director for accuracy. For each lesson topic, the inked video and recorded PowerPoint lectures followed the same script. To produce the inked videos, an Apple iPad and Apple pencil were used, along with Microsoft OneNote and the iPad’s built-in screen capture function. The audio was later recorded with a Snowball USB microphone. To edit the videos, Apple iMovie was used to crop the video clips, speed up the drawings, and overlay the voiceover. To produce the recorded PowerPoint lectures, an Apple MacBook was used, along with Microsoft PowerPoint and the MacBook’s built-in screen capture function. Audio was captured with a Snowball USB microphone. The PowerPoint was delivered and captured in real time. The dynamic, fast-moving illustrations of the inked videos permitted the same scripts to be delivered in less time for the inked videos in comparison to the PowerPoint recordings.

Study Design

At the end of the academic year, we contacted all DGSOM second-year students to solicit participation in a pilot study with 26 students agreeing to participate. Students’ prior knowledge was assessed via self-reported survey ratings on the 2 covered topics, PSVT or diastolic murmurs, on a 5-point scale (1 = none, 2 = basic, 3 = intermediate, 4 = advanced, 5 = expert). Students were to be excluded if they rated their prior knowledge of either didactic topics as advanced or expert.

A balanced, randomized crossover design was employed to examine performance differences resulting from exposure to 2 modalities: inked video versus recorded PowerPoint. This experimental design was chosen to expose students to the intervention (online inked video) while serving as their own control group. Students were randomized into the 2 groups (INKPPT or PPTINK) using the random number generator function in Microsoft Excel. Students in INKPPT received the first lesson on PSVT in an inked video format, whereas students in PPTINK received the same lesson in a recorded PowerPoint format; afterward, students in INKPPT received the second lesson on diastolic murmurs via recoded PowerPoint lecture, and students in PPTINK received the same lesson via inked video. We used 2 sets of dependent variables to test modality effects: immediate post-tests following exposure to the 2 modalities and delayed retention tests administered after 12 days. Each post-test consisted of 10 items, yielding a total of 4 post-tests with 40 items. A 6-item perceptions survey on the 2 modalities was administered following the second immediate post-test, with 4 items on a Likert-type scale from 1 to 5 (1 = not at all satisfied, 5 = extremely satisfied), culminating with 2 open-ended items asking students to describe what features of the 2 modalities helped and interfered with their learning. The post-tests and surveys were administered electronically on Google Forms.

Statistical Analysis

We conducted 3 pairs of matched pairs t-tests to compare performance differences between the 2 modalities on immediate post-test, delayed post-test, and knowledge retention from immediate to delayed post-test. Non-parametric tests were selected as Shapiro-Wilk test of normality was violated for 2 of the 4 post-tests (P = .01 and P = .02). Reliability of the 20-item immediate post-test was .46 (Cronbach alpha) and for the 20-item delayed retention post-test was .68. (Reliability was influenced by the small sample size and lower number of test items.) We also conducted 2 pairs of matched pairs t-tests to assess differences between the 2 modalities regarding student-reported satisfaction with learning and engagement. All tests were considered statistically significant at P = .05; SPSS 24.0 was used for data analysis.

Results

A total of 26 students responded to recruitment media, expressing interest in the study. The 26 subjects were randomly divided into 2 groups, INKPPT and PPTINK, but 4 did not complete the modules. Of the 22 students remaining, none were excluded for prior knowledge of subject material. The final convenience sample of 22 students yielded the groups: INKPPT (n = 12) and PPTINK (n = 10; Figure 1). All 22 students completed the entirety of the study, with one exception: one perceptions survey from a student in INKPPT was returned but only partially completed (the survey included ratings for the inked video format, but not for the recorded PowerPoint format).

Figure 1.

Overview of study recruitment, randomization, and follow-up.

Figure 2 represents the means and standard deviations of all immediate and delayed retention tests. Scores for both modalities on immediate post-tests are on the higher end, indicating a comparable learning effect of these modalities (mean ± SD = 8.09 ± 1.23 and 8.08 ± 1.40 for inked video and recorded PowerPoint, respectively). Delayed post-tests show the same pattern (mean ± SD = 6.27 ± 1.61 and 5.91 ± 2.14 for inked video and recorded PowerPoint, respectively); however, knowledge decay over 2 weeks for both modalities was considerable (loss of 1.82 points of inked video modality versus 2.14 points for recorded PowerPoint modality).

Figure 2.

Mean student performances (n = 22) on immediate and delayed post-tests by teaching modality.

The Related Samples Wilcoxon Signed Rank tests performed to detect differences in all group means showed no significant difference between immediate post-test performance on the 2 modalities (P = .84), or between delayed post-test differences (P = .46), retaining the null hypothesis. Thus, there was no short term or sustained, differential impact of inked video versus recorded PowerPoint on student performance. However, the decline in scores from immediate to delayed post-test was significant for both modalities, demonstrating swift decay in knowledge regardless of type of modality (P < .01).

Table 1 shows the means and standard deviations of the student ratings for both teaching modalities regarding engagement and satisfaction with learning. Responses to the post-test perceptions survey showed that students felt more engaged with the inked video than the recorded PowerPoint (mean ± SD = 4.00 ± 0.76 and 3.00 ± 0.78 for inked video and recorded PowerPoint, respectively) and were more satisfied with their learning (mean ± SD = 3.64 ± 0.90 and 3.00 ± 1.10 for inked video and recorded PowerPoint, respectively, Figure 3). A breakdown of each individual survey reveals that every student rated their personal engagement and learning satisfaction with the inked video at least as highly as with the recorded PowerPoint—with most rating the inked video higher. The Related Samples Wilcoxon Signed Rank tests showed a significant difference between inked video and recorded PowerPoint regarding student-reported engagement (P < .01) and satisfaction with learning (P = .045). In response to the open-ended question asking students which features of the inked videos helped their learning, students reported that the animation of the inked videos allowed for “more active than passive” learning, presented the information in small pieces that were “easier to digest,” and made it “easier to follow along” with the presented information. In response to the open-ended question asking students which features of the inked videos interfered their learning, students reported that the real-time writing and drawing did not allow them to have a preview of the information that would come next and that it was easy to miss concepts that were not explicitly written on the slide.

Table 1.

Mean, standard deviation, and P-values of student ratings for inked video and recorded PowerPoint with respect to engagement and satisfaction with learning, as assessed by a 5-point Likert-type scale (1 = not at all, 5 = extremely).

	Engagement			Satisfaction with learning
	Inked video	PowerPoint	P value	Inked video	PowerPoint	P value
Individual subjects	n = 22	n = 21		n = 22	n = 21
Rating	4.00 ± .76	3.00 ± .78	<.01	3.64 ± .90	3.00 ± 1.10	<.05

Figure 3.

Mean student ratings of inked video and recorded PowerPoint with respect to engagement and satisfaction with learning, as assessed by a 5-point Likert scale. The asterisk(*) indicates statistical significance with p value of 0.05.

Discussion

Few studies have compared the efficiency of different types of online teaching modalities in medical students. Considering the enormous amount of material that medical students learn within a limited time, it is important to find ways to teach efficiently without sacrificing the quality of instruction.

In this pilot study, students performed equally well on lessons taught via the shorter inked videos and lessons taught via the longer, recorded PowerPoint lectures in both the immediately administered assessments and the delayed retention assessments. In the lessons of this study, the durations of the videos between the 2 modalities differed by a couple minutes, a time difference which may appear inconsequential. However, in real practice, students may be watching tens, if not hundreds, of instructional videos over the course of their curriculum. Considering that the inked videos used in our study were 25% and 44% shorter than the recorded PowerPoint lectures in lesson 1 and lesson 2, respectively; one could imagine a significant amount of time saved for students should our findings translate to production of other inked videos covering medical school content. In addition, it is valuable for a curriculum to include teaching modalities that appeal to students, and the perceptions survey showed that students consistently rate higher engagement and learning satisfaction with the inked videos.

However, a potential limiting factor for incorporating inked videos into a curriculum is the production of the videos themselves. As discussed in “Lesson Materials,” the inked videos in this study were produced by a single individual without prior training and utilizing relatively mainstream software. There is a wide range of software capable of producing this type of media. The experience of producing an inked video will be user-dependent, varying widely depending on digital literacy and access to equipment. Undoubtedly, it takes a great deal more effort to produce an inked video than it does to record a PowerPoint lecture delivered in a classroom. This discrepancy is made more striking by the fact that many medical schools already have a collection of recorded classroom-based lectures at their disposal but would likely need to produce an inked video from scratch. Thus, while the inked videos may be time-saving for students in the long run, they will first require a substantial faculty-driven time investment. Ultimately, whether an educator decides to use the inked video format should depend on their own assessment of the costs and benefits.

If an educator were to use inked videos, we see potential for the inked video to be built into the flipped classroom, a popular teaching model that is currently regarded as superior to the traditional classroom. ²³ In the flipped classroom approach, students are primed to the learning topic of the classroom-based session through at-home, self-directed learning.^24,25 Students can use faculty-made inked videos for the self-directed learning before the classroom-based session. In this context, the inked video may allow students to efficiently build a foundation of knowledge, which will later be reinforced with an application-based learning method.

An incidental finding of this study was that in both online teaching modalities, a significant decay in test performances was observed between the immediate post-test and the delayed post-test. In other words, with these 2 common teaching modalities, there is a significant loss of knowledge after a time span as short as 2 weeks. While outside the scope of this study, this finding suggests a need to further investigate how pre-clerkship teaching prepares students for long-term knowledge retention.

Limitations

This study has a number of limitations, one being that its participants were recruited from only a single institution. While this single-site design allowed for a relatively standardized baseline of prior training in cardiac physiology among study participants, it limits the generalizability of the findings. However, the crossover design in which participants serve as their own internal control should help guard against cofounding variables. The small sample size (n = 22) limits the ability to detect statistically significant differences. The lessons in the study also tested only 2 selected cardiac pathologies, which were thought to lend themselves well to a visual and dynamic teaching method. It remains to be tested whether the data can be extrapolated to lessons covering different subject material, especially those that are more difficult to represent visually. The study conditions also may not have fully replicated real-world study conditions. In the study, students watched the lesson videos without rewinding the video, altering the speed, or taking notes, but in real-life study conditions, students may practice different study habits. Study participation was also voluntary, so there was danger of selection-bias. The study did not use a double-blinded design, but the outcome being measured (scores on multiple choice post-tests) was objective. Whether or not the data can be extrapolated to other medical school populations and other subject materials remains to be investigated and further studies with a larger sample size should be conducted.

Conclusions

Given the limited sample size and single-site design of this study, its findings are preliminary, but they suggest that the inked videos are a non-inferior, more efficient teaching method than the traditional recorded PowerPoint lectures. The study also suggests that students have greater satisfaction with their learning when watching inked videos. However, incorporating inked videos into a school curriculum requires a variable, but significant time investment from faculty members. In considering whether or not to employ inked videos in their teaching, an educator should weigh the cost of the time and resources needed to create the videos against the potential benefits of marginally increased efficiency and satisfaction for the students. Future studies should consider testing the efficacy of inked videos in teaching additional subject material with a larger and more diverse study population.