Abstract
OBJECTIVES
The intent of this study is to compare the academic performance of pre-clinical medical students in online teaching with online proctoring settings versus face-to-face (F2F) teaching with traditional F2F exam proctoring settings.
INTRODUCTION
In 2019, the world was faced with a rapidly progressive pandemic of COVID-19. Consequently, numerous institutions have opted to transform from the traditional F2F teaching method to an online modality, ensuring the safety of their students and maintaining the effectiveness of their educational programs. In particular, medical universities faced a unique challenge in creating a new learning system due to their atypical curriculum, which often involves hands-on training and is not easily adaptable to remote teaching.
METHODS
The core of this study is based on the scores of first-year undergraduate medical students across three academic years from the Mohamed Bin Rashid University of Medicine and Health Sciences in Dubai (MBRU), United Arab Emirates (UAE). Course and semester grades were collected and compared in both F2F and online teaching modalities as well as exam settings.
RESULTS
Our results show that major courses with high-credit courses had no difference in students’ performance regardless of the teaching modality, and in low-credit courses, students generally did better in the online teaching modality. Regarding exam settings, no significant difference was observed in students’ performance.
CONCLUSION
With few studies examining the variations between F2F and online teaching, the current study in the UAE is the first study comparing academic performance during the COVID-19 era. Our findings illustrate no significant difference between both online and traditional F2F learning. Therefore, given the considerable potential of alternative teaching modalities, future studies are encouraged to explore the efficacy of online learning and proctoring methods.
Introduction
As the COVID-19 outbreak originated in the Republic of China, it rapidly spread across the globe within a few months, eventually being declared a pandemic on March 11, 2020. In response, numerous countries implemented stringent measures, such as lockdowns, to protect their communities and mitigate the virus' spread. Consequently, the educational sector, including medical schools, was compelled to vacate their physical classrooms and transition from the traditional face-to-face (F2F) instructional model to an online format. 1 This challenge presented a significant hurdle for educational institutions, with limited guidance on how to navigate the situation. In Dubai, for instance, online teaching became the norm for all schools and universities during the peak of the COVID-19 cases.
Medical schools were in a challenging position due to the atypical nature of their curricula, which usually involve hands-on practical training that is not easily adaptable to an online environment. A cross-sectional survey study conducted in Saudi Arabia among medical students revealed that online teaching is perceived as less effective across six parameters, including developing skills and acquiring knowledge. 1 Similarly, in Jordan, the overall satisfaction rate in medical distance teaching was 26.8%. 2 In Iran, on the other hand, a cross-sectional study found that overall students’ average scores increased significantly during e-learning versus in-person education. 3 A study in Pakistan found the end-of-year summative objective structured clinical examination (OSCE) to be displaying mixed results, and they suggested a blended learning environment for teaching clinical skills. 4 Other studies that compared traditional F2F teaching modality with online learning, found no statistical difference in students’ performance. 5 Consequently, medical schools had to rearrange the dynamics of their educational curricula from the traditional platforms to ensure that students continue their education without any disruption. 6
There are different types of exam proctoring, the in-famous being the traditional F2F proctoring modality, where invigilators directly observe students during exams. With the COVID-19 pandemic, universities had to introduce a new way to monitor online exams using remote proctoring systems. 7 These systems serve to verify students’ identities and detect any suspicious behavior during the examination process. Remote proctors can observe students through their webcams, listen to them via their microphones, and monitor their screens to ensure the integrity and honesty of the exam. However, it is important to recognize that some students have reported heightened levels of anxiety because of various factors associated with remote proctoring. These factors include a sense of constant surveillance due to the presence of a camera, concerns regarding potential technical issues, difficulties arising from unreliable internet connections, and disruptions caused by ambient background noise. These subjective elements introduce an additional layer of complexity to the overall examination experience.8,9
Despite the available literature, it is still unclear whether online teaching has a significant impact on medical students’ academic performance. Additionally, no studies have examined the potential influence of exam settings, specifically in terms of exam proctoring and administration, on students’ academic outcomes. Hence, although other studies in the literature compared different teaching styles,2,3 we identified a gap regarding how exam proctoring influences students’ academic performance, as measured by their grade point average (GPA).
The intent of this study is to compare the academic performance of pre-clinical medical students in online versus F2F teaching and proctoring setting among medical students enrolled in the Bachelor of Medicine and Bachelor of Surgery (MBBS) at Mohammed Bin Rashid University of Medicine and Health Sciences (MBRU). Academic performance will be evaluated using the GPA at the end of the semester, since it is a helpful measure in grasping student ability and generating objective performance measurements. 3
While other studies did not explore the impact of credit hours on the course's end-of-year grade when teaching was shifted remotely, we aim to observe the impact of credit hours on the course grades when online teaching was applied. To the best of our knowledge, this is the first study conducted in the Middle East focusing on both teaching and exam proctoring styles on medical students of the bachelor's level.
Several comparisons are made to answer two main questions: Is there a difference in students’ performance if teaching is done F2F or online, and is there a difference in students’ performance if exam settings are conducted F2F or remotely?
Methods
This is a retrospective analytical cohort study carried out at Mohammed Bin Rashid University of Medicine and Health Sciences (MBRU) from 2020 to 2021. The subjects of the study are pre-clinical medical students at Mohamed Bin Rashid University of Medicine and Health Sciences in Dubai (MBRU) in Dubai – United Arab Emirates (UAE). The inclusion criteria were students above the age of 16 years, having access to internet, enrolled as a year-one medical student at MBRU. The exclusion criteria was students below the age of 16, lacking internet access, and dropping out of the program during the first year. Most of the students are high school graduates, with ages that range between 17 and 19 years, with 70% of students being females. Since the employed subjects were first-year medical students, they had no prior experience with online teaching as high schools in the UAE did not offer online teaching. However, due to the UAE being among the top countries with internet access, all students had internet and were able to access the online material without major obstacles that could hinder the students’ overall learning experience. Furthermore, most students at the university were graduates from UAE high schools and were equally accustomed to the use of technology. The core of this study is to compare the course grades among different sub-groups of teaching and proctoring modalities.
Convenience sampling was used in this study. The study sample included all year-one medical students from the university. Our study employed three distinct cohorts: cohort 1 (academic year 2018-2019), cohort 2 (academic year 2019-2020), and cohort 3 (academic year 2020-2021). The study subjects were categorized into 10 sub-groups (labeled as A, B, C, D, E, F, G, H, I, J) based on differences in teaching and proctoring methods, as shown in Figure 1. Using the GPA as a metric for academic performance, there was no missing data. Notably, comparisons between Groups A and B, as well as Groups C and D, were drawn to evaluate the didactic approach under F2F proctoring settings. Alternatively, the comparison of Groups E and F aimed to assess the impact of different teaching methods in an online proctoring setting. Moreover, comparisons between Group G versus H and Group I versus J were done to elucidate distinctions stemming from the proctoring modality. It is noteworthy that the students shared the same course enrollment during each semester, in accordance with the university's curriculum.
A simplified diagrammatic scheme of the sub-groups included in the analysis for F2F versus remote teaching and proctoring.
Students’ grades from different courses were utilized to evaluate their performance. Thus, nine courses were included in the study from the first semester of the first year. The content of the courses has been briefly described as well as the total number of students (N) that took each course in Table 1.
A list of all year-one courses analyzed in the study with a brief description of the content material total number of students that took the course (n = F2F learning and online learning).
Statistical analysis
Grades are derived from tests, homework, class participation, and research project scores designated toward each course. 10 Following the extraction of the raw data, final course grades were then converted into numerical variables such that a grade of A is equivalent to a course score of 4, A− to 3.7, B+ to 3.3, B to 3, B− to 2.7, etc… and analysis was then done using the IBM SPSS statistical software version 24. For data collection, only the faculty could access them, and the anonymity of data is ensured.
Ethical approval
Ethical approval from the MBRU institutional review board has been granted (IRB approval number: MBRU/CM/DO/L-083/2019-05-29). The IRB classifies such work as “IRB exempt” since there is no need for informed consent from the study subjects due to the pre-existing nature of the data, the anonymity of the participants, and the lack of direct human interaction. This study adhered to the tenets of the Declaration of Helsinki.
Results
In total, five comparisons were done, three of them were to evaluate students’ performance in online teaching versus F2F teaching, while the other two were done to evaluate the students’ performance in online exam proctoring versus F2F exam proctoring. Within each set of comparisons, two to four courses were used to measure significance using the two-tailed student t-test. Groups A and B had the courses shown in Table 2, while Groups C and D compared the courses shown in Table 3.
Comparison of the mean course grades for F2F teaching (Group A) versus online teaching (Group B), both having F2F proctoring.
F2F: face-to-face, SD: standard deviation, SE: standard error.
Comparison of the mean course grades for F2F teaching (Group C) versus online teaching (Group D), both having F2F proctoring.
F2F: face-to-face, SD: standard deviation, SE: standard error.
Groups E and F had different courses. Group E had limbs and spine, and thorax while Group F had head and neck, abdomen, pelvis, and perineum as shown in Table 4. The reason for having different courses is the lockdown circumstance in the country.
Comparison of the mean course grades for F2F teaching (Group E) versus online teaching (Group F), both having online proctoring.
F2F: face-to-face, SD: standard deviation, SE: standard error.
Groups G and H had the courses shown in Table 5, while Groups I and J had the courses shown in Table 6. The P-value was considered statistically significant if less than or equal to 0.05. Finally, two mean values for each course were generated, one for online teaching/proctoring and one for F2F teaching/proctoring to identify which method of delivery is better or worse if significance is present.
Comparison of the mean course grades for online proctoring (Group H) versus F2F proctoring (Group G), both having online teaching.
F2F: face-to-face, SD: standard deviation, SE: standard error.
Comparison of the mean course grades for online proctoring (Group J) versus F2F proctoring (Group I), both having F2F teaching.
F2F: face-to-face; SD: standard deviation; SE: standard error.
Two principal questions were studied in this research, one regarding teaching delivery mode and the other regarding exam proctoring environment. Our data revealed that for one course (Fundamentals of Epidemiology and Biostatistics), there was a significant difference where students performed better when teaching was done online, however, when the study was repeated with another set of students, statistical difference was not observed. Data also revealed that students did better in the Thorax course which is taught in the second semester when it was delivered online. The other courses such as Foundations of Medical Sciences, Enzymes and Metabolism, Abdomen, Pelvis, and Perineum, which are considered major courses with high credits, had no difference in students’ performance. Regarding exam settings, there was no significant difference observed in students’ performance if exam proctoring was done F2F or remotely. The online sample size of each group and course are listed in Tables 2–6.
The first research question explores whether there is a difference in students’ performance if teaching is conducted F2F or online. If any significant difference is detected, which mode is better? Tables 2 and 3 show similar comparisons where the teaching mode was different, once online, and once F2F with the exam proctoring setting being F2F. The only difference was the semester. Table 2 involves four courses from the first semester, whereas Table 3 involves four courses from the second semester. As shown in Table 2, there is statistical significance in both courses of Fundamentals of Epidemiology and Biostatistics with a P-value of 0.04 where the students performed better when teaching was delivered online. Students also performed better with online teaching in the English for Health Sciences course with a P-value of 0.01, as well as Thorax with a P-value of 0.01. The remaining courses showed no significant differences in students’ performance between online and F2F teaching. Consequently, Table 4 compares students’ performance in online teaching versus F2F teaching when both groups had exam proctoring done remotely. The final course grades of the first group that undertook both courses of Limbs and Spine, and Thorax are compared with the second group that took the Head and Neck course, as well as the Abdomen, Pelvis, and Perineum course. The final course grades showed no significant difference.
The second research question is aimed to explore whether there is a difference in students’ performance if exam settings are conducted F2F or remotely. Table 5 shows students’ performance during F2F exam proctoring and online exam proctoring with both groups undergoing online teaching. No significant difference was observed in both courses. Moreover, Table 6 shows students’ performance between F2F exam proctoring and online exam proctoring with both groups undergoing F2F teaching. There is no significant difference observed between both courses.
Discussion
The current study examined pre-clinical medical students’ academic performance in F2F teaching and online teaching, in addition to variations present between online proctoring versus traditional F2F proctoring. There was no significant difference found in academic performance between the two teaching modalities. However, it was observed that in some low-credit courses, students generally performed better during online teaching. As for exam settings, there was no significant difference noted in students’ performance if exam proctoring was done F2F or remotely.
Regarding the teaching modality: the course content, tutors, and exam settings were chosen to be similar in both teaching modalities to ensure that the only outcome being studied is the teaching delivery mode. Likewise, for the exam environment, all variables including the teaching delivery method were kept similar and were chosen accordingly so that the only difference was the proctoring style.
Most of the studies investigating the difference between online and F2F teaching used surveys to detect the perception of students and faculty regarding the shift of teaching modality. A study from a South African University used students’ survey responses and compared them with pre-lockdown and post-lockdown assessment grades. The study reported lower grades for students who expressed a preference for self-study over assisted study. 11 The main contributor to poor academic performance during the transition was poor internet access, which highlights the importance of affordable internet access and infrastructure. 12 Access to the internet was not an issue in our study as all students had internet access and information technology (IT) support was offered to all students.
Several studies have employed alternative measures to assess students’ academic performance in online and remote teaching settings. For example, one study conducted in Pakistan utilized the OSCE. 13 Similarly, another study examined the students’ academic performance using the National Board of Medical Examiner (NBME), revealing no statistically significant differences. 14 However, it is important to acknowledge that these measures do not encompass other aspects of assessment, such as projects, assignments, and the OSCE, which collectively contribute to the calculation of the GPA. Therefore, incorporating these additional elements provides a more comprehensive and realistic measure of students’ performance.
A study on medical residents and speech-language pathology graduates compared the effect of F2F teaching versus online, interdisciplinary, and interactive teaching on pre-and post-course knowledge test scores. The study found that online teaching yielded comparable gains to traditional teaching with no significant difference. 15 The findings were similar to our study, although they used data from only one assessment component. One study of eight years duration compared students’ performance in an Environmental Science course and found no significant difference between online and F2F learning. 10 Subsequently, another study involving 64 psychology students compared both teaching modalities and their preferences. The study showed that although students generally preferred to complete activities in a F2F manner, there was no significant difference in their academic performance. 16 It is thus concluded that online and F2F activities can lead to similar academic results which is in line with our findings. Some studies have illustrated improved learning outcomes in the context of online learning. 3 In our study, all courses except low credit courses had no significant difference in academic performance regardless of the teaching method. The difference in academic performance when accounting for the course credits can be pertained to the fact that low-credit courses tend to be less demanding, allowing students to learn at their own pace. Alternatively, high-credit courses typically require more attention and F2F interaction between students and instructors.
Other studies notably demonstrate that there may be minor differences between the two modes of teaching. Many empirical comparisons between online and F2F courses found that online students perform as well as or better than F2F students. 17 The differences in outcomes challenge the idea that teaching can only be delivered via the traditional F2F method once again.
Some may argue that non Science, Technology, Engineering and Mathematics (STEM) subjects are easier to teach via an online setting which cannot be compared to Science, Technology, Engineering and Mathematics (STEM) studies like ours. However, a study that compared students’ academic performance in a biology course, demonstrated the ability to translate traditional F2F teaching to an online environment effectively. 18 In six of the eight comparisons conducted, no significant difference in academic performance was noted, and students performed better in only two comparisons when teaching was online. This supports the results of our study pertaining to the course of Fundamentals of Epidemiology and Biostatistics where students in the first comparison performed better online in accordance with Table 2, while there was no difference in their overall performance in the same course in the second comparison shown in Table 3.
With regards to the study's second research question assessing how students’ performance was affected when exams were done off-campus with online proctoring, there was no significant difference. Traditionally, exams were done strictly in a F2F setting. MBRU ensured that students were divided into small groups and allocated to specific exam venues where at least two invigilators were dedicated to proctor students throughout the entire exam. Every student would be appointed to a specific seat that ensured sufficient distance between each other. The list of students and the venues would then be changed the next day and shuffled accordingly, repeating the same process in every exam to ensure optimal fairness.
Given the pandemic, most institutions, including MBRU, had to shift their exams remotely. Although many typical learning and teaching transactions can be managed using modern online learning technologies, closed-book exams can be quite difficult to implement especially in a short period of time and involving a large number of students. 19 Innovative techniques had to be used to ensure that students’ grades reflected their learning, paving the way for the usage of proctoring software with audio and video monitoring. At MBRU, exams scheduled during the COVID-19 pandemic were done remotely using the Examplify software (version 2.3.2) platform, which utilized picture verification before starting the exam ensuring optimal transparency. A study that was done comparing the traditional assessment versus online assessment revealed that there were no differences regarding the academic performances in accordance with the grades on the test between the two groups of students, however, it was reported that student anxiety before the exam was higher in the online group. 20 On the contrary, the same study reported a higher satisfaction rate after performing the exams in the online group compared to the traditional F2F exam setting.
This disparity brings about the implication of this study: certain factors could be attributed to the difference in students’ learning. Regarding the delivery mode of teaching, instructor teaching methods should always be considered when assessing students’ performance in both the online and F2F modalities. A study demonstrated that the instructor's quality is the most prominent factor that affects the students’ satisfaction during online classes. 21 In other words, the instructor's enthusiasm, and ability to design, plan, and deliver the course's content influences both the students’ satisfaction and performance. Thus, dedicating an ample amount of time to educating instructors and facilitators on how to effectively deliver online materials and endeavor students’ engagement is extremely vital as students’ perspective and ability to handle teaching either F2F or online play an important role in their overall performance. The literature regarding students’ perspectives on online teaching was overall positive during the COVID-19 pandemic where the majority of participants perceived the sessions as relevant to their learning needs.22,23
A study illustrated that academic performance is unlikely to be influenced by students’ preference of the teaching modality. 16 Importantly, the findings also imply that asking students to engage in their non-preferred modality does not necessarily mean that the students’ performance will be affected negatively. Similarly, regarding F2F proctoring versus online proctoring, students initially found the experience challenging and uncomfortable, 19 however, they generally felt better following the exams and their scores revealed that it was as effective as the F2F proctoring. Universities and schools, in accordance with the instructors, should put in a greater amount of effort to make sure the students fully comprehend the instructions before sitting the exam.
According to the American Psychological Association (APA) Work Group of the Board of Educational Affairs, the learner must ensure that he/she is willing to put in the time and effort to grasp the material and to be as effective and efficient as possible, 24 and if the students have any queries, they should dedicate the appropriate time to ask the instructor for further clarification. 25 Therefore, it is important for both the instructor and the student to take responsibility in both cases of teaching modalities and to use online teaching to the students’ advantage which will in turn positively impact their performance.
The limitations of this study revolve around several factors. Firstly, the use of convenience sampling resulted in a sample limited to undergraduate medical students from a single university. Nonetheless, a notable constraint of this study resides in the relatively small size of the sample study population due to the nature of our setting. Consequently, the potential for generalizability within the country and the broader region might be limited. Furthermore, most of the comparisons performed in this study involved different students in the control and exposure groups given that those were the only available matches for comparison at the time. Another limitation of the study is that it did not consider the students’ perception of online versus F2F teaching.
Some of the confounding factors include minor changes in the instructors’ performances across the year, students’ morale to learn during the pandemic, and a general feeling of insecurity during a global pandemic which affects students’ performance. The results of the study should not be influenced by the limitations mentioned, and further studies are encouraged to take place in the future to better aid the understanding of students’ performance and to continue research in the two modalities focused upon in this study.
Further research is warranted to explore the impact of online teaching on students, particularly in disciplines that necessitate hands-on experience, like medicine. Additionally, future studies should compare blended learning approaches with both F2F and online teaching methods. Although this study specifically examined pre-clinical medical students, it is crucial for future research to investigate the effects of online teaching on medical students during their clinical phase.
Conclusion
This study compares the effectiveness of online teaching and online proctoring with traditional F2F teaching. Despite the significant impact of the COVID-19 pandemic on the education sector worldwide, our data demonstrate that the traditional F2F modality of both teaching and exam proctoring could be transformed into an online platform if sufficient attention is displayed from both the instructor and the student. Moreover, the results of this study provide a glimpse of how online teaching and proctoring might be as effective as F2F education. This approach holds considerable potential and warrants further exploration and validation in future studies.
Footnotes
Acknowledgments
We acknowledge Ms. Hanan Faraji for proofreading the manuscript and English language editing.
Author Contributions
RK wrote the main text of the paper, conducted data collection and analysis. SA revised the paper critically and edited it, prepared tables. II is responsible for the conception of the original idea, design, and data collection.
Availability of Data and Materials
The datasets analyzed for this study are available upon request from the corresponding author.
DECLARATION OF CONFLICTING INTERESTS
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical Approval
Ethical approval from the Mohamed Bin Rashid University of Medicine and Health Sciences in Dubai (MBRU) institutional review board has been granted (IRB approval number: MBRU/CM/DO/L-083/2019-05-29). This study adhered to the tenets of the Declaration of Helsinki.
FUNDING
The authors received no financial support for the research, authorship, and/or publication of this article.
