Sage Journals: Discover world-class research

Abstract

OBJECTIVES

Over the years, medical schools have evolved their curricula in response to the medical field, faculty, and students. The current study aims to examine how medical students study, what resources they most commonly use, and how it relates to United States Medical Licensing Examination (USMLE) Step 1 scores.

METHODS

A cross-sectional survey study of United States medical students was distributed via social media, GroupMe school chats, and school listservs from September 8, 2020, to December 12, 2020. The survey gathered data including, demographic and school information, resources students, time spent using each resource, and USMLE Step 1 scores.

RESULTS

The survey was completed by 560 students from 102 different United States medical schools. Study guides as online resources (83.2%) were mostly used, lecture (82.5%), Anki (spaced repetition flashcards, 68.3%), and school-organized sessions (workshops, labs, and small groups) (60.7%). Of the students surveyed, 90% attended schools with a recorded lecture option. Only 54% of these students watched their lectures live. When watched online, the average watching speed was 1.75 ± 0.4× with a mode of 2× speed. In examining different medical school styles, schools with a 1.5-year preclinical curriculum had higher USMLE Step 1 scores (244.5 ± 15.6 vs 236.9 ± 16.2, P-score = .024) compared to schools that did not (1- or 2-year preclinical curriculum).

CONCLUSION

Medical students seem to be using third-party resources in addition to their medical school curriculums. Because students are already broadly using these, medical schools could allow their curriculums to reference, require, or complement third-party online resources. Additionally, because of their increased cost, medical schools could consider ways to alleviate the cost on medical students as a means of equitable support. Lastly, Step 1 scores significantly correlated with schools with a 1.5-year preclinical curriculum, although the exact reasoning for this remains uncertain.

Keywords

Medical education study habits medical school curriculum survey-based study

Introduction

Over the years, medical schools have evolved their curricula in response to the medical field, faculty, and students. Several factors have been clear influences on the evolution of medical school curricula: evolving technology,^1,2 increased emphasis on early clinical exposure,² the integration of problem-based learning,³ and the increasingly common integrated approach to the medical curriculum.⁴ Understanding the evolving educational process in the COVID/post-COVID era, including how medical students learn during their preclinical coursework is of interest to better understand.

As healthcare education has evolved, academic institutions have incorporated new topics and teaching methods including the flipped-classroom model,⁵ bioethic courses,⁶ and an increased flexibility in student schedules by making lecture attendance optional.⁷ Medical schools have also noted that students are reporting high rates (>27%) of burnout and depression.^8-10. In an effort to improve student mental health and increase collegiality and collaboration between medical students, many schools have shifted their grading system from numerical grading to tiered honors/high pass/pass/fail systems to now strictly pass/fail systems.^11,12

Now medical education is evolving yet again. Use of historical print and online textbooks are less common as medical students now seem to be turning to third-party resources focused on “higher-yield” preparation to supplement and sometimes replace finite portions of the standard medical school curriculum. In addition, according to the 2020 AAMC Year Two Questionnaire, 37.0% of second-year medical students reported “almost never” attending in-person pre-clerkship courses or lectures, a steep increase from 2014 when only 16.8% of second-year medical students did so.^13,14 Meanwhile, medical students using third-party online medical education videos on a “daily” basis jumped from 9.1% in 2014 to 40.9% in 2020.¹²

One of the most popular third-party resources include “Anki,” a digital program that uses flashcards to teach content through spaced repetition¹⁵ where thousands of free, premade flashcards have been created by past medical students and are widely accessible. “Boards and Beyond” is another popular resource with a well-organized video platform covering all areas of traditional medical education emphasizing high-yield content.¹⁶ “Sketchy Medical” is another video series using illustrations and pictorial memory devices to help students memorize preclinical and clinical information.¹⁷ These are just a few of the wide selection of resources that medical students can choose.

Recently students have chosen to forgo the classroom, preferring to watch lecture recordings and use outside resources at home.^13,14,18 Prevalence of third-party resource usage has been reported as high as >80%.¹⁹ This has created pressure to use these resources, often at the expense of the classroom's curriculum, as students fear they may fall behind their peers if they do not.²⁰ The cost of these resources should also be considered, as students are often purchasing these resources in addition to their medical school tuition (Boards and Beyond $249/year, Sketchy Medical $399/year, Physeo $249/year, OnlineMedEd $365/year, Osmosis $378/year, as of December 2021), creating an additional burden on students and barriers to those who are unable to afford the resources.

Given the recent increased usage of third-party outside-of-school resources in medical school and the decreased attendance in lecture, the purpose of the current study aimed to better understand how medical students currently study and interact with internal (school) and external curricular resources. To do so, the current study aimed to examine and quantify what resources are most commonly used, how much time students spend on each resource, and how this relates to United States Medical Licensing Examination (USMLE) Step 1 scores as this is the standardized and consistent metric across schools.

Methods

Study design

The current study used a cross-sectional research design to survey United States’ medical students between September 8, 2020 and December 12, 2020. During this time period, the survey was sent out to medical students of all class years except first years. The social and professional networks of two medical student authors (SH and JZ) were utilized to recruit students to participate in the survey. Methods used to send out the survey include school listservs, GroupMe Chats, Facebook groups, Instagram, and medical student Reddit groups.

The survey was administered using Qualtrics Research Suite, which allows researchers to de-identify responses while building, distributing, and analyzing research surveys online. The authors’ Institutional Review Board approved this study for exemption. Written informed consent was obtained using an approved consent letter before starting the survey and participants were able to terminate their participation at any time.

Survey questions

Data collected from the survey (Supplement S1) included basic demographic questions including age, gender identity, school, and year of study. Furthermore, the study collected information on the style of preclinical curriculum the school used. The questionnaire was tested and developed by the authors.

When asking about the student's study strategies, the survey inquired about resources used (including online or school-based resources), lecture usage, and the number of hours spent on each study resource. If students had taken USMLE Step 1, they were asked their score.

Statistical analysis

Statistical analysis was performed using Qualtrics Research Suite and STATA. Values were reported as mean ± standard deviation (SD). Student's T test was used to compare Step scores between study groups. The significance level was set at a = .05.

Results

Respondent characteristics

The survey was completed by 560 students from 102 different medical schools in the United States. Respondent information is delineated in Table 1. As the survey was sent out via social media platforms and school list serves it was not possible to discern how many people may have seen the posts and thus the response rate cannot be reported.

Table 1.

The demographic of the participants in the questionnaire.

Participant demographic
Total participants	567
Agreed to consent	560 (99%)
Age (median ± SD)	25 ± 3
Sex
Female	48.5%
Male	49.0%
Other	2.6%
Class year (by %)
M2	57.9
M3	20.3
M4	16.1
PhD	1.9
Other	3.7
Preclinical length (by %)
2-Year	51.2
1.5-Year	44.9
1-Year	4.0

The average student age was 25 ± 3 years old. In terms of gender, 48.5% identified as female, 49.0% identified as male, and 2% identified as other. In terms of medical school year, the study cohort was 57.9% second years, 20.3% third years, 16.1% fourth years, 1.9% in their PhD years, and 3.7% in their research year/second degree year/another year. Regarding their schools, 2-year preclinical curriculums were reported by 51.2%, 1.5-year preclinical curriculums were reported by 44.9%, 1-year preclinical curriculums were reported by 4.0%.

Survey responses

Student resources used are shown in Table 2. The mostly used resources (in decreasing order) were: online resources (83.2%, average 13.0 ± 11.5 hours/week), lectures (82.5%, average 18.5 ± 13.1 hours/week), Anki repetition flashcards (68.3%, average17.8 ± 14.9 hours/week), and school-organized sessions excluding lectures (60.7%, average 8.0 ± 8.4 hours/week).

Table 2.

The resource utilization data among survey participants.

Medical Resources	Number of Students	Number of Hours Spent on Average
Online resources^a	347 (83.2%)	13.0 ± 11.5
Lecture	344 (82.5%)	18.5 ± 13.1
Anki	285 (68.3%)	17.8 ± 14.9
School-organized sessions	253 (60.7%)	8.0 ± 8.4
First aid	242 (58.0%)	6.1 ± 9.2
Practice questions	204 (48.9%)	8.4 ± 12.2
Textbooks	171 (41.0%)	9.2 ± 12.8
Study groups	125 (30.0%)	8.7 ± 12.4
Reading journal articles	40 (9.6%)	5.5 ± 13.7
Other	16 (3.8%)	2.8 ± 3.7
Total	417

Online Resources noted included: Boards and Beyond; OnlineMedEd; Osmosis; Pathoma; Physeo; Pixorize; Pathoma; SketchyMedical.

Table 3 shows the breakdown of how students watch lecture recordings. Of these, 90% of the students surveyed attended schools with a recorded lecture option. Of those students, 54% ± 38 watched lectures live. Interestingly, students who watched lectures online watched at an average of 1.75 ± 0.4× speed. And the most common watching speed was 2× speed.

Table 3.

The data on lecture-watching habits.

Variable	Value
Percent with recorded lecture	89.8
Percent of lectures watched live (only the recorded population sampled)	54 (38)%
Mean watching speed of recordings	1.75 ± 0.4× speed
Mode watching speed of recordings	2× speed

Correlation with USMLE Step 1 scores

Regarding USMLE Step 1, 116/560 (20.7%) students had already taken USMLE Step 1 (8 students preferred not to share their score). The average step score was 243 ± 17.8. Most curriculum factors were not significantly associated with step scores: pass/fail curriculum, use of specific study aides, study groups, or medical journal articles (P > .05 for each). Notably, schools with a 1.5-year preclinical curriculum had higher USMLE Step 1 scores (244.5 vs 236.9, P-score = .0237) compared to schools that did not.

Discussion

The current study aimed to examine how medical students study and what resources (school or third-party) they most commonly use in the COVID era during the preclinical curriculum. A survey was developed, distributed, and completed by 560 medical students from over 100 schools around the United States. The use of varied study aids was assessed.

Online resources were reported to be the most used by the surveyed medical students (83.2%) at 13.0 ± 11.5 hours per week. The high rate of usage matches the >80% online resource usage rate found by Kauffman et al¹⁹ who found that >80% of students were using four or more online resources and that there was no correlation between class attendance and a student's USMLE Step 1 score.¹⁹

Lectures were also utilized by 82.5% of respondents at 18.5 ± 13.1 hours/week and school sessions (excluding lectures) were used by 60.7% of respondents at 8.0 ± 8.4 hours/week. This suggests that despite a reduction in class attendance experienced at medical schools nationwide, many students still use both lecture/school sessions in conjunction with the above-noted online resources together. This breakdown of lectures and school sessions likely allows for students to continue to learn by different means (didactic vs small group) and our data reflects that the majority of medical student responding still give critical value to learning their institutions curriculum.

In further analyzing students’ lecture routines, the survey revealed that about half watch their lectures live versus in person. Of the students who watch their lectures online, the average watching speed was 1.75 ± 0.4 times speed and the most common watching speed was 2 times speed. These fast-watching speeds seem to be commonplace in medical schools and indicate that students are aiming to maximize their efficiency in their learning. Interestingly, a recent study by Song et al (2018) found that medical students at the University of Kentucky who listened to podcasts at an accelerated speed (1.5×) displayed poorer immediate retention compared to medical students who listened to podcasts at a normal speed (1.0×).²¹

Almost 68% of the students surveyed use a program called “Anki,” which is a spaced repetition flashcard program that allows students to easily share their ready-made flashcards at 17.8 ± 14.9 hours per week. These findings further highlight the varied resources medical students use in attempting to study how they learn best. Other resources utilized by medical students, include Boards and Beyond, SketchyMedical, Pathoma, Pixorize, and more (Table 2). Four of these resources would easily cost >$1250 per year, in addition to medical student tuition. These costs can present barriers to students from low- and mid-socioeconomic backgrounds who are unable to afford desired resources.

Traditionally course learning objectives referred to a textbook often written by authors within, or external to, the institution along with journal articles relevant to a session. In the setting of online disruptive technology frequently being utilized to enhance student learning, we are witnessing a paradigm shift in education and the time is ripe for medical schools to pivot their referenced materials to complement the ones utilized more regularly by the current generation of learners. The options of online medical education third-party resources have steadily increasing over the past two decades (Table 4). These innovative online contents were cofounded, championed, or adapted for medical student use, by medical students, residents, physicians, and medical educators who received training from institutions including Cornell, Harvard, Johns Hopkins, Northwestern, Universities of Cambridge, Chicago and Hamburg, University of California at Irvine, Los Angeles, San Diego and San Francisco, Vanderbilt, and Yale.

Table 4.

The different commonly used resources among medical students and their founders.

Resource; Year Founded	Founder(s)
First Aid for the USMLE; 1990	Vikas Bhushan, MD
Uworld; 2003	Chandra Pemmasani, MD
Anki; 2006	Damien Elmes
USMLE-Rx (Scholar-Rx); 2006	Tao Le, MD, MHS
Pathoma; 2011	Husain Sattar, MD
AMBOSS; 2012	Benedikt Hochkirchen
	Nawid Salimi, MD
	Madjid Salimi, MD
	Jan Sievert Weiss, MD
Osmosis; 2012	Shiv Gaglani, MBA
Osmosis; 2012	Ryan Haynes, PhD/MPhil
SketchyMedical; 2012	Andrew Berg, MD
	Aaron Lemieux, MD
	Bryan Lemieux, MD
	Saud Siddiqui, MD
Boards and Beyond; 2014	Jason Ryan MD, MPH
OnlineMedEd; 2014	Dustyn Williams, MD
Physeo; 2016	Steve Harding
	Michael Christensen, MD
	Rhett Thomson, MD
	Zach Thomson
Pixorize; 2018	Nathan Liu, MD
Pixorize; 2018	David Westfall, MD

To better adapt with the current medical students at our beloved schools we need to make meaningful efforts to close the gap between the medical school curriculum and the way students supplement material online. Rather than shun new technologies and methods, the time has come to embrace them. Looking around the corner and pondering options on the path before us, schools could consider (1) increasing the financial aid allowance for books and general supplies allowing students to better shunt a portion of this budget towards online resources. (2) Alternatively, if schools require select online resources, financial aid offices could earmark specific funds towards them. (3) Lastly, schools could provide preferred online resources for students akin to how e-textbooks are provided by schools through AccessMedicine and ClinicalKey.

Of importance, USMLE Step scores were also considered by the survey. Although USMLE Step 1 has recently gone to Pass/Fail, the data collected here is thought to be a rough marker of information learned and readiness to advance in training. Most curriculum factors were not significantly associated with step scores (pass/fail curriculum, use of specific study aides, study groups, or medical journal articles) but schools with a 1.5-year preclinical curriculum had higher USMLE Step 1 scores compared to schools that did not. This analysis may support the move medical schools are making from a 2-year curriculum to a 1.5-year curriculum. The LCME reported that 29% of US allopathic medical schools already used a 1.5-year curriculum compared to 56% with a 2-year curriculum during the 2019-2020 academic year.²²

No correlation between third-party resource usage and USMLE Step 1 score or between lecture attendance and USMLE Step 1 score was found. While this may intuitively be surprising given that third-party resources are focused on USMLE Step preparation, the findings of the current study mirror those of Hoyo et al (2020) and Kaufmann et al (2019).^19,23 Although these findings were consistent with prior studies, schools are not necessarily teaching specifically for Step 1. This means that schools should not automatically decide to change how they teach because they may be focusing on more intangible topics that are helpful in becoming a physician. These findings do provide a contrasting perspective to the work of Lu et al (2021), who found that higher Anki usage was associated with higher USMLE Step 1 scores.²⁴

Given these findings, the question remains on why students continue to turn to third-party resources. Likely, this is due to medical students wanting to high-yield and concise information to increase efficiency focused on the USMLE Step 1 exam. Further, most of the online resources continue to be updated with medical student feedback. Also, medical students are likely preoccupied with their USMPLE Step exams pushing them to want only the most concise high yield information with memory hooks to make it easier. Even with a few online resources students can try to optimize their learning in different ways. For example, students can use Boards and Beyond to make sure they have seen nearly all of the information once, Pathoma to focus on higher yield topics in pathology, and SketchyMedical to help memorize all of their microbiology and pharmacology.

As with most survey/questionnaire style studies, the current study has its limitations. Most notably, response rate cannot be defined for the current study based on the method of survey distribution. While a limitation that could be associated with response bias, the methodology was selected to reach the maximum number of medical students possible and to achieve the greatest possible number of responses. Because of our methods for achieving the greatest number of responses, a power calculation was not performed. Another limitation is that the questionnaire was not validated, but it was developed by the current authors. Also, because there was no way to confirm a student's Step 1 score, there can be a potential subjectivity in Step 1 score reporting. Further, there may be variables that confound each other such as variability in which medical schools have gone to 1.5-year curricula or use of multiple study aids.

Conclusions

In the current times, medical students seem to be using third-party resources more commonly to supplement and even replace limited portions of their medical school curriculums. Because students are already broadly using these, medical schools could allow their curriculums to reference, require, or complement third-party online resources. Additionally, because of their increased cost, medical schools could consider ways to offset or alleviate the cost on medical students as a means of equitable support.

Supplemental Material

sj-docx-1-mde-10.1177_23821205241228455 - Supplemental material for Are Medical School Curricula Adapting With Their Students? A Survey on How Medical Students Study and How it Relates to USMLE Step 1 Scores

Supplemental material, sj-docx-1-mde-10.1177_23821205241228455 for Are Medical School Curricula Adapting With Their Students? A Survey on How Medical Students Study and How it Relates to USMLE Step 1 Scores by Scott J. Halperin, Justin R. Zhu, John S. Francis and Jonathan N. Grauer in Journal of Medical Education and Curricular Development

Supplemental Material

sj-docx-2-mde-10.1177_23821205241228455 - Supplemental material for Are Medical School Curricula Adapting With Their Students? A Survey on How Medical Students Study and How it Relates to USMLE Step 1 Scores

Supplemental material, sj-docx-2-mde-10.1177_23821205241228455 for Are Medical School Curricula Adapting With Their Students? A Survey on How Medical Students Study and How it Relates to USMLE Step 1 Scores by Scott J. Halperin, Justin R. Zhu, John S. Francis and Jonathan N. Grauer in Journal of Medical Education and Curricular Development

Footnotes

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Scott J Halperin (Jane Danowski Weiss Family Foundation Fund); Jonathan N Grauer (North American Spine Society Journal Editor-in-Chief).

Supplemental material

Supplemental material for this article is available online.

References

Norman

. Medical education: past, present and future. Perspect Med Educ. 2012;1(1):6-14. doi:10.1007/s40037-012-0002-7

Basukala

Chaudhary

. Early clinical exposure in preclinical years of medical school. JNMA J Nepal Med Assoc. 2021;59(242):1072-1074. doi:10.31729/jnma.5341

Neufeld

Norman

Feightner

Barrows

. Clinical problem-solving by medical students: a cross-sectional and longitudinal analysis. Med Educ. 1981;15(5):315-322. doi:10.1111/j.1365-2923.1981.tb02495.x

Brauer

Ferguson

. The integrated curriculum in medical education: AMEE Guide No. 96. Med Teach. 2015;37(4):312-322. doi:10.3109/0142159X.2014.970998

Williams

. The future of medical education: flipping the classroom and education technology. Ochsner J. 2016;16(1):14-15.

Kirch

Gusic

Ast

. Undergraduate medical education and the foundation of physician professionalism. JAMA. 2015;313(18):1797-1798. doi:10.1001/jama.2015.4019

Lamb

Chow

Lindsley

, et al. Learning from failure: how eliminating required attendance sparked the beginning of a medical school transformation. Perspect Med Educ. 2020;9(5):314-317. doi:10.1007/s40037-020-00615-y

Dyrbye

West

Satele

, et al. Burnout among U.S. medical students, residents, and early career physicians relative to the general U.S. population. Acad Med. 2014;89(3):443-451. doi:10.1097/ACM.0000000000000134

Rotenstein

Ramos

Torre

, et al. Prevalence of depression, depressive symptoms, and suicidal ideation among medical students: a systematic review and meta-analysis. JAMA. 2016;316(21):2214-2236. doi:10.1001/jama.2016.17324

10.

Halperin

Henderson

Prenner

Grauer

. Prevalence of anxiety and depression among medical students during the COVID-19 pandemic: a cross-sectional study. J Med Educ Curric Dev. 2021;8:2382120521991150. doi:10.1177/2382120521991150

11.

Moir

Yielder

Sanson

Chen

. Depression in medical students: current insights. Adv Med Educ Pract. 2018;9:323-333. doi:10.2147/AMEP.S137384

12.

Grading Systems Use by US Medical Schools. AAMC. Accessed December 30, 2021. https://www.aamc.org/data-reports/curriculum-reports/interactive-data/grading-systems-use-us-medical-schools.

13.

American Association of Medical Colleges. Medical School Year Two Questionnaire 2016 All Schools Summary Report. Published online February 2017.

14.

American Association of Medical Colleges. Medical School Year Two Questionnaire 2020 All Schools Summary Report. Published online February 2021.

15.

Deng

Gluckstein

Larsen

. Student-directed retrieval practice is a predictor of medical licensing examination performance. Perspect Med Educ. 2015;4(6):308-313. doi:10.1007/s40037-015-0220-x

16.

Welcome to Boards and Beyond. Accessed July 3, 2022. https://boardsbeyond.com/homepage.

17.

Sketchy | Master your medical studies with visual learning. Accessed July 3, 2022. https://www.sketchy.com/.

18.

Cardall

Krupat

Ulrich

. Live lecture versus video-recorded lecture: are students voting with their feet? Acad Med. 2008;83(12):1174-1178. doi:10.1097/ACM.0b013e31818c6902

19.

Kauffman

Derazin

Asmar

Kibble

. Patterns of medical student engagement in a second-year pathophysiology course: relationship to USMLE Step 1 performance. Adv Physiol Educ. 2019;43(4):512-518. doi:10.1152/advan.00082.2019

20.

Ahuja

Peck

Wijesekera

. Screens in the back: medical student perspective on curriculum engagement. Med Teach. 2020;42(4):474-475. doi:10.1080/0142159X.2019.1635249

21.

Song

Chakraborty

Dawson

Dugan

Adkins

Doty

. Does the podcast video playback speed affect comprehension for novel curriculum delivery? A randomized trial. West J Emerg Med. 2018;19(1):101-105. doi:10.5811/westjem.2017.10.36027

22.

Murphy. Medical schools ponder move to shorter pre-clerkship curriculum. American Medical Association. Published 2021. Accessed December 30, 2021. https://www.ama-assn.org/residents-students/medical-school-life/medical-schools-ponder-move-shorter-pre-clerkship-curriculum

23.

Hoyo

Yang

Larson

. Relationship of medical student lecture attendance with course, clerkship, and licensing exam scores. Med Sci Educ. 2020;30(3):1123-1129. doi:10.1007/s40670-020-01022-y

24.

Farhat

Beck Dallaghan

. Enhanced learning and retention of medical knowledge using the mobile flash card application Anki. Med Sci Educ. 2021;31(6):1975-1981. doi:10.1007/s40670-021-01386-9

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.03 MB