Abstract
Purpose
The United States Medical Licensing Examination (USMLE) Step 1 exam changed from a 3-digit score format to pass/fail in 2022. The current study aimed to examine how medical students perceive this change, factors associated with those perceptions, and determine their preferred exam scoring format.
Design and Methods
A cross-sectional survey of US medical students was distributed from March through June 2024. The survey gathered data including demographic and school information, desired specialty, research involvement, and Step 1 scoring preference.
Results
The survey was completed by 192 students from 18 different US allopathic schools. Of students surveyed, 65.5% preferred pass/fail scoring, while 34.5% preferred a 3-digit-score format. Older age increased likelihood of preferring a 3-digit score (β = 0.345, P = .047, odds ratio [OR] 1.41). A later graduation year decreased the likelihood of preferring a 3-digit score (β = −0.576, P = .020, OR 0.56). Gender (β = 0.293, P = .320, OR 1.34), specialty competitiveness (β = −0.095, P = .776, OR 1.10), and research productivity (β = 0.0047, P = .990, OR 1.00) had no significant effect on Step 1 scoring preference.
Conclusion
Medical students seem to prefer a pass/fail Step 1 regardless of gender, desired specialty and research productivity. However, pass/fail preference differed by graduating class year and age, with younger class years and students preferring pass/fail. While prior literature largely surveyed preference among medical educators, student sentiment on Step 1 scoring supports recent changes to the USMLE Step score format.
Introduction
The United States Medical Licensing Examination (USMLE) comes in a 3-part series that all medical students and residents partake in to continue to postgraduate clinical residency training: Step 1, Step 2, and Step 3. 1 As of January 26, 2022, USMLE Step 1 underwent a considerable change: going from a 3-digit score format to pass/fail. 2 The rationale behind this action by the National Board of Examiners (NBME) was to reduce the adverse impact of overemphasis on USMLE performance in residency screening, minimize racial demographic differences, and return to the initial purpose of the exam in assessing adequacy in medical knowledge. 3
With this change, many unanticipated repercussions have surfaced, accompanied by an altered medical education environment. 2 One such repercussion is an increase in student-given importance and program directors’ attention to research.4,5 Additionally, pass/fail preclinical curriculums further drive its importance. 6 Medical students traditionally had relied on USMLE Step 1 to showcase academic aptitude to succeed at the residency level, and consequently, Step 1 scores were previously one of the most important factors influencing residency selection. 1 In the pass/fail era, this cannot be the case, and it seems students have naturally shifted toward other aspects of the application to increase competitiveness, namely research. In addition, there has been an observed, increased difficulty for students in assessing their personal competitiveness for residency.7,8 Interestingly, learner anxiety failed to be reduced by the pass/fail grading of USMLE Step 1.9,10
A recent study found that a majority of medical school deans were in favor of the USMLE Step 1 change to pass/fail, with the caveat that students applying to traditionally more competitive specialties would be most affected. 11 Meanwhile, during the year that the scoring was changed to pass/fail (2022), residency program directors overwhelmingly said they would not adjust their usage of USMLE Step 1 as a metric in candidate evaluation for that year despite a heterogenous population of applicants, both pass/fail and 3-digit scored USMLE Step 1. 12 A collection of studies also found that medical school deans acknowledge the benefits of a holistic application review process but also recognize that removing a numerical Step 1 score may pose challenges in assessing applicants fairly and efficiently.13–15 Thus, current literature indicates mixed sentiment surrounding the scoring of Step 1 among medical educators. However, there remains a lack of data on student perspectives regarding the pass/fail policy and its implications for residency applications.
Therefore, the aim of this study was to uncover sentiment of US medical students surrounding their preferences for USMLE Step 1 scoring with the purpose of determining if it aligns with the NBME's decision to change the scoring format to pass/fail. Additionally, we sought to address the gap in existing literature by examining whether gender, age, specialty choice, research productivity, and class year were associated with Step 1 scoring preference. The hypothesized variables were based on prior findings that students pursuing highly competitive specialties often favor objective benchmarks such as standardized test scores, 16 that research productivity is frequently correlated with competitiveness in residency applications,17,18 and that class year may influence opinion due to differences in exposure to the pass/fail policy. We hypothesized that students would prefer a pass/fail Step 1, with age, gender, specialty choice, research productivity, and class year influencing scoring preference.
Methods
Study Design
A multi-institutional cross-sectional survey study was reviewed by the Inova Health Institutional Review Board (IRB) and deemed to be exempt (#INOVA-2024-120). Written informed consent was not required to participate in the study due to implied participant consent with completion of the distributed survey. The online survey was distributed via REDCap to United States medical students between March 28 and June 16, 2024. Inclusion criteria included current students enrolled in US allopathic institutions from the class years 2025, 2026, and 2027. Exclusion criteria consisted of those who did not fall into a graded preclinical curriculum. The social and professional networks of 2 medical student authors (ST and RC) were utilized to acquire participation in the survey. Methods used to send out the survey included school listservs, social media outreach, and text messaging. The sampling method was done in this manner to attract as many participants as possible, and consequently, a diverse cohort with varying geographic region and class years.
Survey Design
The survey (Supplemental 1) collected demographic information through questions about age, gender identity, school, and graduating year. Furthermore, the study collected desired specialty, research involvement, and Step 1 scoring preference. All collected data was de-identified. The reporting of this study conforms to the TRIPOD-Cluster reporting guideline (Supplemental 2). 19 The questionnaire was tested and developed by the authors in order to maximize efficiency and convenience. Testing involved grammatical checking, measuring time to take the survey, and finetuning the user interface design.
Definition of Outcomes and Variables
The primary outcome of this study was students’ preference for USMLE Step 1 scoring, presented as 2 options: pass/fail or 3-digit score. Highly competitive specialties (HCS) were defined as those with average match rates <85% over a 3-year period from 2022 to 2024, as per the NRMP website. These included orthopedic surgery, ophthalmology, urology, neurological surgery, dermatology, plastic surgery, and otolaryngology. The remaining specialties were categorized as “less competitive specialties” (LCS). If a student selected both HCS and LCS options, it was assigned to the HCS cohort. This was done under the reasoning that a student intending to apply to an HCS would sensibly be expected to work toward that competitive specialty despite a separate interest in an LCS. Significant research productivity (RP+) was defined as having ≥ 3 manuscript publications indexed on PubMed in accordance to prior literature, which determined ≥ 3 publications to be at the 75th percentile among a competitive specialty. 17 The rest were categorized as not research productive (RP−). Categorization of RP + versus RP− was based on student responses and were not verified by the authors.
Statistical Analysis
Statistical analysis was performed using R (2021; R Foundation for Statistical Computing). Respondent preferences were reported as frequencies and percentages. A multivariate logistic regression model was used to evaluate the associations between variables. Variance inflation factors were calculated and demonstrated no multicollinearity. Significance was defined as P < .05.
Results
Respondent Characteristics
The survey was completed by 201 students from 18 different US allopathic schools during the distribution period. Nine responses were excluded in accordance with the criteria delineated in the methods section, leading to a total of 192 responses used in the study. Respondent demographics are summarized in Table 1. Due to the distribution method used such as school listservs, a response rate could not be ascertained since it was not possible to determine how many people may have seen the recruitment message for the study.
Respondent and Institution Demographic Characteristics.
Orthopedic surgery was the most frequently chosen specialty students wished to pursue (n = 33, 17.2%; Figure 1). In addition, 86 respondents (44.8%) desired to match into a highly competitive specialty as defined in the methods above. The most prevalent HCS was orthopedic surgery (n = 33, 38.4%). The most popular LCS was internal medicine (n = 16, 15.1%).
Student-Reported Specialty of Interest—Students Reported the Specialty they are Most Interested in Pursuing. Highly Competitive Specialties (HCS) are Reported in Blue, While Less Competitive Specialties (LCS) are Reported in Yellow.
About three-quarters of the students (74.0%) were involved in research for their respective desired specialty. Yet, only 43 students (22.4%) were categorized as productive in research, RP + (Table 2).
Frequencies of Peer-reviewed Publications of Respondents.
Dashed red line represents the cutoff for research productive and not research productive.
Of the 65.6% of respondents who took Step 1 by the time of the survey, 5 (4.0%) did not pass on their first attempt. The characteristics of these respondents at the time of the survey are reported in Table 3 along with their preferences.
Characteristics of Failures Among USMLE Step 1 Takers.
USMLE Step 1 Scoring Preferences
Overall, 65.6% of respondents preferred a pass/fail Step 1, while 34.4% preferred 3-digit scoring. Older age increased likelihood of preferring a 3-digit score (β = 0.345, P = .047, odds ratio [OR] 1.41) (Table 4). The threshold age corresponding to 50% likelihood to prefer a scored Step 1 was found to be 28.6 (95% confidence interval [CI]: 25.25, 31.93). A later graduation year, with threshold of 2025.8 (95% CI: 2025.7, 2025.9), decreased the likelihood of preferring a 3-digit score (β = −0.576, P = .020, OR 0.56). Gender (β=0.293, P = .320, OR 1.34), specialty competitiveness (β = −0.095, P = .776, OR 1.10), and research productivity (β = 0.0047, P = .990, OR 1.00) had no significant effect on Step 1 scoring preference. Equation (1) displays the logistic regression model that was generated.
Logistic Regression Results for Variables Influencing Step 1 Scoring Preference.
Equation 1. Multivariate logistic regression model for factors influencing Step 1 scoring preference
Discussion
The principal finding of this study was that most medical students (65.6%) preferred a pass/fail scoring format for USMLE Step 1. Furthermore, gender, desired specialty, and research productivity did not influence the students’ scoring preference. However, each class year had different proportions of students supporting a pass/fail Step 1 exam, with students in earlier class years (ie, class of 2028) favoring the pass/fail scoring format as compared to the older class years (class of 2025). In addition, older age (>28.6 years) was positively correlated with a higher preference for a 3-digit scored Step 1. Four out of 5 students who failed Step 1 on their first attempt now prefer a pass/fail Step 1, while 1 preferred a 3-digit scoring format (Table 3).
The change from a 3-digit-scored USMLE Step 1 to a pass/fail format can potentially change how residency applications are reviewed in the future. In the most recent match cycle (2023-2024), a Step 1 score was the only independent predictor associated with middle versus higher applicant tier placement for plastic surgery residency demonstrating the continued weight of Step 1 in residency selection 2 years into the pass/fail era.20,21 Now, residency application preparation seemingly revolves around other variables, such as Step 2 scores and extracurriculars. Moreover, application items such as letters of recommendations have risen in importance due to the adaptation toward a more holistic review. 22 Research has also been emphasized as a significant application in place of Step 1 scores.4,5,23 However, our findings demonstrate that research productivity was not associated with students’ preference for Step 1 scoring (Table 4).
There exists only sparse data that surveys medical student sentiment surrounding the change in Step 1 scoring. Rothka et al looked at 127 medical students at a single institution to determine student sentiment surrounding a pass/fail Step 1. The study found that most medical students (73.2%) agreed with the decision to make Step 1 pass/fail. 24 However, students reported increased anxiety surrounding Step 2 scores. Another result was that competitive versus other specialties bore no difference in preference for Step 1 scoring, with both cohorts preferring a pass/fail Step 1.
In another study, Lichtbroun et al, also showed that students from both allopathic and osteopathic medical schools in the classes of 2023 and 2024 mostly preferred a pass/fail scoring system. 17 Additionally, the authors observed that those going into highly competitive specialties had a difference in opinion on Step 1 scoring compared to those going into less competitive specialties, with highly competitive specialty applicants preferring a 3-digit score.
Our findings corroborate with both Rothka et al and Lichtbroun et al regarding overall student preference. A majority of the students we surveyed preferred a pass/fail scoring format. Thus, it seems that the decision of the NBME to make Step 1 pass/fail may have been an appropriate initiative; it is a policy largely supported by students and faculty alike. 11 While students seem to mostly support the new Step 1 scoring policy, it should be noted that around one-third of surveyed students wished for a 3-digit score format. This is not an inconsequential number and highlights the complexities in medical education policy decision making.
The data of this study also contrasts that of Lichtbroun et al and supports Rothka et al with respect to the role of desired specialty on student Step 1 scoring preference. Lichtbroun et al noticed a difference in preference for Step 1 scoring with those wanting to match into a HCS versus LCS. 18 The authors reported 55.6% of the HCS cohort and 31.7% of the LCS cohort preferring a numerical Step 1 score (P = .001). Our results suggest no difference in scoring preference among HCS and LCS cohorts (Table 4). A key difference that could explain this disparity between the results of the studies is sample composition. Lichtbroun et al surveyed the classes of 2023 and 2024, while our study examined the classes of 2025, 2026, and 2027. Of note, Rothka et al also examined more recent class years: 2024, 2025, 2026, and 2027. This could indicate that recent classes prefer a pass/fail format compared to prior years. This was demonstrated in our study where class years further into the pass/fail era had a significantly greater likelihood of students in favor of pass/fail scoring than older class years (Table 4). However, these differences may be influenced by the point of training of surveyed students rather than differences between class years themselves. For example, with residency programs placing increasing emphasis on Step 2 scores, senior students may recognize the advantage a scored Step 1 previously provided in differentiating applicants, particularly for competitive specialties. Additionally, those in the interview phase may feel that a scored Step 1 would have offered them an early objective metric and allowed them to strengthen their applications, especially if other aspects of their application, such as clinical grades or research, were less competitive. Depending on when they take Step 2, some students may receive their Step 2 score very late in the application process and realize they are not competitive enough for their desired specialty. This perspective could contribute to the observed trend of more senior students favoring a scored Step 1, despite the overall shift toward pass/fail preferences among recent class years.
Educational Implications
The goal of medical education policy is to enhance the learning process of all medical students. The recent Step 1 scoring change appropriately aligns with this goal and corresponds to student sentiment surrounding Step 1 scoring. However, at many institutions, the scored Step 2 exam has effectively replaced Step 1 as the primary metric for stratifying applicants and extending interview offers. While holistic review factors such as research, extracurricular activities, Alpha Omega Alpha (AOA), Gold Humanism award, clerkship performance, and Medical Student Performance Evaluation (MSPE) comments remain crucial components of the residency application process, the shift toward Step 2 as a standardized, quantifiable benchmark has become increasingly apparent. Consequently, it remains to be seen whether this newfound emphasis on Step 2 has resulted in a lateral shift in the residency application review process, or if Step 1 solely going pass/fail is sufficient for an entirely different educational landscape.
It is expected that questions about Step 2's scoring format are soon to be catapulted to the forefront of the test makers’ attention. This shift raises discussions about whether Step 2 might eventually undergo similar modifications to Step 1, given the NBME's existing reasoning for converting Step 1 to pass/fail. The final decision on the scoring of Step 2 remains to be determined; however, the making of Step 1 pass/fail appears to be a step preferred by medical students and educators alike.
Limitations
This study, as with all cross-sectional survey studies, is not without limitations. One limitation was the inability of the authors to ascertain a response rate due to the survey distribution method. The distribution method was chosen to achieve the highest possible number of responses, and consequently, a power calculation was not performed. Recall bias and subjective reporting of Step 1 scores was possible and may have influenced class year perceptions differently. We tried mitigating this through a large sample size of students from various institutions across the United States.
Additionally, many students enter medical school with a strong interest in HCS before clinical exposure and a deeper understanding of the field. As students progress through their training, exposure to various specialties often shifts their career preferences, with some gravitating toward LCS. This evolution in specialty preference likely influences attitudes toward Step 1 scoring. As a result, the conclusions drawn from our study are inherently limited by the shifting career interests of medical students over time. Moreover, categorizing students who were interested in both an HCS and LCS in the HCS cohort poses another limitation. In addition, our sample was composed of a large proportion of students attending an institution in the South or Northeast region. This also limits the generalizability of the findings of this study to the entire US allopathic medical student population.
Finally, while our sampling method was utilized to maximize captured responses, using the social and professional networks of 2 authors may have led to a sample more heavily interested in highly competitive specialties than what is true otherwise. This smaller sample is to serve as a basis for support for a larger study on student Step 1 preferences.
Conclusion
Medical students seem to prefer a pass/fail Step 1 regardless of gender, desired specialty and research productivity. However, pass/fail preference differed by graduating class year and age, with younger class years and students preferring pass/fail. While prior literature largely surveyed preference among medical educators, student sentiment on Step 1 scoring supports recent changes to the USMLE Step score format.
Supplemental Material
sj-docx-1-mde-10.1177_23821205251337125 - Supplemental material for What Do Medical Students Think About a Pass/Fail USMLE Step 1? A Survey of 18 Allopathic Schools
Supplemental material, sj-docx-1-mde-10.1177_23821205251337125 for What Do Medical Students Think About a Pass/Fail USMLE Step 1? A Survey of 18 Allopathic Schools by Shankar S Thiru, Rachel E Cherelstein, Scott M Feeley, Scott J Halperin, Sarah Lucas, Christopher M Kuenze and Edward S Chang in Journal of Medical Education and Curricular Development
Supplemental Material
sj-docx-2-mde-10.1177_23821205251337125 - Supplemental material for What Do Medical Students Think About a Pass/Fail USMLE Step 1? A Survey of 18 Allopathic Schools
Supplemental material, sj-docx-2-mde-10.1177_23821205251337125 for What Do Medical Students Think About a Pass/Fail USMLE Step 1? A Survey of 18 Allopathic Schools by Shankar S Thiru, Rachel E Cherelstein, Scott M Feeley, Scott J Halperin, Sarah Lucas, Christopher M Kuenze and Edward S Chang in Journal of Medical Education and Curricular Development
Footnotes
Acknowledgments
The authors wish to thank Benjamin Neubauer (EVMS), Derek Tran (Hofstra), Corina Dang (FIU), Ethan Belnap (Rush), Erik Holm (USUHS), Garrison Hughes (UVA), Sarah El-Meanawy (University of Wisconsin), Parth Sharma (Inova Sports Medicine), and Steven Hizmi (Epsilon) for their contributions in raising awareness of the study.
Ethical Considerations
A multi-institutional cross-sectional survey study was reviewed by the Inova Health Institutional Review Board (IRB) and deemed to be exempt (#INOVA-2024-120).
Informed Consent
Written informed consent was not required to participate in the study due to implied participant consent with completion of the distributed survey.
Author Contributions
SST contributed to the study design, data acquisition, data analysis, manuscript drafting, and final approval. REC contributed to the data acquisition, data analysis and interpretation, and final approval. SMF and SJH contributed to the data interpretation, manuscript drafting, and final approval. SL contributed to manuscript revision and final approval. CMK contributed to the data analysis and interpretation and final approval. ESC contributed to the study design, data interpretation, manuscript revision, and final approval.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
Appendix 1. Additional data from 192 surveyed responses.
| Question | Yes | No | |||||
| Does your medical school have a home residency program for your specialty selection? | 142 | 50 | |||||
| Are you doing research with your home program? | 125 | 20 | |||||
| Does your school have a mandatory deadline for Step 1? | 126 | 69 | |||||
| Do you prefer to have a mandatory deadline? | 107 | 88 | |||||
| What activities are you involved in? | Education/training | Military | Professional organization | Research | Teaching/mentoring | Volunteering | Work |
| 106 | 22 | 107 | 157 | 86 | 156 | 20 | |
| When do you believe is the ideal time to take Step 1? | Any time | Before clerkships | After clerkships | ||||
| 19 | 139 | 31 |
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
