Abstract
Background:
While gender bias in medicine, including physician training, has been well described, less is known about gender bias in the selection process for post graduate residency training programs. This analysis reviews the potential role of gender on resident selection for an internal medicine residency program.
Methods:
File review and interview overall and component scores were analyzed based on the gender of the applicant. File review scores were further analyzed based on the reviewer’s gender.
Results:
Women applicants scored higher than men applicants on their file review. There were no differences in any one component score except for leadership in art. Women file reviewers scored applicants higher than men file reviewers, but there was no difference between gender scores. There was no difference in overall or component interview scores between men or women applicants. Scoring did not impact the expected rank performance of applicants based on gender at any stage of the selection process.
Conclusions:
While higher scores were observed in women applicants upon their file review, and women reviewers provided higher file review scores, this did not appear to impact the expected number of women and men applicants at each stage of the applicant process. This suggests a potential lack of gender bias at these stages of applicant selection.
Over the last 30 years, there has been a significant shift in the balance between men and women post graduate medical trainees in Canada. Whereas two-thirds of residents were men in 1990, residency better reflects overall society with 53% of residents now women, as of 2016. 1 Despite this, there is much data that suggests ongoing bias against women physicians in practice, be it when it comes to day to day work, promotion, publication or awards.2-5 There is also growing data that gender bias impacts residency training. There appears to be differences in how residents are assessed based on gender during residency.6,7 There also appears to be differences observed in selecting trainees for resident positions themselves, be it differences in letters of reference or the selection process itself.8-11 It is unclear how widespread this is, particularly if there may be differences when it comes to different specialties selecting residents. While there are a number of studies published demonstrating gender bias in selecting residents in surgical specialties,8-10,12 less is known about non-surgical specialties, and internal medicine in particular.
Given this, as a quality assurance project, the Core Internal Medicine residency program at the University of Alberta reviewed its residency selection process based on the 2019 Canadian Residency Match (CaRMS) to determine if there was evidence of gender bias in their resident selection process. We describe the process and results of this project to determine if the current applicant selection process may contain gender bias.
Methods
The residency selection process for the residency program contains 2 components. First, every applicant submits a file package for review through the CaRMS website. At the time of file review, applicant gender is not explicitly provided nor is a photograph available of the candidate; however, gender is suggested by way of pronoun use in letters of reference. Each file is reviewed by 2 physicians independently and scored based on an internally constructed rubric examining file components such as rotation electives in internal medicine, research, community work, leadership, work-life balance, letters of reference, a personal letter, and an overall opinion of the reviewer. Site of medical school training is not considered as part of the file review and therefore not considered for this project. These scores are combined, and applicants are ranked to determine who will be invited for an in-person interview. The in-person interview is comprised of a Multi Mini-Interview (MMI) format, with each candidate completing 4 themed interview stations, each station containing 2 separate components. Each station is scored by 2 interviewers independently, neither of whom have access to the applicant file or file review scores. The total scores of the MMI are added together and then combined with the file review scores for a final candidate ranking.
Independent of and subsequent to the process described above, each file was reviewed to determine applicant gender, based on how the candidate is referred to (pronoun) in their letters of reference. File reviewer gender was also recorded. Given the complexities of the MMI interview process over multiple days, interviewer gender was not identified for this project.
Applicant scores were analyzed using simple statistics to determine differences in file review scores between men and women, as well as for subcomponents of the file review. Further, overall file review scores were analyzed to determine if the gender of the file reviewer may influence applicant scores. Similarly, applicant scores were analyzed to determined differences in overall interview scores as well as scores for each themed station. Finally, the ratio of men to women applicants was calculated at each selection stage to determine if there were changes.
As this was intended as a quality assurance project, medical ethics approval was waived by the University of Alberta Health Ethics board. Raw data from this study may be accessed by contacting the author.
Results
File reviews
There were 378 applicants to the Core Internal Medicine residency program in 2019, of which 165 were women (43.65%) and 213 (56.35%) were men. Each candidate’s file was reviewed independently by 2 physicians who were randomly assigned. There was a total of 26 file reviewers, 15 of whom were women and 11 were men.
Average file review scores for women applicants equaled 23.618, compared to 22.699 for men applicants (P = .0056). However, there was no statistically significant difference in subcomponents of the scores (See Table 1) except when comparing leadership in the arts, where significantly more women applicants demonstrated aptitude in this area compared to men (N = 79 vs 63, P = .002).
Gender differences in applicant file review scores.
Categories with a denominator score have scoring based on a continuous variable whereas the remainder is categorical data, that is the total number of applicants who meet that criteria.
± refers to standard deviation.
Gender of the file reviewer did appear to impact scoring (see Table 2). Overall scores given by men reviewers were significantly lower than women reviewers (22.494 vs 23.493, P = .0148). Further analysis to determine if reviewer gender impacted scoring of applicants based on their gender demonstrated no difference. Men reviewer scores demonstrated no significant difference between applicant gender (women 23.147 vs men 22.011, P = .141), with similar results for women reviewers (women 23.948 vs men 23.128, P = .107).
Gender differences in file reviewer scores.
There was a significant difference (P = .0148) between the overall scores of women and men file reviewers; the remaining scores showed no statistically significant difference.
The top 180 applicants were offered an interview, representing 98 men applicants (54.4%) and 82 women applicants (45.5%), proportions which are not significantly different than the original applicant group (P = .752).
Interviews
There were 177 interviews completed, of which 79 were women and 98 were men. The total overall score for women applicants was 325.59 (Standard Deviation = 29.9) compared to 319.50 (SD = 34.6) for men applicants (P = .1429). An analysis of the scores from each interview station similarly did not demonstrate a significant difference between men and women applicants.
When examining the top 50 applicants based on interview scores, 23 were women (46%). Similarly, 46 women were in the top 100 applicants based on interview scores.
Discussion
The results of this analysis demonstrate that women applicants scored higher than men applicants on their file review, women reviewers provided for higher scores compared to men reviewers regardless of the gender of the applicant, but there was no difference in scoring between genders for the applicant interview. Despite the differences noted, while there was a positive trend towards more women applicants at each stage, it did not appear to have a statistically significant impact on the gender rankings of applicants, as women applicants represent approximately 45% of the overall applicant pool at each stage of the application process.
This study is important as there is still little data reported about gender bias in resident selection, particularly in internal medicine or its subspecialties. A recent review of a radiology program’s resident selection process also demonstrated higher scores for their women applicants compared to men. 12 A 2001 study examining selection in orthopedic surgery also reported no gender bias in their file review process. 13 A recent review of overall resident selection in Canada demonstrated women applicants were less likely to get their first choice in residency program if applying to a surgical program compared to men applicants, although there seemed to be no difference for fields like internal medicine. 11 Outside of medicine, a selection process for a Faculty of Science position however did demonstrate a clear gender bias among candidates. 14 There is more data available on bias favoring men and against women within letters of reference, 8 with differences noted based on gender in selection of words to describe candidates. Similarly, there are reported biases when it comes to medical trainee evaluation.15,16
While it may be reassuring there is no overt gender bias on the resident selection process here, a number of potential weaknesses must be acknowledged. There is no clear association to state that the file review scores were higher in women applicants because of gender; causation is unclear. Further, one could argue that it is concerning that women applicants did not make up a larger pool of interview candidates if their file review scores were on average higher, with a similar trend, albeit not statistically significant for interview scores; with more applicants to power this review, this may have been observed as certainly there was a trend towards more women interviews (45.5%) compared to initial candidates (43.65%). While the top 180 candidates were offered an interview, it was outside the scope of this project to understand why 3 candidates (3women) did not interview.
This study also cannot account for other potential factors that lead to gender bias. Does gender influence applicant interest in the specialty in the first place, as there is data that demonstrates clear differences in the number of applicants based on gender in different specialties? 1 Have these applicants been influenced to apply to internal medicine compared to other medical specialties because of any inherent gender bias or experience that occurred during medical school training? Examining these issues earlier in medical training and linking them to eventual residency training choices may provide insight. While not specifically examined, due the single site nature of this study, are there any inherent differences at this site that would lead to different results elsewhere? In addition, because of the complexity of the MMI interview scheme and how the data is kept, we are unable to comment on the impact of interviewer gender on resident selection. Reassuringly, multiple studies have demonstrated in different health care learner populations that gender does not seem to influence MMI scoring, and applicants do not feel there is bias during MMI interviews.17-19 Finally, because the specific components of the file review and interview may be unique to 1 residency program, it is unknown if these results are generalizable to other internal medicine programs, let alone other specialties.
In conclusion, the ratio of women to men applicants moving through the various stages of the selection process—file review to interview to ranking—remained consistent, with a non-significant trend towards favoring women applicants, suggesting the possibility of no overt bias against women in the internal medicine program residency selection process. Further study to determine the impact the specific components of the selection process may have as it relates to gender, and over a longer period of time, may be helpful to ensure this is truly the case and if so, this information can be shared with other programs to consider for their selection process.
Footnotes
Acknowledgements
A portion of these results were presented by the author in abstract form at The Royal Australasian College of Physicians 2020 Congress. This abstract was subsequently published in Internal Medicine Journal Vol 50(S1).
Funding:
The author received no financial support for the research, authorship, and/or publication of this article.
Declaration of conflicting interests:
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Author Contributions
SJK was responsible for data acquisition, analysis and interpreation, as well as manuscript preparation and revisions.
