Abstract
Introduction
Radiology is a rapidly growing clinical specialty with a critical role in patient care. With the ongoing evolution of imaging technology and newer analysis methods, radiologists have an unprecedented opportunity to participate in a wide variety of impactful research ranging from applied to basic biomedical research.1-4 Importantly, the future of radiology relies on training the next generation of radiologists to engage and conduct research and evidence-based practices.
Despite the importance of research, the number of clinician-scientists in the United States is declining, with the proportion of clinicians engaged in research decreasing from 4.7% in the 1980s to 1.5% in 2019. 5 A similar trend is present in radiology, with the percentage of American radiology clinician-scientists pursuing an academic career being historically low.6-8 An international consensus report from radiologists in Canada, the United Kingdom and the United States noted that research productivity and publications in radiology are lacking compared to other clinical specialties. 9 Given these alarming statistics, an important question arises: what is hindering research engagement in radiology?
In response to the lack of research-oriented radiologists, our Medical Imaging (MI) department at the University of X provides training through the Clinician Investigator Program (CIP) - a Royal College of Physicians and Surgeons of Canada accredited program that allows medical residents to pursue graduate or postdoctoral-level research training alongside their residency program. 10 Completion of a graduate degree is a mandatory requirement. If the resident already holds a graduate degree, then they will be enrolled in a postdoctoral stream. There is a minimum of 24 months of intensive research training that can be continuous, typically between post-graduate year 2 and 3 of residency, or fractionated if research projects require several years to plan and obtain ethics board approval. 10 Specific timelines and details are established with the institution’s program director and CIP director. Although the University of X’s MI-CIP has been available for more than a decade, there has been no evaluation of the program’s outcomes.
In this study, we measured the research productivity of the University of Toronto’s MI-CIP trainees. Our primary hypothesis was that MI-CIP trainees have an increased research output compared to MI-non-CIP trainees. We also hypothesize that MI-CIP trainees will have more research publications than non-CIP trainees before beginning residency. Our secondary hypothesis was that MI-CIP trainees have equivalent research output compared to trainees from other CIPs, including the well-established University of X’s General Surgery (GSx) CIP. We had 2 objectives in this study: (1) to evaluate the outcomes of the University of X’s MI-CIP by measuring trainee research productivity; and (2) to compare the research productivity of the MI-CIP trainees with MI-non-CIP and GSx-CIP trainees.
The findings from evaluating one of Canada’s largest Diagnostic Radiology residency program will be critical for advancing research training and educational initiatives in other national and international residency programs.
Methods
This study used publicly available data and did not involve the investigation of human subjects requiring ethics board approval. Public domain data was used to gather information on past trainees in the University of Toronto’s residency programs. Specifically, we collected data on 3 cohorts from 2006-2016: the Medical Imaging CIP, the Medical Imaging non-CIP, and the General Surgery CIP trainees. Given the fact that the University of Toronto’s Medical Imaging CIP represents a relatively smaller and newer CIP, we compared our program with a well-established General Surgery CIP (started in 1998) to act as a comparative standard.
Data variables included: the trainee’s full name, sex and start/end dates of training (CIP or traditional residency program). Research publications were searched by using the trainee’s information on PubMed, including search terms such as the full name of the trainee, city of training, and co-authors of publications. Specific quantitative measurements of research productivity were retrieved, including the total number of publications, the number of first-author publications and the journal impact factor of published studies. The data were organized according to 2 time points for each group: “Before residency” and “Post-CIP or residency” when comparing MI-CIP vs MI-non-CIP trainees. “Before residency” includes any research output performed before beginning residency training. “Post-CIP or residency” includes research output at the beginning of training to 7 years post-CIP or graduation. Similarly, when comparing MI-CIP vs GSx-CIP trainees, we use “Before CIP” to represent any research output performed before residency and “Post-CIP” to represent research output during training up to 7 years post-CIP. We used a 7 year endpoint in our analysis to fully capture the research productivity as a trainee and junior staff. Most MI-CIP trainees start CIP training after post-graduate year 2. After CIP completion, trainees would finish the remaining 3 years of the traditional residency program, followed by 1 or 2 years of fellowship training. We then assigned an additional 2 to 3 years to capture any publications of studies conducted during their previous training or initial faculty appointment. Thus, 7 years post-CIP completion allows an initial view of an individual’s research productivity after training completion.
Data and Statistical Analysis
The collected data are displayed as column scatter plot graphs. Each dot represents data from an individual trainee and the lines in each dataset represent the median and interquartile range (IQR).
The Shapiro-Wilk normality test was used to show that the collected data is not normally distributed. We then summarized trends with statistical descriptors, including frequencies, medians and interquartile ranges (IQR) of outcomes for the 3 study cohorts. A two-tailed non-parametric test (Mann-Whitney) was used to compare different cohort groups during different time periods, with statistical significance defined by P <.05. All statistical analyses were performed using R version 4.2.0 (April, 2022).
Results
CIP Demographic Data
General Demographic Information of Each Study Cohort: Medical Imaging Clinician Investigator Program (CIP), Medical Imaging non-CIP and General Surgery CIP Trainees.
Abbreviations: IQR, interquartile range; n.s., not statistically significant.
Comparing Research Productivity of the University of Toronto’s Medical Imaging (MI) CIP vs MI-Non-CIP Trainees (Table 2)
Number of Total Publications
Before residency training, the median (IQR) number of total publications of MI-CIP vs MI-non-CIP trainees was 5.0 (8.0) vs 1.0 (2.0) (P = .0001) (Figure 1(A) and Table 2). These results show that MI-CIP trainees had a significantly higher number of research publications than MI-non-CIP trainees before beginning residency. Research productivity of the University of Toronto’s Medical Imaging-Clinician Investigator Program (MI-CIP) trainees (n = 11) vs MI-non-CIP trainees (n = 74) before residency or post-CIP or Residency. Note, the time period for post-CIP or Residency includes the beginning of training to 7-year post-CIP/graduation. Error bars represent the median with interquartile ranges. A Medical Imaging-CIP trainees had a statistically significant higher number of total publications before residency and at 7-years post-CIP/graduation compared to non-CIP trainees. B. Medical Imaging-CIP trainees had a statistically significant higher number of first-author publications before residency and at 7-years post-CIP/graduation compared to non-CIP trainees. C. Medical Imaging-CIP trainees published in journals with a statistically significant higher average impact factor before residency and at 7-years post-CIP compared to non-CIP trainees. Research Outcomes of Medical Imaging Clinician Investigator Program and non-CIP Trainees Before Residency and at 7 Years Post CIP Training/Graduation. Abbreviations: IQR, interquartile range.
At 7 years post-CIP or post-graduation, the median (IQR) number of total publications of the University of Toronto’s MI-CIP vs MI-non-CIP trainees was 6.0 (10.0) vs .0 (2.0) (P < .0001) (Figure 1(A) and Table 2). Fifty-four percent (40/74) of all non-CIP trainees had 0 publications at 7 years post-graduation. These results reflect active research participation of CIP trainees and a lack of research engagement in non-CIP trainees.
Number of First-Author Publications
Before residency training, the median (IQR) number of first-author publications of CIP vs non-CIP trainees was 2.0 (3.0) vs .0 (1.0) (P < .0002) (Figure 1(B) and Table 2). These results show that CIP trainees actively led research projects in the capacity of a first-author before entering Medical Imaging residency compared to non-CIP trainees.
At 7 years post-CIP or post-graduation, the median (IQR) number of first-author publications of MI-CIP vs MI-non-CIP trainees is 2.0 (4.0) vs .0 (1.0) (P < .0001) (Figure 1(B) and Table 2). These results show that CIP trainees were actively performing research post-CIP.
Average Journal Impact Factor
Before residency, the median (IQR) average journal impact factor of Medical Imaging-CIP vs Medical Imaging-non-CIP trainee’s publications was 3.2 (2.0) vs .3 (2.4) (P = .0032) (Figure 1(C) and Table 2). This result suggests that CIP trainees were actively involved in research before entering Medical Imaging residency. In addition, this result implies that many non-CIP trainees were initially involved in research before residency at some capacity.
At 7 years post-CIP or post-graduation, the median (IQR) average journal impact factor of MI-CIP vs MI-non-CIP trainee’s publications was 3.9 (3.2) vs .0 (2.6) (P < .0001) (Figure 1(C) and Table 2). This finding shows that CIP trainees continued to publish in strong peer-reviewed journals following CIP completion.
Comparing Research Productivity of Trainees with Research Involvement Before Beginning Residency
We found that 10/11 (91%) of MI-CIP trainees vs 39/74 (53%) of MI-non-CIP trainees had 1 or more publications prior to entering radiology residency (Figure 2). Of the 39 MI-non-CIP trainees, 26/39 (67%) had 1 or more publications at 7 years post-graduation, and 4/39 (10%) of non-CIP trainees had ten or more publications at 7 years post-graduation. In contrast, of the 10 MI-CIP trainees, 10/10 (100%) had 1 or more publications at 7 years post-CIP, and 5/10 (50%) of MI-CIP trainees had ten or more publications at 7 years post-CIP (Figure 2). These results demonstrate that a higher proportion of MI-CIP trainees continue engaging in research after graduation compared to non-CIP trainees. The 7-year post-CIP or post-graduation research productivity (number of publications) of the 10 MI-CIP trainees and 39 non-CIP trainees who had 1 or more publications prior to starting residency.
Comparing research outcomes from the University of Toronto’s Medical Imaging (MI) CIP vs General Surgery (GSx) CIP Trainees (Table 3)
Number of Total Publications
Before CIP, the median (IQR) number of total publications of MI-CIP vs GSx-CIP trainees was 5.0 (8.0) vs 2.0 (7.0) (P = .075) (Figure 3(A) and Table 3) and 6.0 (10.0) vs 6.0 (10.0) (P = .72) (Figure 3(A) and Table 3) at 7 years post-CIP. There were no significant differences between the number of publications from trainees in both CIPs at either time point. These results indicate that MI-CIP trainees matched GSx-CIP trainees in the total number of publications. Research productivity of the University of Toronto’s Medical Imaging Clinician Investigator Program (MI-CIP) trainees (n = 11) vs General Surgery CIP (GSx-CIP) trainees (n = 41) before CIP and post-CIP completion. Note, the time period for post-CIP includes research completed during CIP training to 7 years post-CIP completion. Error bars represent the median with interquartile ranges. A. MI-CIP vs GSx-CIP trainees did not have a statistically different number of total publications before CIP and at 7-years post-CIP completion. B. MI-CIP vs GSx-CIP trainees did not have a statistically different number of first-author publications before CIP and at 7-years post-CIP completion. C. MI-CIP vs GSx-CIP trainees did not publish in journals with a statistically different average impact factor before CIP and at 7-years post-CIP completion. Research Outcomes of Medical Imaging Clinician Investigator Program trainees vs General Surgery CIP Trainees Before CIP and at 7 Years Post CIP Training/Post-Graduation. Abbreviations: IQR, interquartile range.
Number of First-Author Publications
Before CIP, the median (IQR) number of first-author publications from MI-CIP vs GSx-CIP trainees was 2.0 (3.0) vs 1.0 (3.0) (P = .15) (Figure 3(B) and Table 3), and 2.0 (4.0) vs 3.0 (2.5) (P = .43) (Figure 3(B) and Table 3) at 7 years post-CIP. There were no significant differences between the number of first-author publications from trainees in both CIPs at either time point. These findings show that MI-CIP trainees matched GSx-CIP trainees in driving and contributing to research projects as a first-author.
Average journal impact factor
Before CIP, the median (IQR) average journal’s impact factor published by MI-CIP vs GSx-CIP trainees was 3.2 (2.0) vs 3.2 (3.7) (P = .81) (Figure 3(C) and Table 3), and 3.9 (3.2) vs 4.3 (2.6) (P = .43) (Figure 3(C) and Table 3). There were no significant differences between the average journal’s impact factor from publications of trainees in both CIP at either time point. These findings show that MI-CIP trainees continued to actively participate in strong peer-reviewed journals, similar to graduates of the GSx-CIP.
Discussion
In this study, we identified MI-CIP trainees as having increased research activity and engagement post-CIP compared to non-CIP trainees (Figure 1). Moreover, MI-CIP trainees are likely self-selected for this program given their significantly higher number of publications and research activity prior to beginning residency (Figure 1). This result support previous studies that demonstrated publications obtained before residency are a consistent predictive indicator for trainees to undertake a future academic radiology career. 7 We found no significant difference in measured research outcomes between MI-CIP and GSx-CIP.
Male-to-female ratio imbalance
This study showed that from 2006-2016, the MI-CIP had a significantly higher number of males than females (Table 1). It is interesting to note that this issue was specific to the MI-CIP cohort, as the MI-non-CIP and GSx-CIP cohorts had a balanced and non-statistically significant difference in male-to-female trainee ratio (Table 1). Previous research has found fewer females involved in academic activities during radiology residency compared to males. A recent descriptive bibliometric analysis of manuscripts submitted to the top 50 medical imaging journals showed that women were disproportionately underrepresented in the first and last authorship (32% and 21%, respectively). 11 Moreover, the sex disparity in radiology is greater than in many other medical specialties, with only 30% female radiology trainees and 25% female attending radiologists. 12 This statistic is in stark contrast to the equal sex distribution in medical school enrollment. 12 Overall, the gender imbalance in radiology represents a crucial area for improvement moving forward. Given that women radiologists identified a lack of mentorship as a key barrier to research engagement, 13 we envision that having more female radiologists conducting research and in leadership positions would encourage female trainees to pursue radiology research training.
Research Outcomes of MI-CIP Compared to Non-CIP and GSx-CIP
We found that trainees in MI-CIPs have a statically significant higher research productivity than MI-non-CIPs in all measured outcomes. When comparing research completed before residency, we found that 10/11 (91%) MI-CIP trainees vs 39/74 (53%) of non-CIP trainees had 1 or more publications before entering radiology residency (Figure 2). These publications may represent work from undergraduate studies, medical school, or previous graduate degrees. Of note, 26/39 (67%) non-CIP trainees had 1 or more publications 7 years post-graduation. We suspect that this research productivity is likely linked to curriculum requirements in the radiology residency program to produce a publication during residency. Although about half of all non-CIP trainees participated in research before residency, only 4/39 (10%) had 10+ publications 7 years post-graduation. In comparison, MI-CIP trainees have a statistically significant higher number of publications vs non-CIP trainees at 7 years post-CIP/post-graduation (Figure 1(A)). In addition, 50% of MI-CIP trainees had 10+ publications at 7 years post-CIP (Figure 2). Together, these results align with studies in American radiology residency programs showing that prior publications and additional advanced degrees are associated with an increased likelihood for a trainee to pursue an academic radiology career. 7 Therefore, residency selection committees may consider prior publications as a potential qualifying selection factor if the goal is to recruit and train future academic radiologists.
Interestingly, there were several productive research outliers in the MI-non-CIP group. These outliers may represent trainees with professional degrees (MSc or PhD) acquired before entering radiology residency. In these circumstances, these trainees may pursue a post-doc via CIP if desired. Other possibilities for outliers include non-CIP trainees with a strong research background completing residency (without pursuing a post-doc via CIP) and then completing a fellowship with research opportunities.
Of note, we did not include senior author publications in our analysis since all 11 radiology-CIP trainees had zero senior author publications 7 years post-CIP completion. We believe that this is because of training time and difficulty in establishing a research career early on after completing radiology residency. There is increasing difficulty in acquiring funding from federal and non-federal sources 14 with a general lack of departmental support. 15
There were no statistically significant differences in the number of publications, the number of first-author publications and the average impact factor of journals between the MI and GSx-CIP trainees. This finding reveals that the smaller and relatively newer MI-CIP is training academic radiologists with comparable research productivity to trainees from the established GSx-CIP. Additional research is needed to evaluate the long-term careers of former CIP trainees to understand the impact of the program on continuing academic successes beyond publications. Some of these metrics may include the successful accrual of research funding, evidence of continued academic productivity and the establishment of research labs.
Barriers to Research in Radiology Training Programs
Previous studies have extensively examined potential barriers to research participation in radiology trainees. Common topics identified in the literature include the lack of funding, inadequate research mentorship and lack of departmental support.16-21 Overcoming these barriers in individual Medical Imaging Departments and local and global communities is needed.
To address the lack of research funding in the United States, the American Board of Radiology introduced a specific clinician-scientist program to support research-oriented radiology and radiation oncology residents - The Holman Research Pathway (HRP). 22 On top of the traditional 5-year residency training time, trainees in the HRP can take up to 2416-21months of additional dedicated research time, 22 which may extend their training to a total of 7 years. 22 Recent retrospective reviews of the program demonstrated that a statistically significant higher proportion of HRP trainees have moved on to academic careers compared to non-HRP trainees.19,23 Moreover, graduated HRP trainees have significant research productivity.19,23 These outcomes represent the initial success of the HRP in supporting radiology research training in the United States.
In Canada, the MI-CIP has been available for trainees for many years. Yet, the MI-CIP continues to be one of the lowest numbers of trainees compared to all other CIPs. 24 We believe that a lack of mentorship and uncertainties surrounding job prospects deter interested trainees from pursuing research.16-21 Discussions regarding how to define standards for compensation of protected academic time for radiologists in academic departments may help change trainees’ impressions of medical imaging research. In addition, mentorship regarding family planning and even the optimal time to embark on research training would be invaluable.
When examining well-established CIPs, such as the General Surgery CIP, there is a strong sense of research culture rooted in the faculty and program leadership. Thus, for radiology research programs to succeed, there needs to be leadership promoting research culture. This can take many forms, including academic support for residents and faculty. Although all Canadian Diagnostic Radiology residency programs have a mandatory scholar/research component for accreditation, 20 the research requirements and curriculum between schools are variable. 21 We believe that initiating a standardized research curriculum between all Canadian Diagnostic Radiology programs could enhance the resident research culture and demonstrate the value and importance of research in radiology.
The Academic Institution’s Role in Promoting Research Training
For faculty to support trainees’ research activities, greater institutional support are required. This may include rewarding faculty for research and providing protected research time for academically productive faculty based on merit. In many academic radiology centres, the time allocated for research is obtained based on successful accrual of grant awards. One drawback of this model is the increasing difficulty in obtaining external funding from federal and non-federal sources, 14 which may discourage research. Other funding models have staff members convert their clinical time into academic time, with a salary provided at the clinical rate. 25 However, this salary may be dependent on achieving specific publication benchmarks. In 2010, a Canadian radiology department implemented a metric-based research-protected time program to provide research salary awards along with protected time to faculty radiologists. 26 Metrics include evaluating research productivity using publications, grants (a dollar amount) and citation indices. 26 The metric-based program emphasizes the value of academic radiology as equal to clinical duties, and the salary granted by the program is equivalent to the same value concerning clinical work. The program converts the research salary award into days where research is performed instead of clinical work. This salary is guaranteed and may provide a greater incentive for faculty to engage in research. Over 9 years, the metric-based model was associated with increased departmental publication productivity and publication impact factor. 26 We suggest that academic departments use a similar approach or develop a customized metric-based model to provide protected time for research. By supporting faculty, we believe that trainees will benefit from the increased opportunities for research engagement.
Study Limitations
There are limitations to the study. Firstly, our study did not address the type of research projects that CIP or non-CIP trainees pursued. Different research projects, such as basic science vs clinical projects, would have differences in completion time, number of publications, and journal impact factor. In addition, there is a potential selection bias in the study results since MI-CIP trainees have a greater number of research publications than non-CIP trainees before beginning their residency programs. Secondly, we determined outcomes based on research productivity, such as the number of publications and average journal impact factor. Other markers of research productivity were not evaluated, such as presentations in scientific meetings given the challenges of finding this information in the public domain. Also, this study did not capture unpublished research projects. Thirdly, given the small MI-CIP cohort size compared to the non-CIP and GSx-CIP cohorts, the observed gender imbalance may not as be robust. Lastly, our study did not identify the number of trainees pursuing master’s degrees, PhDs, or post-doctoral work outside of CIP. Pursuing a graduate degree or post-doctoral work post-graduation represents another pathway to engage in research. This may have been represented in our data by several non-CIP outliers. However, we do not expect this number to be significant as our analysis of CIP vs non-CIP publications did not demonstrate any statistical significance.
Conclusions
Our study provides initial baseline information for evaluating MI-CIPs. The initial research productivity of CIP trainees is significantly higher than non-CIP trainees in all measured research outcomes, which represents the success of the MI-CIP. Furthermore, the study results showed the equivalence of measured research outputs of MI-CIP trainees and trainees of the well-established GSx-CIP at the University of X. Although there are areas for improvement, the MI-CIP represents an opportunity for encouraging and developing research-oriented radiologists. Future research is needed to assess the long-term outcomes of MI-CIPs, not only in our institution but also in different geographic locations around the world. Different factors, such as cultural, financial and social-environmental conditions may impact the outcomes of CIPs.
Footnotes
Acknowledgement
The authors would like to thank Drs. Rachel Fleming, Kate Hanneman, and Birgit Ertl-Wagner for their review and feedback on this work. We would also like to thank Alison Clute for help with editing the French abstract.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article. The authors declare that the research was conducted in the absence of any commercial or financial funding supporting this work.
