Abstract
Introduction
Research activity is a fundamental aspect of creating a well-rounded radiologist who can use and apply evidence-based medicine. It is thus vital to incorporate scholarly activities into one’s medical imaging training early on to promote life-long learning. In Canada, the Royal College of Physicians and Surgeons oversees the implementation of a physician competency framework called “CanMEDS” in all residency training programs, with radiology being no exception. 1 This framework defines 7 key roles that encompass a physician, one of which is the role of scholar. 1 The scholar role is summarized as being able “to contribute to the application, dissemination, translation, and creation of knowledge and practices applicable to health and health care.” 1
Each radiology residency program in Canada is therefore tasked to follow this educational framework with no single “gold standard.” The lack of such a gold standard contributes to significant heterogeneity in research requirements, emphasis, and productivity across Canadian Medical Imaging residency programs.2,3 Residents can pursue different routes for research enquiry and knowledge dissemination of findings including presenting at local research days, presenting at external conferences, and publishing in a journal.1,4,5 These are not mutually exclusive as a single research project may be presented both internally and externally as it proceeds through its phases of completion which may ultimately include publication of the results.
The extent to which these different paths lead to one another has been partially studied, with previous studies largely focusing on publication rates of abstracts at large scientific meetings. A Cochrane review demonstrated that 37–46% of abstracts presented at scientific conferences went on to publication. 6 Combining the Cochrane review with newer publications shows the overall abstract to publication conversion rate at radiology conferences is 26%, based on 6651 publications from 25,498 presentations at 74 conferences.5-8 Abstract to publication conversion rates from the 2005 to 2011 Canadian Association of Radiologists Annual Scientific Meetings (CAR ASMs) was 28%. 9
However, there are few studies focusing on resident research projects. A recent review of 287 resident research presentations from 32 local research days at 7 different Canadian programs showed an overall presentation to publication conversion rate of 34%. 5 The only other four publications on internal presentation to publication showed conversion rates of 28%, 39%, 55%, and 64%, where none of the studies involved radiology programs.10-13 There are currently no published studies on rate of external conference presentation for resident research and no studies on the impact of external presentation on publication rates for resident research projects.
The objective of this study was to determine how many Canadian diagnostic radiology resident research day projects are presented at the national CAR ASM and if presentation at that meeting is associated with increased rates of project publication. Our hypothesis was that presenting at CAR ASM is associated with an increased rate of project publication.
Methods
This project was considered exempt from research ethics approval at our institution. A previously generated database of radiology resident presentations at 2012–2017 research days at 7 Canadian radiology programs including publication status was utilized. 5 These comprised 7 of the 13 English speaking Canadian residency programs which are anonymized for this publication. 5 Geographical scope was wide with 3 schools from Western Canada and 4 schools from Eastern Canada. Full resident lists were not available, but approximate percentage of residents from the 13 English speaking schools was estimated by quota of spots at each program as listed in the 2022 Canadian Resident Matching Services (CaRMS) or the number of first year residents on program web sites if not published on the CaRMS webpage.14-16 Assuming relatively consistent resident numbers across years, this estimates that the included programs accounted for 58% of Canadian radiology residents at English speaking schools. Available data provided by the programs was variable ranging from full books of abstracts to only presentation lists including presenters and titles. As previously described, this database was generated by reviewing research day presentation lists or abstracts from the participating schools to identify medical imaging resident research projects with subsequent internet searching of names and presentations via PubMed, Web of Science, and Google Scholar in the summer of 2018 to identify any connected publications. 5 This identified a total of 287 unique resident presentations of which 100 were published, an overall 35% publication conversion rate. 5 Please note this is discrepant from the 99 publications in the initial report because of an error in the initial database which was uncovered in this review.
The programs/books of abstracts from the 2011–2019 CAR annual meetings were reviewed. The 2014–2019 abstracts are publicly available on the CAR website while the 2011–2013 were provided by CAR staff through email query. 17 The meeting programs contain the full abstracts of each presentation, including author name, presentation title, affiliated institution, and date of presentation. Each resident that presented a research day project in the previous database was individually searched through the CAR meeting abstracts. The years 2011 and 2019 were included to capture any research presentations presented at the CAR before or after resident research day presentations. Each CAR presentation including the resident identified through the research day presentation lists was identified, irrespective of author position. This included presentations both related to and separate from the medical imaging internal research day presentations. Data including presentation year, type of presentation (oral vs. poster), and total number of CAR presentations was tabulated. Publication status of projects presented at internal research days was determined from the previous database. Searching for publication status on non-research day projects presented at the CAR ASM by these residents was not performed.
For projects that were presented at both internal research days and the CAR ASM, time between internal and CAR ASM presentation was determined. Since actual dates of several internal research days were unknown, time to presentation was determined as the number of calendar years between the research day and CAR presentations.
Descriptive statistical analysis was used to determine how many research day projects were also presented at CAR and how many related and unrelated presentations were made by these residents. Interinstitutional variability was also assessed for the proportion of residents that presented at CAR ASM, the proportion of research day projects also presented at the CAR ASM, and the proportion of research day projects which were also published using Chi-square statistical testing. Logistical regression analysis was used to calculate likelihood ratios to determine associations between presentation of a research day project at CAR ASM and publication rate of that individual project. Logistical regression analysis was also used to calculate likelihood ratios to determine association between publication of a research day presentation and varying levels of total CAR presentations by the identified residents (zero presentations versus 1, 2–4, or ≥5).
Results
Breakdown of Internal Research Day and CAR Presentations Including Publications by Residents at Programs across Canada.
Note #1: RD = Research Day, CAR = Canadian Association Radiologists Annual Scientific Meeting. *Research Day Presentations for 5 years from all schools except #3 which had only two years of data
There were 195 CAR ASM presentations by 93 of the 208 residents identified through internal research day presentation lists. The mean number of presentations per resident was .9 +/− 1.9 (Figure 1). For those residents who presented at CAR, 55 made a single presentation, 29 made between 2 and 4 presentations, and 9 made 5 or more presentations. Number of CAR ASM presentations per resident.
For the 36 projects presented at both internal research days and the CAR ASM, the mean time to from research day to CAR ASM presentation was 2.3 months +/− 11.6 months. Presentation at CAR ASM was before research day for 6 (17%) projects (one 2 years prior and five 1 year prior), the same year as research day for 21 (58%) of projects, or after research day for 9 (25%) of projects (five 1 year after, three 2 years after, and one 3 years after).
Presentation dates, publication dates, and publishing journal for the 18 research day presentations that were also presented at the CAR ASM and published.
RD, Research day; CAR ASM, Canadian Association of Radiologists Annual Scientific Meeting; CARJ, Canadian Association of Radiologists Journal; Can J Cardiol, Canadian Journal of Cardiology; Curr Probl Diagn Radiol, Current Problems in Diagnostic Radiology; BMC Cardiovasc Disord, BMC Cardiovascular disorders; J Obstet Gynaecol, Journal of Obstetrics and Gynaecology, Adbom Radiol, Abdominal Radiology; AJR, American Journal of Roentgenology; Can J Neurol Sci, Canadian Journal of Neurological Sciences
Presenting a research day project at the CAR ASM was associated with an increased rate of publication for that individual project with Relative Risk (RR) 1.537 (P=.0396, 95% CI 1.059-2.229). However, in general, CAR ASM presentations unrelated to RD projects by an individual resident did not statistically increase the odds of publication for their research day projects. When compared with zero CAR ASM presentations by a resident, the RR for publication of their research day projects were 1.125 (P=.574, 95% CI .669–1.581), 1.289 (P=.362, 95% CI .623–1.955), and 1.688 (P=.141, 95% CI .712–2.663) for residents with 1, 2–4, and 5 or more CAR ASM presentations, respectively.
Discussion
There is limited research analyzing the progression of resident research projects from inception to publication, especially on the progression from local research day presentation to national conference presentation and to publication. Specifically, we identified 5 prior studies on resident internal presentation to publication rates, with only 1 related to diagnostic radiology.5,10-13 These studies range from a 28–64% publication rate, including a 35% publication rate for the data reassessed in this project.5,10-13 However, none of those studies assess how often these research day presentations are also presented at external conferences or how external presentation may be associated with publication of a given project. We have found that only 13% (36/288) of Canadian radiology resident research day projects go on to be presented at our national specialty conference (Canadian Association of Radiologists’ Annual Scientific Meeting). However, there were a total of 195 CAR ASM presentations presented by the same group of residents between 2011 and 2019. This shows that resident research extends beyond the projects presented at research days as 82% of their CAR ASM presentations were unrelated to these projects illustrating how radiology residents are often pursuing multiple research topics and may follow different routes for knowledge translation and presentation on these different projects.
The research day projects presented at the CAR ASM represented only 36 of the 288 (13%) of the studied research day projects. This project was only able to assess for external presentation at the CAR ASM. It is also likely that radiology residents submitted their projects to other meetings, such as the Radiological Society of North America (RSNA), American Roentgen Ray Society (ARRS), or other societal meetings but, unfortunately, those meetings do not have publicly accessible conference abstract databases to enable assessment. Thus, we believe the 13% CAR presentation rate underestimates the true conference presentation rate for research day presentations. Similarly, the total number of external presentations by these residents is likely underestimated as well.
There was heterogeneity in how prolific each resident was in terms of the number of CAR presentations. Amongst the 208 residents identified as being involved with research through participation in local research days. 115 (55%) did not have any presentations at the CAR ASM, 55 (26%) had 1 CAR ASM presentation, 29 (14%) had between 2 and 4 CAR ASM presentations, and 9 (4%) had five or more CAR ASM presentations. The top 3 prolific residents had their names on 18, 12, and 6 CAR presentations, respectively. There was also significant interinstitutional variability in the percentage of research day topics also presented at CAR ASMs (P=.016) and percent of research day projects published (P<.001). This heterogeneity likely boils down to a combination of independent resident research interest as well as individual residency program research requirements and supports.2,5 We had expected residents with more CAR presentations to have a higher likelihood of publishing their research day presented projects as it could be assumed that they would be more interested in research. However, this was not the case, as there was no statistically significant association between project publication and number of presentations at CAR ASM.
There has been relatively little research on research productivity of Canadian radiology residents. The previous study by Melendez et al (which shares a common research day data base with this study) revealed that there were 118 publications generated from research day projects by 82 of the 208 residents identified via presentations at research days. 5 A prior 2017 survey of Canadian residents found that 88% had research experience prior to starting radiology residency and 34% had published radiology research within peer reviewed journals during their residency. 2
Our study showed that if a radiology resident research project was presented at the CAR ASM it was associated with a higher likelihood of publication (RR 1.537, P=.0396), demonstrating the importance of these national meetings for sharing high quality research. There may be an underlying selection bias, with stronger or more completed projects potentially being more likely to pursue conference presentation in keeping with the previously demonstrated effect that abstracts accepted versus rejected for presentation at conferences are more likely to be published (RR=1.65). 6 As mentioned above, this effect was distinct from additional presentations by the residents unrelated to their research day projects, which further strengthens the value of presenting an individual project at the CAR conference. Publication bias could also be a contributing factor if there is a similar selection bias for abstracts accepted for presentation at external meetings such as the CAR ASM as there is for journal publication. Previously investigated selection biases in the Cochrane review by Sheerer et al potentially relevant to our study include tendency to publish studies with positive results (RR 1.31 and 1.37), larger sample sizes (RR 1.09), randomized trial design (RR1.51), and prospective design (RR 1.17). 6 An example of publication bias specific to radiology is from abstracts on diagnostic test accuracy presented at the 2011 and 2012 RSNA meetings where abstracts with positive conclusions were more likely to be published (Odds Ratio (OR) 3.6). 8
There are only two prior published studies on factors affecting publication of resident research projects. A prior study of plastic surgery 2009–2013 resident research presentations found that higher academic rank of research supervisor was associated with greater odds of publishing a project (OR 3.3 if rank of at least associate professor). 11 This same study found internal presentation of research early in training to be associated with better publication rates with OR of publishing decreasing by 40% (OR .56) with each increase in PGY level. 11 Conversely, a study of 1993–1997 internal medicine research presentations at the University of British Columbia (UBC) showed that a higher level of residency was associated with increased rates of publication (RR 2.1 if greater than or equal to Post Graduate Year (PGY) 3). 9 This UBC study also found male gender (RR 2.6) and previous experience with a research project (RR 1.6) were likewise associated with increased publication rates. 10 There have been more studies on factors associated with difficulties in performing research during residency. The most relevant to our study is a 2017 survey of Canadian radiology residents which found the most cited obstacles to conducting research during residency being lack of time (67%), inadequate mentorship (32%), personal disinterest in research (32%), lack of research training (22%), and lack of professional or financial support (17%). 2
In terms of timing from internal presentation at research day to external presentation at CAR ASM, 6 were presented before, 21 the same year, and 9 were presented after CAR ASM. This is contrary to what we had expected. We had expected almost all research day presentations to be before or the same year as the CAR ASM presentations as internal presentations may be more likely to include works in progress and earlier iterations of the projects. This heterogeneity in timing may be because of different thresholds or requirements for research day presentations at each of the 7 schools.
We found that for the 18 projects presented at CAR ASM that were published the mean time from internal presentation to publication was 15.3 +/− 9.0 months and the mean time from CAR presentation to publication was 14.7 +/− 8.8 months. This compares with 12.3 +/− 13.6 months from internal presentation to publication for all 2013–2017 published research day projects and 16.5 months mean time from the 2005–2011 CAR ASMs to publication.5,9 Additionally, of the 18 research day projects also presented at CAR ASM, 17 of them were published after the CAR presentation. Although some of this may be due to publication lag time, it may also point to the refinement of a project which is made when presenting at a meeting such as the CAR ASM. It does certainly reinforce the concept of the CAR ASM being a medium that facilitates and encourages high quality research in radiology with presentation of results before they are in press, and sometimes even before they are presented at their home institution.
Also of note is that 10 (57%) of the 18 published research day projects presented at CAR were published in the CARJ. This compares to 30 (27%) of the 112 publications from 2005 -2011 CAR ASMs being in the CARJ and 16 (13%) of 118 publications from the base resident research day presentations being in the CARJ.5,9 This could indicate residents targeting both the CAR ASM and CARJ with projects that they feel could have an overlapping Canadian target audience.
There are limitations to our study. The main one is that we were limited searching to CAR ASM conferences. We had initially wished to include the RSNA and ARRS conferences, but their annual meeting programs containing presented abstracts from these conferences were not publicly available. Attempts were made to contact these societies and obtain these meeting abstracts but were unfortunately unfruitful. However, despite this we were able to demonstrate a higher likelihood of publication from presentation at the CAR ASM. Additionally, data from this project reflects a relatively short snapshot in time which is a limitation as the radiology residency takes place over 5 years. As such, we may have not captured all the CAR presentations for a given resident depending on the year of residency that residents were in at the time of their research day presentations. Although we did not show a statistically increased likelihood for a research day project to be published when resident researchers are more prolific at the CAR ASM, the assessment of this may be underpowered as there were only 9 residents in the 5 or more CAR ASM presentations group. Although it would have been ideal to include data from all Canadian residency programs and residents, the data provided is only for 7 of 13 schools encompassing approximately 58% of Canadian radiology residents. This compares with a response rate of 66% for UBC internal medicine residents and a 22% response rate in the Canadian Radiology Resident survey on barriers to research.2,10 Response bias cannot be excluded, but it is likely to have been less than if individual residents were surveyed. Finally, it would have been ideal to further assess presentation and publication rates of resident research day projects based on factors such as project methodology (prospective vs. retrospective), sample size and findings (positive vs. negative results) to look for potential publication or presentation bias. However, given the data available from the programs this was not possible.
Overall, 13% of Canadian radiology resident research day projects go on to presentation at the CAR. Presenting at the CAR is associated with an increased likelihood ratio of publication (RR 1.537, P=.0396).
Footnotes
Acknowledgements
We would like to thank Ha Le with the Clinical Research Support Unit at the University of Saskatchewan for work on the statistical analysis and Carissa Piercey from the CAR for her assistance with the meeting book of abstracts.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
