Abstract
BACKGROUND
Recruitment and retention of Pulmonary and Critical Care Medicine (PCCM) trainees into academic research positions remain difficult. Factors influencing graduates, like salary and personal circumstances, remain unchangeable. However, some program-level factors, like research skill acquisition and mentorship, may be modifiable to encourage matriculation into academic research positions.
OBJECTIVE
We aim to identify proficiency in research-specific skills in PCCM trainees and barriers to careers as research-focused academic faculty.
METHODS
We surveyed PCCM fellows in a nationwide cross-sectional analysis including demographics, research intent, research skills self-assessment, and academic career barriers. The Association of Pulmonary and Critical Care Medicine Program Directors approved and disseminated the survey. Data were collected and stored using the REDCap database. Descriptive statistics were used to assess survey items.
RESULTS
612 fellows received the primary survey with 112 completing the survey for a response rate of 18.3%. A majority were male (56.2%) and training at university-based medical centers (89.2%). Early fellowship trainees (first-/second-year fellows) comprised 66.9% of respondents with 33.1% being late fellowship trainees (third-/fourth-year fellows). Most early trainees (63.2%) indicated they intended to incorporate research into their careers. A chi-square testing of independence was performed to examine the relationship between training level and perceived proficiency. Significant relationships in perceived proficiency were identified between early and late fellowship trainees with an absolute difference of 25.3% (manuscript writing), 18.7% (grant writing), 21.6% (study design), and 19.5% (quantitative/qualitative methodology). The most prevalent barriers were unfamiliarity with grant writing (59.5%) and research funding uncertainty (56.8%).
CONCLUSION
With an ongoing need for academic research faculty, this study identifies self-perceived gaps in research skills including grant writing, data analytics, and study conception and design. These skills map to fellow-identified barriers to careers in academics. Mentorship and innovative curriculum focusing on the development of key research skills may enhance academic research faculty recruitment.
Introduction
Recruitment and retention of pulmonary and critical care medicine (PCCM) trainees into research-focused academic careers is an ongoing challenge. These potential physician–scientists would fill the gap between clinical science and research skills to advance their fields. 1 The National Institutes of Health (NIH) and the American Association of Medical Colleges (AAMC) have recognized the need for more effective retention efforts given the high reported rates of physician–scientist attrition.2,3 An important part of addressing this trend needs to include the recruitment of graduating trainees into academic research careers; however, prior investigation has shown that even research-forward sub-specialty training programs recruit less than half of their graduates to academic positions. 4 Similarly, a recent survey study of program directors (PDs) reported that only 20% of PCCM fellowship graduates pursue academic research careers. 5 To optimize recruitment into research careers, it is critically important to understand and identify the acquisition of key research skills during training and barriers to matriculation into academic research careers from current fellows.
Prior work has suggested that several factors may influence a graduating trainee's decision to remain in an academic research career. Some factors are beyond the scope of a training program such as salary differential, prior exposure to research, and reluctance to rely upon competitive grant funding.6,7 However, several identified risk factors may be addressable by fellowship training infrastructure. For example, graduating PCCM fellows have identified a lack of career development skills and research opportunities as significant factors that contributed to career paths. 8 Similarly, the availability of mentoring and acquisition of research skills during training have also been identified as key needs by trainees to pursue academic research careers.7,9
Here, we sought to better understand how current PCCM fellowship training programs are addressing these important and potentially alterable characteristics as well as capture trainees’ current attitudes toward research careers. There are currently over 700 graduating fellows across over 100 academic PCCM training programs yearly.3,10 Additionally, education on scholarly activity is part of the American College of Graduate Medical Education (ACGME) core requirements for PCCM fellowship. 11 We conducted a cross-sectional nationwide survey of these fellows focused on self-assessment of research skills at their current level of training as well as attitudes and perceived barriers to careers in academic PCCM.
Methods
Survey development
A literature review focused on factors relating to preparation for, barriers to, and retention in academic research careers was performed.4,6,8 Using themes identified in the literature, a 39-question survey, with 37 quantitative items and 2 open-ended qualitative questions, was developed through rigorous discussion and review by a multidisciplinary team, including career academic researchers in the basic and clinical sciences, clinician educators, and fellows in PCCM. Questions were assessed for content, clarity, usefulness, and thoroughness through an iterative review process. 12 The survey was piloted on the PCCM fellows (2% of surveyed population) at the Medical University of South Carolina and underwent internal testing to ensure the branching logic was clear and no errors were encountered.
Demographic data was requested to include gender, race, ethnicity, program type, program size, program geographic region, year in fellowship training, previous experience with research before fellowship training, and current research focus. Respondents were also asked to self-assess their research abilities using a 5-point Likert scale ranging from No Proficiency to Expert Proficiency. Specifically, they were asked to rate their abilities concerning the management of a mentor/mentee relationship, time management, manuscript writing, grant writing, abstract writing, technology-assisted web-based teaching, oral presentations, critical evaluation of research results, ability to develop new research directions, data analysis, quantitative methods, qualitative methods, and overall study design. Respondents were further asked to identify perceived barriers to pursuing an academic research career from a pre-populated list using a “select all that apply” format. Open-ended questions regarding barriers were also included to facilitate additional feedback on barriers to academic careers and perceived gaps in academic research training. The survey instrument is available in the data supplement named Appendix E1. Using the MUSC IRB Self-Evaluation Tool, 13 the project was determined to be not human subjects research and, therefore, exempt from IRB approval. An anonymous survey link was sent through national PDs with informed consent implied by initiating the survey.
Survey dissemination
The survey was submitted to the Association of Pulmonary and Critical Care Medicine Program Directors (APCCMPD) survey review committee and board of directors who approved the survey for dissemination to PDs nationally. The ACCPMPD membership is comprised of members from 96% of all fellowship programs in Pulmonary Medicine, Critical Care Medicine and combined Pulmonary and Critical Care Medicine. 14 The APCCMPD disseminated the fellow survey to all PDs via email. Using a convenience network sampling method, 15 the PDs were asked to forward the survey link to all of their current fellows in training. PDs also received a link to a second survey consisting of a single question asking for the number of fellows to whom the survey was distributed to calculate a response rate. The survey request was emailed from the APCCMPD per their guidelines to the PDs every other week for 6 weeks, or three times total, from March 2021 to April 2021. Participation in the survey was voluntary and no compensation was offered for completion of the survey. Survey data were collected using the Research Electronic Data Capture (REDCap) database.16,17
Data analysis
Due to the lower than expected survey response and to improve the chance of statistical significance while still providing meaningful outcomes, Likert scale responses were dichotomized post-survey by the research group to identify current trainee self-assessment of the adequacy of proficiency in skills for a potential research forward career. For Likert scale responses, No proficiency to Emerging proficiency (a score of 1 or 2) were combined as “Inadequate Proficiency” and Moderate proficiency to Expert proficiency (a score of 3, 4, or 5) were combined as “Adequate Proficiency.” Quantitative data were analyzed using descriptive statistics and stratified by year in training to compare the perceptions of skills and barriers between first-/second-year fellows and third-/fourth-year fellows as we a priori hypothesized that stage in training would be associated with differing perceptions. Chi-square was used to compare differences in self-perceived proficiency across these strata. Descriptive statistics and chi-squared anaylsis were analyzed using the QI Macros function of Microsoft Excel. Logistic regression and functional ANOVA analysis were performed using Stata software 18 to evaluate for significant relationships between independent and dependent variables. Qualitative analysis of open-ended text box responses was performed by two researchers with qualitative training. Analysis was performed by two female analysts trained in qualitative methods19,20 using a general inductive approach and thematic analysis. Comments were read multiple times through an iterative process to identify relevant categories related to the attitudes and barriers to research careers. No software was used. The categories were then compared and contrasted across the four fellowship years to identify emerging themes. 21
Results
Survey dissemination and response data
A total of 82 PDs opened the single-item PD survey and 47 responded with the number of fellows to whom they distributed the primary survey. They indicated that a total of 612 fellows were forwarded the primary survey at least one time. Of these, 194 respondents initiated a survey resulting in 101 fully completed surveys and 11 surveys that were >90% completed with few individual questions left unanswered either intentionally or unintentionally. The remaining 82 respondents opened the survey but did not complete a single section of the survey in full and were excluded from the analysis. The overall response rate was 18.3% (112/612).
Demographic data
Of the 112 survey respondents included in the analysis, 63 (56.2%) were male which is comparable but slightly lower than recent nationwide estimates of the proportion of male pulmonary and critical care fellows (66%) 3 (Table 1). There was broad geographic representation as 36 (32.1%) respondents were training at programs in the Northeast, 24 (21.4%) at programs in the Midwest, 29 (25.9%) at programs in the Southeast, and 23 (20.5%) at programs in the West (Table 1). One hundred (89.2%) respondents described their current training environment as a university-based academic medical center and respondents were reasonably distributed from programs of varying sizes. Fifteen (13.4%) respondents had attained an additional research degree before or during training. Seventy-one (63.4%) respondents had no prior first-author publications, whereas 24 (21.4%) had one, 4 (3.6%) had two, and 13 (11.6%) had three or more prior first-author publications. Among the fellows who responded, the majority indicated they are participating in clinical research, while quality improvement, medical education, and translational science were also frequently selected. Basic science and health services research were the least common types of research selected by respondents. Twelve (15.7%) respondents indicated they are not participating in research and 5 (6.4%) indicated they are not interested in doing research.
Respondent demographics data.
Self-Assessment of research skills
While the majority of first- and second-year fellows self-reported adequate proficiency in all queried skills; several specific skills were associated with higher rates of designated inadequate proficiency (Figure 1). Specifically, skills related to research study initiation including “study design” and “developing new research directions” were both identified as self-perceived areas of low proficiency. Similarly, data processing and analysis skills such as “data analytics,” “critical evaluation of results,” “quantitative methods,” and “qualitative methods” were also identified as areas of low proficiency by early career fellows. Manuscript and grant writing were additionally seen as challenges.
Self-Assessment of proficiency in research skill domains across fellowship—early fellowship trainees, defined as first- and second-year fellows, and late fellowship trainees, defined as third- and fourth-year fellows, self-assessments of skill in several research skill domains. 5-point Likert scoring on a proficiency scale converted to dichotomous adequate versus inadequate proficiency.
Third- and fourth-year fellows self-reported higher levels of proficiency in several skills that were reported as inadequate proficiency by early-stage fellows (Figure 1). Notably, “study design,” “critical evaluation of results,” and “manuscript writing” all demonstrated significantly higher rates of adequate proficiency among later-stage fellows. However, despite these trends, proficiency in several skills was still perceived as inadequate by large proportions of the later-stage respondents. These included “grant writing,” “developing new research directions,” “data analytics,” and “methods.”
A chi-square test of independence was performed to examine the relation between training level and perceived proficiency, and several areas showed significant relationships. Significant relationships in perceived proficiency were identified between early and late fellowship trainees for manuscript and grant writing with absolute differences between late fellowship trainees and early fellowship trainees being 25.3% (X2 [1, N = 104] = 6.06, P = .013) and 18.7% (X2 [1, N = 103] = 5.39, P = .02), respectively. Additionally, significant absolute differences were seen for study design: 21.6%, (X2 [1, N = 106] = 4.49, P = .034) and quantitative/qualitative methodology: difference: 19.5%, (X2 [1, N = 106] = 3.85, P = .049).
Logistic regression and factorial ANOVA testing using Stata software 18 were performed testing multiple independent demographic variables for relationship to research skills. Prior first authorship of a manuscript was a significant factor for a trainee to self-evaluate as having adequate proficiency in “study design,” “qualitative methods,” “quantitative method,” “data analytics,” “developing new research directions,” “critical evaluation of results,” “manuscript writing,” and “management of time.” A level of training later in the fellowship is significant for self-evaluation of adequate proficiency in “study design,” “qualitative methods,” “quantitative methods,” “grant writing,” and “manuscript writing.” Gender was found to be a significant factor with females self-evaluating for inadequate proficiency for “developing new research directions,” “critical evaluation of results,” and “manuscript writing.” There were no significant relationships for “poster presentations,” “oral presentations,” “technology and web-based teaching,” “abstract writing,” and “management of mentor/mentee relationship.”
Attitudes and barriers toward academic careers
Early fellowship trainees were queried on a 5-point Likert scale as to their intent to incorporate research into their careers. Nearly two-thirds responded in the affirmative with 63.2% overall indicating their intention to pursue an academic research career with 42.6% agreeing and 20.6% strongly agreeing with this as a career path while only 14.7% reported no intention of pursuing an academic career. Approximately 20.6% of early training fellows were neutral or undecided (Figure 2).
Intent to incorporate research—query of early fellowship trainees, defined as first and second-year fellows, current intent to incorporate research in long-term career planning.
Late fellowship trainees were further asked to identify personal barriers to continuing a career in academic medicine with a focus on research by selecting from a list of common considerations (Figure 3). Unfamiliarity with grant writing (22, 59.5%), lack of certainty about research funds (21, 56.8%), and difficulty developing sustainable and innovative research ideas (12, 32.4%) were frequently identified as barriers that may be associated with inadequate proficiency in research skills. Lack of mentorship availability (5, 13.5%) and diversity (6, 16.2%), lack of networking with other research-focused physicians (8, 21.6%), concerns about job opportunities (8, 21.6%), and local politics/conflict (3, 8.1%) were potential institutional/cultural-based barriers to careers in academic medicine. Work-life imbalance (13, 35.1%), not being motivated by research (11, 29.7%), financial concerns about staying in academia (10, 27.0%), and personal or family circumstances (3, 8.1%), were identified as potential personal barriers.
Barriers to a career in research—late fellowship trainees’ identification of barriers to pursuing research forward academic careers.
Open-ended responses both corroborated the above findings and identified additional barriers. Across all respondents, COVID-19 was a barrier that consumed time and resources for trainees and mentors. Some respondents elaborated that the pandemic increased the amount of time spent clinically, thus reducing the amount of time available for research. In addition, respondents noted that most conferences and trainings were virtual (or postponed) during the pandemic, which limited opportunities for mentoring, networking, and presentations. Multiple comments across all respondents suggested a lack of mentorship or access to and awareness of resources. This crossed a variety of topics and is represented by comments from respondents that mapped to these several key domains as outlined in Table 2.
Thematic description of barriers to training with creative solutions.
Column 1: barriers; Column 2: selected quotes; Column 3: creative solutions.
Discussion
Recruitment of graduating PCCM fellows into academic research positions continues to be challenging. This study surveyed a broadly representative population of fellows in training to characterize their self-perceived proficiency in research skills, compare this proficiency across stages of fellowship training, and identify potential barriers to pursuing an academic research career. Our findings are notable in that they: (a) identify specific research skills that fellows lack proficiency early in fellowship training, (b) identify key research skills that remain inadequate in late fellowship indicating skills that may not be sufficiently addressed during fellowship training, and (c) identify barriers that map to these self-assess deficiencies contributing to a reduction in matriculation to research focused academic faculty. These data may provide targets that training program leadership can use for focused improvement in research career preparation and career development to enhance efforts to recruit graduating trainees into academic research careers (Table 2).
This study evaluated a cross-section of early and late trainees in fellowship to characterize their self-assessment in key research skills. When comparing single time point self-assessment of skill, our results show that late trainees assessed more proficiency than early trainees and this would suggest that fellowship training inherently does improve the self-perception of proficiency in a variety of research skills including study design, quantitative and qualitative methodology, manuscript writing, and grant writing. However, despite this inherent skills gain, the majority of late fellowship trainees still reported not feeling proficient at grant writing, data analytics, or developing new research directions. The extent to which a lack of proficiency in these three skills contributes to low recruitment rates into academic research careers is unclear. However, respondents identified “unfamiliarity with grant writing” as the most common barrier to pursuing a research career. This suggests that low proficiency in this skill could be a significant, yet targetable, contributor to suboptimal recruitment. Fellows also frequently identified “difficulty with developing and sustaining an innovative research career” as a barrier to pursuing a career in research. This may partly stem from the perceived low proficiency in the ability to develop new research directions. Although self-perceived proficiency in several research skills did appear to improve during training, there are still a significant number of fellows with inadequate proficiency in several key skills needed in the transition to early career academic faculty. Research training programs with intentional curricula targeting these skills are a potential strategy for enhancing proficiency and eliminating barriers to recruitment. 22
This study also suggests that PCCM trainees’ interest in pursuing a research career diminishes during fellowship training. Nearly two-thirds (63.2%) of early fellowship trainees surveyed in this study envisioned a career inclusive of research; however, recent work suggests that only 20% of graduating fellows ultimately seek this career path. 5 Although the data collected here do not conclusively identify a cause(s) of this trend, there are observations worth mentioning. First, lack of certainty regarding grant funding was one of the most frequently identified barriers by over half of the respondents. It is possible that early fellowship trainees are not aware of the role that grant funding plays in an academic research career and are subsequently discouraged from pursuing this direction upon learning both grant funding's importance and its competitive nature. Trainees may be unwilling to accept the risk of perceived failure in a competitive funding environment, particularly after investing considerable time and effort into their career training to date. This reluctance may be further exacerbated by fellowship programs’ inability to impart self-perceived proficiency in grant writing to late fellowship trainees. Second, a substantial proportion of respondents also identified both a greater emphasis on clinical skills and growth and a lack of motivation by research. These selections may have been influenced by the COVID-19 pandemic; however, they may also reflect an imbalance in training prioritization independent of the pandemic's unique circumstances. Critical assessment of how much time is protected for research and when these opportunities are provided may also represent an opportunity to optimize research engagement during fellowship training.
Aside from lacking adequate skill development, PCCM fellows have been faced with many barriers to choosing careers in academic medicine. Prior studies have theorized that financial considerations and stability are major barriers to academic recruitment and retention. 6 However, late fellowship trainees in our survey identified personal barriers to recruitment into academic positions such as personal/family circumstances (8.1%) and financial concerns (27.0%) less frequently than has been described previously in the literature. Conversely, work-life balance was frequently perceived to be a potential barrier to a career in academic medicine. Career guidance sessions that compare and contrast the work-life balance between academic and nonacademic careers could be important strategies that allow fellows to make informed decisions.
This study has several potential limitations. These include a modest response rate that was likely influenced inherently by known difficulties with physician surveys as well as by unforeseen external factors during the Spring of 2021.15,23 Historically, physician surveys have notoriously low response rates. 24 Subspecialty fellowship trainees can be even more difficult to assess with average response rates between 35% and 50%.25–27 At the time of survey dissemination to PDs and fellows in PCCM, the COVID-19 pandemic resulted in significant disruptions in PCCM training programs including the invocation of emergency operations in some programs. This unprecedented strain on training programs may have discouraged PDs from forwarding the survey and/or fellows from taking it and resulting in a lower-than-normal response rate. This is also validated by national data showing low survey response rates during the COVID-19 Pandemic.28–30 Secondly, the respondent demographics suggest a potential over-representation of academic training programs possibly due to selection bias. However, this population may represent an ideal target audience for this study as fellow graduates who pursue academic research careers typically train in academic programs. Response bias may also have introduced confounding although we detected no evidence of acquiescence or dissent bias in our results. Lastly, this methodology was a convenience network sampling method of enrollment; therefore, the sample size is fixed to the current fellow population and power analysis was not performed. The authors used the Standards for Quality Improvement Reporting Excellence (SQUIRE) guidelines for the reporting in this manuscript. 31
Conclusion
In conclusion, there is an ongoing need for academic pulmonary and critical care physicians with a focus on research that is currently being underfilled. PCCM fellows self-perceive gaps in adequate proficiency for several skills important to research careers including manuscript and grant writing, study conception and design, and data analytic methodology many of which align with the most prevalent perceived barriers to academic medicine research careers. Fellowship curricula focused on addressing and improving competence and confidence in research skills, and divisional and institutional support for early career transitions in research-focused careers may be ways to enhance recruitment of early career faculty into academic research positions.
Supplemental Material
sj-pdf-1-mde-10.1177_23821205231184704 - Supplemental material for Self-Assessment of Research Skills and Barriers to Research Careers among Pulmonary and Critical Care Fellows
Supplemental material, sj-pdf-1-mde-10.1177_23821205231184704 for Self-Assessment of Research Skills and Barriers to Research Careers among Pulmonary and Critical Care Fellows by Meg K. Scott, Andrew J. Goodwin, Nandita R. Nadig, Jillian B. Harvey and Edward F. Kilb in Journal of Medical Education and Curricular Development
Supplemental Material
sj-docx-3-mde-10.1177_23821205231184704 - Supplemental material for Self-Assessment of Research Skills and Barriers to Research Careers among Pulmonary and Critical Care Fellows
Supplemental material, sj-docx-3-mde-10.1177_23821205231184704 for Self-Assessment of Research Skills and Barriers to Research Careers among Pulmonary and Critical Care Fellows by Meg K. Scott, Andrew J. Goodwin, Nandita R. Nadig, Jillian B. Harvey and Edward F. Kilb in Journal of Medical Education and Curricular Development
Supplemental Material
sj-docx-4-mde-10.1177_23821205231184704 - Supplemental material for Self-Assessment of Research Skills and Barriers to Research Careers among Pulmonary and Critical Care Fellows
Supplemental material, sj-docx-4-mde-10.1177_23821205231184704 for Self-Assessment of Research Skills and Barriers to Research Careers among Pulmonary and Critical Care Fellows by Meg K. Scott, Andrew J. Goodwin, Nandita R. Nadig, Jillian B. Harvey and Edward F. Kilb in Journal of Medical Education and Curricular Development
Supplemental Material
sj-docx-5-mde-10.1177_23821205231184704 - Supplemental material for Self-Assessment of Research Skills and Barriers to Research Careers among Pulmonary and Critical Care Fellows
Supplemental material, sj-docx-5-mde-10.1177_23821205231184704 for Self-Assessment of Research Skills and Barriers to Research Careers among Pulmonary and Critical Care Fellows by Meg K. Scott, Andrew J. Goodwin, Nandita R. Nadig, Jillian B. Harvey and Edward F. Kilb in Journal of Medical Education and Curricular Development
Footnotes
Acknowledgments
We thank the Association of Pulmonary and Critical Care Medicine Program Directors for selecting this survey to disseminate to member Program Directors. This data was previously presented in abstract form at the American Thoracic Society meeting in May 2022. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The authors disclose that this survey was disseminated anonymously to their current Pulmonary and Critical Care Medicine fellows.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was supported, in part, by the National Center for Advancing Translational Sciences of the National Institutes of Health under Grant Number UL1 TR001450.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
