Abstract
Objectives:
The COVID-19 pandemic has disrupted the social life, work environment, and well-being of millions of people. We examined COVID-19’s impact on National Institutes of Health (NIH)–funded extramural principal investigators (PIs) affiliated with public health and preventive medicine departments across the country and their projects; assessed PIs’ confidence in achieving project goals; and investigated the role of age, sex, experience, and team size on PIs’ confidence in achieving project goals during the pandemic.
Methods:
We sent an anonymous online survey in January 2021 to 1076 extramural PIs affiliated with public health and preventive medicine departments at US institutions; 133 (12.4%) responded. We examined the impact of COVID-19 on the PIs, their project team operations, and their confidence that project objectives would be met, using Likert scales based on age, sex, team size, and PI experience.
Results:
Of 126 PIs, 94 (74.6%) felt that their day-to-day professional life was impacted a lot or a great deal by COVID-19. More female PIs than male PIs reported that their level of stress changed because of the COVID-19 pandemic. Of 125 PIs, 67 (53.6%) made major adjustments to research operations, 46 (36.8%) made minor adjustments, 5 (4.0%) halted research, and 7 (5.6%) reported not being affected. Of 123 PIs, 89 (72.4%) reported not using NIH COVID-19 accommodations. PIs who led projects 4 or 5 times felt more confident about meeting their research objectives than PIs who led projects 2 or 3 times.
Conclusions:
Future studies should investigate how to develop more engaging support and communication strategies to assist NIH researchers in mitigating the effects of pandemics or large-scale emergencies.
The World Health Organization first reported COVID-19 on December 31, 2019. 1 From March 20 through June 22, 2020, the National Institutes of Health (NIH) kept only the most critical laboratories and facilities open. 2 NIH offered accommodations for ongoing research, including deadline extensions and budget flexibility. 3 NIH classifies recipients of research funding by department type (eg, biochemistry, family medicine). In 2020, NIH funded 2388 projects totaling $1 745 257 495 under the department classification of public health and preventive medicine (PHPM). 4
Previous research indicated that prolonged periods of isolation significantly influence mental and organizational health.5 -8 Pandemic stressors can exacerbate mental health conditions and cause insomnia, depression, anxiety, and posttraumatic stress disorder, which affect females more than males. 5 Although adverse situations are well known to affect performance, developing a high degree of confidence is considered key to achieving long-term success. 9 Prior studies indicate a relationship among age, sex, and anxiety.10 -14 Team size, structure, and leadership experience are also known to influence performance, creativity, flexibility, and research output.15 -18
The objectives of this study were to (1) examine COVID-19’s impact on NIH-funded extramural principal investigators (PIs) affiliated with their institutions’ PHPM departments; (2) assess PIs’ confidence in achieving project goals; and (3) investigate the role of age, sex, experience, and team size on PIs’ confidence in achieving project goals during the pandemic.
Methods
Study Population
We extracted the list of PIs from the NIH RePORTER web portal 4 in December 2020. We removed duplicate names and located individual emails by browsing their primary workplace websites. In January 2021, we sent an exploratory, anonymous online Qualtrics survey to 1076 current extramural PIs who belong to PHPM departments across the country with at least 1 active grant under 1 of the following NIH funding codes: Research Project Grant (R01), NIH Director’s New Innovator Awards (DP2), First Independent Research Support and Transition (R29; now inactive), MERIT Award (R37; Method to Extend Research in Time), Multi-year Funded Research Project Grant (RF1), Small Research Grants (R03), Research Enhancement Awards (R15 AREA), Exploratory/Developmental Grants (R21), Small Business Technology Transfer Grants–Phase I (R41), Small Business Technology Transfer Grants–Phase II (R42), SBIR Grants–Phase I (R43; Small Business Innovation Research), SBIR Grants–Phase II (R44), SBIR Cooperative Agreements–Phase I (U43), or SBIR Cooperative Agreements–Phase II (U44). The Augusta University Institutional Review Board approved this study.
Survey Characteristics
We searched the literature for validated or previously used survey questions related to the effects of disease outbreaks.19 -21 We used these questions to develop our 31-item online anonymous survey. The survey consisted of 4 domains. The demographic domain included age, sex, work location, team size, and PI experience, measured by years of experience and number of projects led. Two other domains described COVID-19–related impacts on PIs and their research teams’ operations. The last domain contained questions about respondents’ confidence in their projects’ ultimate success and allowed respondents to write, in their own words, how NIH can help them in the future.
We assessed responses using 5-point Likert scales (1 = not at all, 5 = completely; 1 = not challenging, 5 = extremely challenging; 1 = much less, 5 = much more; and 1 = not at all, 5 = a great deal). Items were designed to test the factors contributing to the likelihood of success or failure of the research work after the pandemic and were structured to enable respondents to conduct a rank order. In contrast, an open-ended item allowed free-text responses at the end of the survey.
In December 2020, five NIH-funded researchers from our university tested the survey. We implemented their recommendations for clarity in the final version.
Survey Distribution
On January 15, 2021, we sent an email invitation to 1076 extramural PIs with a clickable link to our survey (open for 6 weeks) and sent reminder emails at 2 and 4 weeks. Participants provided informed consent to participate in the survey.
Data Analysis
Not every respondent answered all 31 questions; we calculated percentages based on the number of respondents who submitted an answer for each question. Some questions allowed the respondent to choose >1 answer.
We categorized respondents according to age, sex, 2 measures of experience, and team size. We based PIs’ experience on the number of years as PI (new [0-1 year], early [2-10 years], mid [11-20 years], and long established [≥21 years]) and the number of times the researcher had been a PI (1, 2 or 3, 4 or 5, or ≥6). We categorized team size as very small (1 or 2 people), small (3-6), medium (7-10), and large (≥11). We used SPSS version 26 (IBM Corp) to analyze responses at a significance level of P < .05. We used the Mann–Whitney U test to analyze differences by sex. We used the independent-samples Kruskal–Wallis H test to analyze differences in age, experience, and team size. For questions with inadequate (<5) responses in the lowest or highest 2 Likert response categories, we interpreted the U test and the Kruskal–Wallis H P value as having little practical significance for differentiating the groups. 22 The Dunn–Bonferroni post hoc tests detected group differences and accounted for multiple-item analysis. Two authors (W.J.B., R.A.) conducted the thematic analysis of PIs’ responses to the free-format question, and differences in the analysis were resolved by consensus among all authors.
Results
Of 1076 PIs surveyed, 133 (12.4%) responded and 130 (97.7%) agreed to participate. However, only 127 PIs proceeded with answering the next question, and for each question that followed, the number of NIH PI responses varied.
Demographic Characteristics
Of 123 respondents, 79 (64.2%) were female and 44 (35.8%) were male (Table 1). Of 127 respondents, 26 (20.5%) were aged 31-40 years, 49 (38.6%) were aged 41-50 years, 24 (18.9%) were aged 51-60 years, and 28 (22.0%) were aged >60 years. No respondents were aged ≤30 years.
Demographic characteristics of National Institutes of Health PIs affiliated with a public health and preventive medicine department surveyed during the COVID-19 pandemic, United States, February 2021 a
Abbreviation: PI, principal investigator.
All values are number (percentage).
Respondents could select >1 award designation.
Of 125 respondents, only 3 (2.4%) led very small teams, 32 (25.6%) led small teams, 37 (29.6%) led medium-sized teams, and 53 (42.4%) led large teams (Table 1). Of 126 respondents, 104 (82.5%) used human subjects in their research, 8 (6.3%) used animal subjects, 4 (3.2%) used humans and animals, and 10 (7.9%) used neither humans nor animals.
Of 120 PIs, 9 (7.5%) reported being diagnosed with COVID-19, 42 of 121 (34.7%) reported a family member having COVID-19, 38 of 122 (31.1%) reported a research team member having COVID-19, 76 of 120 (63.3%) reported a friend having COVID-19, and 104 of 124 (83.9%) reported having an acquaintance diagnosed with COVID-19 (Table 1).
Impact of COVID-19 on Researchers
Based on the 5-point Likert scale, where 1 = none at all and 5 = a great deal, 74.6% (94 of 126) of PIs felt that their day-to-day professional life was impacted a lot or a great deal, 3.2% (4 of 126) rated impact as a little, and 22.2% (28 of 126) rated impact as moderate. Of 126 PIs, 66 (52.4%) felt that their stress levels changed a lot or a great deal, 20 (15.9%) said that stress levels changed a little or none at all, and 40 (31.7%) said that stress levels changed moderately. Of 124 respondents, 74 (59.7%) felt that their research project was impacted by COVID-19 a lot or a great deal, 13 (10.5%) said that it was impacted a little or none at all, and 37 (29.8%) said that it was impacted moderately.
Given category choices of work, family, social, and media, respondents rated work and family as the top 2 reasons for stress. In a related question, of the 110 PIs who ranked work as stressful, 67 (60.9%) ranked it most stressful; 27 (24.5%) ranked it more stressful; and 16 (14.5%) said that it was neither more nor less stressful, less stressful, or least stressful.
Of the 99 PIs who ranked family as the top reason for stress, 34 (34.3%) ranked family as most stressful; 40 (40.4%) ranked it as more stressful; and 25 (25.3%) said that it was neither more nor less stressful, less stressful, or least stressful. Of the 83 PIs who indicated social as a reason for stress, 12 (14.5%) found it most stressful, 25 (30.1%) found it more stressful, 31 (37.3%) found it neither more nor less stressful, and 115 (18.1%) found it less stressful. Of 54 respondents who indicated media as a reason for stress, 13 (24.1%) said that it was most stressful or more stressful, 17 (31.5%) said that it was neither more nor less stressful, and 24 (44.4%) said that it was less or least stressful.
A Kruskal–Wallis H test revealed a significant difference between 2 age groups and levels of stress (H3 = 25.82; P = .04), with stress among PIs aged ≤40 years (mean rank = 80.4) being significantly higher than among PIs aged >60 years (mean rank = 54.6). We also noted a significant difference (U = 2.53; P = .01) in the U test by sex: female PIs (mean rank = 67.8) reported that their level of stress changed more than male PIs (mean rank = 51.5). Among female PIs, 64.1% (50 of 78) indicated feeling a lot or a great deal of impact on their research projects as compared with 48.8% (21 of 43) of male PIs. The U test showed that female respondents (mean rank = 66.4) experienced a significantly higher overall impact (U = 2.37; P = .02) than male respondents (mean rank = 51.2).
Of 125 PIs who responded to the question “Do you find yourself working more than you did before the COVID-19 pandemic?” 23 (18.4%) said that they worked much less or somewhat less, 29 (23.2%) said that they worked about the same, and 73 (58.4%) said that they worked somewhat more or much more. Of 125 PIs who responded to the question “Do you go to your place of work more or less than you did before the pandemic?” 112 (89.6%) found themselves going to their place of work much less than before the pandemic, 7 (5.6%) said somewhat less, 3 (2.4%) said about the same, and 3 (2.4%) said much more (Table 2).
Impact of COVID-19 on National Institutes of Health principal investigators affiliated with a public health and preventive medicine department surveyed during the COVID-19 pandemic, United States, February 2021 a
All values are number (percentage), unless otherwise indicated.
Responses were assessed with a 5-point Likert scale (1 = none at all, 5 = a great deal).
Sex: P = .01 based on Mann–Whitney U test, with P < .05 considered significant. Women (mean rank = 67.8) reported experiencing more stress than men (mean rank = 51.5; U = 2.53).
Age: P = .04 based on Kruskal–Wallis H test, with P < .05 considered significant. Levels of stress were significantly higher among PI aged ≤40 years (mean rank = 80.4) than among those aged >60 years (mean rank = 54.6; H3 = 25.82).
Sex: P = .02 based on Mann–Whitney U test, with P < .05 considered significant. Women (mean rank = 66.4) had a significantly higher overall impact than men (mean rank = 51.2; U = 2.37).
Responses were assessed with a 5-point Likert scale (1 = much less, 5 = much more).
Most respondents (96.4%, 106 of 110) said that they always complied with the Centers for Disease Control and Prevention’s recommended guidelines for interaction.
Impact of COVID-19 on Team Operations
Of 125 respondents, 77 (61.6%) said that they spent about the same amount of time meeting with their team as they had before the pandemic; however, the modality for meetings changed, with 121 of 124 (97.6%) PIs participating somewhat more or much more in video conferencing. Although 38.5% (47 of 122) of PIs indicated using text messaging slightly more or much more than before the pandemic, 27.4% (34 of 124) indicated using the telephone slightly more or much more. Of the 123 PIs who answered the question about the extent of challenge to project team problem solving and adaptation due to the COVID-19 pandemic, 57 (46.3%) said that it was very challenging or extremely challenging, 40 (32.5%) said that it was moderately challenging, and 26 (21.1%) said that it was slightly challenging or not challenging (Table 3). Females (mean rank = 65.7) cited more challenges to project team problem solving and adaptation because of the COVID-19 pandemic than men (mean rank = 51.2; U = 2.25).
Impact of COVID-19 pandemic on the operations of the research team in a survey of NIH principal investigators affiliated with a public health and preventive medicine department, United States, February 2021 a
Abbreviation: NIH, National Institutes of Health.
All values are number (percentage), unless otherwise indicated.
Responses were assessed with a 5-point Likert scale (1 = much less, 5 = much more).
Responses were assessed with a 5-point Likert scale (1 = moderately less, 5 = much more).
Responses were assessed with a 5-point Likert scale (1 = not challenging, 5 = extremely challenging).
Sex: P = .02 based on the Mann–Whitney U test, with P < .05 considered significant. Female respondents (mean rank = 65.7) reported more challenges to project team problem solving and adaptation because of the COVID-19 pandemic than male respondents (mean rank = 51.2; U = 2.25).
Of 125 respondents to a question about the impact of the pandemic on adjustments to research operations, 67 (53.6%) PIs made major adjustments, 46 (36.8%) made minor adjustments, 7 (5.6%) reported being unaffected, and 5 (4.0%) halted research completely.
NIH Accommodations
Although NIH offered PIs COVID-19–related accommodations for their funded projects, including deadline extensions and budget flexibility, these accommodations were not used by most researchers (72.4%, 89 of 123). Of those who answered the question about using accommodations offered by NIH, 26.7% (8 of 30) said that they used extension of application deadlines, 38.7% (12 of 31) used the flexibility of postsubmission materials, 46.9% (15 of 32) used flexibility and project extensions for human subjects, 51.6% (16 of 31) used administrative flexibilities, 77.4% (24 of 31) used accommodations for loss of research time, 40.0% (12 of 30) used the flexibility of salaries and stipends, and 14.8% (4 of 27) used donation of research supplies.
Three themes emerged in the analysis of the free-format written responses to the question “In the future, what kind of assistance or resources can the NIH provide to support you in your NIH-funded research projects?”: additional funding, flexibilities and extensions, and other accommodations and administrative changes. Of 68 PIs, 50 (73.5%) identified additional funding as a critical success factor. According to 1 PI, “additional funding [would help] to cover the months we were not able to conduct research but yet still had to pay research staff.” Similarly, 33 of 68 (48.5%) PIs believed that supplemental funding would allow them to complete their grant-funded projects.
The second-most stated need was for time extensions, as illustrated by 1 PI’s suggestion: “broader approval of multiple (perhaps 3 or more) no-cost extensions and potential supplements since much of human subjects’ recruitment was held up for over 6 months and slowed for at least a year.” Other PIs requested extensions for annual progress and other reporting deadlines, specifically for flexibility in timelines and project deliverables. Some PIs felt that extensions would regain lost research time and meet project requirements.
PIs identified a basic need for official NIH recognition of the scope of the pandemic’s impact on projects and the consequential structural changes made to accommodate new COVID-19 safety rules and regulations. One PI noted desiring “recognition that many types of research have been slowed because of a wide range of factors. Administrative supplements to address these problems.” The lack of involvement by NIH project officers also drew comments from some PIs. One PI stated, “I have not found my project officer to be responsive to my attempts to contact her. It is frustrating that I feel like I don’t have any support in making decisions.” Other PIs stated a need for “responsiveness and flexibility” and “mentoring for new PIs. Program officers that are more responsive and available.” PIs identified the need for changes in NIH administrative processes to accommodate the research teams during stressful situations such as a pandemic.
Confidence That Project Objectives Will Be Met
PIs who led the projects ≥6 times (43 of 122, 35.2%) reported feeling either fairly (n = 24, 55.8%) or completely (n = 6, 14.0%) confident in meeting the original research objectives (Table 4). PIs who had led projects 4 or 5 times indicated feeling significantly more confident than PIs who had led projects 2 or 3 times that their teams would meet their original research objectives (mean rank = 76.3 vs 50.3; Kruskal–Wallis H3 = −26.0; P = .03). However, we found no significant group differences when we considered age, sex, years of experience as a PI, and team size.
Confidence survey of NIH PIs affiliated with a public health and preventive medicine department that their project objectives will be met despite the COVID-19 pandemic, United States, February 2021 a
Abbreviations: NIH, National Institutes of Health; PI, principal investigator.
All values are number (percentage). Responses were assessed with a 5-point Likert scale (1 = not at all, 5 = completely). Percentages are column percentages.
Number of times as PI: P = .03 based on the Kruskal–Wallis H test, with P < .05 considered significant. PIs who had led projects 4 or 5 times (mean rank = 76.29) were more confident than PIs who had led projects 2 or 3 times (mean rank = 50.32; H3 = 25.96).
Discussion
Impact of COVID-19 on Researchers
We found that female researchers were more likely than male researchers to report that COVID-19 was the cause of increased stress levels and that COVID-19 impacted their research projects. Various studies have noted that a significant side effect of an epidemic is an increase in stress or anxiety levels.23,24 Upon further examination of studies on sex and its relation to stress and anxiety, our results demonstrated a similar correlation of increased stress levels among female PIs.23,25,26
We found that older PIs (aged >60 years) reported lower levels of stress than younger PIs (aged <40 years). In contrast, 2 previous studies on a general population identified no age differences in stress levels during COVID-19.12,13 The differences that we observed in the age cohorts could be explained by improved coping skills acquired through accumulated life and research experience in the older population. 27
We observed that respondents identified work and family as primary sources of stress during the pandemic. Family as a source of stress may be linked to parenting and the associated burden of children learning from home during the pandemic.28,29 According to a recent study, work stress has a considerable and detrimental impact on employee performance. 30
NIH Accommodations
Approximately 72% of respondents reported not using available NIH COVID-19 accommodations and flexibilities. PIs conveyed a recurring need for increases to overstretched budgets, mentoring, simplifying processes, and clearer guidance. Because NIH provided some flexibilities and accommodations, these persistent themes may indicate a lack of understanding of available resources or perception of insufficiency, which could be caused by ineffective methods for relaying information about interim or emergency resources and accommodations. We believe that this misunderstanding of NIH-provided accommodations might explain the high proportion of PIs who indicated not using NIH COVID-19 support. Given that the participants in this study identified the need for increased financial support, administrative changes, and other accommodations, proactively addressing these areas of concern through training, operational changes, and improved communication will likely provide support for the success of PIs. Additional financial support will likely reinforce PIs’ confidence and positively impact project completion and success.
Confidence That Project Objectives Will Be Met
This study found that most PIs across sex, age, and team size groups expressed confidence in meeting the ultimate objectives of their projects despite the widespread disruptive effect of the COVID-19 pandemic. Although PIs reported differences in stress levels based on age and sex, we found no significant difference in their confidence in meeting research objectives. Similarly, we found no significant differences in their confidence in meeting research objectives by team size or experience measured by years as a PI. However, experience, measured by the number of times that they served as a PI, had a significant impact. PIs who had led projects 4 or 5 times were more confident than PIs who had led projects 2 or 3 times. We hypothesize that being a PI for a longer time but with a smaller number of projects does not build as much confidence as being a PI for a larger number of projects. In other words, confidence appears to be built on the number of projects that a PI oversees rather than the number of years that they served as a PI.
Confidence is desirable in times of uncertainty and stress because it reflects elasticity. An NIH internal survey indicated that 55% of extramural-funded researchers projected a negative impact from the pandemic on their career trajectories. 30 Yet, only 39% (48 of 123) of PIs reported being not at all confident, slightly confident, or somewhat confident about reaching their project goals. It may be that the long-term outlook on careers may appear more uncertain to PIs than the short-term outlook of their projects. While these initiatives as contingency or fallback options may contribute to PIs’ confidence in their projects, we did not find significant differences attributable to NIH’s accommodations. However, this subject may deserve additional future evaluation.
Limitations
This study had several limitations. First, it had a low response rate (12%), which may have led to nonresponse bias. Second, it had a higher proportion of female respondents (62%) than male respondents (35%), which may indicate an unrepresentative sample because NIH funds significantly more male researchers than female researchers. In 2020, NIH awarded 15 170 (35.3%) research grants to female PIs and 26 044 (60.6%) research grants to male PIs. 31 Lastly, because of a lack of resources and time constraints, we limited our study to PIs affiliated with their institutions’ PHPM departments. As a result, the results may be skewed when compared with a more representative sample of all NIH-funded PIs or even PIs from other departments, such as radiation-diagnostic/oncology or pharmacology.
Conclusions
While COVID-19 impacted all respondents, younger PIs (aged <40 years) felt more stressed than older PIs (aged >60 years), and female PIs felt more stressed than male PIs. To help reduce stress, institutions should encourage collaboration and mentoring among their investigators to help younger investigators. Further studies on support for female PIs would be helpful. Most PIs expressed confidence in surmounting challenges and ultimately achieving the original goals of their projects. We found no difference in confidence levels based on age or sex. However, having served as a PI for more projects appears to build confidence. PIs who have successfully completed several NIH-funded research projects should mentor novice PIs. Research institutions and funding agencies should encourage and reward such mentorship.
Most respondents stated that they did not use the accommodations that NIH offered. Conversely, respondents stated that they wanted more flexibility from NIH for their projects. Consequently, NIH should consider developing accommodations with flexibilities responsive to what PIs might use and desire during a crisis. NIH may also consider developing more engaging support and communication strategies to help researchers mitigate the effects of future pandemics or widescale emergencies.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
