Abstract
Keywords
Introduction
In the wake of the substantial impact that the COVID-19 pandemic had on human life worldwide, 1 the global health system now faces challenges and pressures regarding epidemic prevention, control, and resilience, particularly in hospital systems.2,3 As the main frontline healthcare providers directly engaged in the care and treatment of patients, nurses play a crucial role in the healthcare system’s response to the pandemic. 4 Given that nursing interns are future healthcare providers, there is a growing advocacy for promoting resilience among them. This can equip them with the skills to effectively handle difficult situations and fulfill their professional obligations. 5 However, nursing interns who were present during the COVID-19 crisis witnessed the challenges and sacrifices made by healthcare professionals. As a result, they may have experienced negative emotions, including stress and anxiety, as well as fear of infection and uncertainty about the future.3,6
Resilience is a popular term used to describe the capacity to learn from dilemmas and turn adversity into opportunity; this can assist healthcare providers in successfully overcoming obstacles and moving forward.7,8 Moreover, resilience is a trait that helps individuals effectively respond to negative emotions in clinical practice and difficulties in their future careers. Academic resilience combines resilience with educational background, referring to the ability of the students to successfully cope with adversities and show a high level of academic performance while being exposed to multiple stressors or adverse circumstances and facing the risk of failure. 9 During the COVID-19 pandemic, nursing students encountered new situations in both academic and hospital settings. During the height of the pandemic, many countries suspended in-person classes and shifted to remote learning to protect the health and safety of faculty and students.10,11 Our study population comprised nursing students who were enrolled during the pandemic and received a combination of offline and remote education programs. These students complained that remote learning was not productive and that it was difficult for them to attend lectures and complete assignments, which resulted in insufficient skill-based learning for clinical applications. 12 Therefore, many nursing interns lack the confidence required to cope with clinical internships owing to a paucity of suitable and sufficient operational practices, 10 which also leads to anxiety and uncertainty about the competencies required for hospital interactions and duties.11,13 Moreover, hospitals suspended internships during the height of the pandemic due to COVID-19 transmission concerns. As a result, nursing students were unable to complete their full internship experience. Some families also expressed apprehension regarding their students’ safety during this unprecedented time.
Nevertheless, nursing interns need to adapt and cope well with the challenges caused by unprecedented emergencies such as COVID-19. However, existing research on nursing interns has mainly focused on their experiences, perceptions, and attitudes toward the pandemic.11,13–16 Most studies suggest that nursing interns are not psychologically prepared to cope with such critical situations. For example, Curcio et al. reported that nursing students found their learning experiences very stressful during the pandemic not only because of the fear of infection but also owing to the challenges of remote education. 16 Several studies have also specifically addressed academic resilience among nursing students. For example, Berdida and Grande, 17 Grande, Berdida 18 have both investigated the relationship between quality of life and academic resilience among nursing students during the COVID-19 pandemic. They also explored the mediating role of resilience and academic motivation on academic stress and self-directed learning among nursing students. 19 While these studies focused more on nursing students in schools, the investigation of nursing interns’ academic resilience in the specific context of the COVID-19 pandemic is necessary since it can directly measure their ability to cope with the aforementioned negative emotions in clinical practice such that positive approaches can be implemented in time.
Therefore, in this study, we aimed to establish and utilize a hybrid machine learning and decision analysis model to analyze the academic resilience of nursing interns. First, we used the Nursing Student Academic Resilience Inventory (NSARI) scale developed by Ali-Abadi, Ebadi. 5 Subsequently, a hybrid multi-criteria decision analysis model that combines random forest (RF) and importance-performance analysis (IPA) plots was used to identify the main variables and key improvement gaps affecting the academic resilience of nursing interns, as well as to make targeted suggestions for improvement.
Materials and methods
Research design and data analysis
An online questionnaire survey was administered to 96 nursing interns at a tertiary general hospital in Taizhou, China, via convenience sampling in January 2023. The inclusion criteria were as follows: participants who (1) were officially registered nursing students or (2) were nursing interns in pre-graduation internships. The exclusion criteria were nursing interns who had interrupted practice owing to COVID-19. Questionnaires were collected only from nursing interns who had provided consent before completing the questionnaire, which required approximately 10–15 min. From the collected questionnaires, we analyzed the main variables of the academic resilience of nursing interns using an RF plot. The key improvement gaps were subsequently analyzed using an IPA of the four main characteristic categories. The research design and data analysis flow of this study are shown in Figure 1. Design and data analysis process of this study.
Nursing Student Academic Resilience Inventory scale
Nursing Student Academic Resilience Inventory (NSARI) scale.
Random forest
RF is an efficient prediction method first systematically introduced by Breiman. 20 It is one of the many ensemble learning methods that includes multiple decision trees for bagging integrated learning technology. 21 It not only makes use of modern non-parametric technology but can also effectively extract nonlinear relationships from input data.22,23 Additionally, RF can depict both linear and nonlinear relationships without making any additional assumptions about the independent or dependent variables. 24 A large number of classification and regression trees are built in the RF setup using randomly chosen training datasets and randomly selected predictor variable subsets to model the results. 25 The results for each tree are combined to obtain an overall forecast for each observation. 25 Consequently, RF frequently offers greater accuracy than a single decision tree model while retaining some of the advantages of tree models. 25 Therefore, the RF model has been widely adopted in the environment.26,27 engineering, 28 agriculture. 29 geography. 30 management. 31 medicine. 32 and other fields.
Importance-performance analysis
Formally introduced by Martilla and James in 1977, IPA is a method of conducting business research. It was created as a tool for marketing purposes to evaluate and recommend new management strategies.
33
As a key component of quality assurance, performance evaluation, and the creation of effective improvement programs.
34
IPA is always conducted to verify the performance and importance of each variable.
35
This technique (Figure 2) uses a four-quadrant matrix, with performance listed on the horizontal axis and importance on the vertical axis to identify strengths, weaknesses, and improvement opportunities for developing a strategic plan.34,36,37 IPA establishes a cross coordinate after data input, and the analysis of all elements is directly scattered in each quadrant, making the presented results clear and easy to comprehend; this benefits researchers in determining the essential factors that need to be rapidly and effectively upgraded.
36
Although originally developed for marketing purposes, IPA has been applied to research in various fields, including public health,34,35 tourism,36,38 architecture,
39
medical decision-making,
40
information system,
41
training course 42,43 and media.
44
Importance performance analysis.
Quadrant I is the superior region representing a high level of importance and performance. The client places a high value on this quadrant’s characteristics, and the organization appears to offer a high caliber of performance. This quadrant provides the major strengths and opportunities needed to establish or retain a competitive advantage.
Quadrant II denotes high performance but low importance and includes characteristics that are less significant to the client; however, the organization appears to overemphasize pertinent items in this quadrant to deliver high levels of performance. In this case, the organization should transfer resources from this quadrant to other quadrants where performance improvement is required. Therefore, the second quadrant is considered an opportunity region.
With both low performance and importance, Quadrant III suggests that no improvement is necessarily required to some extent. The attributes in this quadrant are referred to as minor weaknesses; thus, they do not require considerable priority for improvement. This is referred to as the delayed improvement region.
Quadrant IV contains attributes that are important to the client but at a relatively low level of performance. These attributes are referred to as significant weaknesses that must be addressed immediately for improvement. It is considered a key improvement area and is considered able to indicate key improvement gaps.
Results
Data collection and participants
The background description.
Importance degree of academic resilience variables
Academic resilience data from nursing interns were used to construct an analytical model. The value of each factor for overall academic resilience was calculated using RF plots to determine the importance of each item. The out-of-bag estimate of the error rate in the RF model was 25%. In other words, this important result yields an accuracy of 75% and good stability. Figure 3 shows the importance of the academic resilience of the 96 nursing students based on the NSARI scale. The items with the highest degree of importance included “Cooperation by nurses reduces my stress (C3),” “I am motivated to participate in internships (O5),” and “I have sufficient confidence in taking care of patients (SE3),” whereas those with lowest degrees were “My instructors trust my judgment in taking care of the patients (SE1),” “I am looking forward to being a great nurse in the near future (O2),” and “By strengthening my nursing knowledge, I will do my best to take care of the patients (T2).” The importance of each item.
The four quadrants of academic resilience variables
Results of academic resilience of nursing interns in case hospital.

Analysis chart of IPA method in case hospital.
Discussion
Main variables
The study adopted the RF method to determine the weight of the items on the NSARI scale; “Cooperation by nurses reduces my stress (C3),” “I am motivated to participate in internships (O5),” “I have sufficient confidence in taking care of patients (SE3),” “Having good communication with attending physicians reduces my stress (C4),” and “I manage the difficulties of my academic years (SA1)” were the main variables related to the academic resilience of nursing interns.
Rodriguez-Almagro et al. found that some nursing interns reported a lack of supervision or less-than-usual guidance during COVID-19; clinical nurses were often occupied with their responsibilities and unable to provide explanations to interns about their actions. 15 Thus, to a certain extent, nursing interns may lack effective cooperation with nurses in clinical practice. Additionally, nursing interns may struggle to provide care for their patients because of their lack of knowledge and skills, which may have been affected by the attendance of online classes throughout the pandemic, making it challenging to focus on learning. 45 This may cause nursing interns to feel inadequate, resulting in insufficient confidence when caring for patients. Moreover, having sufficient confidence in practicing and communicating with physicians during internships is always challenging for nursing interns. At present, nursing interns generally work under the guidance of clinical nurses, focusing on observation and performing some simple tasks, with little communication with physicians. However, effective communication between nurses and physicians is crucial once nurses begin working professionally after graduation, despite not having had much opportunity to develop this skill during their internships. In addition, although they have honed their knowledge and skills, nursing interns need to show their skills to gain the confidence and trust of existing nurses and confirm their abilities to carry out nursing procedures independently.
Key improvement areas
An IPA was conducted to evaluate the academic resilience of nursing interns working in hospitals during the COVID-19 pandemic under normalization prevention and control. Additionally, an IPA matrix was created to identify the key improvement gaps pertaining to the academic resilience of nursing interns. The items that fell into quadrant IV were of high importance but low performance, meaning that they required the most urgent improvement in performance in the future. These items included “Having good communication with attending physicians reduces my stress (C4),” “I have sufficient confidence in taking care of patients (SE3),” and “I examine all options to reach my goals (SR1),” which were also the key improvement areas in the study hospital. Among these items, two were also the main variables to identify the academic resilience of nursing interns (C4 and SE3), which demonstrates the importance of good communication and sufficient confidence in improving academic resilience among nursing interns. Examining all options to reach goals is a characteristic of persistence and a prerequisite for success, which can provide nursing interns with stronger motivation to practice and complete their studies and internships with greater purpose.
Recommended measures
Owing to the particularity of the pandemic period, the clinical practice of nursing interns is affected. Therefore, before the occurrence of the next unpredictable pandemic, hospital managers should strengthen the management of nursing interns, not only to improve the practical ability of interns but also to improve the management ability of the hospital to respond to sudden crises. Previous studies have shown that nursing instructors should play a significant role in fostering the resilience of nursing interns and helping them preserve their mental and emotional health throughout the pandemic. 46 In other words, nurses should provide positive feedback to interns in clinical work, guide and encourage them to practice, and enhance their confidence. Nurses should also provide nursing interns with the opportunity to interact with physicians and coach them on communication skills. Lastly, nurse leaders should regularly promote and strengthen the resilience of nursing interns through lessons, seminars, peer activities, blended platforms comprising short videos, and other initiatives to equip nursing students to successfully handle the challenges of clinical practice.46–50
Regarding education, it is necessary to review and improve current nursing clinical education to ensure professional readiness. 51 Open communication, collaborative work, and a compassionate attitude toward nursing students can help interns become more resilient, making them feel more confident and trusted, thus helping them feel more supported in coping with challenges. 52 Therefore, we suggest that clinical nursing educators strengthen the daily guidance and teaching, as well as the whole-hospital teaching of nursing interns; contents should include communication skills, confidence cultivation, target management, and positive psychology. It is paramount to integrate such content into the daily education and management of nursing interns to assist them in developing and strengthening their academic resilience.
At the same time, it is also necessary to pay attention to the mental health of nursing interns to help them actively face the difficulties encountered during the internship process. Fixed supervising nurses were set for interns in each department, and the status of interns during the working period was assessed. When the academic resilience of interns was determined not to be optimal, timely communication and guidance were provided to foster a healthy and positive attitude for the successful completion of the internship.
Future studies that focus on the construction of clinical practice and education programs for nursing interns as well as the implementation measures and effect evaluation and to further explore effective methods to improve the academic resilience of nursing interns are necessary.
Research limitations
This study had certain limitations. First, the data collection was limited to nursing interns at a certain stage in one prefecture-level city hospital, which may have restricted its representativeness. Second, sample size calculation and justification were not performed, the sample size was not large enough, the proportion of male-to-female participants was not balanced, and the difference in education levels affected the representativeness of results to a certain extent. Third, all data was gathered through self-reported questionnaires, which inevitably led to response and social desirability biases. Therefore, it is recommended that a larger sample be randomly drawn from a wider group of nursing interns from different regions and countries. Finally, this study focuses on applying machine learning methods to construct decision-analysis models from data to provide decision-makers with new thinking. However, this study does not further discuss innovations that combine digitization with innovation collection methods in the healthcare field.
Further research
Based on the decision analysis model constructed in this study, in future practice, the hospital can design an application for nurse interns to fill in their academic situation. Through the model described in this study, the application can then identify the academic resilience of the interns according to the results of their responses and give appropriate opinions and suggestions. For hospitals and schools, managers can learn about students’ internships from the application and take the initiative to pay attention to their mental and physical health. In addition, hospitals can provide health wristbands for nurse interns, combine them with the application, obtain students' psychological and physiological data immediately through information systems and technology, and build a warning model to increase the dimension of students’ academic resilience and provide more accurate health measures in the future. Moreover, interpretable machine learning methods can be used for further research as well.
Conclusions
This study applied a hybrid multi-criteria decision analysis model combined with RF and IPA to investigate the main variables and key improvement gaps in the academic resilience of nursing interns under the normalized prevention and control of COVID-19. Its findings showed that the academic resilience of nursing interns is primarily affected by self-confidence, motivation, and cooperation with nurses; thus, nursing managers in healthcare facilities can improve the confidence and motivation of interns by strengthening the link of interns with nurses and physicians, promoting their cooperation and communication, encouraging and affirming interns by nurses during the clinical practice, and motivating interns to become more proactive in overcoming obstacles in the future. These findings may also assist nursing managers in targeted training of newly hired nurses while providing a reference for the clinical practice education of nursing interns in hospitals during future unexpected pandemics. Based on this study, we will further build an application to analyze the academic resilience of nursing interns, combine the physiological and psychological data of interns, integrate interpretable machine learning methods, obtain and analyze the interns’ condition over time, and provide corresponding strategies to improve the academic resilience of nursing students in hospitals and schools.
Footnotes
Acknowledgements
We thank the authors who developed the Nursing Student Academic Resilience Inventory (NSARI) and permitted us to use the scale. We also thank the editor and the reviewers for their helpful remarks that improved this paper.
Author contributions
Mao Ye and Yen-Ching Chuang drafted the initial and final manuscript. Siqi Liu and Jianhong Yang collected data. Yen-Ching Chuang analyzed the data and performed the final editing. Lili Feng, Fuqin Tang and Weifang Xu designed and conducted the study. All the authors contributed to the article and approved the manuscript.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study is supported by The Zhejiang Medical and Health Science and Technology Program (No. 2023KY1337); the Education Planning Project of Taizhou City, Zhejiang Province (No. GG22016+GG22019+GG22020); Nursing Discipline Development Special Fund Project of Taizhou University, Zhejiang Province (No. 202201)
