Abstract
Background
The use of virtual patients in nursing education has emerged as a valuable tool for developing clinical decision-making skills. However, little is known about the outcomes of incorporating virtual patients into nursing education curricula. The Nursing School of Porto has dedicated efforts to developing clinical scenarios and integrating virtual patient simulation into its nursing degree curriculum. This study aims to evaluate the outcomes of integrating virtual patient simulation into the nursing degree curriculum, specifically focusing on physical nursing examination, nursing diagnosis, nursing treatment, and overall nursing decision-making scores.
Method
An observational, retrospective, and descriptive study was conducted using a non-probabilistic convenience sample of 521 second-year nursing degree students.
The results
On average, students completed 31 virtual patient simulation training sessions. Students obtained an overall score of 82.5% (SD=8.52) for nursing decision-making, a physical examination score of 85.6%, a diagnostic activity score of 72.1%, and a implemented treatment score of 82.8%. Positive relationships were observed between the number of completed scenarios, the overall average evaluation score, and final marks.
Conclusion
The integration of virtual patients yielded a positive impact on nursing education. However, further evidence is necessary to determine the impact of virtual patient simulation on the clinical competencies of nursing degree students.
Keywords
Introduction
The availability and use of virtual patients in nursing and medical education have gained relevance in recent years, notably influenced by the transformations imposed by the COVID-19 pandemic (Hege et al., 2020; Muller et al., 2021; Plackett et al., 2022).
Virtual patients are computer programs that simulate real-life clinical scenarios in which students play the role of healthcare professionals (Lioce et al (2020). Virtual simulation places students at the forefront, enabling them to exercise decision-making, motor, and/or communication skills (Lioce et al (2020). The use of virtual patients simulator allows nursing and other health students to perform activities such as collecting clinical histories, conducting physical examinations, formulating diagnoses, implementing interventions, and evaluating the outcomes of their clinical decisions.
The evidence shows that the integration of virtual patients in nursing education is effective in knowledge retention, critical thinking, clinical reasoning, instrumental skills, self-efficacy perception, learning satisfaction, interaction and feedback, teamwork, learning experience, and the realism of simulation spaces, thereby enhancing emotional safety (Baumann-Birkbeck et al., 2017; Brown et al., 2021; Dale-Tam et al., 2021; Foronda et al., 2020; Kyaw et al., 2019; Mestre et al., 2022; Padilha et al., 2018; Pence, 2022; Plackett et al., 2022; Sim, et al., 2022).
Currently, virtual patient technology recreates friendly learning environments based on high realism and fidelity, fostering emotionally safe spaces for students learning (Padilha et al., 2024). In nursing education, virtual patients support students in deepening their competencies, cultivating intrinsic motivation, and enhancing active learning (Padilha et al., 2024). Using virtual patients by nursing students results in a greater perception of effectiveness and less likelihood of clinical error (Berman et al., 2016; Padilha et al., 2022). Therefore, this technology plays a pivotal role in fostering friendly learning environments in nursing education nurturing students’ emotional safety throughout their learning process.
The effectiveness of virtual patient simulation (VPS) depends not only on the features of the available technology, but also on the quality and suitability of the clinical scenarios used and the pedagogical strategy guiding the use of this technology. Regarding the features of the virtual patient simulator, the availability of detailed feedback about each decision made, along with supporting evidence for each decision or evaluation is crucial for student active learning. Furthermore, the successful implementation of this technology depends on the continuous assessment of student´s perceptions and learning outcomes. Throughout this process, teachers play a crucial role in facilitating the learning journey using technology to support students in their active learning endeavours (Padilha et al., 2022).
This article describes the outcomes related to physical nursing examination, nursing diagnosis, nursing treatment and global nursing decision-making scores among nursing students of the Nursing School of Porto (ESEP). In 2016, ESEP has initiated the development of clinical scenarios and their integration into virtual patient simulation within the nursing degree curriculum.
Thus, after conducting research and development (R&D) on VPS in nursing education at the ESEP, it is imperative to describe the results stemming from the use of the virtual clinical simulator by students.
Development and Implementation of a Clinical Virtual Patient Simulator in Nursing Education
In 2016, The ESEP began the development of clinical virtual scenarios in the field of Nursing, intended for integration into the virtual patient simulator Body Interact®. The objective was to tailor these scenarios to align with the competencies to be developed in each phase of the nursing degree student´s academic progression.
For the development of clinical scenarios, a R&D methodology was adopted rooted in participatory action research (PAR) (Reason, 2006). The process involved a cycle of action and research juxtaposition guided by the principles of rigour and conduct of action research (AR) defined by McKay and Marshall (1999) and directed at: i) facilitating change in nursing education by developing a set of scenarios for training clinical reasoning and clinical decision-making in nursing, and integrating VPS into nursing education; ii) managing research, contributing to producing synthesis knowledge: and implementing, developing and evaluating virtual patient simulation in nursing education (e.g., Pinto et al., 2020).
A theoretical framework was employed to guide the PAR based on a cyclic process comprising five phases: i) diagnosis, ii) action planning, iii) action implementation, iv) evaluation, and v) identification of the acquired as illustrated in Figure 1 (Padilha et. al., 2016; Susman & Evered, 1978). Design of the R&D path developed between 2016 and 2023 at the ESEP.
In the diagnosis phase, scenarios that contributed to developing clinical reasoning and clinical decision skills were identified by four Nursing teachers at the ESEP, based on the syllabus of the nursing degree, ensuring alignment with its contents and objectives. This process involved establishing consensus among internal and external experts to ESEP (6 experts for each scenario). Experts’ selection criteria were internal expert – PhD in Nursing, specialisation in the clinical field, clinical and teaching experience in the clinical field, publications in the field; external expert – recognized clinical merit, master’s degree and/or speciality in the clinical field, clinical experience and teaching in the clinical field, publications in the field.
In the planning phase, four teachers synthesised the evidence related to each domain outlined in the identified scenarios and then evolved to construct the clinical scenarios. These clinical scenarios were developed and evaluated following INACSL standards (INACSL Standards Committee, 2016a, 2016). Subsequently, the developed scenarios were evaluated by the internal and external experts at the ESEP.
During the implementation phase, the software development team from Body Interact® carried out the necessary work and developments to integrate the scenarios into VPS. Following initial testing by the development team, ESEP researchers and internal and external experts tested each scenario. After the testing phase and the necessary improvements were completed, the clinical scenarios were made available in one curricular unit of the nursing degree at ESEP.
Between 2016 and the end of 2023 we developed more than 40 clinical simulation scenarios. In 2018-2019 we began using VPS in the Nursing Degree curriculum as an integrated pedagogical strategy of the curricular unit Body Processes Responses to Disease 1 (RCD1).
The integration of the VPS was grounded on the simulation theory of Jeffries (Jeffries et al., 2022). This integration was implemented through the following strategic steps: First, theoretical classes of RCD1 on the subject were held as part of the syllabus aimed at presenting students with the theoretical contents that support clinical reasoning and decision-making in the thematic field. Then, students participated in theoretical-practical classes where the principles inherent to clinical reasoning, decision-making, and the demonstration/implementation of technical procedures in the thematic domain were addressed and systematized. After the theoretical-practical class, students were provided with a set of clinical scenarios for clinical reasoning training and decision-making in the thematic domain, distributed one week in advance of the laboratory practice class (subsequent class of the curricular unit RCD1). These scenarios were organized by sessions associated with the thematic domains. During these sessions, students could access each session’s objectives and scenarios through their distance communication devices (e.g., laptops or smartphones), enabling them to engage in clinical reasoning and decision-making training before laboratory classes. These sessions functioned as prebriefing preparation (INACSL Standards Committee et al., 2021a) where 15 clinical scenarios were made available to students. After each simulation session, the virtual patient simulator offers unidirectional feedback based on predefined rationales and evidence to support decision-making. This functionality allows the student to review detailed information about each decision, presented chronologically, acompained by supporting evidence for each decision or evaluation. Also, students can access an overall performance simulation score, categorized by the domain of action (physical examination, diagnostic activity, and implemented treatment).
In the following week, theoretical-practical classes were followed by laboratory practical classes. During these classes, students trained in the implementation of nursing interventions based on the provided clinical scenarios. An ESEP teacher facilitated the training using low-fidelity task trainers to provide hands-on skills alligned with the class objectives (INACSL Standards Committee et al., 2021b). Finally, after each laboratory practice class, the teacher demonstrated the resolution of each clinical scenario in the VPS, followed by a debriefing session, which included discussion and guided reflection to address any queries or concerns raised by the students.
Although virtual patients have been developed and implemented in nursing education in recent years, little is known about the results of incorporating virtual patient simulation into nursing education curricula.
Methods
An observational, retrospective, descriptive study was conducted to examine the use of VPS.
Selection of Participants
A non-probabilistic convenience sample was employed of nursing degree students at ESEP attending the second year of study who utilized the VPS in 2020-2021 and 2021-2022. The inclusion criteria were nursing degree students who had completed attendance of the curricular unit Body Processes Responses to Disease 1, in the academic years 2020-2021 and 2021-2022, regardless of their success in the curricular unit. Exclusion criteria were nursing degree students who have not been enrolled in the course Body Processes Responses to Disease 1 in the academic years 2020-2021 and 2021-2022, and students who have obtained accreditation to the curricular unit. Students had access to 15 virtual patient scenarios during one semester in the curricular unit Body Processes Responses to Disease 1. The use of VPS was not mandatory.
Data Collection
The variables analysed were: global nursing decision-making score; physical examination score; diagnostic activity score; implemented treatment score; and number of completed scenarios. Data were extracted from the Learning Management System (LMS) of the VPS. This LMS serves as an interface accessible to teachers, facilitating the management of students, creation, and release of training sessions with specific timings, objectives, and evaluation. In addition, it enables the monitoring of student/class usage and performance over time. Performance outcomes are presented graphically and can be exported in XLS format. The results are expressed as percentages according to the predefined clinical scenarios. Moreover, this information can be aggregated at different levels of granularity.
Data Analysis
In the initial phase, we analysed frequencies and descriptive statistics for all collected variables. This approach allowed analysing the distribution of variables and identify any potential outliers. The Shapiro-Wilk test assessed the normality of the distribution of variables, using asymmetry and kurtosis values to assess deviation of variables from normal distribution. Then, differential analyses were performed between different groups of students, using parametric and non-parametric tests based on the normality assumption.
Statistical analyses were conducted using JASP, Jamovi, IBM SPSS Amos v.29, and IBM SPSS Statistics v. 29 softwares. The results adhered to APA standards by presenting effect sizes measured by Cohen's d (low: 0.2; medium: 0.5; high: 0.8) and considering a significance level of P<.05.
Ethical Considerations
The study obtained authorization ADHOC_697/2022. Participants were emailed a letter presenting the study and were asked to sign written informed consent.
After extraction from the LMS of the VPS, the data were anonymized. Authorization for the use of data for analysis was granted at the time of the student’s registration in the VPS, and this was stated in the privacy policy.
Results
Descriptive statistics of the variables under analysis.
Students who used the VPS had a mean age of 20.97 years, 83.3% (n=434) were female, 50.3% (n=262) were enrolled in the second year of the nursing degree in 2021/2022, 48.9% (n=255) were enrolled in the third year (students who completed the curricular unit in 2020/2021 and attended the third year) and 0.8% (n=4) attended the fourth year (students who had not completed the curricular unit and attended the fourth year).
For academic performance, the average final mark for the curricular unit RCD1 was of 13.34 (in a scale of 0-20). Also, the overall degree average at the time of data collection was 14.18 (in a scale of 0-20). Each ESEP student completed an average of 31.14 VPS scenarios in the curricular unit RCD1.
Overall, ESEP students obtained an average score of 82.50% in evaluating the training performed in the VPS. When analysing the items incorporating the assessment of VPS usage, students attained an evaluation score of 85.6% for the physical examination, 72.1% for the diagnostic activity, and 82.8% for the implemented treatment.
In this study, students who attended the curricular unit in the academic year 2021-2022 exhibited better results in the overall score of physical examination evaluation in the VPS (M=85.3; SD=9.0), compared to those who attended the curricular units in 2020-2021 (M=80.4; SD=16.6); t (514) =-0.418, p<.001, d=-0.37.
Furthermore, male students obtained a higher score in the implemented treatment evaluation (M=86.3; SD=8.4) compared to female students (M=85.9; SD=11.4); t (471) =-0.601, p=.026, d= -0.077.
Regarding the number of completed scenarios, a positive and moderate relationship was found between the number of completed scenarios and the final grade of the curricular unit RCD1 r (519) =.42; p<.001, and the overall degree average r (370) =.36; p<.001. This result indicates that the greater the number of solved scenarios the higher the likelihood of positive influence of the overall average score of VPS r (521) =.41, p<.001.
No statistically significant differences were found in the number of completed scenarios between students who attended the curricular unit in 2020-2021 (M=31.9; SD=17.4) and students who attended the curricular unit in 2021-2022 (M=30.5; SD=17.7); t (515) =0.949, p =.821, d=0.084.
Also, no significant differences were observed in the overall score of evaluation obtained from the scenarios concluded in the VPS between students who attended the curricular unit in 2020-2021 (M=82.5; SD=8.4) and those who attended the curricular unit in 2021-2022 (M=82.5; SD=8.7); t (515) =0.031; p=.587; d=0.003.
Discussion
In this study 521 nursing degree students used a VPS Body Interact®. On average, students completed 31 virtual patient simulation sessions. Students obtained a global nursing decision-making score of 82.5% (SD=8.52), a physical examination score of 85.6%, a diagnostic activity score of 72.1%, and a implemented treatment score of 82.8%, during their training with virtual patients. These findings suggest that using VPS as a complement to the educational strategy, offers students increasing training opportunities. These opportunities have a positive impact on decision-making skills, as observed in studies conducted by Plackett et al. (2022), Raupach et al. (2021), Padilha et al. (2019), and Plackett et al. (2020).
Positive relationships were observed between the number of completed scenarios, the overall average evaluation score, and final marks. Interestingly, male students attained the highest scores in implemented treatment evaluation. These findings suggest that the completion of more scenarios positively influences the overall evaluation score in the VPS. Moreover, a positive relationship was observed between the number of completed scenarios and the final evaluation of the curricular unit and the course mark.
The study sample comprised over 83% female participants, with a mean age of 20.97 years. These data align with the typical profile of students enrolled in degree courses in Portugal where, at the national level, 82% of students are female (Estatísticas da Educação e Ciência, 2022).
The LMS functionality of the VPS allows teachers to manage each student's learning path, by offering real-time access to metrics on student’s usage and performance. Teachers can analyse the development of students decision-making processes and their engagement in training decision-making using the LMS. This feature enables the teacher to tailor the pedagogical strategy to optimize the student’s learning experience. This can be demonstrated by the improvement of the student’s overall scores in the evaluation of the physical examination in the VPS across the two periods under study. Notably, the global scores of the VPS usage exhibited higher average values, indicating alignment between the skills developed by students and the course's learning objectives, particularly regarding clinical reasoning and the decision-making process. However, in this study, the average evaluations of VPS usage surpassed the final marks of the curricular unit RCD1. This difference can be explained by the fact that this curricular unit aims to develop clinical reasoning and instrumental skills in cardiorespiratory, intestinal and bladder elimination, and fluid volume management. Thus, the final evaluation of the curricular unit is determined by the theoretical and theoretical-practical components (60%) and laboratory practical components (40%). Importantly, in the laboratory practical component, the use of the VPS and/or its evaluation scores do not compete for the evaluation of the component. The VPS serves as a tool for students to autonomously prepare for laboratory practical classes.
The global average assessment scores obtained by students with the use of VPS find parallels in the evidence from several studies on the use of VPS. The VPS is effective in several domains of the development of professional skills in the health area. According to these studies, VPS contributes to the improvement of knowledge, critical thinking, clinical reasoning, instrumental skills, perception of self-efficacy, satisfaction with learning, interaction and feedback, teamwork, learning experience, and the realism of simulation spaces, thus enhancing emotional safety (Baumann-Birkbeck et al., 2017; Brown et al., 2021; Dale-Tam et al., 2021; Fogg et al., 2020; Foronda et al., 2020; Kononowicz et al., 2019; Kyaw et al., 2019; Padilha et al., 2019; Padilha et al., 2019; Pence, 2022; Plackett et al., 2022; Qiao et al., 2023; Sim et al., 2022).
This study revealed that, on average, students experienced at least 31.14 additional learning opportunities when using VPS. This finding is supported by the integration of VPS alongside existing pedagogical strategies adopted during this course. Moreover, data analysis revealed positive relationships between the number of completed scenarios, the overall average evaluation scores using VPS and their relationship with final course grades and degree attainment. However, regarding the relationship with degree marks, further research on evaluating the impact of the use of VPS on the development of competencies transversal to the Nursing Degree course is necessary.
The use of VPS demonstrated high levels of perceived utility, easiness of use and intention of use by VPS students indicating alignment of students perceived needs (Padilha et al., 2018). This promoted an optimized disposition to learn by providing a pedagogical tool for early preparation of classes. This pedagogical approach incites curiosity and fosters active learning, encouraging self-responsibility for learning through realistic clinical scenarios tailored to student development levels and learning objectives. Notwithstanding the abovementioned, the gamification elements inherent in VPS can also enhance students' motivation for their use (Buckley & Doyle, 2016; Padilha et al., 2024).
Using the VPS with the simulator allows automatic unidirectional feedback through the Body Interact® “debriefing feature”. Adding to this functionality is the availability of teachers to solve doubts and demonstrate optimal scenario resolutions during laboratory practice class. This creates an environment where students can openly share doubts and feelings, tailoring their learning experience to their expectations, motivations, and individual circumstances.
This study highlights that the emerging question is no longer about the necessity or the imperative to use VPS in health education. Instead, research should focus on optimizing the development of professional skills before clinical practice.
Study Limitations
This study presents a significant limitation by comparing only two sets of data from two academic years and addressing only one curricular unit from a Nursing degree in a single context. However, this study integrates a substantial dataset of 16,444 virtual clinical scenarios, completed by 521 students - a volume of data unparalleled in existing evidence. The decision to restrict the analysis to the 2020-2021 academic year is justified because the LMS framework had a new structure for data extraction after 2020.
Another limitation of the study is that the evaluation derived from the use of VPS does not carry a specific weight in the overall curricular unit mark, therefore, impeding an analysis of the impact of VPS in the course final mark. Additionally, it is crucial to highlight that VPS represents a pedagogical strategy still in its infancy of integrating the health field curricula, limiting the ability to thoroughly discuss these findings and compare them with similar studies. Irrespective of these results, it cannot be assumed that it would translate to in-person care for actual patients.
Conclusion
This is a pioneer study because, to the best of our knowledge, it marks the first evaluation of the outcomes resulting from the integration of a virtual patient simulator as a pedagogical strategy applied to nursing degree students.
The overall average results exceeding 82% underscore the adequacy of the clinical scenarios used in VPS to the students' competencies development stage, their learning preferences, and the objectives of the curricular unit where they were used.
Integrating VPS as a pedagogical strategy into a study curriculum requires substantial investment from higher education institutions and teachers, encompassing training, technology, and dedicated time for reflection, discussion, and analysis. Our journey of research, development, and use allows us to provide a valuable reference for those embarking on the adoption of this pedagogical strategy.
Footnotes
Acknowledgements
The authors wish to acknowledge the support and role of Professor Paulo Machado, Ana Ribeiro and José Ramos in the conception, development and implementation of the use of VPS. We extend this acknowledgement to all researchers of the Nursing Clinical Reasoning Education Project, all the Professors of the Curricular Unit of Body Processes Responses 1 between 2016-2023, and all nursing students enrolled. The authors wish to acknowledge the contribution of the professional translator Maria do Amparo Alves in editing this article.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article was supported by National Funds through FCT - Fundação para a Ciência e a Tecnologia, I.P., within CINTESIS, R&D Unit (reference UIDB/4255/2020 and reference UIDP/4255/2020).
