Describing How Scenario Topic,Situation Urgency and Complexity Level Impacts Simulation Observers’ Clinical Judgment: A Secondary Analysis

Abstract

Introduction

Nurses use clinical judgment (CJ), or the process of “thinking like a nurse” by recognizing and interpreting a patient's concerns, to guide care decisions. However, nursing students graduate without sufficient CJ for safe nursing practice. A prior investigation found observing simulations leads to significant, but inconsistent CJ gains, depending on learner ability. Observers watched diverse simulation designs with increasing complexity, highlighting the need to understand which conditions best support CJ development.

Objective

This secondary analysis of a prior investigation aims to describe how simulation designs (i.e., scenario topic and situation urgency) and complexity level impacts CJ. A secondary aim is to investigate how simulation designs and complexity level impacts CJ based on learner ability.

Methods

Sixty-one junior-level baccalaureate students participated in the study over a semester. Participants observed eight expert-modeled simulations, each focused on medical-surgical or mental health scenarios in either urgent or routine situations. After each simulation, participants responded to 11 CJ prompts, which were scored using the Lasater CJ Rubric (LCJR). The scenarios were rated for complexity (simple, moderate, difficult, complex) based on author-identified sources. Participants were grouped into low-, medium-, or high-ability categories based on their cumulative LCJR scores. Two separate two-way repeated measures ANOVAs examined how simulation topics and situation urgency influenced CJ, with analyses repeated for each ability group.

Results

Observers’ mean CJ is influenced by both simulation design and complexity level. CJ decreases in complex medical-surgical and mental health scenarios as complexity exceeds a “difficult” rating, regardless of ability. However, CJ improves in simple and moderately complex routine situations. Observing “difficult” and “complex” simulations hindered significant CJ gains, regardless of ability.

Conclusions

Nurse educators should consider learner ability and complexity level when designing simulation observations to prevent overwhelming observers’ CJ, especially during urgent and complex medical surgical or mental health scenarios.

Keywords

clinical judgment education nursing student observation simulation training

Problem

Nurses use clinical judgment (CJ) in nearly every nursing task to make safe patient care decisions (Dickison et al., 2019). CJ, a complex cognitive skill, involves thinking through patient care situations to notice salient cues, interpret meaning, respond appropriately, and reflect on clinical decisions (Klenke-Borgmann et al., 2020; Tanner, 2006). However, novice nurses demonstrate inadequate CJ (Kavanagh & Sharpnack, 2021). Consequently, newly graduated nurses have trouble applying theoretical knowledge, which negatively impacts their clinical judgment and leads to medication errors (Murray et al., 2020) as well as unsafe hospital conditions (Chaboyer et al., 2021; Jessee, 2021; Treacy & Caroline Stayt, 2019). Nurse educators must find ways to improve learners’ CJ to meet essential nursing education requirements (American Association of Colleges of Nursing, 2021).

Review of Literature

Simulation-based learning experiences (SBLE) expose learners to realistic situations replicating real-world practice complexities (Lioce et al., 2025). Active SBLE participation develops CJ (Lee & Oh, 2015; Nielsen et al., 2023) more efficiently than traditional hospital-based clinical learning experiences because learners spend more time-on-task (Sullivan et al., 2019). Therefore, many American nursing boards allow replacing half of hospital-based clinical hours with simulation (Bradley et al., 2019). However, learners spend a majority of simulation time observing in non-active roles to accommodate multiple learners and increase simulation capacity (Hayden et al., 2014; O’Regan et al., 2021; Smiley, 2019). While assigning observer roles improves simulation scalability, learners’ random rotation through simulation roles could lead to inequitable simulation exposure.

Nurse educators assign asynchronous expert-modeled simulation (EMS) observations to increase simulation exposure, create equitable experience, and facilitate simulation CJ development (Dodson, 2023; Huun, 2018; Rogers & Franklin, 2022, 2023). During asynchronous EMS observations, learners independently watch expert providers delivering simulated nursing care (Baldwin et al., 2014). Educators vary simulation observations by altering scenario topics and situation urgency with increasing complexity, assuming exposure to different experiences builds CJ. The original study found 63 junior-level nursing students’ CJ significantly increased after observing eight EMS videos with varying scenario topics, situation urgency, and complexity levels. However, gains were variable and peaked at the fifth video before declining and stalling after the sixth. When learners were stratified into groups by ranking total scores, high- and medium-groups showed more linear improvement trends up to the fifth video, while lower-performing groups exhibited less predictable gains (Rogers & Franklin, 2022). These findings suggest scenario topics, urgency, and complexity could influence CJ.

Nurse educators vary scenario topics by using different medical diagnoses to promote clinical judgment (CJ) development across diverse clinical contexts. Hambach et al. (2023) reported mean CJ improvements when learners engage with varying diagnoses during active simulation participation. In contrast, Fogg et al. (2023) found CJ performance decreased when learners cared for patients with different diagnoses in repeated virtual simulations, raising questions about the transferability of CJ across scenario topics. Notably, no studies have examined how exposure to different scenario topics during observation influences CJ development (Rogers et al., 2020). Understanding the relationship between scenario topics and observers’ CJ is essential to optimizing simulation design.

Novice nurses are most often criticized for having insufficient CJ when caring for deteriorating patients in urgent situations (Kavanagh & Sharpnack, 2021; Lasater et al., 2015; Treacy & Caroline Stayt, 2019). Zulkosky et al. (2016), a seminal study investigating simulation observers’ CJ during urgent situations, found observers outperformed active participants in unfamiliar, urgent situations due to better cue recognition and interpretation. However, in familiar scenarios observers’ CJ was less accurate, suggesting unfamiliarity during urgent situations impacts observers’ cue processing. However, researchers measured CJ within one simulation during reflective pauses, which are known to impact learners’ thinking (Lee et al., 2024). Examining observer CJ across multiple scenarios during varying levels of situation urgency without relying on reflective pauses could clarify how situation urgency impacts observers’ CJ.

Nurse educators expose learners to diverse scenario topics and highly urgent situations to prepare learners for rising patient acuity and complexity levels. However, better preparing nurses for real-world complexities requires understanding what contributes to complex patient care. Recent work to define nursing patient care complexity confirms the multifaceted nature of complex patient care (Huber et al., 2021). Using a hybrid concept development mode, Huber et al. (2021) redefined patient-related complexity as a “relational and dynamic phenomenon with multiple, interconnected processes” shaped by the interplay of several elements such as illness severity, therapy challenges, psychosocial factors, and available resources.

This definition aligns patient care complexity with Cognitive Load Theory (CLT), which explains complexity as the sum of interacting elements within a task or environment (Choi et al., 2014; Sweller et al., 2011). These interacting elements can either enhance or hinder learning by influencing cognitive resources for learning. As the number and interactivity of elements increase, learners must process more information, which can overwhelm their working memory, particularly for novices (Sweller et al., 2011). Previous studies have shown that increasing the number of elements in simulation designs negatively impacts simulation performance (Smith et al., 2022; Tremblay et al., 2019, 2023). Other research consistently demonstrates novices are better equipped to manage simpler tasks, while highly complex tasks exceed their cognitive capacity, hindering performance (Cabrera-Mino et al., 2019; Haji et al., 2016; Shinnick, 2022; Shinnick & Cabrera-Mino, 2021; Smith et al., 2022; Tremblay et al., 2019, 2023). However, there is a gap in the literature comparing how different complexity levels impact observers’ CJ outcomes. This gap limits designing simulation observations that appropriately challenge learners based on their expertise level.

Learners’ expertise, or ability, is shaped by prior knowledge and backgrounds, which impacts CJ development (Klenke-Borgmann et al., 2023; Lasater et al., 2019; Lavoie et al., 2022; Lesā et al., 2021; Roskos et al., 2021; Shinnick, 2022; Shinnick & Cabrera-Mino, 2021; Young et al., 2016). While many studies demonstrate the impact of prior knowledge on CJ development during active simulations, few have explored this relationship in the context of simulation observations (Lasater et al., 2019; Rogers & Franklin, 2022). The original study revealed that while observers across stratified ability levels showed meaningful gains in CJ, these gains were not uniform. Learners in different ability groups exhibited distinct CJ trajectories and responded differently to increasing simulation complexity. Qualitative findings further illustrated learners’ approaches, knowledge application, and thought processes varied by ability level (Rogers & Franklin, 2022). This secondary analysis aims to stratify learners by ability to better understand how prior knowledge influences CJ during simulation observations. By exploring how learners of varying ability levels respond to simulation complexity, nurse educators can gain critical insights into tailoring simulation designs and determining appropriate complexity levels according to learner ability.

The Healthcare Simulation Standards of Best Practice (HSSOBP) emphasize the importance of scaffolding in simulation design, but they do not clarify when or how learners are ready for more complex simulation experiences (Miller et al., 2021). As a result, decisions about what students are ready for are often left to expert discretion. Understanding how simulation design conditions influence CJ enables educators to create more efficient simulation experiences that align with learners’ developmental stages and cognitive capacities (Eppich & Reedy, 2022). Therefore, the purpose of this study is to describe how simulation designs (i.e., scenario topic and situation urgency) and complexity level impacts CJ. A secondary purpose is to investigate how simulation designs and complexity level impacts CJ based on learner ability. Therefore, this study will answer the following research questions:

How do simulation design elements (scenario topics or situation urgency) impact clinical judgment (CJ) outcomes across different complexity levels?

Hypothesis 1: CJ outcomes depend on the simulation design (i.e., scenario topic or situation urgency) and complexity level.

Hypothesis 2: As complexity level increases, learners will demonstrate initial improvements in clinical judgment (CJ) up to an optimal complexity threshold. Beyond this threshold, further increases in complexity will overwhelm learners, resulting in a stalling or declining CJ performance.

How does learner ability influence clinical judgment (CJ) outcomes in response to simulation designs (scenario topics and situation urgency) and varying complexity levels?

Hypothesis 3: CJ outcomes differ based on learner ability.

Theoretical Framework

The NLN Jeffries Simulation Theory (Jeffries, 2021) posits simulation design and participant characteristics impact participant outcomes. Simulation design in this study includes investigating simulation scenario topics, situation urgency, and complexity level. Participant characteristics include learner ability. The outcome of interest is CJ.

Methods

Study Design

This secondary data analysis is a descriptive, one-group quasi-experimental design. The study used a within-subject factorial design with simulation design (i.e., scenario topic and situation urgency) and complexity level as within-subjects factors.

Sample and Setting

The sample included 61 out of the 63 junior-level traditional baccalaureate learners from the original study conducted in a university in the southern United States who voluntarily consented to researchers maintaining data in a repository.

Ethical Considerations

The university institutional review board concluded this study was exempt from board oversight (IRB #2023-429). Participants provided voluntary consent to participate in the original study and to researchers maintaining data in a repository for future use. Deidentified data were stored in a password-protected, double identification cloud system managed by the university IT department according to the IRB protocol. Participation was voluntary and there were no incentives or extra credit offered to participate because all study activities were regularly scheduled coursework.

Data Collection Procedure

Participants observed two asynchronous EMS videos prior to each in-person simulation day throughout the semester, totaling eight observations. Participants provided written responses to 11 CJ prompts aligned, with the LCJR, describing what participants noticed, interpreted, responded, and reflected from each EMS observation using Qualtrics^XM prior to debriefing (Table 1). Written responses were scored using the Lasater Clinical Judgment Rubric (LCJR, Lasater, 2007) after obtaining the author's consent. For this secondary analysis, EMS videos were categorized by scenario topic and situation urgency (Table 2). EMS videos increased in complexity over time.

Table 1.

Clinical Judgment Prompts Matched to the LCJR and Tanner's Model of CJ.

Tanner CJ concept	LCJR Subscale	Prompts Item
Noticing	Focused observation	What did you first notice about the patient?
	Recognizes deviation from expected pattern	What was different than you expected? Have you seen this before in other patients?
	Information seeking	What other information would be helpful? How can you get that information?
Interpreting	Prioritizing data	How did you prioritize the patient information/data? In other words, what was most important for this patient now?
	Making sense of the data	How do you think the nurse based her intervention? If intuition, what kinds of data might offer evidence to support a gut feeling?
Responding	Calm manner	What was the nurse's approach with the patient? How comfortable would you be in this situation?
	Clear communication	How did the nurse gain the patient's trust? What did she say to the patient? To the family member(s)?
	Well-planned intervention	What factors, including patient feedback, impacted the treatment plan?
	Being skillful	How did the nurse's skill compare to nursing standards of care?
Reflecting	Evaluation/self-analysis	What went well? What didn’t go as smoothly? Why or why not?
	Commit to improve	What would you do differently if you had the opportunity?

Note: The table shows how each prompt (adapted from Lasater (2011) with permission from the author) relates to the LCJR subscales and Tanner's (2006) CJ Model.

Table 2.

Expert Modeling Video Scenario Design by Scenario Topic and Situation Urgency.

EM Video	Scenario Topic		Situation Urgency
EM Video	Medical- Surgical	Mental Health	Urgent	Routine
1	Diabetic patient with foot ulcer who is schedule for a femoral-popliteal bypass			Performing pre-operative teaching
2		Depressed patient who recently attempted suicide by acetaminophen		Treating severe nausea prior to giving oral antidote medication
3	Post-operative appendectomy receiving IV antibiotics			Admitting patient from the PACU with minimal pain
4		Patient with bipolar disorder who is experiencing mania	Restarting antipsychotic medication that was erroneously discontinued to manage manic episode
5		Patient who recently lost long-term spouse and has active suicidal ideation	Helping a client in a home health setting expressing suicidal ideation
6		Transgender patient with anxiety and anger from experiencing healthcare discrimination		Needing to restart oxygen after inadvertent removal during patient transport causing desaturation
7	Congestive heart failure exacerbation patient experiencing shortness of breath		Managing fluid overloaded patient with oxygen desaturation and inappropriate IV fluid orders
8	Patient diagnosed with pyelonephritis experiencing anaphylactic reaction to antibiotics		Anaphylactic reaction with profound decompensation

Study Factors

Simulation Design

Simulation observation scenarios were designed according to the Healthcare Simulation Standards of Best Practice (INACSL Standards Committee, 2021). Scenarios are reviewed each semester by expert simulation faculty. Practice content experts review scenarios at the initial design and each semester when faculty have questions or make changes to ensure scenarios align with current practice.

Scenario Topic

Medical-Surgical

Medical-surgical scenarios depict caring for clients with acute or chronic medical conditions from wide-ranging pathophysiological disorders or surgical procedures. In this study, medical-surgical scenarios included clients awaiting surgery, recovering from an appendectomy, experiencing heart failure, and receiving antibiotics for pyelonephritis.

Mental Health

Mental health scenarios depict nursing care for clients experiencing mood alterations, behavioral challenges, emotional instability, or psychiatric disorders. Mental health scenarios depicted clients who attempted suicide, experienced mania, expressed suicidal ideation, and experienced gender dysphoria.

Situation Urgency

Urgent

Urgent situations in this study refer to any situation requiring immediate nursing action to prevent losing life, limb, or hemodynamic stability, or maintaining client or healthcare provider's safety. In this study, urgent situations included observing nursing care for a client experiencing acute mania, suicidal ideation, respiratory distress from pulmonary edema, and anaphylactic reaction.

Routine

Conversely, routine situations involve standard procedures, assessments, or skills without an unexpected client condition change. Routine scenarios included observing nursing care for clients awaiting a femoral-popliteal bypass graft, experiencing nausea, recovering from a recent appendectomy, and being discharged with pneumonia.

Complexity Level

Complexity level was determined post hoc by rating researcher-identified complexity sources from simple, moderate, difficult, to complex. Table 3 provides EMS simulation design examples determining complexity ratings for each complexity source. Table 4 rates each EMS's complexity based on each complexity source. Generally, the first and second simulation were simple, then increased by a complexity level every two simulations.

Table 3.

Example Video Components for Determining Complexity Ratings by Complexity Source.

Complexity Source	Simple	Moderate	Difficult	Complex
Cue recognition	Obvious cues that are very familiar and common	Cue is familiar but is slightly ambiguous	Cue is less familiar, uncommon, or complex	Subtle cues or requires managing multiple cues
Cue significance interpretation	Cues relate to obvious problem with little room for misinterpretation	Requires minimal interpretation	More ambiguous and may be interpreted multiple ways; requires inferences	Requires individualized understanding of patient risk or condition
Patient risk	Probability of adverse events happening is very low	Potential adverse outcomes are manageable	Adverse outcomes are manageable with prompt intervention	Requires immediate intervention to prevent risk
Patient advocacy	Requires ensuring patient receives standard care, education, consent or resources	Patient requires help accessing resources, coordinating care or managing symptoms	Ensuring patient rights are upheld; Addresses ethical or legal issues; Requires mental health referral	Addressing social determinants of health; not intervening risks patient life
Patient stability	Stable; vital signs are within normal limits	Generally stable, but has risks needing monitoring; minor vital sign fluctuations	Could deteriorate without intervention; abnormal vital signs require intervention	Deteriorating patient; patient vital signs are severely abnormal or life-threatening
Order management	Implementing standing orders	Obtaining new orders	Questioning standing orders	Urgent order implementation
Situational awareness	Straightforward, stable environment	Requires monitoring few risk factors or minor/ expected complications	Requires frequent monitoring due to changing patient condition; requires monitoring unexpected complications	Complex and dynamic situation requiring monitoring multiple cues that could be life-threatening
Medical terminology	Common diagnoses and terminology	Terms require specific knowledge about a less common condition or intervention	Includes technical terms or acronyms related to an uncommon diagnosis or intervention	Includes terms or acronyms related to highly complex diagnosis, procedure, lab result, or nursing specialty
Relationship skills	Conversations build basic rapport using conversations or basic therapeutic communication skills	Requires providing emotional support, assessing basic risks, gathering patient history or explaining common procedures	Conducting individualized health histories, risk assessment, or therapeutic communication techniques with multifaceted problems	Managing crisis situations; Coordinating care with healthcare team members; using nursing skills to mange sensitive situations
Care individualization	Scenario relates to standard healthcare with minor adjustments to meet individual needs	Nurse conducts a more holistic assessment and considers physical, emotional, spiritual, or social needs	Nurse manages multiple interrelated health conditions, consulting other disciplines, or an evolving health status	Nurse manages multiple interrelated health conditions, consulting other disciplines, or an evolving health status individualized to the patient in an urgent situation or based on continuous patient evaluation; may also consider long-term needs in an evolving situation
Task management	Nurse can focus on one thing at a time	Nurse completes 2– critical tasks	Multiple competing priorities	Requires prioritizing tasks in an urgent or life-threatening scenario
Care Prioritization	Standard care needed with minimal prioritization	Requires prioritizing tasks using common strategies like airway breathing circulation	Requires prioritizing tasks using less common strategies (e.g., acute vs chronic; urgent vs non-urgent; stable vs. unstable)	Requires prioritizing tasks in an urgent scenario where the consequence of improper prioritization could be life threatening
Critical Behaviors	Standard care needed	Nurse verbalizes priority interventions with rationale	Priority interventions are verbalized without rationale	Priority interventions are not verbal and require observing behavior
Medication knowledge	Administering commonly prescribed medications; No drug allergies; Only one medication due;	Administers less common drug specific to the patient condition; Prioritizing less than three medications; Patient has one allergy that is relevant to the drug administered in the scenario; must consider when last dose was administered	Administers a drug related to a nursing specialty (e.g., emergency psychiatric medications); Needs to prioritize multiple (>3) medications; Patient has one allergy that is irrelevant to the drug administered in the scenario; multiple related drug allergies; must consider when multiple drugs were last considered	Administers a drug rarely given or learner has no experience with in an urgent scenario (e.g., fluid bolus); Needs to prioritize multiple medications and administer rapidly; Multiple unrelated drug allergies; Must question drug appropriateness

Table 4.

Expert Modeling Video Complexity Ratings by Source.

Learner Ability

Participants were classified as having low, medium, or high CJ ability based on ranking cumulative total LCJR scores from all eight EMS observations. Low ability learners cumulative LCJR ranged from 114 to 138, whereas medium and high ability ranged from 140 to 148 and 149 to 172, respectively. Higher cumulative scores indicated high learner ability.

Measures

Clinical Judgment

The LCJR, based on Tanner's CJ model (Tanner, 2006), measures CJ over four phases (i.e., noticing, interpreting, reflecting, and responding). These phases are subdivided into 11 domain items that describe required elements for developing effective noticing, interpreting, responding, and reflecting. For example, effective noticing requires focused observation, recognizing deviations from expected patterns, and information seeking. Each domain item is assessed using a four-point Likert scale, with descriptors corresponding to four performance levels: beginning (1), developing (2), accomplished (3), and exemplary (4). Total scores range from 11 to 44, with higher scores indicating more advanced clinical judgment. LCJR has good internal consistency, with Cronbach alpha ranging between .89 and .974 with measuring CJ in undergraduate nursing students (Adamson & Kardong-Edgren, 2012; Cantrell et al., 2021). LCJR is also valid (Victor-Chmil & Larew, 2013) and sensitive (Shinnick & Woo, 2020) for measuring novice learners’ CJ. LCJR also has acceptable reliability for measuring CJ in observers’ written reflections, with a Cronbach alpha of .67 (Rogers & Franklin, 2023). While Cronbach alpha in observers represent marginally acceptable scores, there are no other current methods in the literature for objectively measuring observers’ clinical judgment.

Demographics

Participants reported age, race, gender, ethnicity, learning style preference, previous degrees, and previous healthcare experience via Qualtrics^XM using a researcher-developed tool.

Statistical Analysis

Data were analyzed using commercially available statistical software (IBM SPSS v.26 and Stata MP v.15). A statistician provided oversight for developing analysis plans and interpreting results. Power calculations using G*Power indicated 36 participants were needed for detecting a medium effect, based on 95% power with an α value 0.05 (Faul et al., 2007). Inter-rater reliability was performed on ten percent of LCJR scores using Gwet's AC statistic (Gwet, 2008) and percent agreement. Gwet's AC serves as an alternative to Cohen's kappa for assessing inter-rater reliability (IRR). Unlike kappa, which can be influenced by biases stemming from observer disagreements or imbalanced marginal distributions, Gwet's AC is designed to mitigate these issues and provide a more robust measure of agreement. Interrater reliability in this study represented substantial agreement with a cumulative AC 0.74 (± 0.045 CD) and acceptable agreement with 78.15 (±3.39% SD) percent agreement.

Participant Demographics

Participants’ demographics were compared between high, medium, and low ability groups using Fisher Exact, chi-square, Kruskal Wallis, and one-way analysis of variance (ANOVA) for dichotomous, multinomial, ordinal, and continuous variables respectively. Kruskal Wallis analysis was used when ANOVA assumptions were violated.

Simulation Design Impact on CJ

After checking test assumptions, the effects of scenario topic and complexity level on CJ were analyzed using a two-way repeated measure analysis of variance (RM ANOVA). This analysis was repeated to examine the effect of situation urgency and complexity level on CJ.

Simulation Design Impact on CJ by Learner Ability

Serial two-way RM ANOVA's were performed to analyze the effect of scenario topic and complexity level for high, medium, and low ability learners’ CJ after checking test assumptions. Serial analyses were repeated to examine situation urgency and complexity level effect on CJ for each learner ability group.

Results

Sample Characteristics

The study population was predominantly white and female. Participants had similar demographic characteristics based on CJ ability (Table 5).

Table 5.

Demographic Characteristics by Ability Groups.

		Learner Ability			Group Differences
		High(n = 21)	Medium(n = 20)	Low(n = 20)	χ²	F	H	p value
Variable		n (%)	n (%)	n (%)	χ²	F	H	p value
Race	Asian	3 (14.3%)	2 (10%)	0	5.016			.286
	African American	0	2 (10%)	1 (5%)
	White	18 (85.7%)	16 (80%)	19 (95%)
Gender	Female	20 (95.2%)	17 (85%)	19 (95%)	1.831			.400
Gender	Male	1 (4.8%)	3 (15%)	1 (5%)	1.831			.400
Ethnicity	Hispanic	2 (9.5%)	5 (25%)	2 (10%)	1.77			.413
Ethnicity	Non-Hispanic	19 (90.5%)	15 (75%)	18 (90%)	1.77			.413
Learning style preference	Read/Write	5 (23.8%)	3 (15%)	4 (20%)	3.566			.468
	Auditory	4 (19%)	5 (25%)	1 (5%)
	Kinesthetic	12 (57.1%)	12 (60%)	15 (75%)
Previous degree	No	16 (76.2%)	17 (85%)	16 (80%)	.505			.777
Previous degree	Yes	5 (23.8%)	3 (15%)	4 (20%)	.505			.777
Previous healthcare experience	No	13 (61.9%)	13 (65%)	16 (80%)	1.77			.413
Previous healthcare experience	Yes	8 (38.1%)	7 (35%)	4 (20%)	1.77			.413
Age mean (SD)		23.2 (4.28)	23.2 (5.06)	23.5 (1.31)			1.4	.497
GPA mean (SD)		3.29 (.041)	3.31 (.375)	3.40 (.243)		.072		.499

The data met 2-way RM ANOVA test assumptions except CJ was not normally distributed as assessed by Shapiro-Wilk's test on the studentized residuals values p > .05. There were also a few studentized residual outliers, with values ranging from 3.08 to 3.43. After consulting with a statistician, non-normally distributed CJ data were not transformed, and outliers were not removed because neither significantly impacted results. Degrees of freedom were corrected using Greenhouse-Geisser method for any test with Mauchly's Test of Sphericity p > .05. Table 6 provides mean LCJR scores by simulation design and learner ability.

Table 6.

LCJR Means by Simulation Design for Overall Group and by Learner Ability.

	Scenario Topic				Situation Urgency
	Med-Surgical		Mental Health		Routine		Urgent
	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Observation 1
Overall (n = 61)	16.43	2.232	15.87	2.837	16.43	2.232	18.20	2.587
High (n = 21)	17.52	2.620	17.57	2.785	17.52	2.620	20.67	1.623
Medium (n = 20)	16.05	1.905	15.05	1.932	16.05	1.905	17.90	1.165
Low (n = 20)	15.65	1.663	14.90	2.936	15.65	1.663	15.90	2.150
Observation 2
Overall (n = 61)	17.93	2.626	18.20	2.587	15.87	2.837	19.70	2.194
High (n = 21)	19.57	2.204	20.67	1.623	17.57	2.785	21.00	1.265
Medium (n = 20)	18.65	1.785	17.90	1.165	15.05	1.932	20.30	1.625
Low (n = 20)	15.50	1.933	15.90	2.150	14.90	2.936	17.75	2.149
Observation 3
Overall (n = 61)	18.70	2.194	19.70	2.194	17.93	2.626	18.70	2.194
High (n = 21)	20.00	1.673	21.00	1.265	19.57	2.204	20.00	1.673
Medium (n = 20)	19.20	1.609	20.30	1.625	18.65	1.785	19.20	1.609
Low (n = 20)	16.85	1.981	17.75	2.149	15.50	1.933	16.85	1.981
Observation 4
Overall (n = 61)	18.20	2.064	17.82	2.247	17.82	2.247	18.20	2.064
High (n = 21)	19.52	1.861	19.33	1.770	19.33	1.770	19.52	1.861
Medium (n = 20)	18.40	1.188	17.95	2.012	17.95	2.012	18.40	1.188
Low (n = 20)	16.60	1.930	16.10	1.714	16.10	1.714	16.60	1.930

Simulation Design Impact on CJ

Scenario Topic and Complexity Level

There was a statistically significant interaction between scenario topic and complexity level on CJ F(3, 180) = 4.773, p = .003. η_p² = .074. There was a significant effect of complexity level on CJ for medical surgical and mental health scenario topics, respectively F(3, 180) = 15.877, p < .001, F(2.479, 148.753) = 36.077, p < .001, ε = .826 (Figure 1). Post hoc complexity rating planned comparisons revealed CJ did not significantly change after the second medical surgical observation complexity level rated “difficult” (M = −0.262, SE .314, p = 1). As complexity increased, CJ significantly increased in mental health scenarios between “Simple” and “Moderate” complexity ratings (M = 3.836, SE .408, p < .001), but significantly decreased when rated “difficult” (M = −1.885, SE = .313, p < .001).

Figure 1.

Clinical judgment means by scenario topic and scaffolded complexity.

Situation Urgency and Complexity Level

There was a statistically significant interaction between scenario urgency and complexity level on CJ F(3, 180) = 20.181, p < .001, η_p² = .252. There was a significant effect of complexity level on CJ for routine and urgent scenarios, respectively F(3,180) = 13.475, p < .001, η_p² = .183; F(3, 180) = 10.741, p < .001, η_p² = .152 (Figure 2). Post hoc complexity rating planned comparisons revealed as complexity increased in routine scenarios, observers’ CJ made significant gains in “moderate” complexity levels (M = 1.508, SE = .409, p < .001), but then decreased insignificantly when advancing to “difficult” complexity levels (M = −.114, SE = .351, p = 1). More notably, during urgent situations, observers made significant gains over the first two “moderate” complexity observations (M = 1.508, SE .301, p < .001), but CJ significantly declined in “Difficult” (M = −1.000, SE .283, p = .005) and “Complex” levels (M = −1.508, SE .283, p < .001).

Figure 2.

Clinical judgment means by situation urgency and scaffolded complexity.

Simulation Design Impact on CJ by Learner Ability

Scenario Topic and Complexity Level

There was not a statistically significant interaction between scenario topic and complexity level on CJ in high, medium, or low ability learners, respectively F(3,60) = 1.324, p = .275, η_p²= 0.062, F(3,57) = 2.561, p = .064, η_p² = .119, F(3,57) = 2.135, p = .106, η_p² = .101 (Figure 1). The main effect of scenario topic did not significantly impact CJ between complexity level in high, medium, or low learners, respectively F(1,20) = 3.027, p = 0.097, η_p² = .131, F(1,19) = 1.174, p = .292, η_p² = .058, F(1,19) = .001, p = .972, η_p² = .000. However, complexity level did significantly impact CJ in high, medium, and low ability learners, respectively F(3,60) = 14.412, p < .001, η_p² = .419, F(3,57) = 42.617, p < .001, η_p² = .692, F(3,57) = 5.991, p = .001, η_p² = .240. Post hoc complexity rating planned comparisons revealed the following CJ impacts by learner level.

High Ability

Observers made significant gains between the first two complexity levels (M = 2.571, SE .578, p = 0.001), but then stalled at “difficult” levels and significantly declined during “complex” levels (M = −1.071, SE .330, p = .024).

Medium Ability

As complexity increased, observers made significant gains over the first three complexity levels (M = 4.2, SE = .413, p < .001), but then significantly decreased at the “complex” level (M = −1.575, SD = .293, p < .001).

Low Ability

Low performers were not able to demonstrate significant gains over the first two complexity levels, but then significantly increased CJ at “Difficult” levels (M = 1.6, SE .392, p = .001) but then lost most CJ gains at the “complex” level (M = −.950, SE .451, p = .379).

Situation Urgency and Complexity Level

CJ outcomes depend on the situation urgency and complexity level in high, medium, and low ability observers, respectively F(3,60) = 31.244, p < .001, η_p² = .306, F(3,57) = 13.741, p < .001, η_p² = .420, F(3,57) = 4.815, p = .005, η_p² = .202 (Figure 2). There was a significant effect of complexity level on high and medium, but not low ability observers’ CJ for routine situations, respectively F(3,60) = 4.549, p = .006, η_p² = .185, F(3,57) = 14.524, p < .001, η_p² = .433, F(1.91, 36.294) = 1.186, p = .315, η_p² = .059, ε = .637. Complexity level significantly impacted high, medium, and low ability observers’ CJ in urgent scenarios respectively F(3,60) = 3.402, p = .023, η_p² = .145, F(3,57) = 10.107, p < .001, η_p² = .347, F(3,57) = 3.425, p = .023, η_p² = .153. Post hoc complexity rating planned comparisons revealed the following CJ impacts by learner level.

High Ability

During routine situations, high-ability learners made the largest CJ gains in “moderate” complexity levels (M = 2, SE .778, p = .109), but increasing complexity level prevented making significant gains from the first to last scenario (M = 1.8, SE .758, p = .162). During urgent situations, CJ significantly declined as complexity increased from “difficult to “complex” levels (M = −1.476, SE .450, p = .023).

Medium Ability

During routine situations, CJ increased significantly in “moderate” complexity levels after observing two simple procedures (M = 3.6, SE .520, p < .001), but then there was no significant increase beyond “difficult” levels (M = 0.7, SE .711, p = 1). As complexity increased in urgent EMS observations, CJ significantly increased significantly in a “difficult” level after one practice observation (M = 2.4, SE .419, p < .001) but then significantly declined as complexity level rose beyond “difficult” (M = −1.9, SE .429, p = .002).

Low Ability

Increasing complexity prevented low-ability learners from gaining significant CJ from the first to last routine EMS observation (M = .45, SE .484, p = 1). As complexity increased in urgent situations, CJ significantly increased between “moderate” observations (M = 1.85, SE .563, p = .023) but then declined as complexity level rose beyond “difficult” (M = −.7, SE .539, p = 1).

Discussion

Nurse educators need to understand conditions impacting CJ, an essential novice nurse competency. Assigning simulation observations improves CJ, but gains are not consistent over time (Rogers & Franklin, 2022) This secondary data analysis is the first to explain how simulation designs (i.e., scenario topics and situation urgency) and complexity level impacts observers’ CJ. Findings from 61 junior-level prelicensure participants indicate observers’ CJ outcomes depend on the scenario topic and situation urgency complexity level. As complexity increases beyond “difficult” levels, CJ is negatively impacted in urgent and complex medical surgical or mental health scenarios, despite ability. Overall, observers’ CJ significantly improves in routine situations but is easily overwhelmed in urgent situations. However, high and low ability learners are vulnerable to premature complexity level increases even in routine situations. Therefore, nurse educators should consider learner ability and simulation designs when increasing complexity.

This study found complexity level impacted junior-level observers’ CJ in all simulation designs, despite learner ability. Specifically, CJ gains were significantly impacted when increasing to “difficult” levels. These findings are consistent with multiple previous studies that found placing healthcare professions learners in increased complexity negatively impacts learning outcomes (Haji et al., 2015; Smith et al., 2022; Tremblay et al., 2019, 2023). Educators need to understand what complexity level is appropriate for learners at all levels to inform simulation design best practice (Miller et al., 2021) and ensure instructional efficiency. This study suggests junior-level learners should have more time in “simple” and “moderate” simulations prior to increasing to higher levels. It is important to note that complexity in this study was determined by grading EMS qualities post hoc based on researcher-identified complexity sources. Learners could identify scenario complexity differently. Future studies investigating simulation complexity from the learners’ perspective could inform simulation design best practice.

Next, this study found observers’ overall CJ depends on the scenario topic and complexity level. Furthermore, overall CJ is negatively impacted in urgent and complex medical surgical or mental health scenarios. Contrarily, a recent study found junior-level nurses’ learning outcomes from complex medical-surgical simulations were significantly improved by repeating the same simulation with deliberate observer role practice after debriefing (Zulkosky et al., 2021). Debriefing and simulation sequencing may explain these differences. Observers in this study received debriefing after measuring CJ outcomes and were then placed in different simulation scenarios instead of repeating the same scenario. This required observers to transfer learning from previous simulation observations instead of applying knowledge gained from debriefing in the same simulation scenario. This study design mimics current practice because most simulation programs randomly assign observers to simulation scenarios thereby increasing simulation capacity but limiting opportunities for deliberate practice following debriefing feedback. It is possible junior-level observers need more time repeating observing complex urgent, medical-surgical, and mental health scenarios after debriefing rather than moving to a new, more complex scenario. Future studies measuring CJ outcomes prior to debriefing following deliberate role practice could help clarify simulation observation best practice.

This study found observers’ CJ significantly declines in urgent situations as complexity increases, despite learner ability. This is consistent with other studies finding novice nurses being ill-prepared when managing deteriorating patients (Kavanagh & Sharpnack, 2021). Klenke-Borgmann et al. (2023) suggest simulation is a method for developing novice nurses’ cognitive resources to effectively notice and respond to patients’ deterioration. Accrediting bodies require nurse educators evaluate learners’ CJ competency prior to graduation (American Association of Colleges of Nursing, 2021). A recent study demonstrates nurse educators can reliably measure and track observers’ CJ outcomes (Rogers & Franklin, 2023). However, this study adds important implications for measuring CJ competency. Since CJ declined with increasing complexity, nurse educators should not measure CJ using one scenario to determine CJ competency. Instead, CJ should be measured in multiple situations to ensure learners are able to display CJ in multiple situations. Furthermore, junior-level observers’ CJ was overwhelmed by observing complex situations indicating this may not be a reasonable learner expectation. Future research should quantify what CJ educators can reasonably expect learners to demonstrate at each level prior to graduation in complex situations to define CJ competency criteria.

Interestingly, even though observers’ overall CJ depends on the scenario topic and complexity level, findings were not the same based on learner ability. High, medium and low ability observers’ CJ did not depend on the scenario topic, only complexity level. Since simulation observations are an important educational tool for developing learners’ competency, educators need to understand more about conditions impacting observer outcomes. Exploring human factors associated with simulation role assignments could be an important next step. For example, simulation designs impact learners’ cognitive load (Rogers & Franklin, 2021). It is postulated observers have less cognitive load and more situational awareness than active participants because there is less performance stress (El Hussein & Ha, 2023; O’Regan et al., 2021). However, there is a dearth of evidence measuring cognitive load and situational awareness in observers. Measuring cognitive load and situational awareness could be an important next step in future studies for explaining this study's findings.

This study compared observers’ longitudinal CJ learning outcomes prior to debriefing over time and discovered simulation designs, most notably complexity level, impact learning outcomes. Collectively this study demonstrates nurse educators need to know more about leveling appropriate simulation complexity. Prematurely increasing simulation complexity either stalled progress or significantly negatively impacted observers’ CJ outcomes. This study used faculty opinions to determine appropriate complexity levels, and the researcher unintentionally put learners in too complex scenarios too soon. There is a conversation equating intentionally placing learners in highly complex situations as learner abuse because this practice causes significant training scars (Kardong-Edgren et al., 2024). This study confirms that prematurely increasing complexity negatively impacts CJ in complex and urgent, medical-surgical, and mental health scenarios. Therefore, it is imperative that future studies explore and quantify what learners perceive as complex and what situations are appropriate based on learner level and ability.

Nurse educators need to rethink how simulation complexity is scaffolded to ensure junior-level nursing students retain and effectively apply clinical judgment (CJ) skills. Instead of varying scenario topics and situation urgency, scaffolding complexity within a single topic and complexity source may be more effective for developing and retaining CJ. This approach could allow learners to build upon foundational skills and apply them to more complex situations, fostering deeper understanding and skill transfer. Research shows gradually increasing complexity levels enhances noticing skills (Poledna et al., 2022) and CJ improves when novices perform similar simulations when only slight changes are made to the patients’ scenario (Williams, 2024). Conversely, abrupt increases in complexity have been shown to negatively impact learning outcomes (Smith et al., 2022; Tremblay et al., 2019, 2023). By adopting a more deliberate and progressive approach to scaffolding complexity within similar scenarios, nurse educators can better support students in developing robust CJ skills that are both retained and readily applicable in practice.

In this study observers were provided simulation preparation, but not specifically preparing for observer roles. The HSSOBP for Pre-briefing recently defined role orientation as a critical step for learner simulation preparation (McDermott et al., 2021). Simulation observation thought leaders suggest observers need different preparation for simulation observation experiences (Johnson & Fey, 2023). It is possible providing individualized observer preparation prior to urgent and complex scenarios could have improved results by focusing observers’ attention to salient cues, thereby improving situational awareness. Future studies investigating observer simulation preparation and pre-briefing could inform HSSOBP for pre-briefing simulation observers.

As nurse educators move to use simulation for evaluating competency, it is important to ensure learners have adequate practice before high stakes simulation testing occurs. Specifically, the HSSOBP for learner evaluation requires learners to have adequate practice in similar experiences before high stakes simulation testing (McMahon et al., 2021). Many programs use simulation observations as preparation for active simulation, assuming this prepares learners for active simulation experiences. This study indicates most observers made CJ gains during less complex practice sessions, but more practice is needed before increasing complexity in complex, urgent, medical-surgical, and mental health scenarios. It is important to note that low and high performers were especially vulnerable to prematurely increasing complexity because they were unable to make CJ gains in routine situations along with complex urgent, medical-surgical, and mental health scenarios. This suggests observers need more practice in less complex scenarios to allow more CJ development and learner ability may influence practice needs. This is consistent with other research indicating starting simulation earlier in the nursing curriculum improves CJ outcomes (Hambach et al., 2023). Specifically, low performers need more time observing simulation and learners may need more tailored simulation observation experience based on learner ability. Nursing simulationists need to understand more about simulation observation dosing prior to using simulation for high stakes competency testing to ensure learners have adequate practice.

This study found when observers are placed in too complex environments, CJ may be easily overwhelmed. Observing expert nurses perform nursing care during hospital-based clinical learning experience rotations is very similar to observing EMS. However, there is a paucity of evidence for learning outcomes from clinical experiences (Leighton et al., 2021, 2022). This study could have implications for observing complex and urgent, medical-surgical, and mental health hospital-based clinical learning experience rotations. Nurse educators often ask charge nurses for input on appropriate patient selection who subsequently suggest patients with “interesting,” unique, or complex health problems. It is possible students need more repetition in routine hospital-based clinical learning experiences before caring for more complex patients. Similarly, when learners observe a complex urgent, medical-surgical, or mental health scenario, important details may be missed resulting in inadequate CJ. In both cases, nurse educators and nurse preceptors must ensure learners have adequate debriefing. Using guided reflective questions from this study could help evaluate learners’ CJ outcomes from hospital-based clinical learning experiences (Billings & Kowalski, 2019; Lasater & Nielsen, 2024; Lee & Wessol, 2022; Schooley et al., 2024; Walsh & Sethares, 2022; Williams & Hence, 2024). Future studies investigating the reliability for LCJR measuring CJ in clinical reflections could fill an important void in the nursing education literature for measuring clinical experience learning outcomes.

Implications for Nursing Education

This study has many important implications for simulation design, competency assessment, and learner simulation experiences. First, nurse educators should consider learner ability and complexity level when designing simulation observations. This study substantiates following the HSSOBP Outcome & Objectives (Miller et al., 2021) for scaffolding simulation objective complexity and helps nurse educators identify potential complexity sources. Next, increasing complexity beyond learner ability consistently negatively impacts learning outcomes (Smith et al., 2022; Tremblay et al., 2019, 2023), just as it did in this study. Nurse educators can use the complexity descriptions in this study to ensure junior level learners have more practice with “simple” and “moderate” complexity levels. Nurse educators should consider using these descriptions for building complexity more gradually to improve skill performance (Haji et al., 2016; Poledna et al., 2022). Additionally, this study found simulation design decisions significantly impact CJ learning outcomes, which is consistent with findings from Fogg et al. (2023). Therefore, as many nursing programs are moving towards using simulation to evaluate learners’ competency, nurse educators need to ensure simulation designs match learner ability. Furthermore, since CJ depends on scenario topics and situation urgency complexity level, nurse educators should evaluate learners using multiple different situations before determining CJ competency. Nurse educators should ask learners to respond to clinical judgment prompts after clinical and simulation for measuring clinical judgment and tracking progress over time (Billings & Kowalski, 2019; Lasater & Nielsen, 2024; Lee & Wessol, 2022; Schooley et al., 2024; Walsh & Sethares, 2022; Williams & Hence, 2024). Finally, this study identified novice nurses who could make significant CJ gains in more simple simulation observations. This suggests junior-level learners need more time in simple scenarios before increasing complexity to prevent overwhelming CJ gains. It is possible starting simulations at an earlier time in the nursing program could help learners build skills more efficiently (Hambach et al., 2023).

Strengths and Limitations

The strengths of this study include the within-subjects design because it allows participants to serve as their own control. This helps reduce individual differences in knowledge, ability, or motivation from impacting results. This secondary analysis is limited by reviewing previously collected data and therefore did not precisely control scenario design conditions. Furthermore, videos design components were not precisely controlled according to LCJR expertise criteria. Even with this limitation, we were able to demonstrate consistency between overall and learner ability levels therefore improving the findings’ trustability. Future studies could more precisely control simulation design according to LCJR expertise criteria to quantify how learners respond to simulation designs. Next, this study is limited by defining complexity post hoc according to expert opinion rather than precisely controlling complexity in the simulation design. Despite this limitation, increasing complexity significantly impacted learners’ CJ no matter how the results were analyzed. Investigating complexity using a priori controlled simulation designs would inform simulation design complexity best practice.

Conclusion

Simulation designs impact observers’ CJ outcomes. Namely, prematurely increasing complexity levels negatively impact observers’ CJ no matter the situation urgency, scenario topic, or learner ability. Nurse educators should use caution when increasing simulation complexity to prevent overwhelming observers’ CJ. Nurse educators need to ensure observers have adequate deliberate practice in simple, routine scenarios before building complexity with complex and urgent, mental health, and medical-surgical scenarios. This article is an important first step to understanding conditions impacting simulation observer learning outcomes. This article also informs the HSSOBP for scaffolding learner SBLE objective complexity. We need more evidence comparing CJ outcomes under different conditions using all learner levels and role assignments to inform simulation design best practice.

Footnotes

ORCID iD

Beth A. Rogers

Ethical Approval and Informed Consent Statements

The Texas Christian University Institutional Review Board determined this study was exempt (TCU#2023-429). Participants provided informed consent electronically and provided additional volunteer consent for data use for secondary analysis.

Author Contributions

Beth A. Rogers was responsible for conceptualization, data curation, formal analysis, methodology, funding acquisition, investigation, data curation, writing original draft., editing revisions, and visualization of this manuscript.

Data Availability Statement

Inquiries to access data and other research-related materials may be submitted for consideration to the corresponding author.

Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This work was supported in part by internal grants from the TCU Research and Creative Activities Fund and TCU Invests in Scholarship Fund. Publication charges for this article were supported by the TCU Library Open Access Fund.

References

Adamson

K. A.

Kardong-Edgren

(2012). A method and resources for assessing reliability of simulation evaluation instruments. Nursing Education Perspectives, 33(5), 334–339. https://doi.org/10.5480/1536-5026-33.5.334

American Association of Colleges of Nursing. (2021). The essentials: Core competencies for professional nursing education. https://www.aacnnursing.org/Portals/42/AcademicNursing/pdf/Essentials-2021.pdf

Baldwin

Mills

Birks

Budden

(2014). Role modeling in undergraduate nursing education: An integrative literature review. Nurse Education Today, 34(6), e18–e26. https://doi.org/10.1016/j.nedt.2013.12.007

Billings

D. M.

Kowalski

(2019). Teaching nurses to make clinical judgments that ensure patient safety. The Journal of Continuing Education in Nursing, 50(7), 300–302. https://doi.org/10.3928/00220124-20190612-04

Bradley

C. S.

Johnson

B. K.

Dreifuerst

K. T.

White

Conde

S. K.

Meakim

C. H.

Curry-Lourenco

Childress

R. M.

(2019). Regulation of simulation use in United States prelicensure nursing programs. Clinical Simulation In Nursing, 33, 17–25. https://doi.org/10.1016/j.ecns.2019.04.004

Cabrera-Mino

Shinnick

M. A.

Moye

(2019). Task-evoked pupillary responses in nursing simulation as an indicator of stress and cognitive load. Clinical Simulation in Nursing, 31, 21–27. https://doi.org/10.1016/j.ecns.2019.03.009

Cantrell

M. A.

Mariani

Lengetti

(2021). Internal reliability estimates of the Lasater clinical judgment rubric among new-to-practice nurses in a simulated setting. Nursing Education Perspectives, 42(6), E49–E51. https://doi.org/10.1097/01.NEP.0000000000000802

Chaboyer

Harbeck

Lee

Grealish

(2021). Missed nursing care: An overview of reviews. The Kaohsiung Journal of Medical Sciences, 37(2), 82–91. https://doi.org/10.1002/kjm2.12308

Choi

H.-H.

van Merriënboer

J. J. G.

Paas

(2014). Effects of the physical environment on cognitive load and learning: Towards a new model of cognitive load. Educational Psychology Review, 26(2), 225–244. https://doi.org/10.1007/s10648-014-9262-6

10.

Dickison

Haerling

K. A.

Lasater

(2019). Integrating the national council of state boards of nursing clinical judgment model into n nursing educational frameworks. Journal of Nursing Education, 58(2), 72–78. https://doi.org/10.3928/01484834-20190122-03

11.

Dodson

T. M.

(2023). Use of expert modeling videos in undergraduate nursing education: A systematic review. The Journal of Nursing Education, 62(2), 89–96. https://doi.org/10.3928/01484834-20221213-04

12.

El Hussein

M. T.

(2023). Experiences of nursing students in observer roles during simulation-based learning and the impact on patient safety: A scoping review. Clinical Simulation in Nursing, 78, 7–17. https://doi.org/10.1016/j.ecns.2023.02.003

13.

Eppich

Reedy

(2022). Advancing healthcare simulation research: Innovations in theory, methodology, and method. Advances in Simulation (London), 7(1), 1–23. https://doi.org/10.1186/s41077-022-00219-y

14.

Faul

Erdfelder

Lang

A.-G.

Buchner

(2007). GPower 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146

15.

Fogg

Yousef

M. G.

Thompson

Bauman

E. B.

Kardong-Edgren

(2023). Fostering clinical decision-making using screen-based simulations aligned with the NCSBN clinical judgment measurement model. Clinical Simulation in Nursing, 84, N.PAG-N.PAG. https://doi.org/10.1016/j.ecns.2023.101452

16.

Gwet

K. L.

(2008). Computing inter-rater reliability and its variance in the presence of high agreement. British Journal of Mathematical and Statistical Psychology, 61(1), 29–48. https://doi.org/10.1348/000711006X126600

17.

Haji

F. A.

Cheung

J. J. H.

Woods

Regehr

Ribaupierre

Dubrowski

(2016). Thrive or overload? The effect of task complexity on novices’ simulation-based learning. Medical Education, 50(9), 955–968. https://doi.org/10.1111/medu.13086

18.

Haji

F. A.

Rojas

Childs

Ribaupierre

Dubrowski

(2015). Measuring cognitive load: Performance, mental effort and simulation task complexity. Medical Education, 49(8), 815–827. https://doi.org/10.1111/medu.12773

19.

Hambach

Cantrell

M. A.

Mariani

(2023). A program of simulated learning experiences to develop clinical judgment and clinical competence among sophomore baccalaureate nursing students. Clinical Simulation in Nursing, 80, 55–63. https://doi.org/10.1016/j.ecns.2023.04.005

20.

Hayden

Smiley

R. A.

Alexander

Kardong-Edgren

Jeffries

P. R.

(2014). The NCSBN National Simulation Study: This award is made possible by the Sigma Theta Tau International/ A Longitudinal, Randomized, Controlled Study Chamberlain College of Nursing Center for Excellence in Nursing Education. Replacing Clinical Hours with. Journal of Regulation Nursing, 5(2). https://doi.org/10.1016/S2155-8256(15)30084-3

21.

Huber

Kleinknecht-Dolf

Kugler

Spirig

(2021). Patient-related complexity of nursing care in acute care hospitals– an updated concept. Scandinavian Journal of Caring Sciences, 35(1), 178–195. https://doi.org/10.1111/scs.12833

22.

Huun

(2018). Virtual simulations in online nursing education: Align with quality matters. Clinical Simulation in Nursing, 22, 26–31. https://doi.org/10.1016/j.ecns.2018.07.002

23.

INACSL Standards Committee. (2021). Healthcare simulation standards of best PracticeTM. Clinical Simulation in Nursing, 58, 66. https://doi.org/10.1016/j.ecns.2021.08.018

24.

Jeffries

P. R.

(2021). The NLN Jeffries Simulation Theory (2nd ed.). National League for Nursing.

25.

Jessee

M. A.

(2021). An update on clinical judgment in nursing and implications for education, practice, and regulation. Journal of Nursing Regulation, 12(3), 50–60. https://doi.org/10.1016/S2155-8256(21)00116-2

26.

Johnson

B. K.

Fey

(2023). Setting observers up for success in simulation. Nurse Educator, 48(6), 321–325. https://doi.org/10.1097/NNE.0000000000001433

27.

Kardong-Edgren

Wells-Beede

Wands

(2024). Student abuse in simulation: Causes and solutions. Teaching and Learning in Nursing, 19(2), 145–149. https://doi.org/10.1016/j.teln.2024.02.007

28.

Kavanagh

Sharpnack

(2021). Crisis in competency: A defining moment in nursing education. OJIN: The Online Journal of Issues in Nursing, 26(1). https://doi.org/10.3912/OJIN.VOL26NO01MAN02

29.

Klenke-Borgmann

Cantrell

M. A.

Mariani

(2020). Nurse educators’ guide to clinical judgment: A review of conceptualization, measurement, and development. Nursing Education Perspectives, 41(4), 215–221. https://doi.org/10.1097/01.NEP.0000000000000669

30.

Klenke-Borgmann

Lineberry

Broski

(2023). Integrative literature review on cognitive science to reconsider failure to rescue in nursing: A call to action. Journal of Continuing Education in Nursing, 54(6), 253–260. https://doi.org/10.3928/00220124-20230511-05

31.

Lasater

(2007). Clinical judgment development: Using simulation to create an assessment rubric. Journal of Nursing Education, 46(11), 496–503. https://doi.org/10.3928/01484834-20071101-04

32.

Lasater

(2011). Clinical judgment: The last frontier for evaluation. Nurse Education in Practice, 11(2), 86–92. https://doi.org/10.1016/j.nepr.2010.11.013

33.

Lasater

Holloway

Lapkin

Kelly

McGrath

Nielsen

Stoyles

Dieckmann

N. F.

Campbell

(2019). Do prelicensure nursing students’ backgrounds impact what they notice and interpret about patients? Nurse Education Today, 78, 37–43. https://doi.org/10.1016/j.nedt.2019.03.013

34.

Lasater

Nielsen

(2024). The lasater clinical judgment rubric: 17 years later. Journal of Nursing Education, 63(3), 149–155. https://doi.org/10.3928/01484834-20240108-05

35.

Lasater

Nielsen

A. E.

Stock

Ostrogorsky

T. L.

(2015). Evaluating the clinical judgment of newly hired staff nurses. Journal of Continuing Education in Nursing, 46(12), 563–571. https://doi.org/10.3928/00220124-20151112-09

36.

Lavoie

Deschênes

M.-F.

Maheu-Cadotte

M.-A.

Lapierre

Mailhot

Rodriguez

Desforges

(2022). Nursing students’ decision-making regarding postpartum hemorrhage: An exploration using the recognition-primed decision model. Nurse Education in Practice, 64, N.PAG-N.PAG. https://doi.org/10.1016/j.nepr.2022.103448

37.

Lee

J. Y.

Donkers

Jarodzka

Sellenraad

Faber

T. J. E.

van Merriënboer

J. J. G.

(2024). The effects of reflective pauses on performance in simulation training. Simulation in Healthcare, 19(2), 82–89. https://doi.org/10.1097/SIH.0000000000000729

38.

Lee

P. J.

(2015). Effects of the use of high-fidelity human simulation in nursing education: A meta-analysis. Journal of Nursing Education, 54(9), 501–507. https://doi.org/10.3928/01484834-20150814-04

39.

Lee

K. C. C. N. E.

Wessol

J. L.

(2022). Clinical reasoning, judgment, and safe medication administration practices in senior nursing students. Nurse Educator, 47(1), 51–55. https://doi.org/10.1097/NNE.0000000000001059

40.

Leighton

Kardong-Edgren

McNelis

A. M.

Foisy-Doll

Sullo

(2021). Traditional clinical outcomes in prelicensure nursing education: An empty systematic review. The Journal of Nursing Education, 60(3), 136–142. https://doi.org/10.3928/01484834-20210222-03

41.

Leighton

Kardong-Edgren

McNelis

Sullo

(2022). Learning outcomes attributed to prelicensure clinical education in nursing: A systematic review of qualitative research. Nurse Educator, 47(1), 26–30. https://doi.org/10.1097/NNE.0000000000001097

42.

Lesā

Daniel

Harland

(2021). Learning with simulation: The experience of nursing students. Clinical Simulation in Nursing, 56, 57–65. https://doi.org/10.1016/j.ecns.2021.02.009

43.

Lioce

(Ed), Lopreiato

(Founding Ed.), Anderson

Deutsch

E. S.

Downing

Robertson

J. M.

Diaz

D. A.

Spain

A. E.

(Assoc. Eds.); the Terminology and Concepts Working Group (2024). (2025). Healthcare Simulation Dictionary .

44.

McDermott

D. S.

Ludlow

Horsley

Meakim

(2021). Healthcare simulation standards of best practice prebriefing: Preparation and briefing. Clinical Simulation In Nursing, 58, 9–13. https://doi.org/10.1016/j.ecns.2021.08.008

45.

McMahon

Jimenez

F. A.

Lawrence

Victor

(2021). Healthcare simulation standards of best practice evaluation of learning and performance. Clinical Simulation In Nursing, 58, 54–56. https://doi.org/10.1016/j.ecns.2021.08.016

46.

Miller

Deckers

Jones

Wells-Beede

McGee

(2021). Healthcare simulation standards of best practice outcomes and objectives. Clinical Simulation In Nursing, 58, 40–44. https://doi.org/10.1016/j.ecns.2021.08.013

47.

Murray

Sundin

Cope

(2020). A mixed-methods study on patient safety insights of new graduate registered nurses. Journal of Nursing Care Quality, 35(3), 258–264. https://doi.org/10.1097/NCQ.0000000000000443

48.

Nielsen

Gonzalez

Jessee

M. A.

Monagle

J. L.

Dickison

Lasater

Huston

C. L.

Phillips

Jeffries

Todero

Rich

Knecht

Sommer

S. K.

Lewis

M. P.

Herron

E. K.

Stuedemann Fedko

Thomas Dreifuerst

Monagle

J. L.

Lasater

(2023). Current practices for teaching clinical judgment results from a national survey. Nurse Educator, 48(1), 7–12. https://doi.org/10.1097/NNE.0000000000001268

49.

O’Regan

Molloy

Watterson

Nestel

(2021). ‘It is a different type of learning’. A survey-based study on how simulation educators see and construct observer roles. In BMJ simulation & technology enhanced learning (Vol. 7, Issue 4, pp. 230–238). Bmj Publishing Group. https://doi.org/10.1136/bmjstel-2020-000634

50.

Poledna

Gómez-Morales

Hagler

(2022). Nursing students’ cue recognition in educational simulation. Nurse Educator, 47(5), 283–287. https://doi.org/10.1097/NNE.0000000000001198

51.

Rogers

Baker

K. A.

Franklin

A. E.

(2020). Learning outcomes of the observer role in nursing simulation: A scoping review. Clinical Simulation in Nursing, 49(49), 81–89. https://doi.org/10.1016/j.ecns.2020.06.003

52.

Rogers

B. A.

Franklin

A. E.

(2021). Cognitive load experienced by nurses in simulation-based learning experiences: An integrative review. Nurse Education Today, 99, 104815–104815. https://doi.org/10.1016/j.nedt.2021.104815

53.

Rogers

B. A.

Franklin

A. E.

(2022). Describing learners’ clinical judgment trajectory after observing expert modeling videos: A mixed methods study. Clinical Simulation in Nursing, 73, 37–47. https://doi.org/10.1016/j.ecns.2022.08.001

54.

Rogers

B. A.

Franklin

A. E.

(2023). Lasater clinical judgment rubric reliability for scoring clinical judgment after observing asynchronous simulation and feasibility/usability with learners. Nurse Education Today, 125, 105769. https://doi.org/10.1016/j.nedt.2023.105769

55.

Roskos

S. E.

Fitzpatrick

Arnetz

Shrotriya

Hengstebeck

(2021). Complex patients’ effect on family physicians: High cognitive load and negative emotional impact. Family Practice, 38(4), 454–459. https://doi.org/10.1093/fampra/cmaa137

56.

Schooley

Spoljoric

Covington

Garwood

Klosinski

Mordi

(2024). Reflective clinical judgment questions to educate the next generation of nurses. Journal of Nursing Education, 63(3), 182–185. https://doi.org/10.3928/01484834-20240108-01

57.

Shinnick

M. A.

(2022). Situational awareness differences between novice and expert nurses: Is there a correlation with clinical judgment? Clinical Simulation in Nursing, 62, 57–65. https://doi.org/10.1016/j.ecns.2021.10.002

58.

Shinnick

M. A.

Cabrera-Mino

(2021). Predictors of nursing clinical judgment in simulation. Nursing Education Perspectives, 42(2), 107–109. https://doi.org/10.1097/01.NEP.0000000000000604

59.

Shinnick

M. A.

Woo

(2020). Comparison of simulation assessments: Can they identify novice/student nurses? Clinical Simulation in Nursing, 46, 40–49. https://doi.org/10.1016/j.ecns.2020.04.002

60.

Smiley

R. A.

(2019). Survey of simulation use in prelicensure nursing programs: Changes and advancements, 2010- 2017. Journal of Nursing Regulation, 9(4), 48–61. https://doi.org/10.1016/S2155-8256(19)30016-X

61.

Smith

N. E.

Barbé

Randolph

(2022). Application of the cognitive load theory in prelicensure nursing education: A quantitative measurement focusing on instructional design. International Journal of Nursing Education Scholarship, 19(1), 1–12. https://doi.org/10.1515/ijnes-2021-0127

62.

Sullivan

Swoboda

S. M.

Breymier

Lucas

Sarasnick

Rutherford-Hemming

Budhathoki

Kardong-Edgren

(Suzie) (2019). Emerging evidence toward a 2:1 clinical to simulation ratio: A study comparing the traditional clinical and simulation settings. Clinical Simulation in Nursing, 30, 34–41. https://doi.org/10.1016/j.ecns.2019.03.003

63.

Sweller

Ayres

P. L.

Paul

Kalyuga

(2011). Cognitive load theory. Springer.

64.

Tanner

C. A.

(2006). Thinking like a nurse: A research-based model of clinical judgment in nursing. Journal of Nursing Education, 45(6), 204–211. https://doi.org/10.3928/01484834-20060601-04

65.

Treacy

Caroline Stayt

(2019). To identify the factors that influence the recognizing and responding to adult patient deterioration in acute hospitals. Journal of Advanced Nursing, 75(12), 3272–3285. https://doi.org/10.1111/jan.14138

66.

Tremblay

Leppink

Leclerc

Rethans

Dolmans

D. H. J. M.

(2019). Simulation-based education for novices: Complex learning tasks promote reflective practice. Medical Education, 53(4), 380–389. https://doi.org/10.1111/medu.13748

67.

Tremblay

Rethans

Dolmans

(2023). Task complexity and cognitive load in simulation-based education: A randomised trial. Medical Education, 57(2), 161–169. https://doi.org/10.1111/medu.14941

68.

Victor-Chmil

Larew

(2013). Psychometric properties of the Lasater clinical judgment rubric. International Journal of Nursing Education Scholarship, 10(1), 30. https://doi.org/10.1515/ijnes-2012-0030

69.

Walsh

J. A.

Sethares

K. A.

(2022). The use of guided reflection in simulation-based education with prelicensure nursing students: An integrative review. Journal of Nursing Education, 61(2), 73–79. https://doi.org/10.3928/01484834-20211213-01

70.

Williams

(2024). The effect of preventable harm simulation on entry-level nurses’ clinical judgment. Nurse Leader, 22(3), 279–284. https://doi.org/10.1016/j.mnl.2024.03.002

71.

Williams

M. S.

Hence

M. S.

(2024). Revising reflection assignments to align with clinical judgment measurement model language: Maximizing critical thinking, feedback, and measurement in simulation. Nursing Education Perspectives (Wolters Kluwer Health), 45(6), 375–376. https://doi.org/10.1097/01.NEP.0000000000001149

72.

Young

J. Q.

Dijk

S. M.

O’Sullivan

P. S.

Custers

E. J.

Irby

D. M.

Cate

(2016). Influence of learner knowledge and case complexity on handover accuracy and cognitive load: Results from a simulation study. Medical Education, 50(9), 969–978. https://doi.org/10.1111/medu.13107

73.

Zulkosky

Minchhoff

Dommel

Price

Handzlik

B. M.

(2021). Effect of repeating simulation scenarios on student knowledge, performance, satisfaction and self-confidence. Clinical Simulation in Nursing, 55, 27–36. https://doi.org/10.1016/j.ecns.2021.03.004

74.

Zulkosky

K. D.

White

K. A.

Price

A. L.

Pretz

J. E.

(2016). Effect of simulation role on clinical decision-making accuracy. Clinical Simulation in Nursing, 12(3), 98–106. https://doi.org/10.1016/j.ecns.2016.01.007