How to improve the Triage: A dashboard to assess the quality of nurses’ decision-making

Abstract

This paper presents a dashboard for assessing the quality of Triage decision-making process in the assignment of the priority code to patients arriving at an emergency department. The quality is assessed through performance indexes aimed at measuring both the nurses’ accuracy and the presence of nurses’ shared behavior in the Triage assignment process. The theoretical perspective of the cognitive heuristic, that is a cognitive shortcut strategy to make decisions, is the framework to design performance indexes of the dashboard. The dashboard has been tested in two Emergency Departments. We gather the data input through the submission to 54 nurses of a questionnaire, simulating 25 patients’ scenarios. The dashboard reveals a disparity between nurses in priority level assignment. This disparity depends on nurses’ capacity in disregarding some information and in considering the right cue. We find the presence of sharing behaviors and inclination to over or under assessment by nurses. This performance analysis, integrated with the nurses’ reasoning in assigning the priority code, can address the management in designing improvement actions at individual level and at the organizational one. In addition, the application to two Emergency Department confirms the influence of organizational aspects in the individual decision process.

Introduction

Triage is the front-office of emergency departments aimed at making a quick assessment of a patient’s urgency to receive a health treatment. Emergency departments’ mission is to guarantee appropriate care to the patient promptly, therefore, the quality of decision about urgency assigned during Triage affects the patient’s safety.^1,2 The patients can exhibit different illnesses, moving from trivial to life threatening, and Triage assigns them a priority level according to their urgency, “determining who will not be disadvantaged by longer waiting times and who requires immediate attention to achieve optimal outcomes” ¹ (page 86). Triage is performed by nurses that could under or over assess the clinical conditions of patients, increasing the waiting time of patients with life threatening conditions.^3,4 Nurses can over assess the priority code of a patient and, consequently, other patients could wait longer than necessary. On the contrary, nurses can under assess the priority code of a patient extending her/his waiting time.

Scholars have given great attention to make Triage more efficient in order to streamline the processing of patients and reduce the waiting time of more urgent patients. Some scholars focused on layout such as standardized pathways for the management of patients with specific disease conditions,^5–7 or fast track areas for low-acute patients.^8–10 Other scholars designed more accurate algorithms for prioritizing patients.^11,12 These studies focus on organizational aspects to reduce the waiting time of patients and on support systems to help nurses in assigning the correct priority level. On the contrary, an aspect not well investigated even if relevant resides on nurse’s reasoning about the patients’ health conditions, prompting her/him to recognize the more urgent patients. ¹³ Instead, understanding what influences nurses’ decisions can support the management in improving Triage effectiveness. ¹⁴ What information do nurses take into account to assign the priority level? Is the assignment of the priority code based on standardized guidelines or does it depend on nurses’ individual characteristics? How can we assess if the nurses assign the priority level based on guidelines or organizational rules rather than on their experience and knowledge?

Scholars remark a nurses’ disparity in priority code assignment due to diversity of factors concerning patients’ urgency above all in clinical cases with not enough information or characterized by information not matching the complexities of patients.^15,16 The disparity also depends on nurses’ interpretation process of patient’s clinical conditions based on few cues, short time, and influenced by experience. ¹⁷

Previous research focused on analyzing the different intensity of organizational and individual factors on nurses’ dynamic decision-making ¹⁸ and on the assessment of factors that affects the nurses’ decision. ¹⁹ This paper aims at designing performance indexes for assessing the quality of nurses’ decision and detecting the prevalence of shared behaviors among nurses in specific situations. The quality of decision is broken down into two elements: the discrimination capacity in perceiving the differences among patients, assigning them a different priority level, and the accuracy in assignment to the patient the correct priority level. The assessment of quality in nurses’ decision-making can be the starting point for managers to improve the Triage by leveraging on nurses’ decision-making.

The paper is organized as follows. Firstly, we introduce the relevant literature on decision-process according to cognitive heuristic concept. Based on the characteristics affecting the Triage decision-making, we develop two indexes for measuring the nurses’ decisions quality. After we design a dashboard based on nurse’s performance indexes. The application of this dashboard to two Emergency Departments (EDs) allows us to show how to combine these two performance indexes to give interesting insights on improvement actions. Discussion and conclusion end up the paper.

Triage as decision-making based on cognitive heuristics

Strategies used by nurses for decision-making rely on cognitive heuristics ²⁰ that are useful decision-making strategies in uncertain and complex environments, used by individuals when the information, time, and processing capacity are limited. ²¹ Ambiguous information, turbulent environments, and time pressure prompt a decision-maker to leverage on few cues to make a decision in a short amount of time.^22,23 Against this backdrop, the heuristic decision-making process is a dynamic and complex process in which the social-organizational context and its rules interact with individual’ s experience and beliefs in order to guide individuals in selecting the subset of the available information, the salient cues, to make a decision quickly. ²⁴ Individuals attribute a different weight to information and base their judgment only on cues considered relevant. ²⁵ Inability to distinguish correctly the relevant information can lead the decision-maker to misinterpret the salient cues and make a wrong decision. ²⁶ In some situations, the individuals can choice the salient information recalling such situation from memory ²⁷ or applying simple organizational rules easily to remember. ²⁸ In an organizational and management perspective, the cognitive heuristics become a way to communicate the rules in decision-making, ²⁹ a coordination mechanism for guiding decision processes and communicate strategies to different stakeholders and organizational groups ²⁸ or, in healthcare, to coordinate nurses and doctors in treating a patient. ³⁰ The social-organizational context, made by formal and informal rules, these last ones socially constructed through sharing practices like imitation, support systems, institutional commitments can provide some directions for the decision. ³¹

In the context of Emergency Departments, for example, there could be an informal but strong rule shared by nurses driving them to do not use priority codes related to low urgency, thus implying an under-assessment of the disease under evaluation. This happens mainly in public structures of National Health Systems where low urgency treatments are often associated to the payment of a fee on behalf of patients. Additionally, the literature ³² outlines that organizational specific guidelines or protocols on Triage, representing the way in which health organizations spread formally their rules, can guide and support fast decisions of nurses, especially the beginners.

Despite the importance of social-organizational context, individuals can leverage also on their discretionary judgment in the final decision to be made. ³³

Therefore, the fact that decision-making is affected by the interplay between individual characteristics and social context^34,35 implies that the wrong decision can depend on organizational rules or on individual discretionary judgment. A discretionary judgment occurs when the individual makes the decision basing on his own reasoning.

Improving the quality of decision-making requires understanding what factors, context, or individual characteristics, prevail on the individual heuristics guiding decision-making.^36,37 The analysis of wrong decisions reveals interesting insights on the prevalence of context or individual characteristics. According to the organizational perspective, we can suppose that the presence of a strong coordination mediated by the social context should lead individuals in a same organization to apply the same heuristic, identifying the same salient cues and, aside from the error, to make the same decision. On the contrary, a disparity of individuals’ decision in an organization could indicate the prevalence of a discretionary judgment.

Existing research on factors affecting nurses’ decision-making in Triage highlights contextual factors and individual nurse’s experience, knowledge, and intuition can cause the variation of nurses’ decision-making in assessing patient urgency^36–38 and the weight of organizational and individual factors can vary according to task complexity. ¹⁸ In addition, given the characteristics of Triage process, the quality of decisions depends also by the capacity of nurses to assign the same priority code to patients with similar urgency. Aside from the wrong priority code, the nurses can exhibit the capacity to discriminate patients according to the different urgency or, on the contrary, put together patients even if they exhibit different clinical conditions. As the access to the treatment areas depends on priority codes, and within the priority code queue by the arrival time, an ineffective capacity to discriminate among patients can affect the quality of Triage.

Based on a set of 25 patient scenarios developed in previous research, ¹⁸ we have interviewed Triage nurses in two Emergency Departments in order to describe how the analysis of cognitive errors in Triage assessment can reveal useful insights on nurses’ behavior and on performances of Triage. The 25 patient scenarios represent 25 real situations of emergency designed in collaboration with a Triage trainer in a previous step of the research. We designed two performance indexes measuring not only the accuracy of nurses’ decision, based on the capacity of the nurse to identify the right cues, but also the discrimination capacity, based on the behavior of nurses in capturing the differences among patients in terms of urgency.

The main contribution of the work resides in proposing a dashboard able to integrate these two performance indexes, in order to address the management toward training actions or to shed light on the impact of organizational aspects on decisions.

Nurses’ Performance Indexes

The dashboard, composed by visualization tools based on performance indexes, makes an assessment of nurses’ decision-making in the Triage process, inferring the prevalence of organizational context or of nurses’ discretionary judgment on decisions.

The performance indexes measure the accuracy of the nurses in the priority code assignment and their discrimination capacity in capturing differences among patients. These indexes are the input of a visualization tool presented in the case studies.

For the computation of the proposed indexes, we compared the priority code assigned by a nurse with the correct code.

Let us consider m patient scenarios and k nurses. A patient scenario is a real Triage situation. Consider a matrix X of k rows and m columns so that the element ${\text{x}}_{ij}$ is the priority code assigned by the i-th nurse (for i = 1, …, k) to the j-th patient (for j = 1, …, m). We introduced a m-dimensional vector y in which the general term $y_i$ is the correct priority code to be assigned to the specific patient i. Then, we consider the priority codes assigned to the patients as N ordinal response classes (for N = 1, …, n).

Accuracy performance index

Concerning the accuracy of a nurse’ decision (nurse i), we compute the Accuracy Index (A) as

$$A_i=\frac{1}{k}{\displaystyle \sum _{j=1}^k{P}_{ij}}$$

(1)

with $0\le A_i\le 1$

$P_{ij}$ is a variable index which is equal to 1 if $x_{ij}=y_j$ and 0 otherwise. The total accuracy of Triage assessment in an emergency department is given by the average of overall nurses accuracies and it is

$$A=\frac{1}{k}{\displaystyle \sum _{i=1}^k{A}_i}$$

(2)

with $0\le A\le 1$ .

An Accuracy (A) equal to 0 means that all the nurses in the Emergency Department assign a wrong code to each analyzed case. When A increases, this means that nurses tend, on average, to be more precise in the identification of the correct priority code. An Accuracy (A) equal to 1 means that all nurses assign the right priority codes to analyzed cases.

Discrimination capacity index

Concerning the discrimination capacity index, it takes over the capacity of nurses to identify the salient right cue that discriminates the cases, putting the patients in different N ordinal response classes (for N = 1, …, n), that nurses assign to a patient scenario according to the urgency to be admitted to emergency department’s treatment area. These classes correspond to the priority codes that the nurses assign to patients. The discrimination index measures how well the nurses separate the cases, if they are different, giving a distribution of cases in the n classes (in our case classes correspond to diverse priority codes) coherently with the correct distribution. The discrimination index is a clustering method, usually applied to compare two partitions in order to resolve rank aggregation problems^39,40 and it is a suitable method for comparing the nurse’ s partition of cases into the n classes with the correct one. Let us consider the row vector $x_i^{\prime }$ (containing for each patient scenario the attribution of the code by the nurse i) as a partition of the m cases into N classes, and the row vector $y^{\prime }$ as the correct partition. The clustering method aims at measuring the concordance of each nurse partition with the correct one, introducing two coding matrixes (m × N), namely, $D_{x_i}$ and $D_y$ , respectively, for partition of the nurses and for the correct partition. Both coding matrixes are dummy as their elements are equal to 1 or 0 (1 if the specific patient scenario has been assigned the specific code of n codes available, 0 otherwise). Considering these matrices, we can define the following equivalence relation for the partition $y^{\prime }$

$$E_y=1-\Vert d_y-d_y^{\prime }\Vert$$

(3)

where $\Vert \text{Qui si è persa la formattazione, }\text{non so se è }\text{un probema del mio word}\Vert$ is the normalized Manhattan distance.

In an analogous way, we defined the nurse’ s equivalence relation for the partition $x_i^{\prime }$ as

$$E_{x_i}=1-\Vert d_x-d_{x_i}^{\prime }\Vert$$

(4)

The Equivalence relation measures the similarity of each couple of patient scenarios in terms of partitioning and gives as result two $\text{m }\times \text{m}$ symmetric matrixes ( $E_y$ , $E_{x_i}$ respectively, for the correct partition and the i-th nurse’s partition). The elements of these matrixes are equal to one if two patient scenarios (row and column of equivalence relation matrix) have the same code, zero otherwise (except the for the main diagonal).

If nurses are able to discriminate the difference of patient scenarios in terms of priority codes, they separate the patient scenarios in the correct partition. This means that nurses distribute correctly the cases into distinct priority codes. Therefore, we computed the Discrimination Capacity Index (D) for the i-th nurse as following

$$D_i=1-\Vert E_y-E_{x_i}\Vert$$

(5)

with $0\le D_i\le 1$ .

D_i is equal to 1 if the i-th nurse provides the correct partition, otherwise it is equal to zero if he/she gives a partition completely wrong.

The total Discrimination capacity of Triage in an emergency department is given by

$$D=1-\Vert E_y-E_x\Vert$$

(6)

with $0\le D\le 1$ .

where the matrix $E_x$ is a measure of similarity by averaging over all discrimination capacities of the n nurses as follows

$$E_x=\frac{1}{n}{\displaystyle \sum _{i=1}^n{E}_{x_i}}$$

(7)

The total Discrimination capacity (D) is equal to 0 when all nurses aggregate in a same class (representing a priority code) cases that refer to different priority levels. When D increases, this means that nurses tend, on average, to aggregate in the same class cases that belong to the same priority level. D is equal to 1 when all nurses aggregate in the same class cases that belong to the same priority level.

Global performance index

We consider the Accuracy and the Discrimination Indexes as two components of a global performance index as follows

$$G{P}_i=h{D}_i+\left(1-h\right)A_i$$

(8)

where the weighting parameter $h$ can be suitably selected.

The $h$ is a tuning parameter that allows to fine tune the weights of the two performance indexes on the global performance. The h parameter can be selected by those using the metrics in order to reflect organizational characteristics and specific information needs.

Finally, we defined an emergency department’s Global Performance Index (GP) as follows

$$GP=\frac{1}{k}{\displaystyle \sum _{i=1}^{k}G{P}_i}$$

(9)

GP ranges from 0 to 1, where a value equal to 0 refers to a limit situation in which all the nurses of the Emergency Department assign all wrong codes and consider in the same aggregation cases belonging to different classes; when GP is equal to 1, instead, the opposite is right and all the nurses assign correct priority codes and group cases that belong to same class or priority level.

The application of the dashboard to emergency departments

The field of study concerns the application of a dashboard to two emergency departments, for privacy concern, namely, Alpha and Beta, located in the South of Italy with organizational differences.

The two Emergency Departments perform the Triage process differently: in Alpha, the priority code is immediately assigned based on qualitative cues and visual signs, whereas in Beta the nurse assigns a temporary priority code based on qualitative cues, and after the measurement of the vital signs the definite priority code is attributed. Beta is a specialized emergency department on cardiovascular diseases and serves a larger population (about 90,000 patients for year), whereas Alpha is not specialized and presents a lower number of patients (about 53,000 patients for year). In Alpha, the management pays a great relevance to training. Both hospitals are public.

The analysis has been performed considering only the nurses regularly involved in the Triage process: 18 in Alpha and 35 in Beta. The average working years in the same hospital is 18 years for Alpha nurses and 20 years for Beta.

In the case of Beta, we consider the definite priority code assigned by nurses in the second step of the Triage assessment in order to make more feasible the homogenous application of the dashboard to the selected cases. In addition, it is important to clarify that the aim of the research is to show that dashboard can be applied in different situations (as, for example, in cases Alfa and Beta that are different with respect to the implemented Triage methodology) and could give useful advice to managers for organizational improvements.

The application of the dashboard has been experimented on 25 patient scenarios that simulate the arrival of patients at the ED. The patient scenarios have been designed by the research team in collaboration with a Triage trainer and concern real Triage situations. Cases are differentiated in terms of urgency (assessed preliminarily by the trainer, considered an expert), in terms of completeness of information, and coherence among information.

The patient scenarios describe the patient’ s conditions trough the information usually recorded during the Triage process.^36,41–43 This information concerns quantitative aspects (vital signs eventually taken by a nurse), qualitative ones (pain, symptoms showed by a patient), and clinical history (clinical documents or other diseases told by a patient).

For each patient scenario, the Triage trainer justified the correct priority code with respect to the salient cues. The patient scenarios report vital signs, complaint, symptoms, pain showed by a patient and, eventually medical history cues. These cues can be incomplete, consistent with each other, or inconsistent in order to capture the capacity of nurses in selecting relevant cues and, consequently, discriminate about patient scenarios giving them the right priority codes.

The research team carried the interviews, after a period of observation of the Triage process, presenting the patient scenarios to each nurse separately and giving 2 minutes for the priority code assignment. An efficient Triage should take not more than 2 minutes. After the assignment of the priority code, nurses explained their choices and identified the chosen salient cues and the reason for the assignment of that priority code. This qualitative information was useful to better understand cognitive errors and to identify more effectively eventual and appropriate improvement actions.

Findings

Nurse’s Performance Indexes combine the accuracy of the decision with the discrimination capacity, that is partitioning correctly patient scenarios (Figure 1).

Figure 1.

Global, Discrimination, and Accuracy Indexes of nurses in the two EDs.

We can observe that in both EDs, the Discrimination Capacity is usually greater than the Accuracy, and this implies that the nurses tend to well aggregate patient scenarios but assign them wrong priority codes.

In the dashboard, the values of Discrimination Capacity and Accuracy have been analyzed through a portfolio matrix, namely, Nurses’ Performance Matrix, in order to detect different strategies supporting nurses’ training (Figure 2).

Figure 2.

Nurses’ performance matrixes and improvement strategy for Alpha and Beta.

Figure 2 reports the values of performance indexes with respect to two dimensions: the horizontal axis is related to the D index, and the vertical axis to the A index. The crossing point of the axes indicates the threshold values for these indices, that we settled equal to the mean values of D and A for nurses of the same ED. We divided nurses in the four quadrants of the matrix. The meaning of the position of a nurse in each quadrant is reported below.

1. Great: proportion of nurses that assign cases into the correct partition, obtaining also an accuracy above the threshold.

2. Questionable: proportion of nurses that assign cases into the correct partition, making more mistakes in the definition of the priority code than the average selected as threshold.

3. Guess: proportion of nurses that doesn’t assign cases into the correct partition, obtaining an accuracy above the threshold.

4. Critical: proportion of nurses that doesn’t assign cases into the correct partition, making more errors than that established threshold.

Most of nurses in the two EDs fall into the Great and Critical quadrants, therefore, a training action should be necessary as the threshold is not very high. We would have expected no cases in the quadrants Guess and Questionable; these expectations were partially confirmed: very few nurses are included in the quadrants above. In fact, in the first case (Guess), we find nurses that exhibit an accuracy close to the threshold and a discrimination capacity lower than the threshold; in the second case, we find nurses with low accuracy and a discrimination capacity slightly major than the threshold. This situation represents a warning for managers and those interesting in improving the quality of the Triage process and, as a consequence, the specific reasoning of nurses could be analyzed in depth before to identify training and organizational interventions.

The last analysis is based on the proportion of patients’ cases into the priority codes. In Italy, the Triage System adopts as priority codes the following four colors: white (absolutely not urgent), green (intermediate value but with a low urgency), yellow (intermediate value but with a high urgency), and red (very urgent).

The proportion is the ratio between the sum of a given priority code with respect to the total patient scenarios. The proportion of the expert is the correct proportion and it is equal to: 20% of whitecodes 36% of green codes, 32% of yellow codes, and 12% of red codes.

In Figures 3 and 4, we represent the proportion of the four priority codes given by each nurse in Alpha and Beta, respectively. The red line is established according to the correct proportion. If the nurse’s proportion is below the red line, this means that the nurse has assigned less priority codes in that color than the expert. Otherwise, the nurse has assigned more priority codes than the expert in the color examined. The absence of the bar implies that the nurse has assigned the correct proportion of such priority code.

Figure 3.

Proportion of priority code assigned by each nurse in Alpha ED.

Figure 4.

Proportion of priority code assigned by each nurse in Beta ED.

All nurses in both EDs tend to over assess the white codes (the bars of most nurses are below of the red line) and to assign more green codes. For these two priority codes, we notice a shared behavior among nurses. Instead, for yellow and red priority codes, assigned to patients with critical conditions, we can notice a different behavior as if the nurse’s discretion prevailed over the organizational context.

Discussion

A growing attention is given to the quality in the healthcare attention and several are the perspective of analysis.^44–47

Few studies investigate the Triage decision-making quality although the relevance of the issue. Moon et al. ⁴⁸ evaluated the accuracy of nurses’ in Triage assessment with the help of an expert in order to determine the causes of mistakes. Tam et al. ⁴⁹ analyzed the accuracy of Triage and they prompted emergency department’s managers to activate training actions. These two works take into account only the accuracy, analyzing retrospectively some cases. Instead, Göransson et al. ¹³ assessed the accuracy and also the concordance among nurses in codifying the patient’s urgency, using patient scenarios. Unlike Göransson et al., ¹³ we don’t assess the concordance among the nurses, but their capacity to discriminate correctly among patient scenarios, comparing the nurses’ classification to the expert one. The capability of discrimination is the ability of nurses to put correctly together the patients. Comparing the accuracy and the discrimination indexes, the Performance Matrix Index is a useful visualization tool for guiding the selection of improvement strategies. Of course, for the design of improvement strategies, the interviews on nurse’s reasoning give relevant insights on the chosen salient cue and the cognitive heuristic framework applied in the specific context.

The Performance Matrix Index can support the management in identifying adequate organizational improvements, as it gives an assessment score for each nurse. For example, intensive training could be addressed to nurses located in the “critical” quadrant; additionally, the management could create mixed groups of nurses located in the four quadrants, in order to facilitate the sharing of experiences and learning mechanisms among the professionals. According to Lancharoen et al., ⁵⁰ the group is an intermediate level between organization and individual: organizational level can give information, the group shares this information with its members to understand roles and practices and comply with organizational policies.

Another interesting factor to be explored concern the difference in contexts of Alpha and Beta EDs. From the case studies, we can infer that Alpha obtains better performances both in Accuracy and Discrimination Capacity indexes. We statistically evaluated these differences with a two sample exact permutation test ⁵¹ on 10.000 permutations. The p-value for the three indexes is less than 0.1 (0.085 for D index, 0.016 for A index, and 0.027 for GP index), and the differences are significative. A great focus on nurses’ training and the smaller population served could be factors that affect the quality of Triage decisions. This is a conclusion that could be further analyzed in future studies.

Furthermore, the dashboard also captures shared behaviors and over or under assessment tendencies. This analysis reinforces what is claimed in the literature on cognitive heuristics: the organizational context plays a relevant role on decision-makers and, in specific situations, it prevails on discretionary judgments. ⁵² In the case studies here presented, this occurs in the assignment of white and green priority codes (see Figures 3 and 4). The interviews to nurses confirmed that an applied informal rule is to assign a green code rather than a white one in order to avoid any discussions with patients. This is because in Italy, white codes are associated with the payment of a fee. We also noticed that main difficulties occurred in the assignment of yellow and red priority codes. Also in this case, the interviews were critical in understanding the reasons behind the choices of nurses. The analysis of salient cues considered by nurses pointed out that nurses tend to weight more vital signs than qualitative symptoms, and this led to an incorrect interpretation of the patient’s urgency. The patient scenarios have been designed in order to capture this tendency, putting in the same case inconsistent qualitative and quantitative information. This finding suggests that a possible improvement could consist in clarifying the relevance of qualitative information in order to make the nurses able to better discriminate the yellow and red priority codes.

Conclusions

Triage plays a relevant role in managing patients suffering of life-threating diseases and, according to cognitive heuristics, the Triage decision-making cannot be simply automatized as the nurses’ discretionary judgment can be relevant; mistakes depend both on organizational context and on nurses’ individual characteristics, such as the experience.

Existing research highlights a complexity of decisional processes in clinical settings, guided by cognitive heuristics to make a decision in uncertain and time constrained situations. These heuristics can be affected by individual and organizational rules.^29,53,54 Nevertheless, methods and applications focused on how to measure the weight of individual and organizational factors on decision-making processes are lacking. ¹⁸ The contribution of this paper is to propose a dashboard based on performance indexes, able to give evidence of the influence of organizational rules on individual decision-making. In addition, the relevance of this dashboard also resides in its capacity to assess the quality of a specific healthcare process, and the quality assessment is becoming a pivotal activity in the governance of a hospital inasmuch it allows to improve efficiency and effectiveness of operations of healthcare organizations. ⁵⁵

The proposed dashboard can support the management in designing appropriate training actions or organizational improvements for increasing the quality of the Triage process.

The performance indexes can be used not only to better identify training actions devoted to nurses but also to introduce organizational rules or create teams aimed at diffusing the right interpretation of patients’ symptoms among nurses.

Even if the patient scenarios are typical of the Triage process, the performance indexes can be applied to other decision-making processes based on cognitive heuristics supporting in the understanding the role played by organizational rules on individual decision-making.

The dashboard could be also used as a tool to assess the nurses’ learning after the improvement actions or to measure the effectiveness of a new organizational rule. This dashboard can be relevant in all contexts characterized by decisions affected by organizational rules (as in the case of low level of white codes’ assignment in the two case studies). On the contrary, when the decision is based mainly on individual factors (i.e., the different behaviors in assignment of yellow and red priority levels), the management could act specific training actions such as job rotation or supervision.

Further developments of this research can be addressed in the following areas. According to management studies on heuristics, a larger sample of Emergency Departments could be useful to test if the specific context can be an effective vehicle to implement simple heuristics. With a larger sample it could be possible to verify if a strong focus on training activities, a smaller number of accesses and a less variable demand (specialization degree) are enablers for an effective use of heuristics as guiding behaviors. Concerning the dashboard, further analytics can be designed to better evaluate nurses’ behavior above all regarding over and under assessment.