Abstract
Theory:
Individuals have different qualities, levels of willingness, and degrees of engagement for working in teams. This behaviour is termed ‘Collective Orientation’ (CO). Collective orientation can be trained and has a positive influence on team processes. Here, we investigated the effect of a simulator-based, Crisis Resource Management team training upon the participants’ CO.
Hypotheses:
We hypothesized (1) the scales of CO and Presence for lab-based microworld research (PLBMR) are applicable to the German anaesthesia teams, (2) the CO can be influenced by means of simulation training, (3) the training effect is dependent on sex and/or profession, and (4) the change of CO depends on the perceived presence of the participants in the scenario.
Method:
In a pre-post study, 66 nurses and doctors from various anaesthetic departments took part in a 1-day training course to improve non-technical skills. The primary outcome was the mean difference between the CO measured (via questionnaires) immediately before (T1) and after (T2) training. The change was then tested for dependence upon other variables, such as sex, professional group, and immersion into the simulation scenarios.
Results:
Collective orientation improved significantly after training (mean difference: 0.2; P < .001; dz = 0.53). Considering the subscales, affiliation increased significantly (P < .001; dz = 0.59), whereas dominance remained unchanged. Furthermore, no correlation was found regarding sex, professional group, or immersion into the simulation scenarios.
Conclusions:
Our study demonstrated that simulation-based training improves the participants’ COs, primarily by increasing affiliation. Subjective scenario reality did not significantly influence this. Nonetheless, it remains unclear as to what factors categorically resulted in this benefit. The shared experience in the course by all team members might trigger the effects. However, further studies are needed to identify the modifiable factors that can improve teamwork attitudes.
Introduction
Analysis of critical incidents in industries that require staff to work as part of high responsibility teams (HRT) 1 resulted in the realization that adverse situations are often caused by ‘human errors’. The principles can also be applied to a health care environment, although there are some differences, eg, the working culture. 2 As HRT have to focus not only on technological issues under stressful situations but also teamwork processes – they also require proficiency in various non-technical skills (NTS). 3 The NHS (National Health Service) Education for Scotland developed the ‘Anaesthetists’ non-Technical Skills-Rating Scale’ with the dimensions of task management, decision-making, situational awareness, and team working to deal with a critical situation. 3 Fletcher et al defined teamwork as skills for working in a group context, in any role to ensure effective joint task completion and team member satisfaction. The European Society of Anaesthesiology (ESA) recommends safety trainings for anaesthesia teams to improve non-technical skills. 4
An early finding regarding teamwork is that every individual brings different qualities, levels of willingness, and degrees of engagement for working in a team. 5 The individual behaviour of team members has been characterized by the term ‘Collective Orientation’ (CO). Diskrell et al 6 defines CO as the propensity to work in a collective manner in team settings. During the development of a measuring scale for CO (in the form of a questionnaire), 2 main factors became apparent. The first main factor was termed ‘Affiliation’ and consisted of 10 items. A high ‘Affiliation’ describes the ability to work both in a goal-oriented manner and with a higher regard for others in a team. Low ‘Affiliation’ is characterized by a preference to work on one’s own. The second factor was labelled ‘Dominance’, which was composed of 5 items. A high degree of ‘Dominance’ demonstrates a priority in having power and control. Low ‘Dominance’ reflects a cooperative working style. All items for both factors are listed in Table 1. The adapted and validated German version was developed by Hagemann. 7 As CO is an attitude which effectively supports teamwork processes,6,8 it can be positively changed by training of health care providers, if the participants believe in it is important to their work. 9
Items of collective orientation. 7
Items marked with (R) are negatively worded and have to be reversed-scored.
Simulation-based training methods enable participants to learn in a virtual work environment, which helps foster their NTS. Despite technological advancements, a degree of artificiality in the simulation’s representation cannot be prevented. Factors that negatively affect the presence can also have a negative impact on the performance of teams in a virtual environment, 10 so it is interesting to consider training effects related to perceived reality. To measure presence, Frank and Kluge 11 devised a scale known as ‘Presence Scale for Lab-based Microworld Research’ (PLBMR) and applied it already in the medical context. 12
The aim of this study was to assess the impact of simulation training for anaesthetic teams upon each participant’s CO in relation to the presence. The hypotheses were (1) the scales of CO and PLBMR are applicable to the German anaesthesia teams, (2) the CO can be influenced by means of simulation training, (3) the training effect is correlated sex and/or profession, and (4) the change of CO correlated with the perceived presence of the participants in the scenario.
Materials and Methods
Study design and population
Our study was a pre/poststudy using survey methodology with participants from various anaesthetic departments in our HAINS (Hannoeverscher Anaesthesie-Intensiv-Notfallsimulator)-Safety simulation programme. The first measurement (T1) was taken immediately prior to the training, and the second (T2) was completed immediately after the course. The items for assessing the presence were also completed after the course. The study was approved by the ethics committee of the Hannover Medical School (no. 2958-2015).
Setting and population
The participants of each course (8 hours) were represented near equally by nurses and doctors, with a maximum combined total of 12 people per date. All were required to be active in the field of anaesthesiology at the time of the study. Participation was voluntary and could be withdrawn at any time without giving reasons.
In terms of content, the topics were designed to increase competencies in teamwork, such as situational awareness, decision-making, and communication. As learning objectives for the competencies on NTS, the participants learned the CRM key principles of Rall et al 13 (Table 2).
Fifteen Crisis Resource Management (CRM) key principles published by Rall et al 13 considered as a template for the debriefing of the scenarios.
Demographic data
Demographic data were gathered by the questionnaire at T1. The professional position (eg, assistant medical director, attending physician, resident physician, specialist nurse, nurse), the level of care (eg, outpatient care, standard-level care, advanced-level care, and maximum-level care [eg, university medical centre]), years of work experience, and sex was inquired.
‘CO’ questionnaire
Collective orientation was measured using the validated German version of a paper-based questionnaire. 7 Items could be answered on a 5-point Likert-type scale from 1 (trifft nicht zu/I totally disagree) to 5 (trifft vollständig zu/I totally agree). The internal consistency of the overall scale (α = .79) as well as the subscales ‘affiliation’ (α = .82) and ‘dominance’ (α = .70) proved to be satisfactory. The mean of all items was used as the marker for the CO.
‘Presence’ questionnaire
Frank and Kluge 11 devised the scale ‘Presence Scale for Lab-based Microworld Research’ (PLBMR) (Table 3). All 6 items of the questionnaire were rated with a 6-point Likert-type scale from 1 (‘totally disagree’) to 6 (‘totally agree’). The internal consistency of the scale proved to be α = .71. The mean of all items was used as the marker for the presence during the simulation scenarios.
Adapted items of the Presence Scale for Lab-based Microworld Research (PLBMR) for German and English trainees. 11
Protocol
The interventions were a single 1-day training course (8 hours), composed of both lecture material and multiple scenarios. Each course started with a lecture with the basics of Crisis Resource Management and non-technical skills, followed by an introduction to the simulation environment. Subsequently, the subjects participated in a total of 6 different medical situations. These critical situations included ‘difficult intubation’, ‘anaphylaxis’, and ‘severe bleeding’. Each scenario required approximately 15 minutes to complete. In case of technical problems with the material, a simulated nurse could help. When not actively participating in a scenario, they would observe the other scenarios via a 1-way mirrored pane. The SimMan3G (Laerdal Medical, Norway) was used as a patient simulator during the scenarios. A 20-minute debriefing took place after each scenario, during which time the participants discussed their performances with the facilitator and the learning group. All debriefings were conducted by a small team of anaesthesiologist, using the debriefing approach described by Rudolph et al. 14
Statistical analysis
Demographic data and survey data were analysed in a descriptive manner. The normal distribution of the values was revised using the Kolmogorov-Smirnov test. For testing hypothesis (a), the reliability of the scales determined by Cronbach alpha was calculated. To test hypothesis (b), a t test for dependent samples was conducted. To test hypothesis (c), whether training effect correlates with professional group or sex, a univariate analysis of variance (ANOVA) with repeated measures was conducted. To test hypothesis (d), if a change in CO correlates with the perceived presence, a Pearson correlation was conducted. We assumed a P < .05 as being statistically significant. As effect size, Cohen d was calculated. All calculations were made using SPSS Statistics 24 (IBM Corporation, Foster City, CA, USA).
Results
Although all 66 participants filled out the questionnaires, not all information was completed – 3 participants provided no information regarding their profession, 10 gave no information regarding their work environment, 4 provided no information about their work experience, 6 had incomplete ‘Presence’ items, and 12 incomplete CO items (9 incomplete ‘Affiliation’ subscales, 3 incomplete ‘Dominance’ subscales). Incomplete questionnaires regarding subgroup ‘Affiliation’ or ‘Dominance’ were not included in the analysis of the respective item. A total of 60 sheets of the survey ‘Presence’ were complete and could be evaluated. In total, 53% of the participants were female, 47% were male. The professional position was 12 nurses, 14 specialist nurses, 5 assistant medical directors, 13 attending physicians, 19 resident physicians. The level of care ranged from outpatient care (1.5%), standard-level care (15.2%), advanced-level care (21.2%), and maximum-level care (31.0%). The work experience was given as an average of 11.65 years (1-36 years, SD: 9.9 years). Subgroups of professional groups and level of care were not statistically evaluated as the standard deviation was too big to process the data.
Reliability of the scales
To test hypothesis (a), Cronbach alpha was calculated. The result for the questionnaire CO was .74 (T1) and .84 (T2). Subdivided by subscale and time of measure, Cronbach alpha was .65 (T1) and .75 (T2), respectively, .65 (T1) for the subscale ‘affiliation’, and .81 (T2) for the subscale ‘dominance’. No item reduction led to a substantial increase in Cronbach alpha.
For the questionnaire PLMBR, Cronbach alpha was .47. The elimination of 1 item increases the value to 0.54. Therefore, we did not include this item in any further calculations.
Change in ‘CO’
To test hypothesis (b), the change in CO through a simulation training was evaluated via CO scores from the questionnaires before (T1) and after (T2) training (Figure 1). The normal distribution was given for all data. Collective orientation was M = 3.58 (SD = 0.43; n = 66) prior to training and M = 3.78 (SD = 0.47; n = 66) after the training. For the subscale ‘Affiliation’, the respective results were M = 3.63 (SD = 0.44; n = 66) and M = 3.88 (SD = 0.48; n = 66). For the subscale ‘Dominance’, we, respectively, found M = 3.51 (SD = 0.61; n = 66) and M = 3.62 (SD = 0.67; n = 66). In comparison with T1, the CO at T2 increased significantly, t(65) = −4.27, P < .001, dz = 0.53. Considering the subscales, the value for the dimension affiliation also increased significantly after training, t(65) = −4.76, P < .001, dz = 0.59. The change from T1 to T2 in the dimension dominance was not significant, t(65) = −1.68, P > .05, dz = 0.21.
Changes in ‘collective orientation’ (CO) with its subscales ‘Affiliation’ and ‘Dominance’ prior to (T1) and after (T2) training. * indicates the significant difference for CO (P < .001; dz = 0.53) and ‘Affiliation’ (P < .001; dz = 0.59). ‘Dominance’ showed no significant change (P > .10).
Impact of profession and sex on ‘CO’
To test hypothesis (c), whether the potential training effect on CO is correlated with profession or sex, univariate ANOVAs with repeated measures were conducted. There were no significant results for the interaction between the change in CO and ‘professional group’ or ‘sex’. However, the regression coefficients show a difference for CO between the professional groups at measurement point T2 (β = 0.237; P < .05) as well as for affiliation (β = 0.247; P < .05). The increase in CO and affiliation was stronger for nurses than for physicians.
Impact of ‘presence’ on ‘CO’
To test hypothesis (d), whether the effect of CO is correlated to the perceived presence, the questionnaire PLMBR was carried out. Figure 2 shows that there is no systematic association between the change in the CO and the presence of the participants.
Change in ‘Collective Orientation’ (CO) in relation to presence. There is no systematic association between the change in the CO and the presence of the participants (n = 54).
Further analyses showed that there was no systematic variation between duration of work experience and CO as well as the change in CO. In addition, no significant correlations were found between presence and duration of work experience.
Discussion
The aim of this study was to investigate the influence of a simulator-based crisis resource management training on the CO of the participants.
Our primary hypothesis of the study was to test the existing scales for measuring CO and presence in our targeted group. All values of Cronbach alpha are higher than .65 which are acceptable for low-stakes instruments designed for programmatic decision-making that do not have direct consequences for individuals. 15 The result of presence is .54, so the interpretation has to be carefully, even more as they are self-reported. Individual differences of the participants, such as real-life work experiences, may have influenced their presence outcomes. It has been described that participants may attempt to mentally ‘recreate’ actual life situations in a simulation 16 – they often experience simulations more realistically if they have undergone similar incidents previously. This may also be the reason of a wide variation of presence and low Cronbach alpha values.
The results supported hypothesis (b), namely that CO can be improved by a simulation-based team training. This improvement was mainly due to an increase in the dimension ‘Affiliation’. The participants experienced a critical situation together, where the importance of teamwork was not only emphasized by a theoretical introduction but also during the debriefings. The importance of team performance for the success of the patient care could be experienced by observing other high-quality teams as well as through participation in their own scenario. Own experience and the relevance of the topic are known determining factors for learning success. 17
The effect size Cohen d shows a medium effect, which is satisfactory. The practiced skills, such as teamwork and communication, are very much shaped by corporate culture and are generally difficult to change. 18 The effect size shows that the simulation course can be a first step in change.
The dominance remained essentially unchanged during our study. As opposed to affiliation, dominance is a more difficult quality to assess regarding non-technical skills. On the one hand, a team leader in a critical situation must make rapid decisions and ensure that these decisions are efficiently executed. On the other hand, the team leader must remain open-minded and receptive to input from all team members so as to ensure that the treatment plan is the optimal strategy. The variable handling with these types of communication and task management will often be dependent upon the actual clinical situation. From the authors’ point of view, a single questionnaire, including those from our study, can hardly remotely predict or accommodate for this dynamic variability in dominance’s value during critical situations. The role of dominance and its relation to team leadership during critical situations is worthy of further study.
In addition, we saw no significant differences between the 2 professions and the effect of CRM training on CO. Both professional groups are likely to be difficult to fully classify due to the variabilities in their professional training, workplaces of employment, and subspecialization. Our result potentially reflects the special work environment for anaesthesiology in Germany which is characterized by a direct one-to-one proportion of nurses to physicians. In contrast to work at the ward level, the cooperation time in anaesthesiology with the respective ‘other’ professional group tends to be more intense than with their own profession.
Nonetheless, the increase in CO was higher (not significant) for nurses than for physicians. Perhaps the nurses became more aware of their own influence on the success of the treatment and their role in the team. Thomas et al 19 showed that physicians and nurses have discrepant attitudes about the teamwork they experience with each other. Further studies with a focus on these relationships would be interesting.
As for our fourth hypothesis, we found no correlation between presence and the effect of training upon CO. Few studies that have compared high- and low-fidelity simulation and their respective effects upon training have not been congruent: some studies have shown no difference at all,20,21 whereas others have found an advantage in high-fidelity simulation 22 and some an advantage in low-fidelity simulation. 23 All of these studies had different study populations, learning objectives, and course structures. The aim of training and kind of participants have an influence. 24 Our explanation is that the interprofessional structure of the simulation course directly affects the CO, rather than the simulation scenarios themselves. In addition, debriefing is known to play a major role: Savoldelli et al 25 have shown that exposure to a simulation crisis without constructive debriefing offers little benefit to the trainees. In the debriefing, the participants have the opportunity to recognize and understand the point of view of the team members observing from outside of the stressful situation. Furthermore, while debriefing, participants learn the reflection process of any demonstrated behaviour. 26 These could be reasons for, or contribute to, improvements in the CO.
Limitations
There are some limitations associated with this study. First, the course content was created by trainers from a single centre and might not be directly comparable to other CRM trainings. Although the participants came from several anaesthesia departments, it is a monocentric study. Potentially, the subgroups were too small to see effects regarding sex and professional groups. Furthermore, the transfer into the clinical routine was not measured. It is possible that the measured effect only occurs in the simulated environment with the participating team members. Nor do we know how long any training effect might persist. The low values of Cronbach alpha in the measurement of presence suggest that there might be better instruments for assessment. Further research has to show whether there could be a better instrument for the participants of health care providers.
Conclusions
Simulation-based crisis resource management training can positively influence the CO of participants. The dimension ‘Affiliation’ is the main goal in a group of experienced health care providers. The factors that influence the training effect remain unclear; the shared experience and review of critical situations can be a reason. Presence does not appear to be an important factor for all participants. Further studies are required to assess for transfer of true benefits to the workplace after improvements in CO by these trainings.
Footnotes
Funding:
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was funded exclusively by the department.
Declaration of Conflicting Interests:
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Author Contributions
MF: conception and design of the study, acquisition of data, analysis and interpretation of data, drafting the manuscript; HE: conception of the study, drafting the manuscript; LS: acquisition of data, drafting the manuscript; VH: interpretation of data, drafting the article; LF: conception and design of the study, analysis and interpretation of data, drafting the manuscript
