Abstract
Clinical reasoning and decision-making in prehospital contexts are complex, and patient assessments may be influenced by stress or biases, thus potentially risking patient safety. Previous research has shown mixed results regarding cognitive interventions designed to counteract biases and improve decision-making. In educational settings, there are no tools that assess clinical reasoning while also measuring important decision-making outcomes. This study employed a mixed-methods design and a novel assessment model to evaluate clinical reasoning and decision-making among Swedish prehospital nurse specialists. Additionally, the effect of the metacognitive TWED mnemonic was investigated. Thirteen participants were randomly assigned to two groups and assessed patients in simulation settings, with groups switching cases after brief training on the TWED mnemonic. The primary outcomes included point-based scoring on decision-making, grading of potential risks of patient harm, and analysis of clinical reasoning through reflections. The results showed large variation, without overall differences between groups or demographics. A complex case presentation resulted in lower scores and greater risks of potential patient harm. Qualitative analysis highlighted participants’ ability to handle conflicting data, which correlated with better outcomes. The use of the TWED mnemonic may have increased commission bias. Further research is needed to validate and understand these findings.
Background
In Sweden, roughly 1.3 million individuals a year require the attendance of emergency medical services. Most of these attendances are a response to an emergency call, and result in a prehospital nurse performing a patient assessment. The most frequent complaints are shortness of breath, abdominal pain, and chest pain (Swedish National Board of Health and Welfare, 2023a). After the prehospital patient assessment, 10–14% of patients are not considered to require conveyance to an emergency department (Höglund et al., 2020; Lederman et al., 2020). From an international perspective, studies on various populations have indicated significant variation in non-conveyance rates, with figures ranging from around four percent to as high as 98% of patients not requiring conveyance to an emergency department. There are concerns regarding competencies, support, and patient safety in non-conveyance decision-making (Ebben et al., 2017).
The Swedish emergency medical services are considered advanced life support units, with registered nurses or nurse specialists expected to deliver advanced medical care (Hjalmarsson et al., 2020). The staffing policy, as outlined in the Swedish regulations on prehospital care (SOFS 2009, p. 10), mandates that every ambulance must be staffed with at least one registered nurse. A nurse specialist, as defined by the Swedish Higher Education Regulation (SFS 1993, p. 100), is a registered nurse with an additional one-year master’s degree. There are a variety of nurse specialist subtypes, such as intensive care or anaesthesiology, but most prehospital nurse specialists have a degree in prehospital emergency care. In some regions of Sweden, there are directives mandating that emergency medical services are staffed with a nurse specialist working together with either a registered nurse or an emergency medical technician (Swedish National Board of Health and Welfare, 2023b). Experienced nurse specialists sometimes work as single responders, performing individual patient assessments without the support of a team member (Carlström & Fredén, 2017).
Patient Assessments in a Prehospital Context
Performing a patient assessment in a prehospital context is complex. Prehospital nurses describe the assessments as influenced by patient-related factors such as vital signs, symptoms, and medical history, with associated issues regarding paradoxes in findings and difficulties obtaining reliable information from intoxicated patients. Other contextual factors, such as home and social situation, or the absence of available options other than transport to an emergency department, also influence the decision-making (Nilsson & Lindström, 2016). Prehospital nurses also address various contributors to stress, including traumatic or chaotic situations involving children or upset relatives, as well as the influence of fear and physical or emotional reactions. Furthermore, challenges in cooperation with team members or disrupting circumstances (e.g. limited resources, adverse weather conditions, the influence of family members, or malfunctioning technology) are highlighted as having a detrimental impact (Glawing et al., 2023).
During an ambulance attendance, the patient assessment relies on the gradual collection of variable and fragmented information. The attendance can be divided into several phases, starting with dispatch information and finding the address, followed by first contact and scene assessment. Primary and secondary surveys are then performed, generating a field diagnosis and possible interventions, followed by a decision on the level of care and transport. The final phases include handover and documentation (Andersson et al., 2022). Throughout the attendance, clues are identified and analysed, and hypotheses are defined that render solutions, actions, and evaluation (Gugiu et al., 2022).
Clinical Reasoning and Decision-Making
In order to conduct high-qualitative patient assessments in complex contexts, prehospital nurses need to develop skills in clinical reasoning and decision-making. The reasoning process could be described as the analysis that leads to a decision (Gugiu et al., 2022). Clinical reasoning is a central and complex process that involves defining of the patient problem, therapeutic decisions, and prognosis (Yazdani & Hoseini Abardeh, 2019). In systems using prehospital nurses, it has been found that nurses are sometimes unaware of their clinical reasoning, and situational signals of lower severity can have a negative impact on the reasoning process (Andersson et al., 2022). Clinical decision-making is a term that highly correlates to clinical reasoning and is often defined as the decision-making process of data gathering, interpretation, evaluation, and consideration (Tiffen et al., 2014). Clinical decision-making is at best based on rational, logical, and evidence-based thinking, but it can be affected by knowledge gaps, the incorrect use of decision aids, or different biases (Croskerry, 2017). Since reasoning and decision-making are highly interdependent, this study will observe and measure clinical reasoning as it occurs during the process of analysis and justification, leading up to the decision-making and its important outcomes.
Dual-Process Theory and Biases
To guide patient assessments, prehospital nurses use subconscious and conscious thinking in the decision-making process (Perona et al., 2019). This corresponds well to dual-process theory, which divides thinking into system 1 and system 2. System 1 is intuitive, fast, subconscious, automatic, requires low effort, and is used in situations that are familiar. System 2 is slow, conscious, controlled, requires higher effort, and is used in new situations (Evans, 2008). In patients with failing vital functions, system 1 is primarily used, while system 2 is more common in unclear patient presentations (Andersson et al., 2019).
Biases can be defined as systematic errors due to inappropriate mental models or processing limitations, and are more common in intuitive system 1 thinking (Hammond et al., 2021). Biases may negatively affect decision-making in healthcare, with stressful emergency care settings being more vulnerable (Daniel et al., 2017; Saposnik et al., 2016). Cognitive biases that influence the thought process include anchoring, availability bias, confirmation bias, diagnostic momentum, and representativeness restraint. All of these biases may lead to search satisficing and premature closure. Other common biases, which are more related to the clinician’s affective response, include the framing effect and visceral biases (Croskerry, 2003; Prakash et al., 2017). Of all biases, overconfidence is most frequently reported, not only in medical literature but in all fields of decision-making research (Berthet, 2022).
Croskerry et al. (2013) discuss a variety of interventions to counteract the influence of biases. Some research has demonstrated potential benefits of cognitive strategy interventions (Al-Khafaji et al., 2022; Lambe et al., 2016; Mamede et al., 2020; Prakash et al., 2019; Staal et al., 2022), but there are also studies that failed to prove the effect of such interventions (Gopal et al., 2021; O’Sullivan & Schofield, 2019; Sherbino et al., 2011; Sherbino et al., 2014). Furthermore, most of these previous studies on the topic were carried out in a non-clinical setting, with the primary measure being performances on written clinical vignettes. Hence, there are challenges in interpreting and validating these results (Berthet, 2022; Croskerry & Campbell, 2021; Graber et al., 2011; Kahneman & Klein, 2009).
One example of an intervention with the potential to reduce bias is the TWED mnemonic. It is a checklist for metacognitive support, developed and tested by Chew et al. (2016b). The letters in TWED stand for ‘Threat’, ‘What else/What if I’m wrong?’, ‘Evidence’, and ‘Dispositional factors’. The TWED mnemonic has shown potential in increasing diagnostic accuracy (Chew et al., 2016a; Chew et al., 2017, 2017, 2019).
Patient Safety
Improving prehospital assessments and decision-making have a great impact on patient safety (Bigham et al., 2011). Moreover, there is increasing pressure on the emergency medical services, particularly in terms of assessing patients with mental health issues, substance abuse, or socio-economic vulnerabilities (Andrew et al., 2020). The complexity in assessing patients within these categories is well-known, with higher risk of biases such as psych-out-error, and additional difficulties regarding intoxicated patients with acute medical conditions (Croskerry & Campbell, 2021; Hoban, 2017; Singh et al., 2017). In Sweden, an estimated 1%–3% of non-conveyance decisions concern patients with an overlooked time-sensitive condition (Magnusson et al., 2022). Concerning patients conveyed to an emergency department with the highest priority (i.e. suspected life-threatening condition), protocol deviations and lapses in care have been identified, potentially compromising patient safety (Hagiwara et al., 2019).
To increase patient safety in emergency care, different types of checklists have been studied, with varying results (Ko et al., 2011). Evidence supports the use of checklists for specific situations or patient groups (Chen et al., 2016; Dryver et al., 2021; Kerner et al., 2017). Similarly, there is a link between correctly assessed prehospital conditions and a higher degree of correctly administered treatment (Ramadanov et al., 2019). However, studies have shown wide variation in the degree of agreement between the prehospital diagnosis and the final diagnosis (Wilson et al., 2019). Hence, checklists for specific conditions or situations may be perfectly designed for aiding decision-making, but they still rely heavily on the assumption that the clinical reasoning process is flawless. If the analysis of patient risk factors and symptom presentation is flawed, and the incorrect medical condition is considered, well-designed specific checklists may misguide the decision-making.
Patient Assessments in Educational Settings
A final consideration pertains to the evaluation and measurement of clinical reasoning and decision-making, within the context of patient assessments. When evaluating medical staff in simulation training, several evidence-based tools are available. A large portion of research has been conducted on the Lasater Clinical Judgement Rubric and Script Concordance Tests (Brentnall et al., 2022). These are both based on Likert-scale observational assessments from one or more observer. The Lasater Clinical Judgement Rubric do not evaluate measurable outcomes, and have been associated with reliability issues (Lee, 2021). On the other hand, Script Concordance Tests produce reliable results, but are advanced, time-consuming, and require extensive panels of experts to ensure high quality scoring (Fournier et al., 2008; Lubarsky et al., 2011). Another well-studied tool is the Health Sciences Reasoning Test, a 33-item multiple choice tool that is mainly used as a pre- and post-test examination (Macauley et al., 2017).
A widely accepted tool for assessment of clinical skills in different health care settings is the Objective Structured Clinical Examination. It was designed to overcome issues with subjective evaluations (Harden et al., 1975). However, this tool offers more of an overall competency assessment than an evaluation of reasoning and decision-making skills, and studies using it also face challenges regarding variations in validity and reliability (Brannick et al., 2011; Daniel et al., 2019; Walsh et al., 2009).
In a prehospital context, the Paramedic Global Rating Scale has recently been translated and validated for use in the Swedish context (Bremer et al., 2020). This Likert-scale observational tool focuses on different dimensions of clinical reasoning and decision-making, but does not assess measurable outcomes (Tavares et al., 2013). Hence, in order to evaluate the effect of a cognitive strategy intervention, such as the TWED mnemonic, on clinical reasoning and the decision-making outcomes, there remains a need for a simple and user-friendly tool.
Methods
Aim
The aim of this study was to assess prehospital nurse specialists’ clinical reasoning and decision-making skills, using a novel combined qualitative and quantitative assessment model, and to investigate the effect of the metacognitive TWED mnemonic.
Study Design
This was an experimental simulation study using a mixed methods convergent parallel crossover design. The quantitative and qualitative results were analysed separately and then discussed in an integrated analysis to demonstrate convergence or divergence (Creswell & Creswell, 2018). Triangulation of the data allows for exploring the divergence of results in mixed-methods research (Williamson, 2005).
Population and Setting
The study was conducted in a region in southern Sweden. Due to regional directives, most of the emergency medical services are staffed with a nurse specialist, working in team with a registered nurse or an emergency medical technician. At the time of the study, the region also employed experienced nurse specialists as single responders. Nurse specialists therefore play a crucial part in patient assessments within the regional prehospital emergency response.
Triage System, Guidelines, and Educational Courses
The emergency medical services in the region uses the Rapid Emergency Triage and Treatment System (RETTS) to support clinical decision-making. RETTS is based on emergency symptoms and signs (ESS), which could be viewed as the patients’ chief complaint (e.g. chest pain, dyspnoea, back pain, vomiting, and intoxication). ESS, in combination with vital signs, assigns each patient with a triage. Triage colour, or priority level, is based on the severity and time-criticality of the patient’s condition, where red is the highest level, followed by orange, yellow, green, and blue (Widgren & Jourak, 2011). This five-level triage system sorts patients into stable and unstable, with red and orange (i.e. priority levels 1 and 2) representing the unstable, time-critical conditions (Wireklint et al., 2018).
Prehospital nurses in the region are expected to work according to RETTS and digitally available treatment guidelines. The treatment guidelines consist of care programmes, information sheets, and directives, based upon the national treatment guidelines for the ambulance organisations (Föreningen för Ledningsansvariga inom Svensk Ambulanssjukvård [FLISA], 2017). The regional guidelines also provide guidance on when to consult a physician for additional decision-support, but the nurse is expected to conduct advanced initial patient assessments. Regarding non-conveyance decisions, some guidelines exist, but nurses are not obligated to consult a physician as a general rule. Employees gradually receive training in specific courses, such as Pre-hospital Trauma Life Support (PHTLS) and Advanced Medical Life Support (AMLS) (NAEMT, 2017, 2020).
Data Collection
The acquisition of data was carried out at a regional training unit during the spring of 2023. On two occasions during a week, 15 nurse specialists were observed in simulations performing two assessments each, and upon completion of each assessment they received a reflection assignment. A total of 30 assessments and 30 reflections were performed. Two participants were excluded due to protocol deviations; thus, information was collected from 26 observed simulations and 26 reflections from 13 nurse specialists. The participants represented nearly all the ambulance districts of the region.
The training unit provided a staged apartment and equipment necessary to carry out this experiment as close to a real-life simulation as possible. Actors were recruited to play the roles of patients and relatives, explicitly instructed on how to respond to the nurses’ questions and assessments. As in reality, the simulation started with a short dispatch notice, all to recreate an atmosphere as close as possible to a real-life patient assessment.
Experimental Design
To address the research question, an experimental crossover design was implemented. A summary of the study background and aims was distributed to all ambulance stations in the region, inviting participation in a research study as part of a regional educational training day. A convenience sample of 15 nurse specialists, employed by the regional ambulance organisation or any of its private partners, was included in the study. Any specialist subtype was accepted, but registered nurses without a specialist degree were excluded from participation.
On-site, a stratified block-randomisation process was conducted, initially dividing participants into two groups with similar prehospital clinical experience. Each participant completed two simulation scenarios with two different patient cases. After completing the first patient case, all nurses received brief training on the metacognitive TWED mnemonic, after which the groups switched cases (Figure 1). Experimental flow chart. * One participant per group excluded due to protocol deviations (final sample, N = 13). ** Case 2 with support from a metacognitive mnemonic. AMI = Acute myocardial infarction. DKA = Diabetic ketoacidosis.
The Patient Cases
The acute myocardial infarction (AMI) case consisted of a patient with chronic back pain and risk factors for cardiac disease, presenting with a change in his back pain due to a posterior myocardial infarction. The key to solving the case was to make a second attempt to obtain a readable ECG, since the first ECG was poorly executed due to motion artefacts. The diabetic ketoacidosis (DKA) case consisted of a patient with type 1 diabetes who was vomiting and experiencing diabetic ketoacidosis. The key to solving the case was to check glucose and ketone levels. Potential biases were induced to interfere with the nurses’ reasoning processes, with more surrounding distractions in the diabetic ketoacidosis case (Appendix 2). The scenarios were inspired by recurrent assessment situations with risks of delayed treatment and where the final diagnosis correlated to a specific treatment guideline. The regional chief medical physician provided anonymous data for the synthesis of the cases.
The participants started off with receipt of the dispatch notice and were asked to proceed to the assessment as usual. The notice contained both relevant and distracting information (Appendixes 1 and 2). There was bedside digital access to RETTS and other implemented guidelines.
Educational Setup and Use of the Metacognitive Mnemonic
After completion of the first case assessment and written reflection, the participants were all introduced to the concepts of clinical reasoning and decision-making, as well as cognitive and emotional biases, along with the Swedish translation of the TWED mnemonic (Appendix 3). The short introduction lasted for 20 minutes and included definitions of clinical reasoning and decision-making, causes of distortion of thinking and reasoning (based on Croskerry [2017], regarding cognitive miserliness, mindware gaps, and mindware contamination), an introduction to cognitive and emotional biases, as well as contextual factors and stressors, and the metacognitive TWED mnemonic. The participants all received a pocket card with the TWED mnemonic on it. They were asked to use this when they assessed the patient in their second case. If the use of the pocket card was not observed, the participants were asked to reassess their final decision with the support of the TWED mnemonic. In most cases, the nurses were observed to explicitly use the TWED mnemonic in the later part of the assessment situation, whereas a few nurses used it earlier in the process.
Instrument and Outcome Measures
Qualitative Outcome Measure
The qualitative outcome measure involved analysing written reflections following completion of the respective patient case assessments. At this stage, participants were unaware of their performance. They were asked to describe the rationale and justify their choice of field diagnosis/ESS, priority, level of care, actions, and treatment. To triangulate the assessments, the questions were chosen to act as analogues for the quantitative measure described below. The Swedish translation of the Structure of Observed Learning Outcomes (SOLO) taxonomy (Swedish National Agency for Education, 2011) was used as support for the analysis. It was adapted to a clinical reasoning context by replacing ‘student’ with ‘nurse’ and by adding a short explanatory text providing the context of clinical reasoning (Appendix 4). The SOLO taxonomy consists of five levels of understanding: prestructural, unistructural, multistructural, relational, and extended abstract. From a clinical reasoning perspective, the emphasis is placed on the ability to effectively manage data, particularly in navigating conflicting evidence. It has been used in various types of research related to the assessment of knowledge, critical thinking, and clinical reasoning (Chrismawaty et al., 2023; Hazel et al., 2002; Ilgüy et al., 2014; Lucander et al., 2010; Pinto et al., 2016).
Quantitative Outcome Measure
Quantitative Point-Based Scoring Model.
aPredefined answer keys for the respective cases are available in Appendix 5.
Ordinal Risk Colour Grading Model.
aConveyance to ED in the DKA case and specific care pathway to percutaneous coronary intervention in the AMI case.
AMI = Acute myocardial infarction.
DKA = Diabetic ketoacidosis.
ED = Emergency department.
ESS = Emergency symptoms and signs (i.e. chief complaint).
Rationale Behind the Point-Based Scoring Model
One of the main reasons for developing a novel assessment tool was to measure important outcomes. Instead of Likert-scale evaluation, or other observational methods, the use of clinical outcome measures may be more relevant when assessing true effects of an intervention. However, no scientific consensus exists on what is considered important outcomes (Kersting et al., 2020). The authors decided on outcomes related to clinical decisions and patient safety. This resulted in the point-based scoring being complemented by the ordinal risk colour grading model. The scoring model included differential diagnosis, determination of correct medical condition, assessment of severity and level of care, as well as decisions on actions and treatment.
The model was based on the assumption that the initial assessment affects the level of care and treatment (FitzGerald & Hurst, 2017). Diagnostic agreement was graded in correlation with the final diagnosis as incorrect, correct ESS (i.e. chief complaint), or correct diagnosis, inspired by Magnusson et al. (2018). Priority level (i.e. the severity of the medical condition) was based on the distinction between stable and unstable conditions in priority blocks. This distinction includes priority level 3 to 5 versus priority level 1 or 2 according to RETTS (Wireklint et al., 2018). It was complemented with the Lund Outcome Set for Evaluation of Triage (LOSET), developed by Johansson et al. (2023), which is a set of 49 clinical outcomes linked to priority level 1 and/or 2 (e.g. receiving acute percutaneous coronary intervention, or that the patient’s pH was measured under 7.3, as examples of priority level 1 correlating to the myocardial infarction and diabetic ketoacidosis cases in the study). The level of care was linked to the priority level according to RETTS/LOSET, as well as to implemented regional guidelines for specific care pathways (i.e. care pathway to percutaneous coronary intervention). Actions and treatment were based on the implemented regional guidelines for the correct diagnosis in each patient case, as well as potentially harmful treatments available in other sections of the guidelines. The elements of the scoring model were reviewed by the main author of the LOSET study and the regional chief medical ambulance physician.
The Metacognitive Mnemonic
The TWED mnemonic (Chew et al., 2016a) was translated into Swedish following the general steps recommended for translating instruments. This included two separate translations, a discussion leading to synthesis, a blind back-translation to the original language, and a comparison with the original version. Subsequently, consensus was reached on a pilot-tested and refined version (Sousa & Rojjanasrirat, 2011). Contextual and linguistic support throughout the process was provided by the main author of the LOSET study, an English-born nurse, and an American-born PhD (Appendix 3).
Pilot Test
Prior to completing the combined qualitative and quantitative assessment model, as well as to test the translated TWED mnemonic, a pilot test was conducted at an emergency department as a small randomised controlled experiment (N = 6). Two groups performed the same patient case assessment in a simulation setting, with Group 2 having support from the metacognitive TWED mnemonic. The case consisted of a patient with several risk factors for cardiac disease presenting at the emergency department with a sudden onset of fatigue. The nurses had bedside access to the implemented triage guidelines. The maximum total score was 11, and there was the possibility of a negative score. The SOLO taxonomy was used in the analysis of the clinical reasoning from the written reflections. The reflections were first analysed separately and then compared, followed by a discussion to reach consensus.
The median total score differed between the two groups (3 vs. 7) in favour of Group 2, but it did not reach statistical significance. Nurses with a higher level of clinical reasoning (i.e. a higher SOLO taxonomy) had a superior clinical decision-making (i.e. higher total score). The correlation was strong and statistically significant (rs = .91, N = 6, p = .012). Similarly, nurses with a greater level of diagnostic agreement to the final diagnosis had a higher total score (rs = .89, N = 6, p = .017). The SOLO taxonomy did not differ between the groups and ranged from a multistructural level 3 to a relational level 4a. These results were used to perform a power calculation of the study design using repeated measures, with group differences regarding the metacognitive mnemonic in mind. The calculated total sample size was 15 using GPower (version 3.1). It was based on matched pairs and the Wilcoxon signed-rank test, with a large effect size of 0.8, a standard deviation of 3, alpha level of .05, and a power of 80%.
After the pilot, three points were included for determining the correct care pathway to secure vertical and horizontal consistency in the point-based scoring model. This was due to three points being given if the field diagnosis was in line with the final diagnosis, which correlates to three points for the care pathway to percutaneous coronary intervention in the acute myocardial infarction case. Points were also added for differentials and associated reasoning/examinations, and the questions in the reflection task were clarified to generate more depth. The same main questions remained (rationale and justification of field diagnosis/ESS, priority, level of care, and lastly actions and treatment), but they were worded otherwise in order to build upon each other (e.g. ‘based on your choice of field diagnosis/ESS, justify your choice of priority level; Based on your field diagnosis and priority level, justify your choice of care level’). These changes were made to generate deeper reasoning, since the pilot test generated a limited text content for analysis. In the main study, the number of differentials with associated actions and reasoning, needed to be included with the qualitative data on the case report forms. Otherwise, a limited text content could produce a less reliable assessment of the nurses’ actual reasoning capabilities regarding differential diagnosis.
Data Analysis
The SOLO taxonomy of the written reflections was assessed individually by O.F. and U.H., and then compared. If the assessment differed, a discussion took place that led to a consensus. Kappa statistics on inter-rater agreement were calculated. Inter-rater agreement on the SOLO taxonomy of the 26 written reflections was moderate to substantial (.586 and .668 on the sublevels and main levels, respectively). After discussion, a complete consensus was reached on all levels. Observations and scoring were made by O.F. and U.H.
The data analysis was carried out using SPSS version 28. Demographic data were analysed by comparing medians, due to skewed distribution in continuous variables, and counts between groups. Total score, risk colour, and levels of SOLO taxonomy and diagnostic agreement were considered ordinal variables; therefore, non-parametric tests were used. The Wilcoxon signed-rank test and McNemar’s test were used for within-group analysis, while the Mann–Whitney U test, Kruskal–Wallis test, chi-square, or Fisher’s exact test were used for between-group analysis. Spearman’s rank test was used for the correlation analysis.
Results
Demographics
Demographic Characteristics of Participants.
Note. N = 13. Continuous variables were binned into equal-sized groups. Median age was 46 years.
(Range = 25) and did not differ significantly between groups. Neither did years of experience in.
prehospital care (Median = 11, Range = 21) or years as a registered nurse (Median = 16, Range = 24).
aNon-significant notable difference between groups (p = .103).
bOther subtypes represented were intensive care and anaesthesiology.
AMI = Acute myocardial infarction.
AMI/DKA = The group that started with the AMI case, followed by the DKA case.
DKA = Diabetic ketoacidosis.
DKA/AMI = The group that started with the DKA case, followed by the AMI case.
AMLS = Advanced medical life support.
PHTLS = Prehospital trauma life support.
Qualitative Results
The SOLO taxonomy of the written reflections ranged from a single unistructural reasoning to 10 relational reasonings. The remaining 15 reflections were on a multistructural level. In the diabetic ketoacidosis case, nurses with 9–11 years of experience in prehospital care reasoned on a relational level, compared to multistructural-level reasoning among the other nurses (p = .01). No other group or demographic differences were found. Regarding the investigated effect of the metacognitive TWED mnemonic, there was a slight overall decrease in the median taxonomy level from a multistructural level 3a to a multistructural level 3 from the first to the second case (p = .124). This decrease occurred in both groups, with no regard to order of patient cases.
Quotes Representing SOLO Taxonomy Levels Revealed in the Results.
Note. This table contains quotes from participants representing the different SOLO taxonomy levels revealed in the results.
ECG = Electrocardiogram.
ESS = Emergency symptoms and signs (i.e. chief complaint).
GCS = Glasgow coma scale.
PCI = Percutaneous coronary intervention.
SOLO = Structure of observed learning outcomes.
2a = Two aspects of the case are discussed, but several aspects are missing and the nurse is unable to draw a correct conclusion.
3 = Multiple aspects of the case are discussed, but conflicts are ignored and a simplified conclusion is drawn.
3a = Multiple aspects of the presentation and conflicts are discussed but then overlooked, and a simplified conclusion is drawn.
4 = Broad reasoning based on aspects from several different types of data, including reflections on conflicts and inconsistencies.
4a = In addition to level 4, the premature nature of the field diagnosis is also suggested.
Quantitative Results
Demographic Characteristics and Total Score (TS).
Note. N = 13. Continuous variables were binned into equal-sized groups. The maximum total score in the AMI case was 12 points, with a median total score of 9 points (range = 14, from −2–12).
The maximum total score in the DKA case was 10 points, with a median total score of 6 points (range = 7, from 3 to 10).
aStatistically significant difference between the experience groups (p = .024).
bOther specialist subtypes represented were intensive care and anaesthesiology.
AMI = Acute myocardial infarction.
AMLS = Advanced medical life support.
DKA = Diabetic ketoacidosis.
PHTLS = Prehospital trauma life support.
Overall, 50% of the assessments resulted in the combination of a correct field diagnosis/ESS, correct priority, and correct level of care. This resulted in a categorisation of green risk colour, indicating no risk of potential harm to the patient. Only three out of thirteen nurses managed both their cases at this level (Table 7). Twenty-one out of the twenty-six assessments posed no or low risk of potential patient harm, with the remaining five assessments posing moderate or high risk. Four of these five assessments concerned the diabetic ketoacidosis case, equalling 31% of the assessments in this case.
Variation in Field Diagnosis/ESS per Case.
Note. Between the AMI case and the DKA case, there was a statistically significant difference regarding correct versus incorrect field diagnosis or ESS meaning that significantly less participants in the DKA case chose the correct field diagnosis or ESS (p = .016).
aField diagnosis correct to final diagnosis.
bESS correct to final diagnosis.
AMI = Acute myocardial infarction.
DKA = Diabetic ketoacidosis.
ESS = Emergency symptoms and signs (i.e. chief complaint).
There was a significant positive correlation between diagnostic agreement and total score, meaning a closer match to final diagnosis resulted in better clinical outcomes of the decision-making and less potential risk of patient harm (Scatter plot 1). In the acute myocardial infarction case, the correlation coefficient was .84 and in the diabetic ketoacidosis case it was .857 (p < .001). DKA case correlation of total score and level of diagnostic agreement. Note. N = 13, rs (11) = .857, r2 = .734, p < .001. The maximum total score was 10 points. The interpolation line shows the mean value of the total score. The level of diagnostic agreement was calculated from the point-based model answer key as follows: 2 points for DKA; 1 point for ESS 49 (Hyperglycaemia); 0 points for all others. DKA = Diabetic ketoacidosis. ESS = Emergency symptoms and signs (i.e. chief complaint).
Lastly, in the acute myocardial infarction case, 12 out of 13 nurses performed an initial ECG, whereas 8 performed a second (readable) ECG, this being the key action to solve the case. Regarding key actions in the diabetic ketoacidosis case, 10 out of the 13 nurses initially checked the plasma glucose level, whereas 6 checked the plasma ketone levels. The number of ECGs performed in the diabetic ketoacidosis case differed significantly between the AMI/DKA and the DKA/AMI group. All the participants who first assessed the patient with an acute myocardial infarction also performed an ECG in the diabetic ketoacidosis case. In the other group, only two participants performed an ECG (p = .021).
Validation of the Point-Based Assessment Model
The results showed a strong positive correlation between total score and risk colour in both cases. Nurses with superior clinical decision-making (i.e. higher scores) posed less risk of potential patient harm (i.e. superior risk colour), overall validating the point-based scoring model from a patient safety point of view (Scatter plots 2 and 3). The correlations were statistically significant, and the strongest correlation was found in the diabetic ketoacidosis case (r = .942). AMI case correlation of total score and risk colour. Note. N = 13; rs (11) = .867, r2 = .752, p < .001. Maximum total score was 12 points. The interpolation line shows the mean value of the total score. AMI = Acute myocardial infarction. ED = Emergency department. ESS = Emergency symptoms and signs (i.e. chief complaint). DKA case correlation of total score and risk colour. Note. N = 13; rs (11) = .942, r2 = .887, p < .001. Maximum total score was 10 points. The interpolation line shows the mean value of the total score. DKA = Diabetic ketoacidosis. ED = Emergency department. ESS = Emergency symptoms and signs (i.e. chief complaint).
Effect of the Metacognitive Mnemonic
Total Score per Participant and Change From Case 1 to Case 2.
Note. N = 13. The maximum score was 12 and 10 points for the AMI case and DKA case, respectively. Median total score in Case 1 was 8 (range = 14), with green risk colour, and median total score in Case 2 was 6 (range = 9), with yellow risk colour. The differences were non-significant (p = .783 for total score, p = .774 for risk colour).
aParticipants #1, #3, and #10, whose patient assessments generated a green risk colour in both cases.
AMI = Acute myocardial infarction
AMI/DKA = The group that started with the AMI case, followed by the DKA case
DKA = Diabetic ketoacidosis
DKA/AMI = The group that started with the DKA case, followed by the AMI case
There was a significant increase in the number of differential diagnoses between the first and second cases, from a median of three (range = 4) to a median of five differentials (range = 8), when using the metacognitive mnemonic (p = .018). The increased number of differential diagnoses had an overall non-significant negative correlation to risk colour (p = .073) and total score (p = .161). However, in the diabetic ketoacidosis case, the correlations were significant (p = .021), establishing a negative relationship between increasing number of differential diagnoses and performance (Scatter plot 4). DKA case correlation of risk colour and number of differential diagnoses. Note. N = 13; rs (11) = −.629, r2 = 0.396, p = .021. The interpolation line shows the mean number of differential diagnoses. DKA = Diabetic ketoacidosis. ED = Emergency department. ESS = Emergency symptoms and signs (i.e. chief complaint).
Integrative Analysis
Validation of the Overall Qual–Quant Assessment Model
The three-part combined qualitative and quantitative assessment model showed strong consistency and convergence between the quantitative measures of clinical decision-making (total score and risk colour) in both patient cases. The SOLO taxonomy level showed a stronger correlation to the quantitative measures in the diabetic ketoacidosis case, indicating clinical reasoning and the outcomes of the decision-making were more interdependent in these assessments (Scatter plot 5). In contrast, the participants were able to demonstrate good clinical decision-making in the acute myocardial infarction case, without exhibiting comprehensive clinical reasoning skills, resulting in a weaker correlation (Scatter plot 6). DKA case correlations of total score, risk colour, and SOLO taxonomy level. Note. N = 13. The table shows the combined qualitative and quantitative assessment model. Maximum total score was 10 points. Data labels show the SOLO taxonomy level per participant. The interpolation line shows the mean value of the total score. The correlation coefficient between total score and risk colour was .942 (p < .001), and the correlation coefficient between risk colour and SOLO taxonomy was .611 (r2 = .37, p = .026). DKA = Diabetic ketoacidosis. ED = Emergency department. ESS = Emergency symptoms and signs (i.e. chief complaint). SOLO = Structure of observed learning outcomes. 3 = Multiple aspects of the case are discussed, but conflicts are ignored and a simplified conclusion is drawn. 3a = Multiple aspects of the presentation and conflicts are discussed but then overlooked, and a simplified conclusion is drawn. 4 = Broad reasoning based on aspects from several different types of data, including reflections on conflicts and inconsistencies. 4a = In addition to level 4, the premature nature of the field diagnosis is also suggested. AMI case correlations of total score, risk colour, and SOLO taxonomy level. Note. N = 13. The table shows the combined qualitative and quantitative assessment model. Maximum total score was 12 points. Data labels show the SOLO taxonomy level per participant. The interpolation line shows the mean value of the total score. The correlation coefficient between total score and risk colour was .867 (p < .001), and the correlation coefficient between risk colour and SOLO taxonomy was .326 (r2 = .11, p = .277). AMI = Acute myocardial infarction. ED = Emergency department. ESS = Emergency symptoms and signs (i.e. chief complaint). SOLO = Structure of observed learning outcomes. 2a = Two aspects of the case are discussed, but several aspects are missing and the nurse is unable to draw a correct conclusion. 3 = Multiple aspects of the case are discussed, but conflicts are ignored and a simplified conclusion is drawn. 3a = Multiple aspects of the presentation and conflicts are discussed but then overlooked, and a simplified conclusion is drawn. 4 = Broad reasoning based on aspects from several different types of data, including reflections on conflicts and inconsistencies.
Exploring DKA Cases of Divergence in the Combined Quantitative and Qualitative Assessment Model.
Note. The maximum score was 10 points in the DKA case. This table shows the use of combined quantitative and qualitative data to explain results through the triangulation of clinical reasoning and decision-making. In this case, the handling of conflicting data explained why a nurse whose assessment rendered a higher score still posed a greater potential risk of patient harm. SOLO taxonomy levels 3 and 3a are both on a multistructural level, meaning a wrong conclusion is drawn, but with the difference of level 3 ignoring and level 3a discussing conflicts. DKA = Diabetic ketoacidosis. ESS = Emergency symptoms and signs (i.e. chief complaint). Risk colour = Risks of potential patient harm; red equals high risk; orange equals moderate risk. SOLO = Structure of observed learning outcomes. 3 = Multiple aspects of the case are discussed, but conflicts are ignored and a simplified conclusion is drawn. 3a = Multiple aspects of the presentation and conflicts are discussed but then overlooked, and a simplified conclusion is drawn.
In the acute myocardial infarction case, the results showed low SOLO taxonomy and high total scores, as well as a case of high SOLO taxonomy and low total score. In these cases of divergence, the combined model was able to explore the clinical reasoning and decision-making. Participant #5 had a low total score of 4, but a SOLO taxonomy level 4 (adequate handling of conflicts), leading to clinical decision-making with a low risk of potential patient harm. The key to solve the case was to perform a second, readable ECG, but the nurse did not complete that task. Despite this, the written reflection and differential diagnosis revealed management of conflicting data, maintaining a suspicion of cardiac or pulmonary cause, resulting in conveyance to the emergency department, with a second ECG likely to be the first action based on the RETTS triage system and the suspected condition.
Exploring AMI Cases of Divergence in the Combined Quantitative and Qualitative Assessment Model.
Note. The maximum score was 12 points in the AMI case. This table shows the use of combined quantitative and qualitative data to explain results through the triangulation of clinical reasoning and decision-making. A high total score and low SOLO taxonomy was possible, and the triangulation of this fact offered an opportunity to further expand the analysis of the case presentation and discuss possible issues with limited clinical reasoning. The SOLO taxonomy also offered an opportunity to analyse a case of low total score yet low risk of potential patient harm. The handling of conflicting data rendered an assessment with correct ESS and correct medication given, thus resulting in lower patient risk than the point-based model would predict. AMI = Acute myocardial infarction. ESS = Emergency symptoms and signs. Risk colour = Risks of potential patient harm; yellow equals low risk; green equals no risk. SOLO = Structure of observed learning outcomes. 3 = Multiple aspects of the case are discussed, but conflicts are ignored and a simplified conclusion is drawn. 4 = Broad reasoning based on aspects from several different types of data, including reflections on conflicts and inconsistencies.
Effect of the Metacognitive Mnemonic
The results did not support the hypothesis that the self-initiated TWED mnemonic would counter the biases induced in the patient cases. Total score, risk colour, and SOLO taxonomy were slightly lower in the second case, while the number of differential diagnoses significantly increased. This indicates that the TWED mnemonic, the way it was investigated within this study design, had a potential negative impact on clinical reasoning and decision-making.
Discussion
Overall Results
Clinical reasoning and decision-making varied, mainly not due to individual demographics but rather between the patient cases. Only three nurses managed both their patient assessments without any risks of potential patient harm. Regarding demographic differences, the only significant result was found in the diabetic ketoacidosis case, where nurses with 9–11 years of experience scored higher on the point-based model, compared to the other experience groups. As it was the only statistically significant difference, it was most likely due to random variation or low statistical power.
Four out of five patient assessments posing moderate to high risks of potential patient harm, was carried out in the diabetic ketoacidosis case. This indicating a higher level of difficulty compared to the acute myocardial infarction case. This could be explained by several intentional factors, one being a more stress-inducing surrounding setting. This is consistent with previous research that indicates a negative impact of stress on clinical decision-making (Leblanc et al., 2012; Morgado et al., 2015; Pottier et al., 2013). Relevant to this study design and the diabetic ketoacidosis case presentation; noise and conflict are found to be two major contributors to impaired clinical decision-making (Groombridge et al., 2019). Stress can also potentiate various biases, possibly explaining why several nurses did not check the glucose level in the diabetic ketoacidosis case, despite the dispatch notice containing information of vomiting, abdominal pain, and a history of diabetes (Yu, 2016). The diabetic ketoacidosis case had a higher complexity in the patient and scene presentation, generating more conflicting data. The patient was intoxicated, with music playing and a barking dog from behind a closed door. An annoying friend was interrupting the assessment, and there were more diverse symptoms to evaluate (Appendix 1).
In the acute myocardial infarction case the patient presented with a thoracic back pain and risk factors for cardiac disease. He appeared outside of his apartment, ready to take a ride to the emergency department, in an attempt to trigger negative emotions among the nurse specialists. Apart from that, there were some minor distractions and overall fewer symptoms to evaluate, along with less surrounding factors interrupting the assessment process (Appendix 2).
The ability to handle conflicting data (i.e. reasoning on a SOLO-taxonomy level 4 or higher) was linked to fewer potential risks of patient harm, as seen in the integrative analysis scatter plots (Scatterplots 5 and 6). In contrast, an inability to handle conflicting data (i.e. reasoning on a SOLO-taxonomy level 3a or less) had a greater potential of posing risks of patient harm. This was especially evident in the diabetic ketoacidosis case, where the SOLO taxonomy levels correlated to the elimination of potential risks of patient harm. In cases of stress, uncertainty and conflicting evidence, previous research has suggested a tendency to accept higher level of risks in the decision-making (Morgado et al., 2015).
Another potential explanation of the decreased performance in the diabetic ketoacidosis case could be a lack of initial risk stratification, either due to individual strategies or due to the design of the current regional guidelines used in the study. The specific guidelines applicable to the cases were based on diagnosis rather than initial risk stratification of patient groups or symptoms. Neither testing of glucose on all diabetic patients, nor performing an ECG on thoracic back pain, was mandatory due to protocol (Föreningen för Ledningsansvariga inom Svensk Ambulanssjukvård, 2017; Predicare, 2023), leaving the decision solely in the hands of the nurses’ individual reasoning. To overcome this, initial risk stratification may be beneficial (Bentivegna et al., 2021; Demandt et al., 2022; Spangler et al., 2019; Yu et al., 2022). Digital support systems that generate initial differential diagnoses may also support better clinical reasoning and decision-making, but this research field is still new and uncertain (Kourtidis et al., 2022). Risk stratification is especially important, when defining a chief complaint proves to be difficult. There is growing evidence on the increased risks of unfavourable outcomes among patients presenting with unclear or undefined symptoms (Djärv et al., 2015; Ibsen et al., 2021, 2022; Ivic, 2020).
Basic guideline overview, with regards to the above-mentioned challenges, could also support clinical reasoning and decision-making. This needs to be further investigated, as adherence to prehospital guidelines have been found to be suboptimal, with applicability issues and with no general effective implementation standard presented in past research (Ebben et al., 2013; Fishe et al., 2018; Martin-Gill et al., 2022). Proposed interventions are the use of reminders, education, end-user involvement, and adaptation of guidelines to the non-linear process that is clinical reasoning and decision-making in a prehospital context (Andersson et al., 2019; Ebben et al., 2018; Hagiwara et al., 2013).
The results of the study showed a strong positive correlation between diagnostic agreement and the quantitative outcome measures (i.e. total score and risk colour). This correlation was expected, but could have been exaggerated by the study design, as well as regional treatment guidelines based on initial detection of a specific diagnosis (Föreningen för Ledningsansvariga inom Svensk Ambulanssjukvård, 2017). There is evidence of great promise in checklists to improve clinical decision-making, following the correct diagnosis of an acute medical condition (Dryver et al., 2021; Hall et al., 2020). However, in a prehospital setting, the diagnostic accuracy may vary between 58% and 70% (Ackerman & Waldron, 2006; Christie et al., 2016; Cummins et al., 2013; Koivulahti et al., 2020; Magnusson et al., 2018) with a wide variation in regards of specific conditions, where some exceptions show a 100% accuracy (Wilson et al., 2019). This raises the question whether prehospital guidelines should be based on symptoms, risk stratification, diagnosis, or a combination of all three. There is no scientific overview available on the topic.
The Metacognitive Mnemonic
The use of the self-initiated metacognitive TWED mnemonic had a potentially negative impact and did not improve patient safety in this study. In fact, having more differential diagnoses may have increased the risk of commission bias (Pelaccia et al., 2011). When anchoring effects are already in place, it also seems difficult to calibrate the decision effectively, even when increasing the number of differential diagnoses (Brewer et al., 2007). Another aspect is the high educational level and long work experience among the participants in the study. Previous research has discussed the more dominant system 1 processes associated with expert decision-making (Evans, 2008), and the educational and short-term setup of this study may have been insufficient to promote change.
These study findings indicated that it was hard to calibrate decisions late in the decision process. A crew member, or maybe a consulting physician, might be a better facilitator of the metacognitive TWED mnemonic. On the other hand, it seems that technological debiasing interventions are more likely to succeed than cognitive strategies (Ludolph & Schulz, 2018), and that this study’s results are consistent with the 50% of studies on cognitive debiasing strategies that fail to show effectiveness. If this instrument is to be tested again in a similar setting, one might consider a different approach to the education and use of the mnemonic.
Validation of the Overall Qual–Quant Assessment Model
The combined qualitative and quantitative assessment model’s outcome measures were based on SOLO taxonomy (analysis of the reasoning process), point-based scoring, and grading of risks of potential harm (outcomes of the decision-making). There was a strong internal consistency within the quantitative measures of the combined model, with a moderate consistency within all three parameters of the model. In cases with inconsistencies, the different parts of the model could be used to triangulate and explore the reasoning and decision-making.
Through observation of multiple patient assessments, and analysis of written reflections, the model provided an opportunity to reflect on impact of stress, biases, and system 1 and system 2 thinking in different situations. As showed in the results, only three nurses managed both cases without potential patient risks. This opportunity of reflection is important for nurses’ professional growth (Miraglia & Asselin, 2015). Using SOLO taxonomy as qualitative outcome measure of the reflections, may offer a possibility to oversee and quantify long-term individual and group progress (Brabrand & Dahl; 2009). Although, the results from this study showed a lower rate of initial inter-rater agreement on SOLO taxonomy levels than expected.
Regarding the quantitative measures, the first item of the point-based model (i.e. number of differential diagnoses, and associated actions and reasoning) may need to be removed. The first reason for this is that a recalculation of total scores without this item didn’t affect the overall results. The second reason is the fact that the other items of the scoring model directly affect the potential outcome for the patient (i.e. field diagnosis/ESS, priority level, level of care, actions, and treatment), whereas the number of differential diagnoses is a surrogate for something not necessarily affecting clinical outcome. The third reason is based on this study findings, namely that explicit focus on increasing differential diagnoses has a potential negative impact on decision-making.
This study should be viewed as a pilot study on a new concept of simulation training assessment in a combined qualitative and quantitative manner. The choice of a different approach or different outcome measures may provide additional insights and quality to the model.
Limitations and Further Research
This study has some limitations within its design. Nurses were evaluated on individual assessments in a simulation setting, while team assessments are more common in a prehospital context. However, nurse specialists active as single responders did not perform better than other participants, which is an important lesson learnt from this study. Convenience sampling makes it difficult to generalise results, despite the strategic representation of gender, age, and experience. Additionally, the main authors work in the same organisation as the participants, which may have influenced the data collection and analysis.
The authors lacked prior SOLO taxonomy experience, potentially impacting the analysis. Despite a thorough analysis process, additional analysis made by a senior researcher with prior experience of SOLO taxonomy would provide more reliability to the qualitative results. Other qualitative approaches could also be considered (e.g. a talk-out-loud design with audio/video recording). However, due to the limited number of participants, audio or video recordings might have further reduced the sample size.
The sample size was calculated for a between-group comparison based on the pilot test results, while the main findings of the study were correlational in nature. The internal consistency of the quantitative measures was strong, but a larger sample size is needed to confirm the results in the overall combined model. This study should, therefore, be considered a pilot study, and further research with a larger sample size is needed to validate and understand the results.
This study suggests negative short-term effects of brief metacognitive training, and further research is warranted to confirm these results. Also, further research is needed on other possible interventions, such as guideline adaptation and implementation, or research with a long-term perspective on the effects of cognitive strategies on nurse specialists’ patient assessments.
Finally, to understand the results and its potential implications, one needs to be reminded of the context of Swedish prehospital healthcare, as described in the background section of this article. The ambulance crew is expected to work highly independently, making advanced patient assessments according to guidelines. The possibility to consult a physician by telephone exists but is not obligated in these guidelines, for instance regarding non-conveyance decisions. This study puts some light on aspects such as stress, cognitive biases, emotional response, and guideline adherence. All within the context of patient assessments, clinical reasoning and decision-making among nurse specialists within the Swedish prehospital emergency care.
Conclusions
Within a simulation setting, clinical reasoning and decision-making vary among prehospital nurse specialists, without apparent demographic differences. A higher complexity in the case presentation causes higher rates of assessments posing potential risks of patient harm. To facilitate good clinical decision-making in these circumstances, nurses need highly developed reasoning abilities to understand and interpret conflicting data. To assess reasoning abilities, and the subsequent outcomes of the decision-making, a novel combined qualitative and quantitative assessment model was developed. The model was based on SOLO taxonomy analysis and point-based scoring on important clinical decisions.
The novel assessment model showed satisfying internal consistency, especially in a complex case presentation. The combined measures of the model integrated the objectivity of predefined answer keys with the depth of written reflections. It measured clinical outcomes and, from an educational point of view, offered opportunities for feedback and professional growth.
Based on the study findings, self-initiated use of the metacognitive TWED mnemonic did not show any benefits. In fact, the results indicated decreased performance and commission bias, due to a significant increase of differential diagnoses.
Supplemental Material
Supplemental Material - Assessing Clinical Reasoning and Decision-Making in Swedish Prehospital Emergency Care: A Mixed Methods Study With an Experimental Design
Supplemental Material for Assessing Clinical Reasoning and Decision-Making in Swedish Prehospital Emergency Care: A Mixed Methods Study With an Experimental Design by Olof Fager, Ulrika Hindsberg, André Johansson, Christer Axelsson and Magnus Andersson Hagiwara in Journal of Cognitive Engineering and Decision Making
Consent for Publication
All participants received information on the planned publication of the study results. This was clearly stated in the written information received prior to giving informed consent. All data were also pseudonymised to secure anonymity for the participants.
Footnotes
Acknowledgments
Special thanks to Christoffer Nordin and Fredrik Hindsberg, who, together with A.J., did tremendous work as actors during the simulation sessions. Also, thanks to registered nurse Louise Godden and Professor Stephen Marr for helping with the translation process of the mnemonic. We also want to thank the entire regional ambulance organisation with its educational coordinator and chief medical physician, along with the most important persons of all: the participating specialist nurses.
Authors’ Contributions
O.F. and U.H. are the main researchers and authors of this article. O.F. cared for all statistical analyses, tables, and figures. Professor M.A.H. supervised the entire process, and A.J. contributed with additional knowledge and input regarding methods from the perspective of emergency medicine triage, as well as critical appraisal during the process. C.A. provided further valuable feedback on the content. All authors read and approved the final manuscript before submission.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Ethical Statement
Data Availability Statement
The source data, in the form of pseudonymised transcriptions on all written reflections and the SPSS dataset with all analysed variables, is stored on secure servers at the University of Borås and will be made available by the corresponding author upon reasonable request.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
