Competency requirements for the assessment of patients with mental illness in somatic emergency care: A modified Delphi study from the nurses’ perspective

Abstract

Patients suffering from mental illness are vulnerable, and they do not always have access to proper emergency care. The aim of this study was to identify competency requirements for the assessment of patients with mental illness by soliciting the views of emergency care nurses. A modified Delphi method comprising four rounds was used. Data were collected in Sweden between October 2018 and March 2019. The data were analyzed using content analysis and descriptive statistics. The panel of experts reached the highest level of consensus regarding basic medical knowledge: the capability to listen and show respect to the patient are essential competency requirements when assessing patients with mental illness in emergency care. Awareness of these competency requirements will enhance teaching and training of emergency care nurses.

Keywords

competency requirements emergency care emergency departments emergency medical services mental illness

Introduction

Mental illness is considered to be a global public health problem,^1,2 and patients with mental illness are a significant group in emergency care.^3–6 In this article, ‘mental illness’ is used as an umbrella term for a spectrum that includes both severe disorders/diseases and common mental health problems and/or mild symptoms of distress; examples include schizophrenia, anxiety, bipolar disorder, substance use disorder, depression, self-harm, sleeping difficulties, and distress. The term ‘emergency care’ refers to the care and treatment provided by nurses in emergency medical services (EMS) or emergency departments (EDs).

In emergency care, both registered nurses (RNs) and specialist nurses (SRNs) assess patients with diverse symptoms and conditions in order to determine the patients’ care needs.^7–9 Patient assessment involves identifying the immediate problem or detecting signs of a patient in distress and determining the acuity of their condition.¹⁰ The emphasis is typically on physical symptoms¹¹ and is generally based on vital signs, chief complaints, medical history, and the resources needed to treat and care for the patient.¹² Patient assessment is important because it forms the basis for the patient’s care and treatment during emergency care.¹³ However, as the patient’s condition may not always originate in physical discomfort,¹⁴ nurses also need mental healthcare competencies when assessing patients in emergency care.

In this regard, patients suffering from mental illness have reported that they cannot always access proper care and treatment in emergency care.^15–17 Stigmatization has a profound effect on these patients, creating barriers that undermine patient safety and provisions for mental and physical care.^18,19 In assessing patients with mental illness, nurses working in emergency care are reported to lack sufficient education or experience and may exhibit uncertainty,^20–22 manifested for example in frustration and anger toward patients who self-harm.²³ Consequently, there is a risk that these patients may not be appropriately assessed.

To ensure appropriate patient assessment, nurses must meet the competency requirements for assessing patients with mental illness. In this study, the term ‘competency requirements’ refers to nurses’ ability to manage work-related demands and situations and to understand the consequences of their actions.²⁴ Among these work-related demands, patient assessment includes situation awareness, history gathering, decision making, resource utilization, communication, and procedural skills.²⁵ However, the competency requirements considered essential in patient assessment are to some extent determined by institutions of higher education²⁰ and professional organizations, and are specified by national competency descriptions.^26,27

In summary, patients suffering from mental illness are vulnerable and do not always have access to proper patient assessment. Whether the patient is suffering from a disorder/disease, common mental health problems, or distress, nurses must have the requisite mental healthcare competencies to ensure appropriate patient assessment. Research indicates that there is a correlation between the level of nurses’ education and hospital mortality.²⁸ Therefore, there is a risk that inadequate teaching and training of emergency care nurses can influence the safety of patients. Simultaneously, the current knowledge gap makes it more difficult to specify the prerequisites for competence development and education planning. The aim of this study was to identify competency requirements for the assessment of patients with mental illness by soliciting the views of emergency care nurses.

Methods and design

A modified Delphi method was used to elicit views and reach a consensus among a panel of experts comprising emergency care nurses. This method assumes that the opinion of a group – in this case, a panel of emergency care nurses – is more valid than individual opinions.²⁹ The method’s structured and interactive group process is appropriate when existing knowledge of a research topic is limited.³⁰ The Delphi method involves repeated questionnaire rounds; responses from each round are analyzed, summarized, and returned to the expert with a new questionnaire. The process continues until consensus is achieved.³¹ This design was chosen since it is a way to identify and estimate competency requirements without requiring the experts to attend physical meetings. The anonymity among the experts was also a way of avoiding any bias that might be caused by the experts’ backgrounds and opinions.³² In this study, four rounds of questionnaires were considered necessary for the desired consensus level. However, there are no universally agreed guidelines for the Delphi method, and it can be adapted to the study’s aims.²⁹ The data collection was conducted between October 2018 and March 2019.

Recruitment

Round 1

This round was carried out in a region of Eastern Sweden. The following approach was used to recruit the experts: EMS or ED managers were contacted via an email that included information about the study. Nurses were verbally informed at workplace meetings about the study’s aim and procedures. Those interested in participating then advised their manager, who emailed the nurses’ contact details to the researchers. The researchers in turn emailed written information about the study to the interested nurses. Some days later, the researchers established contact with prospective participants by phone or email to ask whether they would consent to participate. The inclusion criteria were 1) formal education as an RN or SRN; 2) employment in EMS or ED; and 3) willingness to share their views on the competency requirements for assessing patients with mental illness. In total, 25 nurses who met the inclusion criteria gave their consent and were included in the study.

Rounds 2–4

These rounds were carried out in two regions of Eastern Sweden: the region included in Round 1 and another neighboring region. To recruit the experts, the following approach was used: EMS or ED managers were contacted via an email that included information about the study. The managers then emailed a web link for the questionnaire to their nurses. Interested nurses responded in the questionnaire directly to the researchers. The questionnaire (see ‘Data collection’) included information about the study; those who were interested in participating gave their consent by responding to the questionnaire. The inclusion criteria were the same as in Round 1.

Participants

Nurses working in emergency care are either RNs or SRNs, and their education differs accordingly. In Sweden, RNs’ education includes three years of university studies, leading to a Bachelor of Science degree in Nursing. RNs can then continue their education for one additional year to become SRNs, leading to a Postgraduate Diploma in Specialist Nursing Prehospital or In-hospital Emergency Care and a Master of Science degree with a major in Caring/Nursing Science.³³ However, there is no national requirement that emergency care nurses must be qualified as SRNs.

Round 1

The experts comprised 25 nurses: 16 female and 9 males, the age range was 32–61 years. Of these, 11 experts were working in EMS and 14 experts at EDs. Three of the experts had previous professional experience in psychiatric care (Table 1).

Table 1.

Overview of the participants’ demographics and the response rate between the rounds.

	Round 1 n	Round 2 n	Round 3 n	Round 4 n
Nurses	25	81	59	48
Women	16	54	38	31
Men	9	27	21	17
Dropout (Rounds 2–4)		6	16	11

Age (years)
Range	32–61	24–60	25–60	26–60

Employed
EMS	11	33	20	18
EDs	14	48	39	30

Professional experience (min–max years)
EMS	3–12	<1–25	<1–25	<1–20
EDs	<1–25	<1–20	<1–20	<1–20

Professional experience in psychiatric care
Yes	3	11	8	7

EMS: emergency medical services; EDs: emergency department.

Round 2

The experts comprised 81 nurses: 54 female and 27 males, the age range was 24–60 years. Of these, 33 experts were working in EMS and 48 experts at EDs. Eleven of the experts had previous professional experience in psychiatric care. There were dropouts in the panel of experts between Rounds 2 and 4. Of the original 81 experts, 48 (59%) completed all the rounds (Table 1).

Data collection

Round 1

Data for Round 1 were collected simultaneously with another study exploring assessments of the care needs of patients with mental illness in emergency care. For the preparation of the questionnaire, data were collected through individual interviews.²⁹ The interviews were conducted by an experienced research assistant and educated SRNs and were digitally recorded. The purpose of Round 1 was to collect statements about issues related to the study topic.³⁴ The decision to conduct open-ended interviews was based on a desire to understand the experts’ viewpoints and develop meaning from their experiences.³⁵ The interview questions were informed by theories of competencies³⁶ and occupational qualification.²⁴ The initial interview question was: ‘What competency requirements for the assessment of patients with mental illness are needed for nurses working in emergency care?’ Depending on the individual responses, probing questions were posed, for example, ‘What knowledge do you think is necessary?’ ‘What skills do you think are necessary?’ and ‘What attitudes do you think are necessary?’ After 25 interviews, no further variations were noted, and no further interviews were conducted. No transcripts were returned to participants for comment.

Rounds 2–4

Statements derived from the analysis (see ‘Data analysis’) of Round 1 were used to develop a questionnaire, which was designed using Survey Monkey^© web-based questionnaire software. The questionnaire was first run through a small test comprising six nurses in emergency care to investigate its feasibility and validity.³⁷ The test group was given the opportunity to provide comments and point out flaws in the questionnaire. This test resulted in adjustments in the formulation of categories in the questionnaire design. The data were then collected using Survey Monkey^©, and in each round, the panel of experts received an email link to the active questionnaire.

In Round 2, the experts’ demographic information was collected. The questionnaire consisted of 37 statements, and the experts were asked for their opinion as follows: ‘In your opinion, how important do you consider the following statement for your assessment of patients with mental illness in emergency care?’ The experts were asked to rate the importance of each issue on a seven-point Likert scale (from 1 = not important at all to 7 = very important). To capture additional competency requirements, at the end of the questionnaire, the experts were given the opportunity to describe, in free text, any additional knowledge, skills, attitudes, and/or experiences that they considered important for assessing patients with mental illness. The free-text comments were a way of highlighting further statements that were not previously noticed.

The questionnaire was available for 17 days. Of the 81 experts, 75 provided complete answers to the questionnaire. The six experts who did not answer the questionnaire completely were excluded from continuing their participation (Table 1).

In Round 3, the questionnaire was adjusted based on both the statements that did not reach consensus and the free-text responses in Round 2. New statements were formulated from the free-text responses and added to the questionnaire. Together with this, information about the no consensus statements’ median, minimum and maximum values from the previous round was provided, in order to supply the participants with the group’s average responses. The experts were then asked again to rate the issue’s importance on a seven-point Likert scale based on the abovementioned question, including the new statements. The questionnaire was available for 19 days. Of the 75 experts, 59 provided complete answers to the questionnaire. The dropouts were 16 experts; three reminders were sent by email; no dropout analysis was performed (Table 1).

In the final round (Round 4), the questionnaire was adjusted based on the statements that did not reach consensus and on the free-text responses in Round 3. New statements were formulated from the free-text responses and added to the questionnaire, and the experts were again asked to rate their importance on a seven-point Likert scale. Together with this, information about the no consensus statements’ median, minimum and maximum values from the previous round was provided, in order to supply the participants with the group’s average responses. The questionnaire was available for 12 days. Of the 59 experts, 48 provided complete answers to the questionnaire. The dropouts were 11 experts; three reminders were sent by email; no dropout analysis was performed (Table 1).

Data analysis

Round 1

Interviews were transcribed verbatim and were subjected to content analysis by authors JC and LK.³⁸ The analysis intended to capture the experts’ views about the competency requirements for assessing patients with mental illness.³⁹ Initially, all the transcripts were read several times to gain a sense of the whole text. In the second step, units of meaning, such as words, sentences, or paragraphs, were identified based on the study’s aim. The condensed units of meaning were then abstracted and labeled with codes.³⁹ Finally, the codes were compared for differences and similarities. Codes that appeared to deal with the same topics were arranged into 37 statements. These statements were then organized into three domains: knowledge, skills, or attitudes.²⁴

Rounds 2–4

The questionnaires in Rounds 2–4 were analyzed using Survey Monkey^© to provide statistical data, namely the mean and standard deviation. The experts again used a seven-point Likert scale to rate the statements. To analyze the level of consensus, the Likert scale was formatted into a three-grade scale (1–2 = not important, 3–5 = important, 6–7 = very important). The required level of consensus for each statement was defined when designing the study. The necessary condition was that, for each statement, a consensus percentage of at least 70% had to be achieved on one of the alternatives on the three-degree scale.²⁹

Ethical consideration

This study conforms to the ethical principles for research outlined in the Declaration of Helsinki⁴⁰ and adheres to Swedish laws and regulations concerning informed consent and confidentiality.^41,42 An advisory statement from the Regional Ethics Committee in Uppsala (Ref. 2018/005) was obtained prior to the study. All experts received both verbal and written information about the study before they gave their consent to participate.

Results

In Round 1, 131 meaning units were identified in the transcribed interviews. An example of meaning unit was ‘A capability to encounter people, to see and to talk. Most important is the capability to listen’. The meaning units were then condensed into 37 statements. The distribution of the 37 statements was as follows: knowledge 9, skills 17, and attitudes 11. In Round 2, 14 new statements were given by the panel of experts, and these were distributed as follows: knowledge 8, skills 5, and attitude 1. In total, there were 51 statements. For further information, see Tables 2 and 3.

Table 2.

The panel of experts’ views of competency requirements that reached consensus.

Domains of competency requirements	Statement	Mean¹	SD ¹
Knowledge	To have basic medical knowledge	6.10	1.11
	To have knowledge of what it means to be a human being	5.83	1.26
	To have knowledge in general psychiatric care	4.69	1.24
	To have experience working in psychiatric care	3.73	1.75
Skills	To be able to listen to the patient	6.63	0.71
	To be able to judge when a situation is unsafe or hostile to oneself and colleagues	6.55	0.82
	To be able to be calm in front of the patient	6.51	0.81
	To be able to read the situation wherein the patient encounter occurs	6.40	0.94
	To be able to communicate with the patient	6.29	0.93
	To be able to be empathetic	6.27	1.00
	To be able to be safe in your actions and encounters	6.25	0.99
	To be able to have self-insight	6.25	1.18
	To be able to interpret other people’s thoughts and behaviors	6.25	1.02
	To be able to create a calm atmosphere around the patient*	6.14	1.00
	To be able to adapt the approach to the patient	6.13	1.20
	To be able to use common sense in the patient encounter	6.08	1.40
	To be able to have patience with the patient	6.08	1.17
	To be able to socialize with the patient	6.05	1.35
	To be able to know oneself	5.89	1.46
	To be able to demonstrate that you are harmless to the patient	5.83	1.55
Attitude	To show respect to the patient	6.77	0.48
	To show a nonjudgmental attitude toward the patient	6.59	0.82
	To show basic human respect to the patient	6.57	0.77
	To show a willingness to encounter the patient	6.52	0.84
	To show an open-minded attitude toward the patient	6.39	0.98
	To show understanding toward the patient	6.37	1.01
	To show a nonevaluative attitude toward the patient	6.28	1.42
	To show a humble attitude toward the patient	6.25	1.03
	To show a helpful attitude toward the patient	6.21	1.17
	To show a willingness to give the patient time	6.05	1.28

¹ Values from Rounds 2–4.

*Competency requirements gained by panel of experts in Rounds 2–4.

Table 3.

The panel of experts’ views on competency requirements that did not reach consensus.

Domains of competency requirements	Statement	Mean¹	SD ¹
Knowledge	To have knowledge in legislation regarding nurses’ rights to protect themselves against physical attacks from the patient*	5.63	1.48
	To have knowledge in how to care for patients suffering from a crisis	5.57	1.09
	To have knowledge in legislation regarding compulsory psychiatric care*	5.47	1.32
	To have knowledge in legislation regarding nurse’s rights to protect a patient from hurting themselves*	5.39	1.25
	To have knowledge in acute psychiatric care	5.33	1.15
	To have knowledge in nonverbal communication*	5.27	1.48
	To have knowledge in conversation technology	5.00	1.34
	To have knowledge based on scientific evidence	4.98	1.46
	To have life experience	4.82	1.52
	To have knowledge in the suicide risk assessment tool*	4.73	1.75
	To have knowledge in how to draw conclusions about the patients’ condition based on experience*	4.24	1.72
	To have knowledge in psychiatric care based on one’s own experience as a relative*	2.08	1.88
	To have knowledge in psychiatric care based on one’s own experience of mental illness*	2.08	1.94

Skills	To be able to identify somatic illness at the same time as mental illness*	5.76	1.32
	To be able to assess the risk of the patient committing suicide*	5.63	1.51
	To be able to support the patient in his or her experience*	5.12	1.38
	To be able to handle violent patients	5.02	1.41
	To be able to support the patient in his or her conversation	4.92	1.43
	To be able to affirm the patient’s experience*	4.55	1.72

Attitude	To respect personal privacy*	4.98	1.75
	To show determination in caring for the patient	4.63	1.56

¹ Values from Rounds 2–4.

* Competency requirements gained by panel of experts in Rounds 2–4.

Based on Rounds 2–4 and the experts’ ratings of competency requirements, a consensus was reached on 30 statements. Competency requirements such as basic medical knowledge, being able to listen to the patient, and showing respect to the patient reached the highest level of agreement. The lowest level of agreement was on work experience in psychiatric care, demonstrating a harmless attitude toward the patient, and a willingness to give the patient time (Table 2).

However, some statements did not reach a consensus among the experts. Competency requirements such as knowing legislation regarding the right to protect against physical attacks from patients, identifying somatic illness at the same time as mental illness, and respecting personal privacy reached the highest agreement among the no consensus statements. The lowest agreement among the no consensus statements was on experience in psychiatric care as a patient or relative and skills such as being able to support the patient’s conversation or to affirm the patient’s experience (Table 3).

Discussion

The results identified three key areas of competency for assessing patients with mental illness in emergency care: 1) theoretical and practical knowledge; 2) communication skills; and 3) respectful attitudes.

Theoretical and practical knowledge

The results indicate that basic medical knowledge is essential when assessing patients with mental illness. This finding aligns with the traditional view of patient assessment in emergency care, which defines patient care needs on the basis of physical illness or injury.^14,43 According to previous research, basic medical knowledge can be understood as lifesaving and disease-oriented knowledge. This addresses the patient’s signs and symptoms, focusing on physiology, pathophysiology, and an understanding of diagnostic tests such as electrocardiography.⁴⁴ However, assessment must extend to all patients whatever their problems, which means considering differential diagnoses by examining signs and symptoms in an organized way.²⁵ Surprisingly, there was no consensus regarding knowledge of acute psychiatric care or the importance of being able to identify both somatic and mental illness. Although any explanation remains speculative, one possible reason is that patient assessment in emergency care is informed by a biomedical perspective.^43,45 This view is supported by existing evidence that education currently tends to focus on biomedical knowledge.²⁰ This emphasis on biomedical knowledge^11,44 may inhibit a more open and sensitive approach to patient assessment.⁴⁶ The results also identified work experience in psychiatric care as another key competence requirement, as patient assessment depends in part on previous experience of similar situations. According to Benner, experience is built on long-term professional activity and multiple patient encounters, but this does not always produce new knowledge.⁴⁷ There is evidence that reflection is of value to nurses as a means of developing new knowledge.⁴⁸ However, one important challenge presented by patient assessment is that the encounter is often rapid and short. This means there is limited time for reflection, and this may inhibit learning.⁴⁴

Communication skills

The study results indicate that it is essential for nurses to be able to communicate effectively with the patient by listening, creating a calm atmosphere around the patient, and interpreting the patient’s thoughts or behaviors. These results align with existing evidence that listening to the patient, reducing external stimuli (e.g. noise) and being able to understand the patient are crucial aspects of patient management.⁴⁹ The present results also align with earlier findings that acknowledging the patient’s experience is an important element of the encounter.⁷ However, there was no consensus regarding support or affirmation of the patient’s experience of mental illness or supporting their ability to participate in the conversation. Several previous studies have noted that patients who seek emergency care are dependent on nursing care and want be treated as persons with individual care needs.^50,51 By implication, an encounter that does not meet requirements such as listening may leave the patient feeling that they are not being taken seriously or that they are being rejected by the nurse.⁵²

Respectful attitudes

The present findings confirm that it is essential to show respect when assessing patients with mental illness by maintaining an open-minded, humane, and nonjudgmental attitude and a willingness to engage effectively with the patient. This makes sense from a caring perspective, as the encounter between nurse and patient is always influenced by the nurse’s commitment and their willingness to be open and sensitive to the patient and their vulnerability.⁵³ The present findings align with previous evidence that the quality of the encounter depends on the nurse’s professional relationship with the patient,⁵⁴ and nurses must be prepared to take responsibility for this relationship.⁵⁵ As a patient may present with signs or symptoms of mental illness, emergency care nurses must be sensitive to the patient’s privacy during assessment.⁵⁶ Surprisingly, there was no consensus here regarding respect for personal privacy despite previous research that identifies respect for personal privacy as an important part of emergency care.⁵⁷ However, there is also evidence that the content of the patient–nurse exchange does not always remain confidential, as the conversation can also be heard by others.⁵⁸ The lack of consensus in this regard may reflect the fact that patient assessment is not always conducted in an appropriate setting – for example, the space may be overcrowded.⁵⁹ These and other factors may influence nurses’ perceptions of personal privacy.

Limitations and strengths

In Round 1 of this study, the experts were interviewed about the competency requirements for assessing patients with mental illness. However, the terms ‘competency requirements’ and ‘mental illness’ may not be sufficient to specify the relevant issues.⁶⁰ For example, competency requirements might reflect the experts’ own experiences and education level rather than the work-related demands associated with patient assessment. Similarly, the term mental illness can be understood in different ways, ranging from distress to a specific diagnosis. The 70% consensus cut-off was arbitrary; however, the percentage was considered reasonable and supported by the literature.^31,61 In Rounds 1 and 2, prospective participants made their interest known to their manager, who then replied to the researchers. This approach entails a risk of bias and may have influenced the availability of participants;³⁷ for example, managers may not have circulated the request to all nurses, and potential participants may not have come to the researchers’ attention. This modified Delphi study involved a four-round approach. However, there was a significant dropout in this study. The Delphi technique can be time-consuming since considerable time can pass between rounds.⁶¹ It can be assumed that the number of rounds contributed to the dropout among the panel of experts. Since there are no strict guidelines regarding the appropriate number of rounds, it is possible that a three-round approach would be better. It is also possible that a more careful selection of appropriate experts in combination with a personalized invitation to this study might have stimulated a higher response rate.³¹ Another aspect is that the dropout rate might have influenced the rigor of this study, for example, experts with a minority opinion may not have contributed to the same extent as experts with a majority opinion.⁶² Therefore, the absence of a dropout analysis might be considered a limitation. The present findings do not conclusively establish the competency requirements for assessing patients with mental illness; nor do they provide the basis for a competency tool. While expert opinion is the lowest level in the hierarchy of evidence,⁶³ it does at least enhance our understanding of what should be considered when specifying competency requirements. After all, a strength is that all the nurses that participated in this study were active clinical nurses in their primary occupation. This means that there is an applicability of the results to practicing nurses in emergency care.

Conclusions

This study has identified theoretical and practical knowledge, communication skills, and a respectful attitude to the patient as essential competency requirements for assessing patients with mental illness. To ensure that patient assessment is adequate and appropriate, nurses’ mental health competencies must correspond to the requirements of their work. Awareness of the requisite competency requirements will help to advance the teaching and training of emergency care nurses; the present findings represent a first step toward specifying those requirements. However, further research is needed to develop a comprehensive account of the requisite mental health competencies for the assessment of patients with mental illness in somatic emergency care.

Footnotes

Acknowledgements

We want to express our most sincere thanks to all the nurses who shared their views on competency requirements for the assessment of patients with mental illness in emergency care. We would also like to sincerely thank Staffan Hammarbäck (RN MSc) for his contribution to the data collection in Round 1.

Author contributions

All authors meet the criteria for authorship and have approved the final manuscript. HA: study design, preparing the manuscript, interpretation, and critical revision of the manuscript for important intellectual content. JC: data collection, data analysis, and preparing the manuscript. LK: data collection, data analysis, and preparing the manuscript. MH: study design, data analysis, preparing the manuscript, interpretation, and critical revision of the manuscript for important intellectual content. All authors read, edited, and approved the final manuscript.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Conflict of interest

The authors declare that there is no conflict of interest.

ORCID iDs

Henrik Andersson

Mats Holmberg

References

World Health Organization. Mental health action plan 2013–2020. Geneva: WHO, 2013.

World Health Organization. World health statistics 2017: monitoring health for the SDGs, Sustainable Development Goals. Geneva: WHO, 2017.

Schönfeldt-Lecuona

Gahr

Schütz

, et al. Psychiatric emergencies in the preclinical emergency medicine service in Ulm, Germany in 2000 and 2010, and practical consequences. Fortschr Neurol Psychiatr 2017; 85: 400–409.

Owens

Mutter

Stocks

Mental health and substance abuse-related emergency department visits among adults. Rockville, MD: Agency for Healthcare Research and Quality, 2007.

Downey

Zun

Burke

Undiagnosed mental illness in the emergency department. J Emerg Med 2012; 43: 876–882.

Andrew

Roggenkamp

Nehme

, et al. Mental health-related presentations to emergency medical service in Viktoria, Australia. MBJ Open 2017; 7(Suppl 3): A1–A18.

Elmqvist

Fridlund

Ekeberg

More than medical treatment: the patient’s first encounter with prehospital emergency care. Int Emerg Nurs 2008; 16: 185–192.

Widgren

Jourak

Medical emergency triage and treatment system (METTS): a new protocol in primary triage and secondary priority decision in emergency medicine. J Emerg Med 2011; 40: 623–628.

Carter

Thompson

Defining the paramedic process. Aust J Prim Health 2015; 22: 22–26.

10.

Wolf

Acuity assignation: an ethnographic exploration of clinical decision making by emergency nurses at initial patient presentation. Adv Emerg Nurs J 2010; 32: 234–246.

11.

Andersson

Jakobsson

Furåker

, et al. The everyday work at a Swedish emergency department: the practitioners’ perspective. Int Emerg Nurs 2012; 20: 58–68.

12.

NAEMT. AMLS Advanced Medical Life Support. Burlington, MA: Jones and Bartlett Publishers, 2018.

13.

Widgren B. RETTS- Akutsjukvård direkt [RETTS-Emergency care directly]. Lund: Studentlitteratur, 2012.

14.

Swedish Agency for Health Technology Assessment and Assessment of Social Services. Triage methods and patient flow processes at emergency departments: a systematic review. Stockholm: SBU, 2010.

15.

Ferguson

Savic

McCann

, et al. ‘I was worried if I don’t have a broken leg they might not take it seriously’: experiences of men accessing ambulance services for mental health and/or alcohol and other drug problems. Health Expec 2019; 22: 565–574.

16.

Wise-Harris

Pauly

Kahan

, et al. ‘ Hospital was the only option’: experiences of frequent emergency department users in mental health. Adm Policy Ment Health 2017; 44: 405–412.

17.

Clarke

Dusome

Hughes

Emergency department from the mental health client’s perspective. Int J Ment Health Nurs 2007; 16: 126–131.

18.

Knaak

Mantler

Szeto

Mental illness-related stigma in healthcare: barriers to access and care and evidence-based solutions. Healthc Manage Forum 2017; 30: 111–116.

19.

Crowe

Averett

Glass

, et al. Mental health stigma: personal and cultural impacts on attitudes. JCP 2016; 7: 97–119.

20.

Sjölin

Lindström

Hult

, et al. What an ambulance nurse needs to know: a content analysis of curricula in the specialist nursing program in prehospital emergency care. Int Emerg Nurs 2015; 23: 127–132.

21.

McCann

Savic

Ferguson

, et al. Paramedics’ perceptions of their scope of practice in caring for patients with non-medical emergency-related mental health and/or alcohol and other drug problems: a qualitative study. PLoS One 2018; 13: e0208391.

22.

Clarke

Boyce-Gaudreau

Sanderson

, et al. ED triage decision-making with mental health presentations: a ‘think aloud’ study. J Emerg Nurs 2015; 41: 496–502.

23.

Rayner

Blackburn

Edward

, et al. Emergency department nurse’s attitudes towards patients who self-harm: a meta-analysis. Int J Ment Health Nurs 2019; 28: 40–53.

24.

Ellström

P-E.

The many meanings of occupational competence and qualification. J Eur Ind Train 1997; 21: 266–273.

25.

Tavares

Boet

Theriault

, et al. Global rating scale for the assessment of paramedic clinical competence. Prehop Emerg Care 2013; 17: 57–67.

26.

Swedish Emergency Nurses Association. Kompetensbeskrivning: Legitimerad sjuksköterska med specialistsjuksköterskeexamen med inriktning mot akutsjukvård [Competency description: qualified nurse with specialist nursing degree with specialization in emergency care]. Stockholm: The Swedish Society of Nursing, 2017.

27.

Swedish Ambulance Nurses Association. Kompetensbeskrivning: Legitimerad sjuksköterska med specialistsjuksköterskeexamen med inriktning mot ambulanssjukvård [Competency description: qualified nurse with specialist nursing degree with specialization in ambulance care]. Stockholm: The Swedish Society of Nursing, 2012.

28.

Aiken

Cimiotti

Sloane

, et al. The effects of nurse staffing and nurse education on patient deaths in hospitals with different nurse work environments. Med Care 2011; 49: 1047–1053.

29.

Keeney

Hasson

McKenna

The Delphi technique in nursing and health research. Chichester: Wiley-Blackwell, 2011.

30.

Clibbens

Walters

Baird

Delphi research: issues raised by a pilot study. Nurse Res 2012; 19: 37–43.

31.

Keeney

Hasson

McKenna

Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs 2006; 53: 205–212.

32.

Kennedy

Enhancing Delphi research: methods and results. J Adv Nurs 2004; 45: 504–511.

33.

Swedish Code of Statutes. Högskoleförordning (1993:100) [The higher education ordinance (1993:100)]. Stockholm: Ministry of Education and Research, 2019.

34.

Keeney

Hasson

McKenna

A critical review of the Delphi technique as a research methodology for nursing. Int J Nurs Stud 2001; 38: 195–200.

35.

Kvale S and Brinkman S. Den kvalitativa forskningsintervjun [The qualitative research interview]. Lund: Studentlitteratur, 2014.

36.

Eraut

Concepts of competence. J Interprof Care 1998; 12: 127–139.

37.

Polit

Beck

CT.

Nursing research: generating and assessing evidence for nursing practice. 10th ed. Philadelphia, PA: Lippincott Williams & Wilkins, 2016.

38.

Patton

MQ.

Qualitative research and evaluation methods. 4th ed. London: SAGE, 2014.

39.

Erlingsson

Brysiewicz

A hands-on guide to doing content analysis. Afr J Emerg Med 2017; 7: 93–99.

40.

World Medical Association. Declaration of Helsinki: ethical principles for medical research involving human subjects. Fortaleza: WMA, 2013.

41.

Swedish Code of Statutes. Lag (2003:460) om etikprövning av forskning som avser människor [Act on ethical review of research concerning people (2003: 460)]. Stockholm: Government Offices, 2003.

42.

Swedish Code of Statutes. Lag om ändring i lagen (2003:460) om etikprövning av forskning som avser människor [Amendments to the law 2003:460]. Stockholm: Government Offices, 2008.

43.

Clarke

Hughes

Brown

, et al. Psychiatric emergency nurses in the emergency department: the success of the Winnipeg, Canada, experience. J Emerg Nurs 2005; 31: 351–356.

44.

Andersson

Sundström

Nilsson

, et al. Competencies in Swedish emergency departments: the practitioners’ and managers’ perspective. Int Emerg Nurs 2014; 22: 81–87.

45.

Swedish Agency for Health Technology Assessment and Assessment of Social Services. Ätstörningar En sammanställning av systematiska översikter av kvalitativ forskning utifrån patientens, närståendes och hälso- och sjukvårdens perspektiv [Eating disorders: a compilation of systematic reviews of qualitative research from the perspective of the patient, relatives, and health care]. Stockholm: SBU, 2019.

46.

Wireklint Sundström

Dahlberg

Caring assessment in the Swedish ambulance services relieves suffering and enables safe decisions. Int Emerg Nurs 2011; 19: 113–119.

47.

Benner

From novice to expert: excellence and power in clinical nursing practice. Upper Saddle River, NJ: Pearson, 2000.

48.

Wihlborg

Edgren

Johansson

, et al. Reflective and collaborative skills enhance ambulance nurses’ competence: a study based on qualitative analysis of professional experiences. Int Emerg Nurs 2017; 32: 20–27.

49.

Shirzad

Hadi

Mortazavi

, et al. First line in psychiatric emergency: prehospital emergency protocol for mental disorders in Iran. BMC Emerg Med 2020; 20: 19.

50.

Nyström M. Möten på en akutmottagning. Om effektivitetens vårdkultur [Encountering at an emergency department: about efficiency care culture]. Lund: Studentlitteratur, 2003.

51.

Holmberg

Forslund

Wahlberg

, et al. To surrender in dependence of another: the relationship with the ambulance clinicians experienced by patients. Scand J Caring Sci 2014; 28: 544–551.

52.

Rantala

Ekwall

Forsberg

The meaning of being triaged to non-emergency ambulance care as experienced by patients. Int Emerg Nurs 2016; 25: 65–70.

53.

Arman

Ranheim

Rydenlund

, et al. The Nordic tradition of caring science: the works of three theorists. Nurs Sci Q 2015; 28: 288–296.

54.

Wireklint Sundstrom

Bremer

Lindström

, et al. Caring science research in the ambulance services: an integrative systematic review. Scand J Caring Sci 2019; 33: 3–33.

55.

Wireklint Sundström

Dahlberg

Being prepared for the unprepared: a phenomenology field study of Swedish prehospital care. J Emerg Nurs 2011; 38: 571–577.

56.

Emergency Nurses Association. Nursing code of ethics: provisions and interpretative statements for emergency nurses. J Emerg Nurs 2017; 43: 497–503.

57.

Torabi

Borhani

Abbaszadeh

, et al. Experiences of pre-hospital emergency medical personnel in ethical decision making: a qualitative study. BMC Med Ethics 2018; 19: 95.

58.

Nayeri

Aghajani

Patients’ privacy and satisfaction in the emergency department: a descriptive analytical study. Nurs Ethics 2010; 17: 167–177.

59.

Calleja

Forrest

Improving patient privacy and confidentiality in one regional emergency department: a quality project. Australas Emerg Nurs J 2011; 14: 251–256.

60.

Malterud

Qualitative research: standards, challenges, and guidelines. Lancet 2001; 358: 483–488.

61.

Hsu

Sandford

BA.

The Delphi technique: making sense of consensus. PARE 2007; 12: 10.

62.

Meijering

Tobi

The effect of controlled opinion feedback on Delphi features: mixed messages from a real-world Delphi experiment. Technol Forecast Soc Change 2016; 103: 166–173.

63.

Burns

Rohrich

Chung

KC.

The levels of evidence and their role in evidence-based medicine.

Plast Reconstr Surg 2011; 128: 305–310.