Abstract
Background:
Artificial intelligence-based decision support systems have been suggested as possible aids for decision makers in emergency care. To balance patient safety, cost, efficiency, and professional integrity, we need to understand the views and arguments health professionals have for and against the implementation of such systems. The current study aimed to explore emergency primary care personnel’s perspectives on artificial intelligence-based decision support systems in emergencies in the municipality.
Method:
This study used a qualitative design with online, semistructured interviews with 12 primary emergency health care personnel. A purposive sampling strategy was used, and participants were recruited either from EPC center or municipal healthcare institutions receiving services from the EPCRU. The data were analyzed following thematic analysis.
Results:
Four main themes were identified, namely, “Human need, clinical evaluations,” “Balancing skepticism and confidence,” “Digital sparring partner and alternative hypotheses,” and “Health personnel’s role in procurement and development.” The participants emphasized a need for human and clinical assessments to detect illness and initiate appropriate actions. Aspects of skepticism and confidence were also discussed. However, they perceived that AI-based decision support systems could be ideal digital sparring partners. Moreover, all the participants underlined the importance of involving intended users when developing and implementing decision support systems.
Conclusions:
This study emphasizes the essential role of health personnel in decision-making processes in emergency primary care, as well as in the implementation processes of AI in these services. AI is suggested as a supportive tool that provides safe and trustworthy solutions. These aspects are useful for managers and other decision makers in the transformation of health services for the future.
Background
As public health improves and populations worldwide are growing older, health systems face multiple challenges. 1 Quality care must be accessible for the entire population, and the cost must be kept at acceptable levels, despite an increasing shortage of health care professionals.1,2 As such, governments internationally struggle to find viable solutions to avoid the overcrowding of hospitals, including onsite diagnostics and prehospital emergency treatment. 3 However, prehospital care providers receive varied training, and the types of prehospital services vary across countries.4,5
Artificial intelligence (AI) and AI-based decision support systems have been suggested as possible aids to decision makers in prehospital care. 6 The World Health Organization (WHO) publicly supports the “science-based adoption of AI for health.” 7 The presumed benefit of using AI systems in acute care medicine is to reduce decision-making time and reliance on external expertise while increasing precision. 8 However, implementation is challenging, as clinically trained health personnel are expected to provide rationales for the decisions they make. 9 Furthermore, there are multiple barriers to the implementation of AI systems in health care in general and in prehospital care specifically. Typical concerns relate to data security, AI trustworthiness, operability, and interpretability. 10 Concerning interpretability, Chee et al 6 argue that although AI systems have clear advantages, health practitioners are reluctant to use them in the absence of a clear rationale and opt for solutions that perform worse but are easier to interpret.
In Norway, emergency care services outside hospitals are under the legal responsibility of municipalities. 11 Municipal emergency care is organized in 167 emergency primary care (EPC) centers, which also offer on-call services to the public. In addition, some regions have implemented emergency primary care response units (EPCRUs). The EPCRU team is at the forefront of balancing the needs of multiple stakeholders, and every decision comes with potential downsides. Undertriaging threatens patient safety, over-triaging increases cost and pressure in emergency rooms, and misdiagnosis leads to potential maltreatment.12,13 Research has shown that health personnel underperform under these types of contingencies. 14 To balance patient safety, cost, efficiency, and professional integrity, we need to understand the attitudes and arguments health professionals have for and against implementing AI-based decision support systems.15,16 The aim of the current study was to explore the perspectives of emergency primary care personnel receiving services from the EPCRU and EPCRU team members on AI-based decision support in emergency situations in the municipality.
Methods
Study Design
The study had a qualitative design using online, semistructured interviews with personnel directly involved in the EPCRU. The study adheres to the Consolidated Criteria for Reporting Qualitative Research (COREQ). 17
Setting
The study was conducted in an EPC region covering a medium-sized and small municipality in southeastern Norway. In October 2022, the EPC center introduced an EPCRU staffed by a selected and trained team of EPC nurses and an EPC physician with significant experience in general practice. The EPCRU operates from 08:00 to 22:00 on weekdays and from 11:00 to 21:00 on weekends and holidays. Whether the EPCRU is dispatched is based on the individual judgment of the EPC physician on call. Once on site, the EPCRU team diagnoses, triages, and, if necessary, provides emergency care. As in all primary care, this work is performed under time pressure and at times with limited access to information about the patient.
Participants
We used a purposive sampling strategy, recruiting participants through previously established professional networks in the EPC center or municipal healthcare institutions receiving services from the EPCRU. The last author (female nurse anesthetist and professor) contacted managers at the EPC, a nursing home and a home nursing service, who agreed to recruit participants according to the inclusion criteria. The inclusion criteria were “nurses,” “health personnel,” or “physicians” working at the EPCRU or working in healthcare institutions receiving services from the EPCRU and having experienced the EPCRU services. There were no exclusion criteria.
The managers then shared email addresses with personnel willing to participate with the last author (unfamiliar to the individual participant in advance). The participants who agreed to participate were linked to one of the interviewers (1 male nurse/PhD, 2 female nurses/PhD, all acknowledged with interviewing, as well as a nurse working at the EPC/master’s student who was unexperienced with interviewing), who set up the time and place for the interview.
In total, 4 nurses and 2 physicians from the EPC, 4 nurses from the nursing home and 2 nurses from the home nursing service participated. Of these, 4 were male, their ages ranged from 30 to 55 years, and their experience as nurses or physicians ranged from 1 to 30 years.
Data Collection
An interview guide was developed in collaboration with the research team on the basis of background literature6,8,9 and clinical experience (as a registered nurse, nurse anesthetist, and ambulance nurse; Supplemental Material). The interviewers were all licensed nurses with wide experience in the field. The interviewers were also experienced interviewers, apart from 1 interviewer who was conducting the interviews under supervision of an experienced supervisor. There were no personal or professional relations between interviewers and participants of the study. The input was also provided by a medical doctor working at the EPCRU. The overall topic of the interviews was emergency primary care personnel’s attitudes toward the possible future introduction of decision support systems based on AI (hereafter “AI-based decision support systems”). AI-based decision support systems were described as “Decision support systems based on large amounts of research-based data, that can help health professionals make decisions in various situations. The system will calculate what is most likely the correct assessment in each situation, on the basis of the information made available.” The interview guide included questions related to perceived positives and negatives, who should develop AI systems, perceived levels of possibilities for participation in development, and how AI systems should be utilized. The interview guide was piloted with an ambulance worker with experience from the EPCRU, who reported the guide to be relevant and understandable.
The interviews were either conducted digitally (Teams®) or in the participant’s workplace in an isolated office, as decided by the participant. The interviews were conducted between November 2023, and February 2024.
The interviewers were allowed to ask follow-up questions and provide clarifications. The interviews lasted between 45 and 60 min and were audio recorded via the University of Oslo’s “Nettskjema” dictaphone application, which also provides automatic transcriptions.
Data Analysis
The automated transcriptions were edited by the interviewers while listening to the recordings to ensure correspondence. The transcriptions were coded and analyzed manually, following thematic analysis. 18 In step 1 (Familiarization), transcriptions were further checked against the audio recordings by the first author. In step 2 (coding), 3 transcriptions (P1, P2, and P3) were pilot coded by the first author and discussed with last author. When consensus was achieved, the remaining transcriptions were coded on by one by the first author. Codes from all transcripts were then discussed with last author until consensus was achieved. Discussions included attempts to identify and avoid biases grounded in professional backgrounds, the last author being a licensed nurse, and the first author being a philosopher of science. These codes were subsequently compared across the dataset to identify subthemes related to the same topics (step 3, Searching for themes). On this basis, the material was recoded into 126 codes, which were organized under 8 themes. Initially, several codes were treated as representative of multiple themes and eventually placed under the theme that best captured the main content of the verbatim. In step 4 (Reviewing themes), the themes were discussed between the first and last authors, and 4 main themes were identified. Main themes were chosen from 2 main criteria, (1) Themes that were discussed by the most participants, and (2), themes that emerged without having been part of the interview guide but were surprisingly often discussed. For instance, the theme “digital sparring and alternative hypotheses” was chosen from this criterion. Step 5 comprised the final definition and naming of the themes along with discussions concerning saturation. This step included all the authors. Saturation was assumed to have been achieved as no new topics presented themselves in new interviews, and reanalysis indicated no further or more informative themes. 19
To ensure confidentiality and anonymity, all participants were assigned codes (P1-P12). The study was evaluated by the Norwegian Agency for Shared Services in Education and Research (ref: 952723), and an Institutional Review Board for research ethics.
Results
In total, 4 main themes were identified, namely, “Human need, clinical assessment,” “Balancing skepticism and confidence,” “Digital sparring partner and alternative hypotheses,” and “Health personnel’s role in procurement and development.” Table 1 provides an overview of the themes and selected quotes.
Overview of Themes and Selected Quotes.
The Need for Human Clinical Assessment
A common theme among all participants, whether generally positive or negative to the possible use of AI-based decision support systems, was that human assessment is crucial in all clinical encounters. There is a shared concern that if healthcare personnel rely too much or exclusively on AI-based decision support systems, the quality of services will suffer. The participants reported at least 2 different types of quality concerns in relation to this. The first, and most prevalent, was the concern that information is lost when a professional reduces the overall assessment to bits of information that are suitable for the support system. For instance, P10 stated: You take away the human dimension, but you also reduce a human being, which is an unimaginably complex system, to something that is easily understood and easily digestible. However, this means that you miss a lot of important things, and that is why I do not think you will ever be able to do something that provides proper quality, if you do not go out to the patients and touch them, and above all look at the context.
Some participants worried that no matter how detailed and precise their descriptions were, they would never be able to fill in all the relevant information. Important information might then be lost in the data entry process, which influences the support system’s assessment of the situation.
A second concern was the knowledge level of the healthcare personnel themselves. The participants reported not expecting all healthcare personnel to have a deep and detailed understanding of the inner workings of algorithms. Therefore, they expressed concerns that healthcare personnel will have to start treatment for which they cannot provide explanations. This was illustrated by, for example, P8: . . . healthcare personnel should not start making decisions based on something they do not understand themselves
For the participants, this was a concern not only for the quality of the treatment provided but also for questions concerning responsibility and accountability in general. Overall, the participants insisted that if AI-based decision support systems are introduced, they must be treated as suggestion makers rather than as providers of “correct solutions.”
Balancing Skepticism and Confidence
Skepticism and confidence issues were common themes among the participants, and they presented arguments from both perspectives. There was, for example, a concern that it would be more difficult to detect mistakes, that there could be low-quality or biased input in the development of the support system, and that practitioners might develop a false sense of security while simultaneously losing some of their clinical skills. On the other hand, several of the participants focused on the benefits of receiving suggestions that are emotionally detached and that mistakes will occur whether decisions are based on support systems or healthcare personnel’s assessments. Typically, some participants were preoccupied with the question of whether the support system would perform better than available experts in the field did. P7 speculated that “Perhaps there will be even fewer misjudgments.”
When discussing trust and safety issues, most participants insisted that any source of knowledge must be treated with some level of skepticism and that any decision must ultimately depend on the person who is there, in the situation, with the patient. Most participants valued the possibility of alternative views and approaches to increase safety without fully trusting the support system. P7 summarized the situation: We are human, and it is human to make mistakes. Therefore, there I think maybe the percentage will decrease, with regard to the physician making his assessment. The nurses make their assessment, the private individual makes their assessment, and then they also obtain help from an artificial intelligence that either confirms or says something different.
Digital Sparring and Alternative Hypotheses
Most participants found potential benefits in using AI-based decision support systems; specifically, the topic of digital sparring partners arose in the majority of the interviews. The general idea expressed by the participants was that support systems are valuable conversation partners that can help validate a specific diagnosis or course of action. For instance, P1 said, “If AI says the opposite and disagrees with me. Then, maybe I should make a new assessment,” implying that disagreement motivates further investigation.
Notably, several of the participants insisted that the suggestions from the support system must be treated as suggestions and not as solutions. The participants maintained their evaluation of a clinical gaze as the ultimate decider but were open to suggestions from a potential support system. A version of this, discussed by many participants, was to use the support system for generating alternative hypotheses that the clinician did not think of but would consider if it were suggested. The idea would then be that the clinician and the support system make independent assessments and that whenever there is divergence between these assessments, the clinician can reassess and come up with a better solution than she would on her own. As P5 put it, If you’re the kind of person who gets very stuck on a previous diagnosis, for example, and thinks that [the previous diagnosis] could be distorting your evaluation, and preventing the consideration other possibilities, then it might be a good idea that you get some input on tentative diagnoses.
In general, most participants agreed that there are possible benefits to receiving alternative suggestions from an AI-based decision support system. Furthermore, some participants focused on the reassurance of finding that the support system agreed with their assessment. Some participants also focused on the potential learning outcomes of working with a support system, where the main idea was that new employees or inexperienced workers can learn and find reassurance in AI-based suggestions. P3 stated: Unskilled workers who visit different places in the municipality and apprentices who visit here with us might benefit greatly from such a tool when they come in and take measurements of the patient. Maybe they can use this to learn as well, not only in emergency situations but also to try to learn things without having to explain or read them in a book, which can be heavy.
Overall, the participants tended to think of AI-based decision support systems as a potential form of digital sparring partner or colleague that can provide reassurance, guidance, and alternative hypotheses.
Health Personnel’s Role in Procurement and Development
When asked directly about their opinions about who should be involved in the procurement and development process, the participants unanimously argued that healthcare personnel should be involved at all stages. The primary motivation given was that only health care professionals know what types of decision systems they need, what kinds of decisions they find difficult, and what to avoid. As P8 put it, It is very important [that healthcare personnel are involved] because healthcare personnel know how they would use it and what is truly needed and what will be easiest to use on a hectic workday. Only health professionals themselves know this.
Many of the participants argued against selecting health care professionals on the basis of education level. Instead, it was suggested that every AI-based decision support system should be developed in collaboration with practitioners who use the system and who are in direct contact with patients since they are the ones who know what, where and how of the clinical encounter. In general, participants focused on the different types of knowledge involved in theoretical knowledge, data-handling, and programming skills, and practical skills when directly involved with a patient. If this type of knowledge was excluded from the procurement and development process, the participants feared that the resulting decision support systems would be left unused. As P10 said, “If people are to want to use it, then there must also be someone who is a user who helps develop the technology.”
Regrettably, most participants were under the impression that it is difficult to be included in such processes and that decisions are typically made far from practice by people without direct involvement in relevant situations. P10 said, “It is a bit like a problem with the health care system in general; it is controlled by someone who does not work in the field, and then you see how it goes.”
Discussion
Emergency care personnel involved with the EPCRU emphasized the need for human and clinical evaluations to detect illness and initiate appropriate actions. Aspects of skepticism and confidence were also discussed, reflecting concerns about errors in AI-based decision support systems, the loss of clinical skills, and the potential benefits of AI-based decision support systems. However, the participants perceived that AI-based decision support systems could be ideal digital sparring partners. Moreover, all the participants underlined the importance of involving intended users when developing and implementing AI-based decision support systems. Our results may guide future implementation of AI in primary healthcare.
The main concern relating to information loss and loss of contextual sensitivity was related to the fundamental difference between an inherently qualitative observation situation and the reduction of that situation into bits of information that can work as data input. These worries are contrasted by general decision-making theory, which, among other issues, focuses on how humans irrationally reduce complexity through heuristics, tend to give unreasonable credence to their initial assessment, and overemphasize the importance of information that is immediately available.20,21 Several studies also indicate that algorithms’ diagnostic performance is generally comparable to that of clinicians.22,23 Indeed, some of the motivations for introducing AI-based decision support systems are to mitigate human errors under pressure. 24 However, to achieve this our results indicate a need to meet healthcare personnels perceptions and attitudes.
Our results point toward a clear preference for decision support systems as supports, while final decisions are left to clinicians. This is in line with findings from, for example, Khan et al, 25 who reported that ethical, social, data collection, and algorithm, and clinical implementation concerns could be possible drawbacks of implementing AI in healthcare. A possible issue is that the irrational aspects of human reasoning under pressure also affect the data-input process, thus exacerbating the main problems. Recent studies have argued that using AI-driven triaging systems can result in major benefits “ . . . including improved patient prioritization, reduced wait times, and optimized resource allocation,” 26 whereas others maintain that there is still much to learn before AI-based decision support systems are fit for triaging. 27 Whatever the performance of future AI-based decision support systems, it is still unclear what role they should play in the clinical decision-making process. Hence, this needs to be further explored in order to implement AI in primary healthcare services in the future.
One motivation for making final decisions to clinicians is that some AI-based decision support systems provide suggestions without thorough explanations. In effect, the clinician might then start a course of action that they cannot motivate or explain. This was seen as a threat to the quality of the intervention, as well as to the accountability and responsibility of the clinician. Accountability, explainability, and responsibility in connection with AI-based decision support systems are thoroughly debated in the literature on AI and health, and there are no clear-cut answers.28,29 Traditionally, clinicians have had to answer for decisions made in clinical encounters. When a AI-based decision support system is used, the clinician must rely not only on the reliability of their own data input but also on the programming performed by the technician responsible for the algorithm behind the decision support system. It is not apparent how this affects responsibility and accountability. 30
A further concern among the participants was that once errors are made, they might be harder to detect when AI-based decision support systems are used. A reason for this could be that once the algorithm is trained on a specific dataset, the suggestions made by the AI-based decision support system will be systematic. In other words, similar evaluations will be made for similar situations. Moreover, although this ensures lower levels of variability, it could also conceal systematic errors. This kind of concern mirrors the concerns of systematic biases in AI-based systems in general and the worry that such systems “ . . . have the potential to perpetuate and even amplify existing biases, particularly those related to race, gender, and other societal constructs.” 31
One trend of research attempting to address these potential negative aspects of AI-based decision support systems is the development of explainable AI systems that reduce the black box of decisions and increase transparency. Such systems could ideally “ . . . help to ensure that patients remain at the center of care and that together with clinicians they can make informed and autonomous decisions about their health.” 32 Additionally, as argued by our participants, the errors and lack of transparency in AI-based systems should be evaluated against errors and lack of transparency in human decision-making in general.
Many participants saw the potential benefit of AI-based decision support systems in terms of digital sparring partners and generators of alternative hypotheses that the clinicians themselves might not initially consider. This approach is well established as a beneficial strategy for counteracting confirmation bias and is a central aspect of scientific thinking. 33 When practitioners are presented with specific information about alternative hypotheses, their willingness to seek alternative explanations increases. 34 If used in the way participants suggest, that is, as support systems rather than decision-makers, AI-based decision support systems might alleviate rather than perpetuate problems related to healthcare bias. This is in direct opposition to the worries presented in Ferrara 30 above. As such, for example, Ghafarollahi and Buehler 35 presented an example of how human-AI collaboration can be used to generate hypotheses.
All the participants in our study insisted that health personnel should be involved in the procurement, development, evaluation, and refinement of AI-based decision support systems for health care. Many of the experts expressed that there was little involvement from health personnel in the current framework. This mirrors the findings of Khan et al, 36 who argue that health personnel’s involvement is limited and call for a more comprehensive and inclusive framework for stakeholder involvement. Three Finnish studies exemplify how complicated involvement can be: Martikainen et al 37 surveyed 2484 Finnish physicians who reported that they experienced health personnel as largely absent from all aspects of the procurement and development process, as well as being neglected when presenting feedback. A survey of developers, however, revealed that developers experience strong involvement from health professionals and that this involvement is central to development and feedback processes. 38 It is therefore not clear that stakeholders have a similar understanding of what involvement implies and how procurement and development processes should unfold. These issues are central when aiming at implementing AI in primary healthcare services in the future.
Although important, patient involvement in these processes was not identified in our interviews, which illustrates the difficulties concerning who, how, and what is to be considered relevant for stakeholder involvement in the development of health technologies. 39
Limitations and Strengths
The results of this study are not generalizable because of the qualitative design. Moreover, transferability may be questioned due to the nature of the EPCRU, which includes participants across primary care services and professional backgrounds. However, this may also be seen as a strength due to the similarities across the sample, indicating a maximum variation. 40 Also, such units may differ both between regions nationally and in primary healthcare internationally.
One of the interviewers was inexperienced with the interviews; however, it was supervised by an experienced qualitative researcher to support the validity and rigor. Additionally, several interviewers were involved, and the interview locations varied (physically or digitally). Heath et al 41 suggested that flexibility regarding which participants are interviewed may improve participant access to research, recruitment, and response rates. The sample is also limited, although the interviews were information rich and adequate.
A transparent presentation of the analysis and quotes may increase the trustworthiness of the study.
Conclusion
This study emphasizes the essential role of health personnel in decision-making processes in emergency primary care, as well as in the implementation processes of AI in these services. AI is suggested as a supportive tool, providing safe and trustworthy solutions. These aspects are practical for managers and other decision makers in the transformation of health services for the future.
Supplemental Material
sj-docx-1-jpc-10.1177_21501319251408672 – Supplemental material for Emergency Primary Care Personnel’s Perspectives Toward AI-Based Decision Support Tools: A Qualitative Study in Norway
Supplemental material, sj-docx-1-jpc-10.1177_21501319251408672 for Emergency Primary Care Personnel’s Perspectives Toward AI-Based Decision Support Tools: A Qualitative Study in Norway by Fredrik Andersen, Stine Eileen Torp Løkkeberg, Camilla Hardeland and Ann-Chatrin Linqvist Leonardsen in Journal of Primary Care & Community Health
Footnotes
Acknowledgements
Tommy Utgaard is acknowledged for conducting interviews with 6 of the participants. Vivian Nystrøm (PhD experienced with qualitative studies) is acknowledged for supervising Utgaard in this process.
ORCID iDs
Ethical Considerations
The work was evaluated by the Norwegian Agency for Shared Services in Education and Research (reference number 952723). The study was conducted in line with the Declaration of Helsinki and was based on anonymity and confidentiality (World Medical Association. According to Norwegian legislation, no ethical approval is needed when health personnel are interviewed.
Consent to Participate
All participants provided written informed consent to participate.
Consent for Publication
Not applicable.
Author Contributions
ACLL conceived the study. SL and CH acquired the data. FA and ACLL performed the initial analysis, in which all the authors discussed and agreed upon the final draft. FA wrote the initial draft of the manuscript, with contributions from ACLL. All the authors (FA, SL, CH, and ACLL) critically reviewed the manuscript and are accountable for the submitted version.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was funded by Interreg Norway-Sweden. The funders did not take part in planning or conducting the study, interpreting the results or writing the paper.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
The datasets generated and analyzed during the current study are not publicly available due to confidentiality issues and being in Norwegian. However, they are available from the corresponding author upon reasonable request.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
