Knowledge and perception of primary care healthcare professionals on the use of artificial intelligence as a healthcare tool

Abstract

Objective

The rapid digitisation of healthcare data and the sheer volume being generated means that artificial intelligence (AI) is becoming a new reality in the practice of medicine. For this reason, describing the perception of primary care (PC) healthcare professionals on the use of AI as a healthcare tool and its impact in radiology is crucial to ensure its successful implementation.

Methods

Observational cross-sectional study, using the validated Shinners Artificial Intelligence Perception survey, aimed at all PC medical and nursing professionals in the health region of Central Catalonia.

Results

The survey was sent to 1068 health professionals, of whom 301 responded. And 85.7% indicated that they understood the concept of AI but there were discrepancies in the use of this tool; 65.8% indicated that they had not received any AI training and 91.4% that they would like to receive training. The mean score for the professional impact of AI was 3.62 points out of 5 (standard deviation (SD) = 0.72), with a higher score among practitioners who had some prior knowledge of and interest in AI. The mean score for preparedness for AI was 2.76 points out of 5 (SD = 0.70), with higher scores for nursing and those who use or do not know if they use AI.

Conclusions

The results of this study show that the majority of professionals understood the concept of AI, perceived its impact positively, and felt prepared for its implementation. In addition, despite being limited to a diagnostic aid, the implementation of AI in radiology was a high priority for these professionals.

Keywords

Artificial intelligence knowledge perception survey technology digital health primary care

Introduction

Artificial intelligence (AI) is defined as ‘the ability of machines to mimic human behaviour’.¹ It is a set of mathematical models, expressed in the form of algorithms, capable of learning and analysing large amounts of data, in different formats and at high speed, to help in the accuracy and speed of diagnosis, facilitate and streamline clinical care, and support public health interventions, among many other applications in the field of health.^2,3 Despite being recent concepts, the rapid digitisation of healthcare data and the sheer volume being generated means that artificial intelligence (AI) is becoming a new reality in the practice of medicine.⁴

Currently, AI is already able to support in patient triage, suggest diagnoses and result alerts,⁵ process natural language to automate clinical documentation by text summarisation,⁶ make diagnoses from radiological images⁷ or drawings such as Archimedes’ spiral,⁸ etc. Several studies have demonstrated the potential usefulness of these AI algorithms in many medical specialties such as ophthalmology,⁹ dermatology,¹⁰ neurology¹¹ and radiology,¹² the latter being one of the healthcare fields where AI has had a greater impact, especially in chest radiographs.^13,14

Despite being one of the most widely used typologies,^15,16 chest radiography is a complex modality to interpret¹⁷ and often creates discrepancies between the radiologist's and family physician's assessment.^18,19 The region of Central Catalonia has the particularity of having few radiologists. As a result, primary care (PC) physicians are increasingly required to interpret X-rays. For this reason, the use of AI algorithms capable of interpreting thoracic radiographs approaching the efficiency of a radiologist could represent a great benefit in the clinical setting as a diagnostic support tool.

Knowing how healthcare professionals perceive the use of AI in the healthcare setting is of vital importance in order to tailor the technology to the needs of professionals, align it with their values and make it easy to use.^20–23 Several studies have detected hope and optimism among healthcare professionals in its application, but also some concern about its ethical implications, lack of knowledge and training on its use, which may contribute to some reluctance among professionals.^24–26

Diaz et al.,²⁷ through a survey, observed that the majority of healthcare professionals agree that AI will improve their day-to-day lives, but that work needs to be done on the need for specific AI training. An integrative review by Shinners et al.²⁸ showed that healthcare professionals were less likely to use AI if they did not trust the technology or understand how it was used to improve patient outcomes. The study by Sit et al.²⁹ demonstrated that students were aware of the importance of AI in their careers and selected the radiology specialty less due to the perception that AI would replace them in the future. Thus, it is necessary to introduce AI directly into the academic training of healthcare professionals so that they can enter the working world with knowledge of this tool.^30,31

Jha et al.,³² when asked about the impact of AI in primary healthcare information systems, observed that physicians were sceptical about the ability of AI to perform better than humans. Another study³³ found that healthcare professionals share concerns about data management, knowledge development, deterioration of the doctor–patient relationship, and alteration of the diagnostic and decision-making landscape.

AI applied to healthcare has great potential, although analysing the perception of stakeholders is key to ensuring its correct implementation. Thus, the aim of this study is to describe the perception and knowledge of PC healthcare professionals on the use of AI as a healthcare tool, and its impact in radiology, through a validated survey. The results obtained will be essential to develop strategies aimed at increasing the acceptance of AI use and reducing possible resistance to change.

Methods

Study design and sample

Observational cross-sectional study aimed at all PC medical and nursing professionals in the Central Catalonia Health Region. This region includes the counties of Anoia, Bages, Berguedà, Moianès and Osona, and has approximately 1000 professionals. The survey was sent via corporate email to all these professionals up to three times to obtain the maximum response rate. The period during which the survey was active was from 2 November 2022 to 31 January 2023. The survey was open to all PC medical and nursing professionals of the Catalan Institute of Health who were over 18 years of age and accepted the invitation they received by email. Responses from other professional categories have been excluded.

A minimum of 283 surveys, distributed in the study region were required, in order to estimate with a 95% confidence and a precision of 0.1 points, the values of the two dimensions of the questionnaire, assuming a standard deviation of 1 point.²²

Sources

A validated survey known as Shinners Artificial Intelligence Perception (SHAIP)³⁴ was used. The survey deals with two dimensions defined in order to describe Professionals impact of AI and Preparedness for AI with 10 questions using the Likert scale (1 means strongly disagree and 5 means strongly agree). Additionally, there are two descriptive multiple-choice questions about the challenges of incorporating AI and the training they would like to receive, and an initial part was added to obtain sociodemographic information (sex, age, professional category and years of professional experience), as well as questions related to knowledge of the concept of AI, one closed and the other open, training and/or preparation for it, and its impact on radiology.

Permission was requested from the author to use it and translate it into Catalan. Two researchers translated it separately and pooled it, and a third helped to reach consensus in cases of discrepancies in translation. The survey was sent to health professionals in digital format.

Statistical analysis

Categorical variables have been described with absolute frequency and percentage, and continuous variables with mean and standard deviation. Factor analysis (FA) and Cronbach's alpha was used to determine the validity and reliability of two dimensions. In general, Cronbach's alpha of 0.7 is taken as indication of good internal consistency. In some cases, an alpha of 0.5 or 0.6 can be acceptable.^35,36 For the bivariate analysis between the dimensions and the sociodemographic variables, the Student's t-test or analysis of variance with multiple comparisons was used. The analyses were performed with R statistical software version 4.2.1, and the significance level was set at 5%.

Ethical considerations

The study protocol was approved by the University Institute for Primary Care Research (IDIAP) Jordi Gol Health Care Ethics Committee (Code 20/177-PCV). The survey was anonymous, so the patient consent was not necessary.

Results

The survey was sent to 1068 health professionals, of whom 301 responded, obtaining a response rate of 28.2%.

Of the professionals who responded to the survey, the mean age was 46.0 years (SD = 11.0) with 81.1% female, 47.8% were nursing professionals and 48.5% were medical professionals, the vast majority with more than 10 years of professional experience in their current job (62.5%) (Table 1).

Table 1.

Descriptive analysis of the sample.

	Total sample (N = 301)
How old are you, in years (N = 292)**	46.0 (11.0)*
Categorised age (N = 292)
[18–30)	22 (7.53%)
[30–40)	62 (21.2%)
[40–50)	101 (34.6%)
[50–60)	70 (24.0%)
> = 60	37 (12.7%)
What is your gender?
Female	244 (81.1%)
Male	57 (18.9%)
What health discipline do you work in?
Nursing	144 (47.8%)
Medicine	146 (48.5%)
Others	11 (3.65%)
How many years of experience do you have in your job?
Less than 1 year	11 (3.65%)
Between 1 and 2 years	24 (7.97%)
Between 3 and 4 years	43 (14.3%)
Between 5 and 10 years	35 (11.6%)
More than 10 years	188 (62.5%)
Do you understand what the concept of AI means?
No	43 (14.3%)
Yes	258 (85.7%)
In your work do you use AI to perform or plan care?
No	99 (32.9%)
I do not know	71 (23.6%)
Yes	131 (43.5%)
AI education or training:
None	198 (65.8%)
Self-starting online courses, webinars, conferences, etc.	63 (20.9%)
Job training	30 (9.97%)
University education	10 (3.32%)
Would you like to receive education on AI in health?
No	26 (8.64%)
Yes	275 (91.4%)

Absolute frequency and percentage; n (%).

* Mean and standard deviation. ** Only 292 professionals answered the age correctly.

AI: artificial intelligence.

With regard to the use and knowledge of AI, 85.7% indicated that they understood its concept but saw discrepancies in its use due to the execution and/or planning in their work; 43.5% indicated that they used it, 32.9% that they did not and 23.6% that they did not know. More than half of the respondents indicated that they had not received any training on AI (65.8%) and the vast majority responded that they would like to receive it (91.4%) (Table 1).

Figure 1 describes the main challenges of incorporating AI into the workplace and topics they would like to know more about. On the one hand, the main incorporation challenges were the knowledge and skills of the professional staff (58.5%), interoperability with current systems (48.2%), infrastructure (46.8%), implementation cost (44.2%) and organisational support (43.2%). On the other hand, the main training topics were AI application in healthcare (85.0%), AI ethics in healthcare (50.8%) and general education on AI capabilities (36.2%).

Figure 1.

Challenges of incorporating artificial intelligence into the workplace and topics they would like to know more about.

Across the SHAIP questionnaire items (1: strongly disagree and 5: strongly agree), two dimensions were obtained; professional impact of AI as the mean of the individual scores of questions 1 to 6, and the preparedness for AI as the mean of the individual scores of questions 7 to 10. In a preliminary analysis, we examined the Cronbach's alpha estimation of internal consistency and reliability of two dimensions. The Cronbach's alpha estimation of professional impact of AI was 0.84 (95% confidence interval (CI) 0.79–0.88) which suggests that items had very little variance specific to individual items. The Cronbach's alpha estimation of preparedness for AI was a 0.56 (95% CI 0.46–0.65). Figure 2 shows the results of the FA to validate the two dimensions of survey with study data and study language.

Figure 2.

Factor analysis to validate dimensions of questionnaire.

Table 2 describes the two dimensions related to the professional impact of AI and the preparedness for AI. Professionals scored different items (on a Likert scale) related to each dimensions. The mean score of responses on the professional impact of AI was 3.62 points (standard deviation (SD) = 0.72) out of 5 on the Likert scale. Thus, they consider that AI will improve their workflow, population health indicators and the sustainability of the healthcare system. Specifically, the two items with the highest scores meaning that they agree were ‘I believe that the use of AI in my specialty could improve clinical decision making’ (3.87 out of 5 mean score with SD = 0.87) and ‘I believe that AI can improve population health indicators’ (3.81 out of 5 mean score with SD = 0.86). In contrast, the two lowest scoring items indicated that they were unsure were ‘I believe that one day AI will be able to be part of my job as a healthcare professional’ (3.40 out of 5 mean score with SD = 1.10) and ‘I believe that the introduction of AI can reduce the economic cost associated with my work’ (3.37 out of 5 mean score with SD = 097) (Table 2).

Table 2.

Descriptive analysis of the items and dimensions.

	Total sample (N = 301)
Professional impact of AI	3.62 (0.72)
(1) I believe that the use of AI in my specialty could improve patient care.	3.78 (0.95)
(2) I believe that the use of AI in my specialty could improve clinical decision making.	3.87 (0.87)
(3) I believe that AI can improve population health indicators.	3.81 (0.86)
(4) I believe that AI may change my role as a healthcare professional in the future.	3.51 (0.98)
(5) I believe that the introduction of AI can reduce the economic cost associated with my work.	3.37 (0.97)
(6) I believe that one day AI will be able to be part of my job as a healthcare professional.	3.40 (1.10)
Preparedness for AI	2.76 (0.70)
(7) I believe that most healthcare professionals are prepared for the introduction of AI technology.	2.65 (1.07)
(8) I believe that I have been adequately trained to use AI in my job.	2.02 (1.01)
(9) I believe there is an ethical framework for the use of AI technology in my workplace.	3.23 (1.05)
(10) I believe that if the AI technology makes a mistake, all the responsibility lies with the healthcare professional.	3.15 (1.14)

Mean of scores and standard deviation (SD); m (SD).

AI: artificial intelligence.

On the other hand, the mean score of the responses on the preparedness for AI was 2.76 points (SD = 0.70) out of 5 on the Likert scale. Thus, professionals are unsure about if they are well prepared to use IA in their workflow, yet they assume their responsibility in the use of AI. Of the four items in the dimension, the one with the highest score meaning that they agree was ‘I believe there is an ethical framework for the use of AI technology in my workplace’ (3.23 out of 5 mean score with SD = 1.05) and the one with the lowest score indicating that they were unsure was ‘I believe I have been adequately trained to use AI in my job’ (2.02 out of 5 mean score with SD = 1.01) (Table 2).

Table 3 shows the results of the bivariate analysis between the two dimensions and the main explanatory variables. With regard to the professional impact of AI, healthcare professionals who understand the concept of the AI reported a greater impact of AI than those who did not. Those who have received training such as self-starting online courses, webinars, lectures or conferences have significantly different perceptions (p = 0.012) of the AI impact to their professional role compared with those that have not had any training. In addition, professionals who want to receive training have reported a greater impact of the AI than those who did not. Otherwise, nursing professionals’ perception of preparedness for AI is significantly different than the ones from medical professionals, with nurses feeling more prepared than doctors (p = 0.024). Furthermore, professionals who are using AI or did not know have reported that they feel more prepared for AI than those who did not.

Table 3.

Bivariate analysis between dimensions and sociodemographic variables.

	Professional impact of AI	p-value	Preparedness for AI	p-value
Categorised age (N = 292)		0.628^α		0.053^α
[18–30]	3.74 (0.59)		3.16 (0.66)
[30–40]	3.63 (0.82)		2.67 (0.86)
[40–50]	3.54 (0.69)		2.72 (0.66)
[50–60]	3.65 (0.75)		2.74 (0.64)
> = 60	3.70 (0.57)		2.86 (0.59)
What is your gender?		0.205^β		0.435^β
Female	3.59 (0.67)		2.74 (0.66)
Male	3.75 (0.89)		2.84 (0.84)
What health discipline do you work in?		0.427^α		0.024^α,2
Others	3.86 (1.24)		2.93 (1.11)
Nursing	3.64 (0.69)		2.86 (0.69)
Medicine	3.59 (0.69)		2.65 (0.66)
How many years of experience do you have in your job?		0.758^α		0.969^α
Less than 1 year	3.52 (0.96)		3.0 (0.85)
Between 1 and 2 years	3.68 (0.91)		2.94 (0.69)
Between 3 and 4 years	3.77 (0.63)		2.81 (0.76)
Between 5 and 10 years	3.52 (0.84)		2.76 (0.81)
More than 10 years	3.61 (0.67)		2.71 (0.66)
Do you understand what the concept of AI means?		0.032^β		0.128^β
No	3.41 (0.69)		2.91 (0.69)
Yes	3.66 (0.72)		2.74 (0.70)
In your work do you use AI to perform or plan care?		0.194^α		<0.001^α,3
No	3.60 (0.82)		2.49 (0.66)
I do not know	3.51 (0.63)		2.80 (0.64)
Yes	3.70 (0.68)		2.94 (0.71)
AI education or training:		0.012^α,1		0.349^α
None	3.54 (0.72)		2.71 (0.67)
Self-starting online courses, webinars, lectures, conferences	3.86 (0.63)		2.80 (0.72)
Job training	3.74 (0.56)		2.96 (0.77)
University education	3.45 (1.27)		2.80 (0.86)
Would you like to receive education on AI in health?		0.006^β		0.544^β
No	3.21 (0.75)		2.68 (0.68)
Yes	3.66 (0.70)		2.77 (0.70)

^α

p-Value of ANOVA contrast.

^β

p-Value of the T student contrast.

Post hoc comparison: only none vs self-starting online courses, webinars, lectures, conferences has obtained a significant p-value (p = 0.011).

Post hoc comparisons: only nursing vs medicine has obtained a significant p-value (p = 0.028).

Post hoc comparisons: only I don’t know vs no (p = 0.006) and yes vs no (p < 0.001) has obtained a significant p-value.

AI: artificial intelligence; ANOVA: analysis of variance.

At the end of the survey, they were asked about the priority level for three possible applications of AI in radiology. This was because radiology is one of the specialties where AI has made a significant impact and the region of Central Catalonia is currently in the process of validating AI algorithms for use in radiology. Of the three possible applications, the high percentage of professionals who consider it a high priority to implement AI for image interpretation to obtain immediate preliminary results (41.2%) and the high percentage of professionals who consider that AI should not be implemented for image interpretation to obtain final and definitive patient results (18.9%) stands out (Figure 3).

Figure 3.

Descriptive analysis of priority in radiology artificial intelligence applications.

Discussion

The aim of this study was to characterise the understanding and perception of PC healthcare professionals regarding the application of AI as a tool in healthcare, and to assess its potential impact on radiology. The findings of the present study provide insight into the perception and knowledge of AI as a healthcare tool among PC healthcare professionals in the region of Central Catalonia. Conducting such analyses is relevant for developing future strategies aimed at improving the acceptance and application of AI in a specific region.

Regarding the professional's knowledge of the concept of AI, most of them feel that they understand what it means, and the majority indicates that they do use it in their work. However, although the Catalan Institute of Health, a health provider for 5.8 million users in Catalonia,³⁷ is working on the validation of AI models applied in imaging with the aim of implementing it in consultations^38–40 and is in an incipient development of different applications,⁴¹ currently there are still none integrated in the patient's clinical history. Therefore, although the vast majority indicates that they are aware of the concept of AI, there is a general lack of knowledge about its applicability and its possible uses.

In fact, Chen found in the meta-analysis⁴² that only 10% to 30% of all healthcare professionals used AI in their clinical practice. Moreover, these results are consistent with the results of the meta-analysis⁴² where it was observed that 74.29% of the included studies indicated a lack of knowledge of the professionals and 15 studies suggested the urgent need to integrate AI into the training programmes of residents. Also, a survey of 3018 medical students in Turkey showed that only 2.8% felt well informed about the use of AI in medicine,⁴³ and a survey answered by 484 UK medical students showed that 89% of students felt that receiving AI training would be beneficial to their career, as well as 78% believing that AI training was necessary as part of a university degree.²⁹ Otherwise, various studies in the global literature suggest that incorporating AI education is crucial in overcoming barriers and limitations in the use and understanding of AI.^44–47

In terms of professionals’ perceptions, a positive impact of AI was reported in the survey (scored 3.62 out of 5) similar to that obtained in the Shinners et al.’s²² study (scored 3.66 out of 5). This agreement was higher for professionals who had some prior knowledge and interest in AI, answering affirmatively that they understood the concept of AI, would like to be trained in AI, or had received some type of training in AI. Therefore, these results may suggest that receiving training may increase sensitivity to how AI can help in their clinical practice. Otherwise, with regard also to the professional's perceptions, over half of professionals considered that they were prepared for AI (scored 2.76 out of 5 in the Likert scale). This result is similar to the one reported in the Shinners et al.’s²² study that reported a score on the Likert scale of 2.55 out of 5. It has been seen that the nursing category believes it is better prepared for the application of AI than the medical category. It is true that this category is one of the groups that can generate the most data in healthcare systems, as they play a fundamental role in data collection.⁴⁸ In this context, data collection can become key in the creation of AI algorithms, and it is therefore interesting to take it into account when establishing strategies for adopting AI in the Catalan healthcare system.

Furthermore, it has also been observed that professionals have felt that using AI makes them feel more prepared. Although it has been mentioned that there is currently no integrated AI model in the management programme of the Catalan healthcare system, work is being done on the development of different applications, and there is therefore some awareness of the concept of AI among professionals. Sheng et al. also show that previous experiences with applied health technology and adequate support for professionals make them more interested in using it and training.⁴⁹

Interestingly, professionals in the Central Catalonia region have considered AI as an ally for improving the quality of care they can provide and have not been so concerned about the possibility of being replaced in the future by AI. This could be due to the long tradition of e-health innovation in this region, being one of the pioneers in the use and application of telemedicine.⁵⁰ In contrast are the responses from a survey of 3018 medical students from 103 medical schools in Turkey, where the results showed that 58.6% agreed that AI would devalue the medical profession and 45.5% were concerned that the application of AI would damage the fundamental value of trust in medicine.⁴³

It should be mentioned that the region of Central Catalonia is currently working to validate an AI algorithm for chest X-rays.³⁸ This region is unique in that it has a shortage of radiologists, which means that PC physicians are increasingly responsible for interpreting X-rays. Given this context, the development of AI algorithms to support X-ray diagnosis, as well as the knowledge and acceptance of healthcare professionals, is essential to integrate and improve workflow for professionals and enhance patient safety. In this context, a question was asked about the priority of incorporating different forms of AI in radiology. It should be taken into account that several studies have mentioned a certain reluctance to adopt AI in radiology due to the fact that in the future there will be a decrease in the demand for radiologists to interpret radiographs, as well as the definitive replacement of radiologists in the future.^29,51 However, both in our study and in other articles in the literature it has been observed that health professionals see clearly that AI should not give a definitive diagnosis to the patient but should only be a tool for quick help and support.^52,53

In this context, it seems that professionals in the territory understand AI as an ally to support diagnosis, rather than as a threat, and therefore strategies for the widespread implementation of AI in this territory should be aimed more at training and infrastructures than at resistance to change.

Limitations and strengths

It is possible that there is a bias in the representativeness of the sample, since 81.1% of the professionals who responded to the survey were women.⁵⁴ It should be taken into account, however, that this study was carried out in the PC setting of the Catalan Health Institute of Central Catalonia, where 89.55% of the nursing category and 65.72% of the family medicine category are women.⁵⁵

Furthermore, this is a study with little external validity of the results given the particularities of the territory, which is mostly made up of rural areas. However, this fact brings strength and innovation, as there are few studies on the perception and knowledge of AI in rural areas.

Finally, the SHAIP tool has only be validated in Australian healthcare settings. To control this limitation, the questionnaire was validated with the study sample and language using Cronbach's alpha and factor analysis.

Conclusion

The results of this study show that most professionals indicated that they had a good understanding of the concept, considered AI to have a positive impact, and felt prepared for its implementation. Additionally, although only as a diagnostic aid, the implementation of AI in radiology was considered a high priority by professionals. Observational studies in selected regions are relevant in order to create strategies that can offer training to professionals to increase their acceptance, reduce resistance to change and prepare them for the near future.

Footnotes

Acknowledgements

The authors would like to thank all the medical and nursing professionals of the Territorial Management of Central Catalonia of the Catalan Institute of Health for their participation in the survey. In addition, this study has been carried out as part of the Industrials Doctorats programme of Catalonia.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Jordi Solé-Casals

Queralt Miró Catalina

References

Shelmerdine

Rosendahl

Arthurs

. Artificial intelligence in paediatric radiology: international survey of health care professionals’ opinions. Pediatr Radiol 2022; 52: 30–41.

Ramesh

Kambhampati

Monson

JRT

, et al. Artificial intelligence in medicine. Ann R Coll Surg Engl 2004; 86: 334–338.

Ávila-Tomás

Mayer-Pujadas

Quesada-Varela

. Artificial intelligence and its applications in medicine II: current importance and practical applications. Aten primaria 2021; 53: 81–88.

Karimi

HaddadPajouh

. Artificial intelligence, important assistant of scientists and physicians. Galen Med J. 2020; 9: e2048.

Tomašev

Glorot

Rae

, et al. A clinically applicable approach to continuous prediction of future acute kidney injury. Nature 2019; 572: 116–119.

Fernandes

Vieira

Leite

, et al. Clinical decision support systems for triage in the emergency department using intelligent systems: a review. Artif Intell Med 2020; 102.

Goldstein

Shahar

. An automated knowledge-based textual summarization system for longitudinal, multivariate clinical data. J Biomed Inform 2016; 61: 159–175.

Solé-Casals

Anchustegui-Echearte

Marti-Puig

, et al. Discrete cosine transform for the analysis of essential tremor. Front Physiol 2019; 9: 1947.

Ting

DSW

Pasquale

Peng

, et al. Artificial intelligence and deep learning in ophthalmology. Br J Ophthalmol 2019; 103: 167–175.

10.

Hogarty

Phan

, et al. Artificial intelligence in dermatology—where we are and the way to the future: a review. Am J Clin Dermatol 2020; 21: 41–47.

11.

Duan

Huang

Sun

, et al. Topological network analysis of early Alzheimer’s disease based on resting-state EEG. IEEE Trans Neural Syst Rehabil Eng 2020; 28: 2164–2172.

12.

Dutta

Long

Brown

DFM

, et al. Automated detection using natural language processing of radiologists recommendations for additional imaging of incidental findings. Ann Emerg Med 2013; 62: 162–169.

13.

Johnson

AEW

Pollard

Berkowitz

, et al. MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Sci Data 2019; 6: 1–8.

14.

Spear

Nikpanah

, et al. Augmented radiologist workflow improves report value and saves time: a potential model for implementation of artificial intelligence. Acad Radiol 2020; 27: 96–105.

15.

Hwang

Park

Jin

K-N

, et al. Development and validation of a deep learning-based automated detection algorithm for major thoracic diseases on chest radiographs. JAMA Netw Open 2019; 2: e191095.

16.

Wong

KCL

Gur

, et al. Comparison of chest radiograph interpretations by artificial intelligence algorithm vs radiology residents. JAMA Netw Open 2020; 3: e2022779.

17.

Gatt

Spectre

Paltiel

, et al.

Chest radiographs in the emergency department: is the radiologist really necessary?

Postgrad Med J 2003; 79: 214–217.

18.

Preston

Marr 3rd

Amaraneni

, et al. Reduction of “callbacks” to the ED due to discrepancies in plain radiograph interpretation. Am J Emerg Med 1998; 16: 160–162.

19.

Espinosa

Nolan

. Reducing errors made by emergency physicians in interpreting radiographs: longitudinal study. Br Med J 2000; 320: 737–740.

20.

Martínez-Torres

Toral Marín

García

, et al. A technological acceptance of e-learning tools used in practical and laboratory teaching, according to the European higher education area. Behav Inf Technol 2008; 27: 495–505.

21.

Stevenson

Nilsson

Petersson

, et al. Nurses’ experience of using electronic patient records in everyday practice in acute/inpatient ward settings: a literature review. Health Informatics J 2010; 16: 63–72.

22.

Shinners

Aggar

Grace

, et al. Exploring healthcare professionals’ perceptions of artificial intelligence: validating a questionnaire using the e-Delphi method. Digit Heal 2021; 7: 20552076211003430.

23.

Nadarzynski

Miles

Cowie

, et al. Acceptability of artificial intelligence (AI)-led chatbot services in healthcare: a mixed-methods study. Digit Heal 2019; 5: 1–12.

24.

Fan

Liu

Zhu

, et al. Investigating the impacting factors for the healthcare professionals to adopt artificial intelligence-based medical diagnosis support system (AIMDSS). Ann Oper Res 2020; 294: 567–592.

25.

Kim

Choi

, etal. Physician confidence in artificial intelligence: an online mobile survey. J Med Internet Res 2019; 21: e12422.

26.

Sarwar

Dent

Faust

, et al. Physician perspectives on integration of artificial intelligence into diagnostic pathology. NPJ Digit Med 2019; 2: 1–7.

27.

Diaz

Guidi

Ivashchenko

, et al. Artificial intelligence in the medical physics community: an international survey. Phys Med 2021; 81: 141–146.

28.

Shinners

Aggar

Grace

, et al. Exploring healthcare professionals’ understanding and experiences of artificial intelligence technology use in the delivery of healthcare: an integrative review. Health Informatics J 2020; 26: 1225–1236.

29.

Sit

Srinivasan

Amlani

, et al. Attitudes and perceptions of UK medical students towards artificial intelligence and radiology: a multicentre survey. Insights Imaging 2020; 11: 14.

30.

Health Education England. The Topol review: preparing the healthcare workforce to deliver the digital future. An independent report on behalf of the Secretary of State for Health and Social Care. Nhs. 2019:102.

31.

Fast

Horvitz

. Long-term trends in the public perception of artificial intelligence. 31st AAAI Conf Artif Intell AAAI 2017. 2016;963–9.

32.

Jha

Topol

. Adapting to artificial intelligence: radiologists and pathologists as information specialists. JAMA 2016; 316: 2353–2354.

33.

Laï

Brian

Mamzer

. Perceptions of artificial intelligence in healthcare: findings from a qualitative survey study among actors in France. J Transl Med 2020; 18: 14.

34.

Shinners

Grace

Smith

, et al. Exploring healthcare professionals’ perceptions of artificial intelligence: piloting the Shinners Artificial Intelligence Perception tool. Digit Heal 2022; 8: 1–8.

35.

Peterson

Kim

. On the relationship between coefficient alpha and composite reliability. J Appl Psychol 2013; 98: 194–198.

36.

Ware

Gandek

. Methods for testing data quality, scaling assumptions, and reliability: the IQOLA project approach. J Clin Epidemiol 1998; 51: 945–952.

37.

Coma

Miró

Medina

, et al. Association between the reduction of face-to-face appointments and the control of patients with type 2 diabetes mellitus during the Covid-19 pandemic in Catalonia. Diabetes Res Clin Pract 2021; 182: 109127.

38.

Catalina

Fuster-Casanovas

Solé-Casals

, et al. Developing an artificial intelligence model for reading chest X-rays: protocol for a prospective validation study. JMIR Res Protoc 2022; 11: e39536.

39.

Escalé-Besa

Fuster-Casanovas

Börve

, et al. Using artificial intelligence as a diagnostic decision support tool in skin disease: protocol for an observational prospective cohort study. JMIR Res Protoc 2022; 11: e37531.

40.

Vidal-Alaball

Fibla

Zapata

, et al. Artificial intelligence for the detection of diabetic retinopathy in primary care: protocol for algorithm development. JMIR Res Protoc 2019; 8(2): e12539.

41.

Programa per la promoció i desenvolupament de la Intel·ligència Artificial al Sistema de Salut de Catalunya IA.

42.

Chen

Zhang

Cai

, et al. Acceptance of clinical artificial intelligence among physicians and medical students: a systematic review with cross-sectional survey. Front Med 2022; 9: 990604.

43.

Civaner

Uncu

Bulut

, et al. Artificial intelligence in medical education: a cross-sectional needs assessment. BMC Med Educ 2022; 22: 1–9.

44.

Wartman

Donald Combs

. Medical education must move from the information age to the age of artificial intelligence. Acad Med 2018; 93: 1107–1109.

45.

Liaw

Kakadiaris

. Artificial intelligence and family medicine: better together. Fam Med 2020; 52: 8–10.

46.

Katznelson

Gerke

. The need for health AI ethics in medical school education. Adv Health Sci Educ Theory Pract 2021; 26: 1447–1458.

47.

Lee

, et al. Artificial intelligence in undergraduate medical education: a scoping review. Acad Med 2021; 96: S62–S70.

48.

Collins

Couture

Kang

, et al. Quantifying and visualizing nursing flowsheet documentation burden in acute and critical care. AMIA Annu Symp Proc 2018; 2018: 348–357.

49.

Liu Sheng

PJH

Wei

, et al. Adoption and diffusion of telemedicine technology in health care organizations: a comparative case study in Hong Kong. Journal of Organizational Computing and Electronic Commerce, 2009; 8(4): 247–275.

50.

Vidal-Alaball

Seguí

Domingo

JLG

, et al. Primary care professionals’ acceptance of medical record-based, store and forward provider-to-provider telemedicine in Catalonia: results of a web-based survey. Int J Environ Res Public Health 2020; 17: 1–13.

51.

Kurowecki

Lee

Monteiro

, et al. Resident physicians’ perceptions of diagnostic radiology and the declining interest in the specialty. Acad Radiol 2021; 28: 261–270.

52.

Pinto Dos Santos

Giese

Brodehl

, et al. Medical students’ attitude towards artificial intelligence: a multicentre survey. Eur Radiol 2019; 29: 1640–1646.

53.

Akudjedu

Torre

Khine

, et al. Knowledge, perceptions, and expectations of artificial intelligence in radiography practice: a global radiography workforce survey. J Med Imaging Radiat Sci 2022; 54: 104–116.

54.

Lundeberg

Fox

Punćochaŕ

. Highly confident but wrong: gender differences and similarities in confidence judgments. J Educ Psychol 1994; 86: 114–121.

55.

Institut Català de la Salut. L’ICS, una empresa femenina. Institut Català de la Salut [Internet]. 2017 [cited 2023 Apr 27]. Available from: https://ics.gencat.cat/es/detall/noticia/infografia_feminitzacio_ics.html.