Abstract
Introduction
The use of artificial intelligence (AI), which can emulate human intelligence and enhance clinical results, has grown in healthcare decision-making due to the digitalization effects and the COVID-19 pandemic. The purpose of this study was to determine the scope of applications of AI tools in the decision-making process in healthcare service delivery networks.
Materials and methods
This study used a qualitative method to conduct a systematic review of the existing reviews. Review articles published between 2000 and 2024 in English-language were searched in PubMed, Scopus, ProQuest, and Cochrane databases. The CASP (Critical Appraisal Skills Programme) Checklist for Systematic Reviews was used to evaluate the quality of the articles. Based on the eligibility criteria, the final articles were selected and the data extraction was done independently by 2 authors. Finally, the thematic analysis approach was used to analyze the data extracted from the selected articles.
Results
Of the 14 219 identified records, 18 review articles were eligible and included in the analysis, which covered the findings of 669 other articles. The quality assessment score of all reviewed articles was high. And, the thematic analysis of the data identified 3 main themes including clinical decision-making, organizational decision-making, and shared decision-making; which originated from 8 subthemes.
Conclusions
This study revealed that AI tools have been applied in various aspects of healthcare decision-making. The use of AI can improve the quality, efficiency, and effectiveness of healthcare services by providing accurate, timely, and personalized information to support decision-making. Further research is needed to explore the best practices and standards for implementing AI in healthcare decision-making.
Introduction
The literature suggests that the COVID-19 pandemic, by enforcing physical distancing measures, facilitated the digitalization and virtualization of health and social services. 1 This phenomenon has prompted researchers to characterize the period following the COVID-19 pandemic as the era of digitalization, artificial intelligence (AI), and substantial increase in the adoption of AI digital platforms, particularly in the provision of healthcare services. 2
AI refers to the ability of a machine or system to perform tasks that require human intelligence, such as reasoning, learning, and decision making. 3 AI encompasses various technologies, such as machine learning, deep learning, natural language processing, robotics, speech processing, and other automation technologies. 4
Various AI-based digital platforms have attracted considerable attention from service providers and users, as they can offer the opportunity to implement patient-centered care within the healthcare systems, owing to a number of potential benefits that these technologies provide. For service users from diverse populations, settings, and conditions, potential benefits include convenience and enhanced accessibility, especially where factors such as physical mobility challenges and more importantly the opportunity to provide services based on the preferences of patients as distinct individuals with distinct differences.5,6
In the field of clinical decision making, AI has demonstrated remarkable capabilities for predicting and classifying diagnoses, as well as providing recommendations and insights.4,7 The increasing empirical evidence suggests that knowledge-based computerized decision support (CDS) and, in particular, knowledge-based clinical decision support systems, have the potential to enhance practitioner performance.8,9 The literature indicates that AI systems have been successful in multiple medical imaging use cases, such as detecting mitosis in breast cancer histology images, 10 classifying skin cancer with dermatologist-level accuracy, 11 diagnosing diabetic retinopathy from retinal fundus photographs, 12 and predicting cardiovascular risk factors from retinal fundus photographs. 13 These studies demonstrate the potential of AI systems to assist healthcare providers in improving diagnosis and prognosis, as well as providing personalized and customized information. Moreover, they have been demonstrated to reliably predict the risk of imminent suicide attempts, 14 and they have been employed to assess the likelihood of patients developing serious conditions or being transferred to palliative care. 15
The decisions within the healthcare service delivery system are often based on the physicians’ own opinions. 16 Meanwhile, many patients lack an advance directive or a surrogate who can decide for them, so strangers have to make the choices 17 ; Patients also have difficulties with planning their future care and understanding resuscitation scenarios, so they may not communicate their wishes or may have unrealistic expectations (eg, overestimating the success of cardio-pulmonary resuscitation). 18 Moreover, relatives who act as surrogates may experience stress and may not respect the patient's preferences. 19 These are only a few examples that demonstrate the need for novel approaches to decision making within the healthcare service delivery system. And, one of the potential options is known to be the use of AI within the decision-making process. AI is changing the way clinical providers make decisions in healthcare. AI can assist healthcare professionals in diagnosing diseases, planning treatments, predicting outcomes, and managing population health. AI can also improve the quality and efficiency of healthcare decision making, as well as the satisfaction and involvement of users. 20
AI is a rapidly evolving field that has the potential to transform various aspects of healthcare, such as diagnosis, treatment, prevention, and management. However, the literature on the application domains of AI in healthcare decision-making is scattered and fragmented, making it difficult to obtain a clear and comprehensive picture of the current state and future directions of this emerging field. This phenomenon motivated us to conduct a systematic review of the literature on the use of AI in the decision-making process of healthcare service delivery in the healthcare system. The purpose of this research was to accumulate and determine the application areas of AI in the decision-making process of providing health care through conducting a review of reviews.
Materials and Methods
This article was a qualitative content analysis that conducted a systematic review of the existing reviews in the literature from 2000 to 2024. The aim of this study was to review systematically the existing reviews on the utilization of AI in decision-making processes within the delivery of healthcare services. A review of reviews is a type of secondary article that synthesizes and evaluates the methods and quality of other review articles on a specific topic. Review articles are systematic summaries of the current state of knowledge and evidence on a topic. Review of reviews aims to compare and contrast the different types of review articles, assess their reliability and validity, and identify the best sources of evidence for researchers, practitioners, and policymakers. Review of reviews can also help to identify gaps and controversies in the literature, and suggest directions for future research. Review of reviews are also referred to as overviews of reviews, umbrella reviews, or systematic reviews of systematic reviews. 21
Data Gathering and Search Strategy
The present study adhered to the PRISMA 2020 (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines for systematic reviews to ensure the quality and validity of the research. 22 A systematic search was conducted to identify all published review articles related to the utilization of AI in decision-making processes within the delivery of healthcare services, from 2000 to 2024 and in English only. The databases searched were PubMed, Scopus, Cochrane, and ProQuest. We used MeSH terms to categorize all keywords into 3 groups: AI, delivery of healthcare, and decision-making. The synonymous keywords were combined using the logical operator “OR.” The first, second, and third groups of keywords were then merged using the logical operator “AND.” EndNote X7.1 software was used to organize the references. And, the search was conducted on February 3, 2024. The search strategy is presented in Table 1.
The Search Strategy Used for Conducting the Systematic Review.
Inclusion and Exclusion Criteria
We included articles published in English from 2000 to 2024 that focused on decision-making using AI in healthcare delivery. We excluded articles that: (1) did not cover a relevant AI topic in healthcare, (2) lacked a title or abstract describing an AI application in healthcare, or (3) lacked a title, abstract, or full-text providing any decision-making usage of AI in healthcare application in any healthcare system. Also, the articles that were recognized as low quality based on the quality assessment score using CASP Systematic Review Checklist, 23 were excluded. Also, some types of articles including short communications, letters to the editor, and other irrelevant publications were removed.
Screening and Data Extraction
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart 24 was followed to select the articles for the review. Any duplicates were removed and the remaining articles were screened by their title and abstract. Any articles that were not relevant to the research objective were discarded, and the full text of the rest was read. Only those that met the eligibility criteria were included in the final analysis. The whole process was carried out independently by 2 researchers. In case of any conflict regarding the results of the procedure, the other authors were consulted to complete the screening process.
The data from the final articles that aligned with the study objective were extracted independently by 2 authors. They used Microsoft Office Excel 2013 to create a data extraction form for data collection. The form consisted of sections such as authors, title, year, country, journal, indexed databases, study type, digital tool, disorder type, number of final articles, and summary of results.
Quality Assessment of Final Articles Using the CASP Checklist
Since this is a relatively new research field, we deemed it important to conduct a qualitative assessment of the presented articles. The quality of the selected studies was assessed using the Critical Appraisal Skills Program (CASP) for review articles (CASP Systematic Review Checklist). 21 The main CASP checklists (for randomized controlled trials and systematic reviews) were adapted from the Journal of American Medical Association (JAMA) User's Guides to the Medical Literature in 1994. The CASP checklist is a tool used to assess the quality of research studies. It comprises various categories and questions that help evaluate the validity, relevance, and applicability of a research study. The checklist is usually evidence-based in medicine for determining the strength of evidence for clinical decision-making. 25
The checklist consists of 10 questions that cover 3 broad issues: are the results of the review valid, what are the results, and will the results help locally. Two questions are descriptive and 8 questions are scored, with 2 for yes, 1 for I can’t say, and 0 for no. The quality of the studies is measured by the percentage of the total score out of 16. The percentage range is between 0 and 100, where higher scores indicate higher quality. The studies are classified into 4 quality levels: very low (0-25), low (26-50), medium (51-75), and high (76-100). Only studies with medium or high quality are deemed eligible for research purposes.
Data Analysis
A qualitative thematic analysis and an inductive approach 26 were used to analyze the data extracted from the previous steps. The authors repeatedly read all the included articles and extractions to gain a thorough understanding of the data and then assigned initial codes to each meaningful extraction. All initial codes were examined and finalized before grouping them into subthemes and main themes. The subthemes and main themes were then named, described, and presented in a table. Lincoln and Guba's 27 criteria were used to evaluate the trustworthiness of the research. Trustworthiness refers to the extent to which the findings of a study are credible, transferable, dependable, and confirmable.
Results
Review
As depicted in Figure 1, we found 14 219 records in the database searching. After duplicate removal, we screened 6483 records. Finally, the quality of the articles was assessed using the CASP checklist, and 18 articles were included.
Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart for study selection.
Table 2 presents a summary of the characteristics of 18 studies that examined the application of AI-based systems in various domains of healthcare decision-making. The studies included in this research covered the results of 669 articles.
Summary of the characteristics of the18 included studies .
Study No. 1 was a systematic review published in 2023. The study aimed to review AI-based clinical decision support systems in the development stage, specifically regarding methodological robustness and constraints for clinical implementation. The review included 121 articles. 28
Study No. 2 was a systematic review published in 2023. The study aimed to assess the performance and applicability of machine learning algorithms in early childhood caries (ECC) prediction and detection. The review included 6 articles. 29
Study No. 3 was a scoping review published in 2023. The study aimed to identify the recent developments in vision transformer-based AI methods for lung cancer imaging applications. The review included 34 articles. 30
Study No. 4 was a systematic review published in 2023. The study aimed to explore whether the use of AI can improve antibiotic prescribing for human patients. The review included 5 articles. 31
Study No. 5 was a systematic review published in 2023. The study aimed to provide a comprehensive overview of the reasons given in the academic literature for and against the use of AI-based applications in clinical ethical decision making. The review included 44 articles. 32
Study No. 6 was a systematic review published in the United Kingdom in 2020. It reviewed 5 final articles that assessed the effectiveness of AI-based computerized decision support systems in health and social care settings. This study was the only one that employed a systematic review method, which is regarded as more rigorous and comprehensive than a scoping review method. However, the low number of final articles reviewed indicates a lack of evidence on the effectiveness of AI-based computerized decision support systems within health and social care settings. 33
Study No. 7 was a scoping review published in 2023. The study aimed to explore the extent of the actual use of AI/machine learning protocols for diagnosing cancer in prospective settings. The review included 18 articles. 34
Study No. 8 was a scoping review published in 2019. It reviewed 62 final articles that assessed the impact of AI-based decision support systems on various outcomes and populations in emergency department triage settings. This study was the only one that focused on emergency department (ED) triage, which is a process that involves prioritizing patients based on their urgency and severity of conditions. While the high number of final articles reviewed indicates a wide range of AI-based decision support systems in ED triage settings. 35
Study No. 9 was a scoping review published in 2023. The study aimed to investigate the incorporation of AI and ML-based decision support systems in mental health care settings. The review included 4 articles. 36
Study No. 10 was a systematic review published in 2023. The study aimed to review the development, adoption, and utilization of medical decision-support systems in diagnosing febrile diseases. The review included 128 articles. 37
Study No. 11 was a scoping review published in 2023. The study aimed to provide an overview of the current status of AI-/information technology-based clinical decision support tools in China. The review included 37 articles. 38
Study No. 12 was a scoping review published in 2024. The study aimed to identify literature gaps for the future development and evaluation of clinical decision-support systems for emergency telephone triage. The review included 19 articles. 39
Study No. 13 was a systematic review published in 2023. The study aimed to investigate studies and tools that address AI/machine learning-based approaches towards clinical decision support (CDS) for monitoring cardiovascular patients in intensive care units (ICUs). The review included 21 articles. 40
Study No. 14 was a scoping review published in 2019. It reviewed 80 final articles that explored the use of artificial neural networks (ANNs) across all levels of healthcare organizational decision-making. This study was the only one that focused on ANN, which is a type of AI system that can learn from data and tailor solutions to a given problem. Moreover, the high number of final articles reviewed indicates a growing interest and potential of ANN in healthcare organizational decision-making. 41
Study No. 15 was a scoping review published in 2022. It reviewed 6 final articles that investigated the use of AI systems to support shared decision-making in healthcare settings. This study was the only one that focused on shared decision-making (SDM), which is a process that involves patients and practitioners making decisions together based on evidence and preferences. However, the low number of final articles reviewed indicates a lack of evidence on AI systems to support SDM in healthcare settings. 42
Study No. 16 was a systematic review published in 2023. The study aimed to identify the application of AI in the diagnosis of various forms of mouth ulcers. The review included 23 articles. 43
Study No. 17 was a scoping review published in 2023. It reviewed 20 final articles that identified and assessed the strategies for the implementation of machine learning tools in hospital settings. This study was the only one that focused on implementation strategies, which are methods and approaches used for introducing and integrating machine learning tools into healthcare settings. Meanwhile, the moderate number of final articles reviewed indicates some evidence of implementation strategies for machine learning tools in hospital settings. 44
Study No. 18 was a systematic review published in 2023. The study aimed to provide detailed insights into AI applications in breast cancer diagnosis, categorize key specializations of interest to the academic community, and identify research themes in each category through citation and thematic issue analysis. The review included 36 articles. 45
Quality Assessment of Selected Review Articles
In Table 3, the result of the qualitative assessment of the selected review articles in this study is reported. Using the CASP checklist, we evaluated the articles and found that all 18 articles scored full marks in the “result validity” dimension (including Q1 and Q2), which indicates a clear focus on the research question and appropriate selection of papers. For the “worth continuing” dimension (including Q3, Q4, and Q5), we found that all articles obtained a full score in the “combination of review results” question (Q5). However, despite these positive findings, none of the articles fully met our criteria in the “inclusion of all the important, relevant studies” question (Q3). Only 6 studies received a full score in the “assessment of quality of included studies” question (Q4). Moreover, all articles received full marks in the “local impact of results” dimension (including Q8, Q9, and Q10). In general, the quality assessment score of all reviewed articles was high.
Quality Assessment of Review Articles Using the CASP Checklist.
Thematic Analysis of the Findings
After thoroughly reviewing the articles included in the study, we have identified 3 main themes: “clinical decision-making,” “organizational decision-making,” and “shared decision-making.” Additionally, there are 8 subthemes related to these main themes: remote monitoring, computerized interpretation of graphs, facilitating diagnosis and prognosis, forecasting administrative and quality indicators, offering cost-effective solutions for time and resource management, providing personalized and customized information, the possibility of patient self-management, enhancing patient medication adherence. In Table 4, the main themes and subthemes, and brief definitions for each theme are presented.
Findings From the Thematic Analysis.
Discussion
The aim of the study was to determine the application domains of various types of AI in the decision-making process in the healthcare system. Three main themes including clinical decision-making, organizational decision-making, and shared decision-making were identified, which originated from 8 subthemes. Each is discussed in detail below:
Clinical Decision-Making
As per the existing scholarly works, the theme of clinical decision-making encompasses various subthemes to where AI can be applied. These subthemes include remote monitoring, computerized interpretation of graphs, and facilitating diagnosis and prognosis.
One of the subthemes that refers to the application of AI in the clinical field is remote monitoring. AI can help classify physical activities and detect abnormal events or behaviors using wearable sensors and cameras. This can help remote-monitor the patients’ mobility, safety, and adherence to treatment plans. 46 One study evaluated the performance of algorithms enabling automated triaging based on existing data sets. Despite the reported lack of statistical significance of the effect of AI on the outcomes, the increased availability of remote monitoring was reported to potentially enhance the outcomes and efficiency through the use of such AI platforms. 47 Interestingly, a study has reported that AI has also the potential to perform in emergency circumstances, assisting in monitoring cardiovascular patients in ICUs. 40
The next subtheme is a computerized interpretation of graphs. The task of identifying the discriminatory and latent attributes from large-scale heterogeneous data is challenging for both nodal graphs and graph images (also known as chart images). Therefore, interpretation-based methods using AI emerged as a promising solution. 48 One possible application of AI in the computerized interpretation of graphs in the clinical field is to use machine learning algorithms to analyze electrocardiogram signals. 49 A study evaluated the performance of AI facilitating the interpretation of cardiotocographs through image pattern recognition by applying computerized interpretation. Surprisingly, the study reported that the application of such AI tools does not enhance the clinical outcomes for maternal or neonatal health. 50 Another paper declared that various tasks, such as analyzing medical images or associating symptoms and biomarkers from electronic medical records with the disease's characterization and prognosis, have shown that AI algorithms can perform equally or better than humans. 51 Moreover, vision transformers, a deep learning model, have become popular in medical imaging, especially for lung cancer. They analyze image classification as a sequence prediction task, capturing long-term dependencies within image patches. 30 These results demonstrate the considerable potential of employing AI in the analysis and interpretation of medical graphs.
The third subtheme is facilitating diagnosis and prognosis. AI can help prevent and diagnose various diseases, such as mental disorders, cancer, diabetic retinopathy, Alzheimer's disease, etc, using deep learning models. This can help provide faster and more reliable diagnosis and reduce human errors.28,29,34,36-38,43,45,52 Besides that, AI has shown the capability to streamline the drug prescription process. 31 Moreover, one paper reported that AI can aid in patient triage by offering guidance and evaluating the severity and priority of cases. 39 Another paper described the application of ANNs in the process of clinical decision-making in healthcare and reported the potential promise of their use in situations involving complex, unstructured, or limited information. 41 Similarly, a study evaluated the performance and benefit of ANNs as clinical decision-making tools in cancer. It indicated that ANNs had demonstrated promising outcomes in diagnosis and prognosis. 53 These findings delineate the potentially significant contribution of AI in the process of diagnosis and prognosis of diseases.
Organizational Decision-Making
Evidence shows that AI can also be used in organizational decision-making. This main theme includes 2 subthemes which are forecasting administrative and quality indicators, and providing cost-effective solutions for time and resource management.
The first subtheme is forecasting administrative and quality indicators. AI can help forecast the performance and outcomes of healthcare processes, such as patient satisfaction, safety, quality of care, and efficiency, using machine learning algorithms and clinical data. This can help identify best practices, monitor progress, and provide feedback and recommendations for improvement. 54 It is shown that ANNs have been effectively applied to forecast quality factors (responsiveness, security, and efficiency) that affect the adoption of e-government services. 55 Moreover, it is reported that some forms of AI tools are used for distinct organizational levels of healthcare systems; in this regard, it was reported that ANNs were mostly used for microlevel decisions between patients and providers, and less for meso- and macrolevel decisions. 41
The next subtheme is providing cost-effective solutions for time and resource management. AI can help automate and streamline administrative tasks, such as scheduling, invoicing, reporting, and customer service, using natural language processing and machine learning. This can help save time and improve efficiency and productivity. 56 It is shown that nonclinical applications have involved enhancing the management of healthcare organizations, as well as forecasting important indicators such as cost or resource utilization.57,58 In such context, ANNs have been employed in decision support models that offer cost-effective solutions for time and resource management to healthcare providers and the healthcare system. 59 AI can help predict the demand and supply of healthcare resources, such as beds, staff, equipment, and drugs, using historical data, current trends, and external factors. This can help optimize the allocation and utilization of resources and reduce costs and waste. 60
Shared Decision-Making
Shared decision-making is the last main theme, and it includes 3 subthemes: providing personalized and customized information, enabling patient self-management, and enhancing patient medication adherence.
The first subtheme is providing personalized and customized information. One study delineated that AI has the potential to enhance patient autonomy by enabling patients to receive their treatment of choice. 32 Meanwhile, a research proposed a system that integrated AI estimations, physician's interpretation, and parents’ perspectives to support ethical treatment decisions for newborn patients; The results showed that the system had delivered personalized and customized information to different users and facilitated shared decision-making involving parents. 61 AI can help create adaptive and personalized educational software that can tailor the learning content, pace, feedback, and assessment to each individual learner. 62 This technology can help deliver customized education and guidance to patients and caregivers using natural language processing and chatbots. 60 In such context, “Software as a Medical Device” (SaMD) is software intended for medical purposes, independent of hardware. It spans from computer applications to mobile apps, aiding in disease diagnosis, treatment, and prevention. 63 Interestingly, through the provision of guidance and regulations for the development, evaluation, and approval of Software as a SaMD, the FDA and other global regulators facilitate stakeholders in promoting safe innovation while ensuring patient safety. 63
The second subtheme is enabling patient self-management. One study investigated the use of learning algorithms to enhance patient self-management; The system reduced the time and visits required by clinicians, and patients expressed satisfaction with it. Moreover, no loss of monitoring was observed. 64 AI can help patients with musculoskeletal pain, such as neck or lower back pain, to improve their physical activity, posture, and coping skills using a smartphone app that adapts to their individual needs and preferences. 65 Similarly, AI can help patients with diabetes to manage their blood glucose levels, diet, and exercise using a chatbot that provides education and coaching based on natural language processing and machine learning. 66
The last subtheme is enhancing patient medication adherence. A study evaluated the use of an AI platform on mobile devices in measuring and enhancing medication adherence in stroke patients. The study demonstrated that the platform was effective and improved adherence by 50%. 67 Even though, contrary to such finding, a trial that tested an algorithm for warfarin dosing in self-managing patients; found no difference in warfarin treatment quality between the algorithm and standard care. 68 AI can help patients with chronic diseases, such as diabetes, hypertension, or cancer, to adhere to their medication regimens by providing reminders, alerts, education, and coaching using mobile phone applications, chatbots, or wearable devices.69,70 Furthermore, AI can prevent medication nonadherence using machine learning algorithms and clinical data, such as patient characteristics, medication history, disease severity, or psychosocial factors. 69
Limitations and Implications
This study has several strengths, such as conducting a review of reviews and using a thematic analysis approach, applying a rigorous and transparent search strategy and quality assessment of selected articles, and providing a comprehensive and detailed synthesis of the themes and subthemes. However, this review has limitations that must be acknowledged. The first and most important limitation of the current research is that the selected studies were only in English. Second, it did not explore the perspectives or experiences of the end-users of the AI systems, such as patients, providers, managers, or policymakers. Therefore, the findings may not capture the acceptability, usability, feasibility, or satisfaction of the AI systems in healthcare decision-making domains.
Conclusions
This article showed that AI systems are useful in various areas of healthcare decision-making, including clinical, organizational, and shared decision-making. The findings indicated that AI has the potential to enhance the quality, efficiency, and effectiveness of healthcare decisions.
Footnotes
Acknowledgments
Hereby the authors acknowledge the contribution of the Bing AI chatbot in preparing the text of the manuscript in terms of English grammar and nativity.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Data Availability Statement
There is no data affiliated with the research.