Sage Journals: Discover world-class research

Abstract

Introduction

ChatGPT, as an AI tool, has been introduced in healthcare for various purposes. The objective of the study was to investigate the principal benefits of ChatGPT utilization in healthcare services and to identify potential domains for its expansion within the building blocks of the healthcare industry.

Methods

A comprehensive three-phase study was conducted employing mixed methods. The initial phase comprised a systematic review and thematic analysis of the data. In the subsequent phases, a questionnaire, developed based on the findings from the first phase, was distributed to a sample of eight experts. The objective was to prioritize the benefits and potential expansion domains of ChatGPT in healthcare building blocks, utilizing gray SWARA (Stepwise Weight Assessment Ratio Analysis) and gray MABAC (Multi-Attributive Border Approximation Area Comparison), respectively.

Results

The systematic review yielded 74 studies. A thematic analysis of the data from these studies identified 11 unique themes. In the second phase, employing the gray SWARA method, clinical decision-making (weight: 0.135), medical diagnosis (weight: 0.098), medical procedures (weight: 0.070), and patient-centered care (weight: 0.053) emerged as the most significant benefit of ChatGPT in the healthcare sector. Subsequently, it was determined that ChatGPT demonstrated the highest level of usefulness in the information and infrastructure, information and communication technologies blocks.

Conclusion

The study concluded that, despite the significant benefits of ChatGPT in the clinical domains of healthcare, it exhibits a more pronounced potential for growth within the informational domains of the healthcare industry's building blocks, rather than within the domains of intervention and clinical services.

Keywords

Artificial intelligence ChatGPT Chatbot healthcare services gray SWARA gray MABAC

Introduction

In the wake of the COVID-19 pandemic, there has been a marked acceleration and application of Artificial Intelligence (AI) in diverse fields such as healthcare, public safety, and business operations.^1,2 Additionally, the existing evidence emphasizes the transformative potential of AI and advanced technologies on business practices in the post-pandemic world.³

ChatGPT, a product of OpenAI, is an advanced artificial intelligence (AI) chatbot. It employs natural language processing techniques to facilitate dialogues that mimic human conversation. ChatGPT, which is constructed on the basis of the generative pre-trained transformer (GPT) model, possesses the ability to respond to inquiries and generate an extensive variety of written content. This includes, but is not limited to, articles, updates for social media, essays, code, and emails.^4,5 This phenomenon highlights the critical importance of identifying areas with potential for the utilization and expansion of ChatGPT. Such recognition is vital for the formulation of future strategies and interventions aimed at improving services within sectors that stand to benefit from ChatGPT.

ChatGPT is pretrained on an extensive text corpus, which enables it to comprehend context and meaning. During this phase, the model discerns patterns and relationships within the language, thereby facilitating the generation of coherent responses to user inputs. Upon receiving a prompt, ChatGPT tokenizes the text and processes these tokens to predict the subsequent token in the sequence, selecting the one with the highest probability. This iterative process continues until a stopping criterion is met, such as a maximum length or a specific end token.⁶ Furthermore, ChatGPT is capable of producing a variety of outputs, including concise answers, essays, and conversational interactions.⁶ In this context, the utilization of ChatGPT has been identified as a valuable strategy to enhance healthcare services, offering personalized care, 24/7 availability, cost reduction, improved efficiency, and enhanced communication, education, and diagnosis.^7–9

The use of ChatGPT in the field of education solves educational problems in less than a few minutes.¹⁰ In the field of transportation, it can control traffic.¹¹ It also improves the quality of products during the production process.¹² In the field of business, it improves the quality and effectiveness of services provided to customers.¹³ ChatGPT, as an AI application, is utilized in healthcare for various tasks like analyzing literature, functioning as a dialogue agent, aiding in medical education, research, and clinical management. It collects patient data, updates medical professionals with recent advancements, and suggests treatments based on symptoms.^14,15 In this regard, as presented in a study, ChatGPT-3.5 and ChatGPT-4 have demonstrated strong performance on fundamental healthcare leadership and management inquiries. Notably, ChatGPT-4 has been presented to outperform its predecessor, making it the preferred choice for leadership and management training as well as for acquiring information on various hospital leadership and management topics.¹⁶

As mentioned, all these articles address specific applications or benefits of ChatGPT in healthcare services. This particular paper, however, employs a systematic review to identify the benefits of ChatGPT in the healthcare sector and then prioritizes the healthcare building blocks based on these identified benefits. This approach aims to provide a comprehensive understanding of the benefits of ChatGPT in enhancing healthcare delivery.

Several review studies have investigated the advantages of employing ChatGPT in healthcare services.^9,17,18 Conversely, other publications have underscored the limitations and challenges associated with the use of ChatGPT. One systematic review conducted by Sallam (2023), explored the potential benefits and limitations of ChatGPT in healthcare education, research, and practice. It discussed its applications, such as efficient data analysis, code generation, and personalized medicine, and raised concerns about ethical issues, lack of originality, and inaccurate citations.¹⁸ Another study in the field of medical education has shown that ChatGPT scored near the passing threshold on important medical exams. It has also demonstrated the ability to write scientific abstracts at an acceptable level. However, the results of this paper have shown that the professional use of ChatGPT for academic writing should be approached with caution.¹⁹ Another systematic review conducted by Garg et al. (2023) analyzed 118 articles and found that ChatGPT could assist with patient care and research tasks, but also highlighted concerns about its accuracy, reliability, and ethical implications. The review concluded that while ChatGPT could serve as a clinical assistant and aid in medical research and scholarly writing, it also presented limitations and ethical considerations that needed to be addressed.¹⁷ Bias, transparency, privacy, accountability, equity, trust, and replacement have been identified as major ethical challenges associated with the use of AI in healthcare.²⁰

Furthermore, Sadaghat (2024) has underscored the challenges associated with the utilization of ChatGPT in medical research. A significant limitation is the risk of presenting incorrect or unverified content, which could undermine credibility. Additionally, the absence of transparent references and the potential for copyright or plagiarism issues present substantial challenges to the reliable use of ChatGPT in professional medical studies.²¹ In another work, Sadaghat indicates that the primary challenge of using ChatGPT in daily medical practice is the necessity to carefully validate its outputs and ensure it is employed to complement, rather than replace, physicians’ clinical expertise and decision-making.²²

In this context, it has been reported that the responsibility for patient harm resulting from ChatGPT-generated advice remains unclear, necessitating the establishment of clear legal frameworks to define accountability and protect patient data. The use of ChatGPT may disrupt the traditional physician-patient relationship, potentially undermining essential compassion and trust. Conversely, overreliance on AI could diminish the humanistic aspects of care, leading to reduced patient adherence to treatment plans. Transparency regarding AI-generated content is crucial to maintaining trust and integrity in healthcare. Additionally, algorithmic biases in AI systems, such as those in ChatGPT, can perpetuate existing inequalities in healthcare. Finally, ensuring the accuracy and reliability of AI-generated information requires continuous updates and rigorous validation against clinical standards.^23–25

Gala and Makaryus (2023) investigated the benefits and concerns of using ChatGPT in the fields of cardiology and their findings discussed the importance of providing adequate training to healthcare professionals who use artificial intelligence tools. In their research, they found out that the incorrect use of artificial intelligence technology can have negative consequences such as wrong diagnosis or incorrect treatment decisions for the patient. Additionally, if an AI model is not properly trained or validated, it may produce false or misleading results that could lead to unintended consequences.²⁶ In the course of this research, we employed the framework of building blocks as established by the World Health Organization (WHO) to comprehend the categorized domains within the network of healthcare service delivery. This framework is architecturally designed with six fundamental components: provision of services, healthcare personnel, health data systems, medical commodities, immunizations and technologies, funding, and leadership/administration. The framework offers a collection of metrics and evaluative indicators to gauge the efficacy of these fundamental components and to monitor advancements in fortifying healthcare systems.^27,28

As observed, while multiple systematic reviews have explored the advantages of employing ChatGPT in healthcare services, none have aimed to prioritize the most significant benefits. This can be regarded as a significant gap within the literature; Therefore, this study sought to address this gap by providing scientific evidence on the existing benefits of using ChatGPT within the healthcare system. Thus, the ultimate aims of the study are

Explore and prioritize the benefits of utilizing ChatGPT in healthcare services.

Determine the potential domains of ChatGPT expansion within the healthcare industry.

This study employs a mixed-methods approach, integrating findings from existing literature with expert opinions in the field through the application of multi-criteria decision-making techniques. This methodology represents a novel contribution to the relevant literature and offers valuable and comprehensive insights for stakeholders regarding the utilization of ChatGPT and its potential applications within the healthcare sector. It provides the findings of other papers within the literature and the views of experts within a single document. The findings can be utilized by managers and policymakers within the healthcare sector, identifying potential areas where ChatGPT can be employed and ultimately achieving the envisioned outcomes through such utilization. Furthermore, the findings of this paper can be utilized by future researchers, highlighting the most important areas for conducting research and offering detailed information on the potential impacts of ChatGPT utilization.

Material and method

Methodology

This research, conducted in 2023, utilized a mixed-methods approach, integrating both qualitative and quantitative methodologies, and was executed in three distinct phases. The primary objective of the study was to investigate and prioritize the advantages of implementing ChatGPT in healthcare services during phases one and two. In phase three, the study aimed to identify potential areas for the expansion of ChatGPT within the healthcare sector, as detailed in the subsequent section on objectives and research questions.

Objectives and research question

Phase one

The objective of the systematic review was to systematically examine the benefits of employing ChatGPT within healthcare services, as documented in the literature. The Joanna Briggs approach was employed for this purpose.²⁹ Initially, a research question was formulated for this phase of the study in collaboration with the authors, which was: “What are the benefits of utilizing ChatGPT within healthcare services according to the existing literature?”

Phase two

In this phase of the study, we aimed to prioritize ChatGPT benefits utilizing the gray SWARA (Gray Stepwise Weight Assessment Ratio Analysis) methodology. The research question for this phase of the study was formulated as: “How ChatGPT benefits are prioritized in terms of their importance according to the study experts?”

Phase three

In this phase of the study, we aimed to determine the potential building blocks for ChatGPT expansion within healthcare systems using the Gray MABAC (Gray Multi-Attributive Border Approximation Area Comparison) methodology. The research question for this phase of the study was formulated as: “What are the potential building blocks for ChatGPT expansion within healthcare systems according to the study experts?”

Data collection

Phase one

A systematic exploration was undertaken to locate all published articles pertinent to the research topic, within the timeframe of 2000–2023, and exclusively in English. The databases of PubMed, Scopus, Cochrane, and ProQuest were utilized for this search since they are considered the most prominent databases indexing studies regarding the context.³⁰ Medical Subject Headings (MeSH) terms were employed to classify all keywords into two categories: benefits, and ChatGPT. Synonymous keywords were amalgamated using the logical operator “OR”. Subsequently, the logical operator “AND” was applied to consolidate the first, second, and third groups of keywords. The references were managed using EndNote 21.2 software. The search strategy employed to identify relevant literature is depicted in Table 1, and was registered in GitHub with the following link:

Table 1.
The search strategy utilized within the systematic review.

Research question What are the benefits of using ChatGPT in healthcare?

Key concepts or terms Benefits, ChatGPT

Databases or sources Cochrane Library, PubMed, web of science, and Scopus.

Time-period 2022–2023

Language English

#1 Benefit* OR advantage* OR outcome* OR merit* OR opportunity*

#2 ChatGPT

Final strategy #1 AND #2

https://github.com/mohsenkhosravi3913/ChatGPT-Utilization-within-the-Building-Blocks-of-the-Healthcare-Services-

The inclusion criteria encompassed:
Papers addressing the research question.

Papers published within the timeframe from 2000 to 2023.

Papers written in English.
The exclusion criteria encompassed:
Brief communications, letters to the editor, and conference papers.

Papers identified as duplicates.
In this stage of the study, duplicate studies were eliminated, and the remaining ones were screened based on their titles and abstracts. Studies not pertinent to our research objective were discarded, and the full text of the remaining articles was reviewed. Only those that satisfied our eligibility criteria were included in the final analysis. Furthermore, only studies providing data relevant to the research question were included in the study. This entire process was independently executed by two researchers. Data corresponding to the study objective were independently extracted from the final studies by two authors. A third author was consulted in case of any conflict of views between the initial two authors.

Phase two

In this phase of the study, the data was derived from a sample of experts in the medical sciences field who have proficiency in ChatGPT and its utilization in healthcare services. In this regard, eight experts were utilized comprising of physicians, pharmacologists, healthcare practitioners, and researchers. The experts were free to withdraw from the study if they presented their unwillingness to participate in the research (exclusion criteria). The criteria for inclusion in the study were as follows:
General familiarity with ChatGPT and its application in healthcare services.

General experience in using ChatGPT in the delivery of healthcare services or conducting healthcare research.

Possession of a PhD. degree in a healthcare field or being a practicing physician.
Data collection was conducted using a questionnaire formulated based on the data obtained from the review conducted in the initial phase of the study. During this process, the benefits of ChatGPT derived from the data were integrated into the questionnaire to be assessed and ranked using the gray SWARA (SWARA-G) method. Consequently, data were collected through two different questionnaires. The first questionnaire was designed for the initial ranking of the identified benefits, while the second questionnaire was used for the final weighting of the benefits. These questionnaires were subsequently administered to experts in two consecutive stages. Ultimately, the benefits were weighted and ranked based on their importance according to the opinions of these experts.

Phase three

In this phase of the study, data collection was conducted using the same sample of experts as in phase two. Meanwhile, the inclusion and exclusion criteria remained consistent. The data collection process was executed via a questionnaire. This questionnaire was constructed by leveraging the data obtained from the thematic analysis conducted in the initial phase of the study. Additionally, it incorporated the framework of monitoring the building blocks of health systems, a concept published by the World Health Organization (WHO) in order to prioritize potential healthcare domains for ChatGPT expansion.²⁷ This study utilized several items from the framework, including the following:
Infrastructure, Information and Communication Technologies, which underscore the need for robust and reliable infrastructure and information technologies in health systems.²⁷

Health Workforce, focusing on the accessibility, skill diversity, and distribution of the healthcare workforce, representing one of the six fundamental components of the WHO framework for monitoring healthcare system building blocks.²⁷

Information, highlighting the need for reliable and robust data for effective health system monitoring.²⁷

Intervention Access and Services Readiness, aligning with the service delivery block of the WHO's building blocks, focusing on the accessibility and readiness of healthcare services for its users.²⁷

Intervention Quality, Safety, aligning with the service delivery block of the WHO's building blocks, focusing on the quality and safety of services provided to beneficiaries within healthcare systems.²⁷
Upon integrating the benefits derived from the thematic analysis of the systematic review and the building blocks proposed by the WHO framework into the final questionnaire, the Gray MABAC (Multi-Attribute Border Approximation Area Comparison) methodology was employed to facilitate the ranking and prioritization of the health blocks. Therefore, the required data has been collected through a questionnaire that was designed for this method. At the next step, the questionnaire was disseminated to the experts. Appendix A (Questionnaire) presents the content of the questionnaire.

Data analysis

Phase one

During this phase of the research, the thematic analysis approach formulated by Braun and Clarke was utilized. This approach consists of six stages: data familiarization, data coding, theme generation, theme review, theme definition and naming, and the final report.³¹

At the outset, the authors initiated a process of becoming acquainted with the subject matter and the research context by conducting a comprehensive examination of the content pertinent to the topic. Following this, the texts relevant to the research question in the data from the final studies were subjected to coding. In the subsequent stage, sub-themes and themes were derived from the coded data through a process of categorization and grouping of the codes. The authors then undertook multiple reviews of the generated themes to ensure the process's validity and reliability and to avoid any risk of bias. In the next step, the authors delineated and assigned names to the themes and their sub-themes based on their intrinsic and existential attributes. Finally, the authors amalgamated the generated themes, sub-themes, and their respective codes into a unified document. MAXQDA 2020, a prominent software for conducting qualitative analysis, was utilized for the analysis.³²

Phase two

The Swara-G method combines gray theory with the Swara approach to achieve the benefits of this combination. As a valuable decision-making methodology, the Swara-G method proves instrumental in analyzing situations characterized by uncertainty. Also, this approach has a simpler evaluation process than other methods such as analytical hierarchy process (AHP).³³ For instance, in scenarios where there are 11 criteria requiring evaluation, the Swara-G method necessitates a mere 10 pairwise comparisons, whereas gray’s Analytic Hierarchy Process (AHP-G) entails 55 pairwise comparisons by the expert for problem resolution.

Swara's approach incorporates the expert's perspective regarding the validity of weight criteria. In other words, Swara's method provides the possibility to evaluate and determine the weight for the criteria and considers the expert's opinion about the correctness of this weight. Moreover, within this methodology, experts possess the opportunity to engage in collaborative consultations and mutually cooperate, consequently yielding more precise and robust outcomes in contrast to alternative approaches employed in multi-criteria decision-making.³⁴ In addition, Swara's method requires simple computational steps, which makes it very user-friendly.³⁵ Moreover, gray theory can model and predict in uncertain and incomplete conditions.³⁶

Compared to the standard SWARA method, the gray SWARA approach provides a more robust and comprehensive evaluation of the criteria, leading to better-informed decisions. The key steps of gray SWARA are as follows: identify the criteria and have decision-makers provide assessments using gray numbers; rank the criteria from most to least important; determine the comparative importance of the average gray value for each criterion; calculate the gray coefficient and gray weights; and normalize the gray weights to obtain the final prioritization.

This gray SWARA analysis enabled decision-makers to make more informed decisions in a military context, considering the uncertainty and vagueness in the evaluation criteria. The use of gray numbers helps decision-makers better reflect the real-world complexities they face.³⁶

In our study, we have employed the gray SWARA method to weigh the benefits of ChatGPT in the field of healthcare. So, through the utilization of gray theory, the analysis of uncertain information can be effectively incorporated into modeling processes, enabling the amalgamation of such data with definitive information to yield more precise outcomes.

Steps of Swara-G

Step 1: The first step in the Swara-G method is to identify and define the elements that must be weighted.

Step 2: Ranking factors

In this step, the experts assign ranks to the criteria in descending order of importance, from the most significant criterion to the least significant criterion.
$j : criterion; j = 1, 2, 3, \dots, n$

$d : decision maker; d = 1, 2, 3, \dots, d$

${\begin{matrix} j = 1 \to the most important criterion \\ j = n \to the least important criterion \end{matrix}$
Step 3: Determining the relative importance of indicators

During this stage, decision-makers assess the gray relative importance values. Specifically, the relative significance of each criterion or sub-criterion is evaluated in relation to its preceding criterion or sub-criterion, based on the prevailing research conditions.

$\underline{s_{j d}} :$ lower limit of gray evaluation according to decision-maker d criterion j

$\bar{s_{j d}}$ : upper limit of gray evaluation according to decision-maker d criterion j

Step 4: Calculation of comparative gray coefficients

The initial phase of the mathematical procedure in the Swara-G method involves the computation of gray comparison coefficients utilizing equations (1) and (2):
$\underline{k_{j d}} : lower limit of gray comparative coefficient$

$\bar{k_{j d}} : upper limit of gray comparative coefficient$

${\begin{matrix} j = 1 \to \underline{k_{j d}} = 1 \\ j > 1 \to \underline{k_{j d}} = 1 + \underline{s_{j d}} \end{matrix};$
(1)
${\begin{matrix} j = 1 \to \bar{k_{j d}} = 1 \\ j > 1 \to \bar{k_{j d}} = 1 + \bar{s_{j d}} \end{matrix};$
(2)
Step 5: Obtaining unscaled gray weights for criteria

In this step, using equations (3) and (4), unscaled gray weights of criteria and sub-criteria are obtained.
$\underline{q_{j d}} : lower limit of gray unscaled weight$

$\bar{q_{j d}} = upper limit of gray unscaled weight$

${\begin{matrix} j = 1 \to \underline{q_{j d}} = 1 \\ j > 1 \to \underline{q_{j d}} = \frac{\underline{q (j - 1) d}}{\bar{k j d}} \end{matrix};$
(3)
${\begin{matrix} j = 1 \to= \bar{q_{j d}} = 1 \\ j > 1 \to \bar{q_{j d}} = \frac{\bar{q_{(j - 1) d}}}{\underline{k_{j d}}} \end{matrix};$
(4)
Step 6: Obtaining the scaled gray weights

In this step, the scaled gray weights are calculated using equations (5) and (6).
$\underline{w_{j d}} : lower limit of gray scaled weight$

$\bar{w_{j d}} : upper limit of gray scaled weight$

$\underline{w_{j d}} = \frac{\underline{q_{j d}}}{\sum_{j = 1}^{n} \underline{q_{j d}}};$
(5)
$\bar{w_{j d}} = \frac{\bar{q_{j d}}}{\sum_{j = 1}^{n} \bar{q_{j d}}};$
(6)
Step 7: Obtaining the scaled weights

In this step, the scaled weights are calculated using equation (7).
$w_{j d} = \frac{\underline{\underline{w_{j d}} \bar{w_{j d}}}}{\sum_{j = 1}^{n} [\underline{w_{j d}} + \bar{w_{j d}}]};$
(7)
Step 8: Aggregation of experts’ opinions.

In this step, Experts’ opinions are integrated using equation (8).
$w_{j} = \frac{\sum_{d = 1}^{D} w_{j d}}{D};$
(8)

Phase three

The framework for utilizing the MABAC (Multi-Attributive Border Approximation Area Comparison) method involves defining the distance of the criterion function from each alternative, based on the BAA (Boundary Approximation Area). The MABAC-G represents an extended version of the crisp MABAC. The gray MABAC is a multi-criteria decision-making technique used to evaluate and rank a set of alternatives based on various performance criteria. The gray MABAC method involves several steps, including the normalization of the decision matrix, the calculation of the weighted normalized decision matrix, the determination of the ideal and anti-ideal solutions, and the ranking of the alternatives based on their proximity to the ideal solution. It is particularly useful when dealing with uncertain or incomplete information, as it can incorporate gray numbers into the decision-making process.³⁶

Our study employs the gray MABAC multi-criteria decision-making technique to systematically prioritize the healthcare building blocks according to the potential benefits of integrating ChatGPT. In other words, the gray MABAC analysis determines the healthcare domains where ChatGPT can be most advantageously utilized. By calculating the distance of each healthcare building block from the ideal and anti-ideal solutions, the gray MABAC method identifies and ranks the top-priority domains where ChatGPT can be most beneficially utilized.

The procedure for implementing the MABAC-G, encompasses the following steps³⁷:

Step 1: The construction of the aggregate gray decision matrix is the initial step. Given “m” alternatives, “n” criteria, and “k” experts, the aggregate gray decision matrix is derived through the application of equation (8).
$\begin{aligned} \hat{X} = & [\otimes x_{i j}]_{m \times n} \\ = {[\begin{matrix} [{\underline{x}}_{11}, {\bar{x}}_{11}] & [{\underline{x}}_{12}, {\bar{x}}_{12}] & \dots & [{\underline{x}}_{1 n}, {\bar{x}}_{1 n}] \\ [{\underline{x}}_{21}, {\bar{x}}_{21}] & [{\underline{x}}_{22}, {\bar{x}}_{22}] & \dots & [{\underline{x}}_{2 n}, {\bar{x}}_{2 n}] \\ \dots & \dots & \dots & \dots \\ [{\underline{x}}_{m 1}, {\bar{x}}_{m 1}] & [{\underline{x}}_{m 2}, {\bar{x}}_{m 2}] & \dots & [{\underline{x}}_{m n}, {\bar{x}}_{m n}] \end{matrix}]}_{m \times n} \\ {\underline{x}}_{i j} = & \sum_{k = 1}^{K} σ_{k} . {\underline{x}}_{i j}^{k}; and {\bar{x}}_{i j} = \sum_{k = 1}^{K} σ_{k} . {\bar{x}}_{i j}^{k} \end{aligned}$
(8)
Step 2: Normalization of the gray decision matrix (⊗N). Normalization for both benefit and cost indicators is performed using equation 9. In the context of this research, it is important to note that the first and fourth indicators represent higher emissions and pollution (negative), while the remaining indicators are associated with less emission and pollution (positive). The normalized gray decision matrix is represented as per equation 10.
$\begin{aligned} \otimes y_{i j} = & [{\underline{y}}_{i j}, {\bar{y}}_{i j}] = [\frac{{\underline{x}}_{i j}}{x_{j}^{max}}, \frac{{\bar{x}}_{i j}}{x_{j}^{max}}] if j \in B \\ \otimes y_{i j} = & [{\underline{y}}_{i j}, {\bar{y}}_{i j}] = [\frac{x_{j}^{min}}{{\bar{x}}_{i j}}, \frac{x_{j}^{min}}{{\underline{x}}_{i j}}] if j \in C \end{aligned}$
(9)
$\hat{Y} = [\otimes y_{i j}]_{m \times n} = {[\begin{matrix} [{\underline{y}}_{11}, {\bar{y}}_{11}] & [{\underline{y}}_{12}, {\bar{y}}_{12}] & \dots & [{\underline{y}}_{1 n}, {\bar{y}}_{1 n}] \\ [{\underline{y}}_{21}, {\bar{y}}_{21}] & [{\underline{y}}_{22}, {\bar{y}}_{22}] & \dots & [{\underline{y}}_{2 n}, {\bar{y}}_{2 n}] \\ \dots & \dots & \dots & \dots \\ [{\underline{y}}_{m 1}, {\bar{y}}_{m 1}] & [{\underline{y}}_{m 2}, {\bar{y}}_{m 2}] & \dots & [{\underline{y}}_{m n}, {\bar{y}}_{m n}] \end{matrix}]}_{m \times n}$
(10)
Step 3: Calculation of the weighted normalized decision matrix. The elements of the weighted matrix, denoted as ⊗f_ij, are derived using equation 11. Here, W_j represents the weight of the “j-th” criterion, which is calculated based on the gray MEREC method.
$\begin{aligned} \otimes f_{i j} = [{\underline{f}}_{i j}, {\bar{f}}_{i j}] = W_{j} \times \otimes y_{i j} = [W_{j} . {\underline{y}}_{11}, W_{j} . {\bar{y}}_{11}] \\ \hat{F} = [\otimes f_{i j}]_{m \times n} \\ = {[\begin{matrix} [{\underline{f}}_{11}, {\bar{f}}_{11}] & [{\underline{f}}_{12}, {\bar{f}}_{12}] & \dots & [{\underline{f}}_{1 n}, {\bar{f}}_{1 n}] \\ [{\underline{f}}_{21}, {\bar{f}}_{21}] & [{\underline{f}}_{22}, {\bar{f}}_{22}] & \dots & [{\underline{f}}_{2 n}, {\bar{f}}_{2 n}] \\ \dots & \dots & \dots & \dots \\ [{\underline{f}}_{m 1}, {\bar{f}}_{m 1}] & [{\underline{f}}_{m 2}, {\bar{f}}_{m 2}] & \dots & [{\underline{f}}_{m n}, {\bar{f}}_{m n}] \end{matrix}]}_{m \times n} \end{aligned}$
(11)
Step 4: Determination of the gray border approximation area (BBA) matrix $\hat{G}$ . The determination of this matrix is predicated on the geometric mean for each criterion, as outlined in equation (12).
$\begin{aligned} \otimes g_{j} = [{\underline{g}}_{j}, {\bar{g}}_{j}] = [{(\prod_{i = 1}^{m} {\underline{f}}_{i j})}^{\frac{1}{m}}, {(\prod_{i = 1}^{m} {\bar{f}}_{i j})}^{\frac{1}{m}}] \\ \hat{G} = {[\begin{matrix} [{\underline{g}}_{1}, {\bar{g}}_{1}] & [{\underline{g}}_{2}, {\bar{g}}_{2}] & \dots & [{\underline{g}}_{n}, {\bar{g}}_{n}] \\ [{\underline{g}}_{1}, {\bar{g}}_{1}] & [{\underline{g}}_{2}, {\bar{g}}_{2}] & \dots & [{\underline{g}}_{n}, {\bar{g}}_{2 n}] \\ \dots & \dots & \dots & \dots \\ [{\underline{g}}_{1}, {\bar{g}}_{1}] & [{\underline{g}}_{2}, {\bar{g}}_{2}] & \dots & [{\underline{g}}_{n}, {\bar{g}}_{n}] \end{matrix}]}_{m \times n} \end{aligned}$
(12)
Step 5: Calculation of the preference index matrix (Q). The computation of this matrix involves the use of the Euclidean distance between the gray numbers ⊗f_ij and ⊗g_j, as detailed in equation (13).
$Q = \hat{F} - \hat{G} = [q_{i j}^{k}]_{m \times n} = {[\begin{matrix} d (\otimes f_{11}, \otimes g_{1}) & d (\otimes f_{12}, \otimes g_{2}) & \dots & d (\otimes f_{1 n}, \otimes g_{n}) \\ d (\otimes f_{21}, \otimes g_{1}) & d (\otimes f_{22}, \otimes g_{2}) & \dots & d (\otimes f_{2 n}, \otimes g_{n}) \\ \dots & \dots & \dots & \dots \\ d (\otimes f_{m 1}, \otimes g_{n}) & d (\otimes f_{m 2}, \otimes g_{n}) & \dots & d (\otimes f_{m n}, \otimes g_{n}) \end{matrix}]}_{m \times n}$
(13)
The calculation of preference indices for both benefit and cost criteria is conducted using the respective equations provided below:
$q_{i j} = {\begin{matrix} d (\otimes f_{i j}, \otimes g_{j}) & i f \otimes f_{i j} > \otimes g_{j} \\ - d (\otimes f_{i j}, \otimes g_{j}) & i f \otimes f_{i j} < \otimes g_{j} \end{matrix}$
(14)
$q_{i j} = {\begin{matrix} - d (\otimes f_{i j}, \otimes g_{j}) & i f \otimes f_{i j} > \otimes g_{j} \\ d (\otimes f_{i j}, \otimes g_{j}) & i f \otimes f_{i j} < \otimes g_{j} \end{matrix}$
(15)
Step 6: Prioritization of the alternatives. The Closeness Coefficient (CC) for each alternative, relative to the boundary approximation area (BAA), is computed by summing the elements of each row in the matrix (Q), as defined in equation (16). It should be noted that an alternative's priority increases with the value of its CC.
$C C (A_{i}) = \sum_{j = 1}^{n} q_{i j} = \sum_{j = 1}^{n} d (\otimes f_{i j}, \otimes g_{j}); i = 1, 2, \dots, n .$
(16)

Ethical considerations

All procedures were conducted in strict adherence to relevant ethical guidelines and regulations, including the Helsinki Declaration of 1975.³⁸ In this regard, Informed consent was obtained from all members of the expert panel, who were also given the freedom to withdraw from the study at any time upon request.

Results

The findings of the study are presented in the subsequent sections.

Phase one: identification of ChatGPT benefits through systematic review

As demonstrated in Figure 1, from the total of 1049 studies retrieved from various databases, 86 were identified as duplicates. Upon meticulous screening of the titles, abstracts, and full texts of the remaining manuscripts, a final selection of 74 studies was included in the research. All of the studies were published in 2023 and consisted of both qualitative and quantitative methodologies. Furthermore, the studies were conducted in diverse regions, including the United States, the United Kingdom, Saudi Arabia, and multiple other locations.

Figure 1.
PRISMA diagram of the systematic review.

Thematic analysis

As delineated in Table 2, the thematic analysis of the data acquired through the final studies yielded 11 distinct themes. These themes encompassed Medical Documentation and Insights, Healthcare Information and Education, Clinical Decision-Making, Healthcare Research, Healthcare Writing, Medical Diagnosis, Medical Procedures, Healthcare Surveys, Privacy, Patient-Centered Care, and Administrative Tasks.

Table 2.
Thematic analysis on the data acquired from the included studies.

No. Theme Sub-theme Reference

1 Medical documentation and insights Documenting and summarizing medical records ^39–69

2 healthcare information and education Providing information, updates, and explanations to healthcare professionals ^{39,41,43,44,47,48,50–56,58–64,66,67,69–95}

Helping healthcare professionals stay informed about new developments in their respective fields

Providing patient communication support

Answering common questions

3 Clinical decision-making Assisting healthcare professionals in making clinical decisions ^{26,43,49,52–54,62,67,69,72,74,75,78,80,88,91,96–101}

Providing evidence-based recommendations

4 Healthcare research Answering questions and providing feedback to medical students ^{9,14,40,41,45,53,55,57,58,63,64,70,72,73,76,77,83,86,91,94,96,98,102–104}

Assisting researchers in analyzing large datasets

Generating hypotheses

5 Healthcare writing Assisting in healthcare writing ^{40,41,43,45,51,53,57,58,66,67,72,75,77,96,102,103}

6 Medical diagnosis Assisting medical diagnosis ^{14,50–52,54,59–61,70,75,89,90,101,104–107}

7 Medical procedures Improving patient outcomes through AI technology ^{39,55,62,70,75,83,85,93,95,104,108,109}

8 Healthcare surveys Assisting in drafting patient surveys ^40,48,75

Streamlining the process of collecting and analyzing data

9 Privacy Assisting in deidentifying patient data ^48,60,86

Maintaining patient privacy and confidentiality

10 Patient-centered care Providing personalized and timely responses to patients’ inquiries ^{41,50,72,83,87–90,92,97,98}

Supporting symptom tracking and medication adherence

Offering mental health support through conversational interfaces

11 Administrative tasks Automating administrative tasks such as scheduling appointments, managing medical records, and handling insurance. ^{9,41,48,70,85,87,94,98,108}

Medical documentation and insights

ChatGPT can quickly summarize the documented medical records, making it easier for doctors to review patient histories and make informed decisions. Furthermore, ChatGPT can assist with report summaries, making it easier for doctors to review and interpret imaging and examination results.^39–69

Healthcare information and education

ChatGPT can aid in medical education by providing information, updates, and explanations to healthcare professionals and patients, helping them stay informed about new developments in the professionals` respective fields and various health conditions and treatment options for patients while answering common questions and providing information about procedures and treatments.^{39,41,43,44,47,48,50–56,58–64,66,67,69–95}

Clinical decision-making

ChatGPT can assist healthcare professionals in making clinical decisions by providing them with evidence-based recommendations.^{26,43,49,52–54,62,67,69,72,74,75,78,80,88,91,96–101}

Healthcare research

ChatGPT can support medical students in their learning process by answering questions and providing feedback. It can also assist researchers in analyzing large datasets and generating hypotheses.^{9,14,40,41,45,53,55,57,58,63,64,70,72,73,76,77,83,86,91,94,96,98,102–104}

Healthcare writing

ChatGPT can assist in healthcare writing, making it easier for healthcare professionals to draft reports and other documents.^{40,41,43,45,51,53,57,58,66,67,72,75,77,96,102,103}

Medical diagnosis

ChatGPT has the potential to assist with diagnosis of multiple illnesses, helping doctors to identify the conditions and diseases.^{14,50–52,54,59–61,70,75,89,90,101,104–107}

Medical procedures

ChatGPT can be utilized in clinical procedures, improving patient outcomes through AI technology.^{39,55,62,70,75,83,85,93,95,104,108,109}

Healthcare surveys

ChatGPT can assist in drafting patient surveys, streamlining the process of collecting and analyzing data.^40,48,75

Privacy

ChatGPT can assist in deidentifying patient data, ensuring compliance with HIPAA requirements, and maintaining patient privacy and confidentiality.^48,60,86

Patient-centered care

ChatGPT can provide personalized and timely responses to patients’ inquiries, support symptom tracking and medication adherence, and offer mental health support through conversational interfaces.^{41,50,72,83,87–90,92,97,98}

Administrative tasks

ChatGPT can automate administrative tasks such as scheduling appointments, managing medical records, and handling insurance.^{9,41,48,70,85,87,94,98,108}

Phase two: weighting of ChatGPT benefits through gray SWARA

In this phase, the gray SWARA method was used to weigh the benefits of ChatGPT in the field of healthcare. Based on the first and second stages of the gray SWARA method, the experts have ranked 11 potential benefits of ChatGPT in healthcare as indicators, ordered from the most important to the least important index. Then, the experts evaluated the relative importance of each indicator compared to the previous indicator. This evaluation was conducted using the linguistic terms of “much less important,” “less important,” “moderately less important,” and “relatively equal importance. Subsequently, the table of the relative importance of the indicators was converted into a table of gray relational importance values according to gray numbers. The final results are presented in Table 3. The average of the weights for each indicator is presented in the last column of this table as the final weights.

Table 3.
Calculation of scaled weights and final weights.

Expert 1 Expert 2 Expert 3 Expert 4 Expert 5 Expert 6 Expert 7 Expert 8 final weights

Indicator W1 W2 W3 W4 W5 W6 W7 W8 average

c3 0.142 0.14 0.142 0.127 0.133 0.143 0.105 0.153 0.135

c6 0.105 0.104 0.105 0.088 0.099 0.099 0.078 0.106 0.098

c7 0.073 0.072 0.073 0.061 0.083 0.068 0.058 0.073 0.07

c10 0.054 0.053 0.054 0.051 0.057 0.051 0.052 0.051 0.053

c4 0.04 0.037 0.04 0.043 0.039 0.042 0.045 0.037 0.04

c2 0.027 0.027 0.027 0.035 0.029 0.029 0.037 0.026 0.03

c1 0.019 0.022 0.019 0.026 0.02 0.021 0.032 0.018 0.022

c5 0.014 0.015 0.014 0.019 0.014 0.016 0.026 0.012 0.016

c8 0.01 0.011 0.01 0.016 0.01 0.011 0.018 0.009 0.012

c9 0.008 0.008 0.008 0.011 0.007 0.007 0.012 0.007 0.008

c11 0.005 0.006 0.005 0.009 0.005 0.006 0.009 0.005 0.006

As depicted in Table 4, certain benefits of ChatGPT were accorded significantly greater priority than others. The benefits that were given higher precedence encompassed clinical decision-making (weight: 0.135), medical diagnosis (weight: 0.098), medical procedures (weight: 0.070), and patient-centered care (weight: 0.053). This indicates that ChatGPT appears to confer substantial benefits to the medical domain of healthcare services, far exceeding its impact on other service domains such as administrative tasks, which was ranked as the least prioritized benefit (weight: 0.006). Figure 2 illustrates the final weights of the benefits of using ChatGPT in the healthcare sector. As the figure presents, clinical decision-makings and medical diagnosis significantly surpass the rest of ChatGPT benefits in terms of importance and priority.

Figure 2.
Radar chart of the weights of the benefits of using ChatGPT.

Table 4.
Ranking of ChatGPT benefits in terms of importance and priority.

Benefits Weight Ranking

C1 Medical documentation and insights 0.022 7

C2 healthcare information and education 0.030 6

C3 Clinical decision-making 0.135 1

C4 Healthcare research 0.040 5

C5 Healthcare writing 0.016 8

C6 Medical diagnosis 0.098 2

C7 Medical procedures 0.070 3

C8 Healthcare surveys 0.012 9

C9 Privacy 0.008 10

C10 Patient-centered care 0.053 4

C11 Administrative tasks 0.006 11

Phase three: prioritization of building blocks according to ChatGPT benefits through MABAC-G

As illustrated in Table 5, the domains of Information and Infrastructure, Information and Communication Technologies were identified as the most critical domains of healthcare system building blocks, possessing the greatest potential for ChatGPT expansion. Conversely, the domain of Intervention Access and Services Readiness was regarded as having the least potential.

Table 5.
Ranking of building blocks according to ChatGPT benefits.

Indicator domain (building blocks) CC Ranking

A1 Infrastructure, Information, and communication technologies 0.0333 2

A2 Health workforce 0.0155 3

A3 Information 0.0348 1

A4 Intervention access and services readiness −0.0100 5

A5 Intervention quality, safety 0.0074 4

Sensitivity analysis of the results

The key limitation of using the gray MABAC method in this study is the inherent subjectivity involved in the decision-making process. The gray MABAC method used to evaluate the potential expansion domains of ChatGPT is sensitive to the choice of reference points. The selection of these reference points can significantly impact the final rankings, introducing another source of subjectivity that may limit the reliability and replicability of the findings. To address these limitations and to enhance the validity of the results, in this study, the healthcare system building blocks were prioritized using several methods. The results are presented in Table 6.

Table 6.
Sensitivity analysis of option rankings.

Indicator domain (building blocks) Ranking (MABAC) Ranking (TOPSIS) Ranking (COPRAS)

A1 Infrastructure, information, and communication technologies 2 2 2

A2 Health workforce 3 5 4

A3 Information 1 1 1

A4 Intervention access and services readiness 5 4 5

A5 Intervention quality, safety 4 3 3

According to Table 6, in all three methods, ChatGPT provides the greatest benefit in the information block. Additionally, in all three methods, the facilities and information and communication technology block hold the second position.

Discussion

In this research, the SWARA-G method was utilized to weigh the benefits of using ChatGPT in the health system, and the MABAC-G method was employed to prioritize the health building blocks within these benefits in a gray environment. Additionally, Microsoft Excel was used to implement the SWARA-MABAC method in our study. The mathematical structure of the MABAC approach remains constant, regardless of the number of alternatives and criteria. This approach demonstrates the capability to be applied to a greater multitude of alternatives and criteria, thereby offering a distinct ranking of alternatives accompanied by numerical values, which enhances the comprehensibility of the results. Furthermore, this method can be used for both qualitative and quantitative criteria.¹¹⁰

The incorporated gray SWARA-MABAC approach has significant advantages over other multi-criteria decision-making methods. In this manner, the relationship between the benefits of ChatGPT within the health blocks can be accurately identified. This methodology holds significant potential in informing managerial and strategic decisions within the healthcare domain, enhancing both accuracy and predictability. By providing decision-makers with a comprehensive understanding of the interrelationships among the investigated factors, it enables more optimal decision-making processes.

In this segment of the study's outcomes, a detailed analysis and discussion will be conducted, specifically focusing on the prioritization of benefits derived from ChatGPT and potential healthcare building blocks for the expansion of ChatGPT.

Review of ChatGPT benefits

As presented within the results section, the thematic analysis of the data acquired through the review yielded 11 themes. The themes included Medical Documentation and Insights, Healthcare Information and Education, Clinical Decision-Making, Healthcare Research, Healthcare Writing, Medical Diagnosis, Medical Procedures, Healthcare Surveys, Privacy, Patient-Centered Care, and Administrative Tasks. Among the themes, Healthcare Information and Education had the highest share of citations among the included studies, being cited by 70% of them. Medical Documentation and Insights was another theme with a significant share of citations, cited by 41% of the studies.

The results corresponding to this section of the study highlighted that ChatGPT's benefits, including clinical decision-making, medical diagnosis, medical procedures, and patient-centered care, were prioritized as the most important. These findings indicated significantly higher potential benefits of ChatGPT in the medical domain of healthcare services, surpassing its benefits in administrative tasks suggesting that policy makers, manufacturers, and researchers in the healthcare industry should focus on enhancing ChatGPT's benefits within clinical domains rather than administrative and nonmedical ones, which can be a significant implication derived from the findings of the study. As the results of the study indicated, the highest potential benefits of ChatGPT within the realm of healthcare services was proposed to be within the scope of clinical domains of healthcare systems. In such context, ChatGPT's capability for continuous clinical decision assistance is demonstrated to possess an overall precision of 71.7% across a variety of clinical scenarios. This indicates that ChatGPT exhibits remarkable accuracy in clinical decision-making, with its proficiency escalating as it acquires more clinical data.¹¹¹ Nonetheless, it has been disclosed that healthcare professionals exhibit moderate to low degrees of confidence in ChatGPT's capacity to formulate medical decisions. Moreover, there are abundant apprehensions regarding the accuracy, dependability, and medicolegal consequences of the data provided by ChatGPT.⁸⁹

Another prioritized potential benefit of employing ChatGPT within healthcare services was the enhancement of patient-centered care. Patient-centered care is an approach that values patients’ experiences, needs, and preferences and necessitates active understanding from healthcare organizations and professionals; It fosters partnerships among practitioners, patients, and their families, ensuring decisions align with patients’ desires and needs.^112,113 In this regard, ChatGPT has the potential to enhance the conveyance of discoveries, foster interdisciplinary cooperation, and streamline workflow, thereby augmenting the efficacy in pinpointing and directing patients in their postoperative care.^114,115 Moreover, ChatGPT is suggested as a solution to surmount linguistic obstacles encountered by patients who are not native English speakers, thus promoting communication that is centered around the patient.¹¹⁶ Furthermore, ChatGPT can provide support to healthcare professionals in addressing patient inquiries, composing medical notes, discharge summaries, and treatment plans, which in turn bolsters the efficiency and precision of healthcare provision.¹⁷

Administrative tasks were regarded as the least significant benefit of ChatGPT within the context of healthcare services. The low prioritization of utilizing chatbots for administrative tasks appears to be associated with the seemingly less significant outcomes of artificial intelligence applications in administrative domains compared to those in clinical settings, as evidenced by the existing literature.^117,118 Another factor contributing to the low prioritization of AI in healthcare may be the ethical concerns associated with its implementation. These concerns encompass issues such as data privacy, algorithmic bias, and accountability for decisions made by AI systems.^119,120 Furthermore, the potential replacement of humans by AI technology in administrative and decision-making roles is expected to be limited due to these ethical considerations.^121,122

Review the prioritization of potential healthcare building blocks for ChatGPT expansion

The study results highlighted that the building blocks of Information, and Infrastructure, Information and Communication Technologies, hold significant potential for ChatGPT expansion. However, the Intervention Access and Services Readiness domain was found to have the least potential.

The findings in this section of the study are supported by multiple researches within the literature.^107,123 ChatGPT's application in the domain of medical information is significant, particularly in aiding patients’ health management. It can automate the summarization of patient interactions and medical histories, thereby streamlining recordkeeping for healthcare professionals. By transcribing their notes, clinicians can use ChatGPT to summarize essential details like symptoms, diagnoses, and treatments, and extract pertinent patient record information. It can also facilitate clinical trial recruitment by analyzing patient data to identify eligible individuals. Furthermore, ChatGPT can help patients manage their medications by providing reminders, dosage instructions, and information about potential side effects and drug interactions. As per a recent article, ChatGPT can act as a reliable agent to gather information from patients with various diseases.¹⁴

Another finding of the study within this section was the low priority of Intervention Access and Services Readiness domain for ChatGPT expansion. In this regard, several limitations of ChatGPT have been reported that may hinder its potential for providing immediate and accessible clinical services to users. These limitations encompass a lack of human-like comprehension and a propensity for generating irrelevant or unoriginal text. Moreover, while ChatGPT can contribute to medical education, research, and clinical management, it is not a substitute for human expertise and knowledge, and thus, must be employed judiciously.¹⁵

Limitations and implications

This study presented several limitations and implications that warrant attention. The experts involved in the study were exclusively based in Iran, and we were unable to engage with international experts. Consequently, our findings concerning the prioritization of ChatGPT benefits and potential domains for ChatGPT expansion within healthcare building blocks may be influenced by the socioeconomic factors and other characteristics specific to Iran and its healthcare system. We, therefore, encourage future researchers to conduct similar studies in other countries to gain a more accurate understanding of the topic with reduced bias.

Additionally, there are limitations in the chosen methodology. The gray SWARA and gray MABAC methods rely on subjective weightings and reference points, which could introduce uncertainties and inconsistencies in the prioritization of benefits and expansion domains for ChatGPT in healthcare. To address these limitations, we used several methods as sensitivity analysis of the results. However, future research can employ more diverse decision-making techniques. Despite our efforts, there may still be inherent biases or limitations in the expert selection process that could have influenced the findings. Future researchers can expand the expert panel to include a more diverse range of stakeholders and perspectives.

The study's findings could have substantial implications for healthcare policymakers, manufacturers, and researchers by providing a prioritized list of ChatGPT benefits within healthcare systems and potential domains for ChatGPT expansion. We recommend that future researchers conduct studies connecting the findings of studies similar to this one with those conducted in other sectors beyond the healthcare industry, as such results would be fruitful in providing a more detailed view of the benefits of ChatGPT. Further research is required to deepen and refine our understanding of these findings.

Conclusion

The study was executed in three phases. The first phase involved a systematic review that resulted in 74 studies. A thematic analysis of the data from these studies revealed 11 unique themes, including Medical Documentation and Insights, Healthcare Information and Education, Clinical Decision-Making, Healthcare Research, Healthcare Writing, Medical Diagnosis, Medical Procedures, Healthcare Surveys, Privacy, Patient-Centered Care, and Administrative Tasks. In the second phase, the views of multiple experts were utilized to prioritize the clinical domains of ChatGPT benefits, such as clinical decision-making, medical diagnosis, medical procedures, and patient-centered care, as the most important domains of ChatGPT benefits in healthcare. Concurrently, the study results underscored that the information building blocks of healthcare systems, including Information, Infrastructure, and Information and Communication Technologies, possess substantial potential for ChatGPT expansion. However, the Intervention Access and Services Readiness domain was identified as having the least potential. This finding suggests that ChatGPT has a higher potential for expansion in the information domains of the healthcare industry's building blocks rather than the intervention and clinical service-based domains.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241297059 - Supplemental material for ChatGPT utilization within the building blocks of the healthcare services: A mixed-methods study

Supplemental material, sj-docx-1-dhj-10.1177_20552076241297059 for ChatGPT utilization within the building blocks of the healthcare services: A mixed-methods study by Payam Shojaei, Mohsen Khosravi, Yalda Jafari and Amir Hossein Mahmoudi, Hadis Hassanipourmahani in DIGITAL HEALTH

Supplemental Material

sj-pdf-2-dhj-10.1177_20552076241297059 - Supplemental material for ChatGPT utilization within the building blocks of the healthcare services: A mixed-methods study

Supplemental material, sj-pdf-2-dhj-10.1177_20552076241297059 for ChatGPT utilization within the building blocks of the healthcare services: A mixed-methods study by Payam Shojaei, Mohsen Khosravi, Yalda Jafari and Amir Hossein Mahmoudi, Hadis Hassanipourmahani in DIGITAL HEALTH

Research question	What are the benefits of using ChatGPT in healthcare?
Key concepts or terms	Benefits, ChatGPT
Databases or sources	Cochrane Library, PubMed, web of science, and Scopus.
Time-period	2022–2023
Language	English
#1	Benefit* OR advantage* OR outcome* OR merit* OR opportunity*
#2	ChatGPT
Final strategy	#1 AND #2

No.	Theme	Sub-theme	Reference
1	Medical documentation and insights	Documenting and summarizing medical records	^39–69
2	healthcare information and education	Providing information, updates, and explanations to healthcare professionals	^{39,41,43,44,47,48,50–56,58–64,66,67,69–95}
Helping healthcare professionals stay informed about new developments in their respective fields
Providing patient communication support
Answering common questions
3	Clinical decision-making	Assisting healthcare professionals in making clinical decisions	^{26,43,49,52–54,62,67,69,72,74,75,78,80,88,91,96–101}
Providing evidence-based recommendations
4	Healthcare research	Answering questions and providing feedback to medical students	^{9,14,40,41,45,53,55,57,58,63,64,70,72,73,76,77,83,86,91,94,96,98,102–104}
Assisting researchers in analyzing large datasets
Generating hypotheses
5	Healthcare writing	Assisting in healthcare writing	^{40,41,43,45,51,53,57,58,66,67,72,75,77,96,102,103}
6	Medical diagnosis	Assisting medical diagnosis	^{14,50–52,54,59–61,70,75,89,90,101,104–107}
7	Medical procedures	Improving patient outcomes through AI technology	^{39,55,62,70,75,83,85,93,95,104,108,109}
8	Healthcare surveys	Assisting in drafting patient surveys	^40,48,75
Streamlining the process of collecting and analyzing data
9	Privacy	Assisting in deidentifying patient data	^48,60,86
Maintaining patient privacy and confidentiality
10	Patient-centered care	Providing personalized and timely responses to patients’ inquiries	^{41,50,72,83,87–90,92,97,98}
Supporting symptom tracking and medication adherence
Offering mental health support through conversational interfaces
11	Administrative tasks	Automating administrative tasks such as scheduling appointments, managing medical records, and handling insurance.	^{9,41,48,70,85,87,94,98,108}

	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	Expert 6	Expert 7	Expert 8	final weights
c3	0.142	0.14	0.142	0.127	0.133	0.143	0.105	0.153	0.135
c6	0.105	0.104	0.105	0.088	0.099	0.099	0.078	0.106	0.098
c7	0.073	0.072	0.073	0.061	0.083	0.068	0.058	0.073	0.07
c10	0.054	0.053	0.054	0.051	0.057	0.051	0.052	0.051	0.053
c4	0.04	0.037	0.04	0.043	0.039	0.042	0.045	0.037	0.04
c2	0.027	0.027	0.027	0.035	0.029	0.029	0.037	0.026	0.03
c1	0.019	0.022	0.019	0.026	0.02	0.021	0.032	0.018	0.022
c5	0.014	0.015	0.014	0.019	0.014	0.016	0.026	0.012	0.016
c8	0.01	0.011	0.01	0.016	0.01	0.011	0.018	0.009	0.012
c9	0.008	0.008	0.008	0.011	0.007	0.007	0.012	0.007	0.008
c11	0.005	0.006	0.005	0.009	0.005	0.006	0.009	0.005	0.006

	Benefits	Weight	Ranking
C1	Medical documentation and insights	0.022	7
C2	healthcare information and education	0.030	6
C3	Clinical decision-making	0.135	1
C4	Healthcare research	0.040	5
C5	Healthcare writing	0.016	8
C6	Medical diagnosis	0.098	2
C7	Medical procedures	0.070	3
C8	Healthcare surveys	0.012	9
C9	Privacy	0.008	10
C10	Patient-centered care	0.053	4
C11	Administrative tasks	0.006	11

	Indicator domain (building blocks)	CC	Ranking
A1	Infrastructure, Information, and communication technologies	0.0333	2
A2	Health workforce	0.0155	3
A3	Information	0.0348	1
A4	Intervention access and services readiness	−0.0100	5
A5	Intervention quality, safety	0.0074	4

	Indicator domain (building blocks)	Ranking (MABAC)	Ranking (TOPSIS)	Ranking (COPRAS)
A1	Infrastructure, information, and communication technologies	2	2	2
A2	Health workforce	3	5	4
A3	Information	1	1	1
A4	Intervention access and services readiness	5	4	5
A5	Intervention quality, safety	4	3	3

Footnotes

Acknowledgments

We would like to acknowledge the Bing AI chatbot for its contribution to rewriting the manuscript in terms of English grammar and words.

Availability of data

The research data can be accessed by contacting the corresponding author of the research.

Contributorship

PSH initiated the research, conducted the data analysis and revised the manuscript. MKH conducted the review and cooperated in writing of the manuscript. YJ cooperated in conduction of the data analysis and writing of the manuscript. AHM cooperated in conducting the review and analysis. HH cooperated in conducting the review and analysis.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

Not applicable to this methodology of research.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Guarantor

PS.

ORCID iDs

Payam Shojaei

Mohsen Khosravi

Supplemental material

Supplemental material for this article is available online.

References

Hajj

Samad

Stechel

, et al. How AI can empower a post COVID-19 world. Strategy and Middle East, 2020.

Khosravi

Zare

Mojtabaeian

, et al. Artificial intelligence and decision-making in healthcare: a thematic analysis of a systematic review of reviews. Health Serv Res Manag Epidemiol 2024; 11: 23333928241234863.

Agarwal

Swami

Malhotra

. Artificial intelligence adoption in the post COVID-19 new-normal and role of smart technologies in transforming business: a review. J Sci Technol Policy Manag 2024; 15: 506–529. ahead-of-print.

Hetler

. ChatGPT. 2024.

Ortiz

. What is ChatGPT and why does it matter? Here's what you need to know. ZDNET. 2023. https://www. zdnet. com/article/what-ischatgpt-and-why-does-it-matter-heres-everything-you-need-to-know

Huang

Tan

. The role of ChatGPT in scientific communication: writing better scientific review articles. Am J Cancer Res 2023; 13: 1148–1154.

Montazeri

Galavi

Ahmadian

. What are the applications of ChatGPT in healthcare: gain or loss? Health Sci Rep 2024; 7: e1878.

. The potential applications and challenges of ChatGPT in the medical field. Int J Gen Med 2024; 17: 817–826.

Sallam

. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare (Basel) 2023; 11: 887.

10.

Adeshola

Adepoju

. The opportunities and challenges of ChatGPT in education. Inter Learn Environ 2023: 1–14.

11.

Teng

Chen

, et al. Chat with ChatGPT on intelligent vehicles: an IEEE TIV perspective. IEEE Trans Intell Veh 2023; 8: 2020–2026.

12.

Wang

Anwer

Dai

, et al. ChatGPT for design, manufacturing, and education. Procedia CIRP 2023; 119: 7–14.

13.

George

. A review of ChatGPT AI's impact on several business sectors. Partners Univers Int Innov J 2023; 1: 9–23.

14.

Dave

Athaluri

Singh

. ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front Artif Intell 2023; 6: 1169595.

15.

Khan

Jawaid

Khan

, et al. ChatGPT - reshaping medical education and clinical management. Pak J Med Sci 2023; 39: 605–607.

16.

Leutz-Schmidt

Grözinger

Kauczor

H-U

, et al. Performance of ChatGPT on basic healthcare leadership and management questions. Health Technol (Berl) 2024; 14: 1161–1166.

17.

Garg

Urs

Agarwal

, et al. Exploring the role of ChatGPT in patient care (diagnosis and treatment) and medical research: a systematic review. Health Promot Perspect 2023; 13: 183–191.

18.

Sallam

. The Utility of ChatGPT as an Example of Large Language Models in Healthcare Education, Research and Practice: Systematic Review on the Future Perspectives and Potential Limitations. medRxiv 2023: 2023.2002.2019.23286155.

19.

Sedaghat

. Early applications of ChatGPT in medical practice, education and research. Clin Med 2023; 23: 278–279.

20.

Khosravi

Zare

Mojtabaeian

, et al. Ethical challenges of using artificial intelligence in healthcare delivery: a thematic analysis of a systematic review of reviews. J Public Health 2024: 1–11.

21.

Sedaghat

. Plagiarism and Wrong Content as Potential Challenges of Using Chatbots Like ChatGPT in Medical Research. Journal of Academic Ethics 2024: 1–4.

22.

Sedaghat

. Future potential challenges of using large language models like ChatGPT in daily medical practice. J Am Coll Radiol 2024; 21: 344–345.

23.

Haltaufderheide

Ranisch

. The ethics of ChatGPT in medicine and healthcare: a systematic review on large language models (LLMs). NPJ Digit Med 2024; 7: 183.

24.

Kapsali

Livanis

Tsalikidis

, et al.

Ethical concerns about ChatGPT in healthcare: a useful tool or the tombstone of original and reflective thinking?

Cureus 2024; 16: e54759.

25.

Wang

Liu

Yang

, et al. Ethical considerations of using ChatGPT in health care. J Med Internet Res 2023; 25: e48009.

26.

Gala

Makaryus

. The utility of language models in cardiology: a narrative review of the benefits and concerns of ChatGPT-4. Int J Environ Res Public Health 2023; 20: 6438.

27.

WHO. Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies. Geneva, Switzerland: WHO, 2010.

28.

Manyazewal

. Using the world health organization health system building blocks through survey of healthcare professionals to determine the performance of public healthcare facilities. Arch Public Health 2017; 75: 50.

29.

Porritt

Gomersall

Lockwood

. JBI's systematic reviews: study selection and critical appraisal. Am J Nurs 2014; 114: 47–52.

30.

Medicine: Top health and medicine databases, 2024, https://lancaster.libguides.com/medicine/databases.

31.

Byrne

. A worked example of braun and clarke’s approach to reflexive thematic analysis. Qual Quant 2022; 56: 1391–1412.

32.

Kuckartz

Rädiker

. Using MAXQDA for mixed methods research. In: The Routledge reviewer’s guide to mixed methods analysis. Routledge, 2021, pp.305–318.

33.

Kurnaz

Özdağoğlu

Keleş

. Method of evaluation of military helicopter pilot selection criteria: a novel Gray SWARA approach. Aviation 2023; 27: 27–35.

34.

Thakkar

. Stepwise weight assessment ratio analysis (SWARA). In: Multi-criteria decision making. 2021, pp.281–289.

35.

Das

Chakraborty

. Application of gray-PROMETHEE method for parametric optimization of a green powder mixed EDM process. Process Integr Optim Sustain 2021; 5: 645–661.

36.

Julong

. Introduction to gray system theory. J Gray Syst 1989; 1: 1–24.

37.

Debnath

Roy

Kar

, et al. A hybrid MCDM approach for strategic project portfolio selection of Agro by-products. Sustainability 2017; 9:1302.

38.

General Assembly of the World Medical Association. World medical association declaration of Helsinki: ethical principles for medical research involving human subjects. J Am Coll Dent 2014; 81: 14–18.

39.

Abdelhady

Davis

. Plastic surgery and artificial intelligence: how ChatGPT improved operation note accuracy, time, and education. Mayo Clin Proc 2023; 1: 299–308.

40.

Al-Worafi

Hermansyah

Tan

, et al. Applications, benefits, and risks of ChatGPT in medical and health sciences research: an experimental study. Prog Microbes Mol Biol 2023; 6.

41.

Berşe

Akça

Dirgar

, et al. The role and potential contributions of the artificial intelligence language model ChatGPT. Ann Biomed Eng 2024; 52: 130–133.

42.

Bosbach

Senge

Nemeth

, et al. Ability of ChatGPT to generate competent radiology reports for distal radius fracture by use of RSNA template items and integrated AO classifier. Curr Probl Diagn Radiol 2024; 53: 102–110.

43.

Chatelan

Clerc

Fonta

. ChatGPT and future AI chatbots: what may be the impact on credentialed nutrition and dietetics practitioners? J Acad Nutr Diet 2023; 123: 1525–1531.

44.

Chervenak

Lieman

Blanco-Breindel

, et al. The promise and peril of using a large language model to obtain clinical information: chatGPT performs strongly as a fertility counseling tool with limitations. Fertil Steril 2023; 120: 575–583.

45.

Corsello

Santangelo

. May artificial intelligence influence future pediatric research?—the case of ChatGPT. Children 2023; 10: 757.

46.

Cunningham

Behm

, et al. Long-Term survival of patients with glioblastoma of the pineal gland: a ChatGPT-assisted, updated case of a multimodal treatment strategy resulting in extremely long overall survival at a site with historically poor outcomes. Cureus 2023; 15: e36590.

47.

Currie

Singh

Nelson

, et al. ChatGPT in medical imaging higher education. Radiography 2023; 29: 792–799.

48.

Ebrahimi

Howard

Carlson

, et al.

ChatGPT: can a natural language processing tool be trusted for radiation oncology use?

Int J Radiat Oncol Biol Phys 2023; 116: 977–983.

49.

Haemmerli

Sveikata

Nouri

, et al.

ChatGPT in glioma adjuvant therapy decision making: ready to assume the role of a doctor in the tumour board?

BMJ Health Care Info 2023; 30: e100775.

50.

Elyoseph

Hadar-Shoval

Asraf

, et al. ChatGPT outperforms humans in emotional awareness evaluations. Front Psychol 2023; 14.

51.

Gill

Kaur

. ChatGPT: vision and challenges. Int Things Cyber-Phys Sys 2023; 3: 262–271.

52.

Kuang

Y-R

Zou

M-X

Niu

H-Q

, et al. ChatGPT encounters multiple opportunities and challenges in neurosurgery. Int J Surg 2023; 109(10): 2886–2891.

53.

Khan

Jawaid

Khan

, et al. ChatGPT-Reshaping medical education and clinical management. Pak J Med Sci 2023; 39: 605–607.

54.

Eggmann

Weiger

Zitzmann

, et al. Implications of large language models such as ChatGPT for dental medicine. J Esthet Restor Dent 2023; 35: 1098–1102.

55.

Deik

. Potential benefits and perils of incorporating ChatGPT to the movement disorders clinic. J Mov Disord 2023; 16: 158–162.

56.

Hügle

. The wide range of opportunities for large language models such as ChatGPT in rheumatology. RMD Open 2023; 9: e003105.

57.

Lee

S-W

Choi

W-J

. Utilizing ChatGPT in clinical research related to anesthesiology: a comprehensive review of opportunities and limitations. Anesth Pain Med 2023; 18: 244–251.

58.

Javan

Kim

Mostaghni

, et al. ChatGPT’s potential role in interventional radiology. Cardiovasc Intervent Radiol 2023; 46: 821–822.

59.

Kemp

Logan

SJS

, et al. ChatGPT outscored human candidates in a virtual objective structured clinical examination in obstetrics and gynecology. Am J Obstet Gynecol 2023; 229: 172.e1–172.e12.

60.

Wang

Yan

, et al. Potential applications of ChatGPT in endoscopy: opportunities and limitations. Gastroenterol Endosc 2023; 1: 152–154.

61.

Liu

Wang

Liu

. Utility of ChatGPT in clinical practice. J Med Internet Res 2023; 25: e48568.

62.

, et al.

Artificial intelligence in intensive care medicine: toward a ChatGPT/GPT-4 way?

Ann Biomed Eng 2023; 51: 1898–1903.

63.

Mohammad

Supti

Alzubaidi

, et al. The pros and cons of using ChatGPT in medical education: a scoping review. Stud Health Technol Inform 2023; 305: 644–647.

64.

Mondal

Podder

. Using ChatGPT for writing articles for patients’ education for dermatological diseases: a pilot study. Indian Dermatol Online J 2023; 14: 482–486.

65.

Naik

Prather

Gurda

. Synchronous bilateral breast cancer: a case report piloting and evaluating the implementation of the AI-powered large language model (LLM) ChatGPT. Cureus 2023; 15: e37587.

66.

Parikh

Shah

Parikh

, et al. ChatGPT-Preliminary overview with implications for medicine and oncology. Indian J Med Paediatr Oncol 2023; 44: 377–383.

67.

Sorin

Klang

Sklair-Levy

, et al. Large language model (ChatGPT) as a support tool for breast tumor board. npj Breast Cancer 2023; 99(1): 44.

68.

Srivastav

Chandrakar

Gupta

, et al. ChatGPT in radiology: the advantages and limitations of artificial intelligence for medical imaging diagnosis. Cureus 2023; 15: e41435.

69.

Sinha

Deb Roy

Kumar

, et al. Applicability of ChatGPT in assisting to solve higher order problems in pathology. Cureus 2023; 15: e35237.

70.

Alhaidry

Fatani

Alrayes

, et al. ChatGPT in dentistry: a comprehensive review. Cureus 2023; 15: e38317.

71.

Ali

Barhom

Tamimi

, et al. ChatGPT-A double-edged sword for healthcare education? Implications for assessments of dental students. Eur J Dent Educ 2024; 28: 206–211.

72.

Alkhaqani

. ChatGPT and nursing education: challenges and opportunities. Al-Rafidain J MedSci 2023; 4: 50–51.

73.

Cascella

Montomoli

Bellini

, et al. Writing the paper “unveiling artificial intelligence: an insight into ethics and applications in anesthesia” implementing the large language model ChatGPT: a qualitative study. J Med Artif Intell 2023; 6: 9.

74.

Kao

H-J

Chien

T-W

Wang

W-C

, et al. Assessing ChatGPT's capacity for clinical decision support in pediatrics: a comparative study with pediatricians using KIDMAP of Rasch analysis. Medicine (United States) 2023; 102: E34068.

75.

Huang

Zheng

Wang

, et al. ChatGPT for shaping the future of dentistry: the potential of multi-modal large language model. Int J Oral Sci 2023; 15: 29.

76.

Hristidis

Ruggiano

Brown

, et al. ChatGPT vs google for queries related to dementia and other cognitive decline: comparison of results. J Med Internet Res 2023; 25: e48966.

77.

Gravel

D’Amours-Gravel

Osmanlliu

. Learning to fake it: limited responses and fabricated references provided by ChatGPT for medical questions. Mayo Clin Proc 2023; 1: 226–234.

78.

Lukac

Dayan

Fink

, et al. Evaluating ChatGPT as an adjunct for the multidisciplinary tumor board decision-making in primary breast cancer cases. Arch Gynecol Obstet 2023; 308: 1831–1844.

79.

Lum

. Can artificial intelligence pass the American board of orthopaedic surgery examination? Orthopaedic residents versus ChatGPT. Clin Orthop Relat Res 2023; 481: 1623–1630.

80.

Mann

. Artificial intelligence discusses the role of artificial intelligence in translational medicine: a JACC: basic to translational science interview with ChatGPT. J Am Coll Cardiol Basic Trans Sci 2023; 8: 221–223.

81.

Panthier

Gatinel

. Success of ChatGPT, an AI language model, in taking the French language version of the European board of ophthalmology examination: a novel approach to medical knowledge assessment. J Fr Ophtalmol 2023; 46: 706–711.

82.

Ray

Majumder

. Assessing the accuracy of responses by the language model ChatGPT to questions regarding bariatric surgery: a critical appraisal. Obes Surg 2023; 33: 2588–2589.

83.

Sallam

Salim

Barakat

, et al. ChatGPT applications in medical, dental, pharmacy, and public health education: a descriptive study highlighting the advantages and limitations. Narra J 2023; 3: e103.

84.

Samaan

Yeo

Rajeev

, et al. Assessing the accuracy of responses by the language model ChatGPT to questions regarding bariatric surgery. Obes Surg 2023; 33: 1790–1796.

85.

Santandreu-Calonge

Medina-Aguerrebere

Hultberg

, et al.

Can ChatGPT improve communication in hospitals?

Prof Inf 2023; 32(2).

86.

Shahsavar

Choudhury

. User intentions to use ChatGPT for self-diagnosis and health-related purposes: cross-sectional survey study. JMIR Hum Factors 2023; 10: e47564.

87.

Sharma

Pajai

Prasad

, et al. A critical review of ChatGPT as a potential substitute for diabetes educators. Cureus 2023; 15: e38380.

88.

Strong

DiGiammarino

Weng

, et al. Performance of ChatGPT on free-response, clinical reasoning exams. medRxiv 2023. 20230329.

89.

Temsah

M-H

Aljamaan

Malki

, et al. ChatGPT and the future of digital health: a study on healthcare Workers’ perceptions and expectations. Healthcare (Switzerland) 2023; 11: 1812.

90.

Temsah

M-H

Jamal

Aljamaan

, et al. ChatGPT-4 and the global burden of disease study: advancing personalized healthcare through artificial intelligence in clinical and translational medicine. Cureus 2023; 15: e39384.

91.

Thapa

Adhikari

. ChatGPT, bard, and large language models for biomedical research: opportunities and pitfalls. Ann Biomed Eng 2023; 51: 2647–2651.

92.

Tustumi

Andreollo

Aguilar-Nascimento

. Future of the language models in healthcare: the role of chatgpt. Arq Bras Cir Dig 2023; 36: e1727.

93.

Xie

Seth

Hunter-Smith

, et al. Aesthetic surgery advice and counseling from artificial intelligence: a rhinoplasty consultation with ChatGPT. Aesthetic Plast Surg 2023; 47: 1985–1993.

94.

Yang

Wei

. The impact of ChatGPT and LLMs on medical imaging stakeholders: perspectives and use cases. Meta-Radiology 2023; 1: 100007.

95.

Yeo

Samaan

, et al. Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma. Clin Mol Hepatol 2023; 29: 721–732.

96.

Cascella

Montomoli

Bellini

, et al. Evaluating the feasibility of ChatGPT in healthcare: an analysis of multiple clinical and research scenarios. J Med Syst 2023; 47(1): 33.

97.

Cohen

. What should ChatGPT mean for bioethics? Am J Bioeth 2023; 23: 8–16.

98.

Loh

. ChatGPT and generative AI chatbots: challenges and opportunities for science, medicine and medical leaders. BMJ Leader 2024; 8: 51–54.

99.

Rajjoub

Arroyave

Zaidat

, et al. ChatGPT and its role in the decision-making for the diagnosis and treatment of lumbar spinal stenosis: a comparative analysis and narrative review. Global Spine J 2024; 14: 998–1017.

100.

Rao

Pang

Kim

, et al. Assessing the utility of ChatGPT throughout the entire clinical workflow. medRxiv 2023. 20230226.

101.

Tomar

Govil

Dhawan

. Closed negative suction drain entrapment in total knee arthroplasty: a report on the implications of a broken drain based on the ChatGPT outlook. Cureus 2023; 15: e36290.

102.

Bom

H-SH

. Exploring the opportunities and challenges of ChatGPT in academic writing: a roundtable discussion. Nucl Med Mol Imaging 2023; 57: 165–167.

103.

De Angelis

Baglivo

Arzilli

, et al. ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health. Front Public Health 2023; 11: 1166120.

104.

. ChatGPT has made the field of surgery full of opportunities and challenges. Int J Surg 2023; 109: 2537–2538.

105.

Antaki

Touma

Milad

, et al. Evaluating the performance of ChatGPT in ophthalmology: an analysis of its successes and shortcomings. Ophthalmol Sci 2023; 3: 100324.

106.

Balas

Ing

. Conversational AI models for ophthalmic diagnosis: comparison of ChatGPT and the isabel pro differential diagnosis generator. JFO Open Ophthalmol 2023; 1: 100005.

107.

Homolak

. Opportunities and risks of ChatGPT in medicine, science, and academic publishing: a modern promethean dilemma. Croat Med J 2023; 64: 1–3.

108.

Gebrael

Sahu

Chigarira

, et al. Enhancing triage efficiency and accuracy in emergency rooms for patients with metastatic prostate cancer: a retrospective analysis of artificial intelligence-assisted triage using ChatGPT 4.0. Cancers (Basel) 2023; 15: 3717.

109.

Janamla

Daram

Rajesh

, et al. Response of ChatGPT for humanoid robots role in improving healthcare and patient outcomes. Ann Biomed Eng 2023; 51: 2359–2361.

110.

Pamučar

Ćirović

. The selection of transport and handling resources in logistics centers using multi-attributive border approximation area comparison (MABAC). Expert Syst Appl 2015; 42: 3016–3028.

111.

Rao

Pang

Kim

, et al. Assessing the utility of ChatGPT throughout the entire clinical workflow: development and usability study. J Med Internet Res 2023; 25: e48659.

112.

Edgman-Levitan

Schoenbaum

. Patient-centered care: achieving higher quality by designing care through the patient's eyes. Isr J Health Policy Res 2021; 10: 21.

113.

Reynolds

. Patient-centered care. Radiol Technol 2009; 81: 133–147.

114.

Fink

. [Large language models such as ChatGPT and GPT-4 for patient-centered care in radiology]. Radiologie (Heidelb) 2023; 63: 665–671.

115.

Soto-Galindo

Capelleras

Cruellas

, et al. Effectiveness of ChatGPT in identifying and accurately guiding patients in rhinoplasty complications. Facial Plast Surg 2024; 40: 623–627.

116.

Teixeira da Silva

. Can ChatGPT rescue or assist with language barriers in healthcare communication? Patient Educ Couns 2023; 115: 107940.

117.

Spear

Ehrenfeld

Miller

. Applications of artificial intelligence in health care delivery. J Med Syst 2023; 47: 121.

118.

Rahman

Victoros

Ernest

, et al. Impact of artificial intelligence (AI) technology in healthcare sector: a critical evaluation of both sides of the coin. Clin Pathol 2024; 17: 2632010–241226887.

119.

Sezgin

. Artificial intelligence in healthcare: complementing, not replacing, doctors and healthcare providers. Digit Health 2023; 9: 20552076231186520.

120.

Davenport

Kalakota

. The potential for artificial intelligence in healthcare. Future Healthc J 2019; 6: 94–98.

121.

Mohanasundari

Kalpana

Madhusudhan

, et al.

Can artificial intelligence replace the unique nursing role?

Cureus 2023; 15: e51150.

122.

Aung

YYM

Wong

DCS

Ting

DSW

. The promise of artificial intelligence: a review of the opportunities and challenges of artificial intelligence in healthcare. Br Med Bull 2021; 139: 4–15.

123.

Ruksakulpiwat

Kumar

Ajibade

. Using ChatGPT in medical research: current Status and future directions. J Multidiscip Healthc 2023; 16: 1513–1520.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.19 MB

	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	Expert 6	Expert 7	Expert 8	final weights
Indicator	W1	W2	W3	W4	W5	W6	W7	W8	average
c3	0.142	0.14	0.142	0.127	0.133	0.143	0.105	0.153	0.135
c6	0.105	0.104	0.105	0.088	0.099	0.099	0.078	0.106	0.098
c7	0.073	0.072	0.073	0.061	0.083	0.068	0.058	0.073	0.07
c10	0.054	0.053	0.054	0.051	0.057	0.051	0.052	0.051	0.053
c4	0.04	0.037	0.04	0.043	0.039	0.042	0.045	0.037	0.04
c2	0.027	0.027	0.027	0.035	0.029	0.029	0.037	0.026	0.03
c1	0.019	0.022	0.019	0.026	0.02	0.021	0.032	0.018	0.022
c5	0.014	0.015	0.014	0.019	0.014	0.016	0.026	0.012	0.016
c8	0.01	0.011	0.01	0.016	0.01	0.011	0.018	0.009	0.012
c9	0.008	0.008	0.008	0.011	0.007	0.007	0.012	0.007	0.008
c11	0.005	0.006	0.005	0.009	0.005	0.006	0.009	0.005	0.006