Sage Journals: Discover world-class research

Abstract

Objective

The integration of medical large language models (MLLMs) into healthcare has garnered global interest, however, the determinants of their adoption by medical professionals remain underexplored. This study aims to elucidate the factors influencing doctors’ intention to utilize MLLMs, encompassing both psychological determinants and demographic attributes.

Methods

An extended theoretical model was developed using constructs derived from the Technology Acceptance Model (TAM) and five constructs. A hybrid online and offline survey was conducted from March to December 2023, including 955 Chinese medical practitioners. Structural equation modeling was utilized to test the research hypotheses.

Results

The measurement model exhibited satisfactory reliability and validity, with fit indices meeting scholarly standards. Perceived ease of use emerged as a significant predictor of both perceived usefulness and satisfaction. Content quality was identified as a substantial influence on perceived satisfaction but did not significantly predict perceived usefulness. Technical support and social influence were found to significantly affect perceived usefulness without directly impacting satisfaction. Perceived usefulness positively influenced both satisfaction and usage behavior, while perceived risk had a negative effect. A significant relationship between perceived satisfaction and usage behavior was established, with gender, age, education, and professional title moderating this relationship.

Conclusions

The study provides empirical evidence for understanding the adoption of MLLMs by Chinese doctors, offering management implications for future technical research, development, and implementation in the medical field.

Keywords

Medical large language models usage behavior content quality technical support Chinese doctor

Introduction

The emergence of medical large language models (MLLMs), as sophisticated deep learning-based natural language processing (NLP) tools, has marked a significant advancement in the field of healthcare. These models have demonstrated exceptional capabilities in clinical decision support and patient interaction, showcasing a potential that rivals, and in some instances, surpasses the performance of medical experts in specific medical tasks.¹ For instance, models such as Google's PaLM2 and Hippocratic AI have been recognized for their high accuracy in these domains.² As intelligent assistants, MLLMs offer substantial benefits to the medical community by aiding physicians in rapid and precise diagnosis, clinical decision-making, and patient care delivery.^3,4 Their integration into healthcare systems promises to enhance the efficiency and accuracy of medical practice, mitigate the occurrence of medical errors, and facilitate improvements in remote consultations and the management of medical documentation.⁵ In China, with its extensive population base and abundant medical data, presents a unique landscape for the application of MLLMs.^6,7 The deployment of these models in China holds the promise of addressing disparities in the distribution of medical resources, alleviating the workload of healthcare professionals, and catering to the growing demand for efficient healthcare services. Furthermore, MLLMs are expected to contribute significantly to chronic disease management, a critical area within the healthcare sector.⁸ The role of MLLMs in the Chinese medical and health sector is set to become increasingly pivotal, heralding a new era of technological integration in healthcare delivery.

Despite the potential benefits, the adoption of MLLMs by doctors is a critical factor determining their practical effectiveness in medicine. While current research predominantly concentrates on the technical aspects of MLLMs, there is a significant gap in understanding the behavioral aspects of their adoption, especially within the Chinese context, where cultural and systemic differences may present unique challenges. This study addresses this research gap by examining the behavioral intentions and influencing factors of Chinese doctors regarding the adoption of MLLMs. Utilizing surveys and interviews, the study will develop a comprehensive model that incorporates individual and psychological factors, shedding light on the mechanisms of adoption. The insights gained from this research will offer tailored recommendations for the Chinese medical context, thereby facilitating the effective integration and utilization of MLLMs in clinical practice.

Methods

Theoretical model

The theoretical model underpinning the study is primarily based on the technology acceptance model (TAM), which traditionally emphasizes perceived usefulness (PU) and perceived ease of use (PEOU) as determinants of technology adoption.^9,10 The present study extends it by incorporating additional constructs that emerged as context-specific predictors within the domain of MLLMs. Content quality, another construct added to the model, originates from the information quality famework and is critical in the context of healthcare technology, where the accuracy, relevance, and timeliness of information can significantly impact decision-making.^11–13 Technical support was added to account for the facilitating conditions that can influence adoption.^11,14 Social influence was included as the interconnectedness of the modern digital age facilitates the rapid dissemination of information and opinions among peers.^15,16 Perceived risk was included to account for potential negative implications associated with technology use, such as privacy concerns or data security issues, which have been shown to influence satisfaction and usage behavior.^17,18 Perceived satisfaction reflects the affective evaluation of technology use, which is increasingly recognized for its influence on user engagement and long-term adoption.^12,19,20 The theoretical model is shown in Figure 1.

Figure 1.

Theoretical model.

Hypotheses development

The hypotheses of this study are grounded in a robust theoretical framework and supported by empirical evidence from the literature. Each hypothesis is developed with reference to specific constructs and relationships identified in the existing body of knowledge.

The positive impact of perceived ease of use on both perceived usefulness and perceived satisfaction is posited based on the foundational work of Davis (1989), which established perceived ease of use as a key determinant of perceived usefulness within the technology acceptance model (TAM).²¹ Perceived ease of use can lead to a more positive user experience, thereby increasing perceived satisfaction with technologies.²² Therefore, the following two research hypotheses are proposed:

H1a: Perceived ease of use has a positive impact on doctors’ perceived usefulness in adopting MLLM.

H1b: Perceived ease of use has a positive impact on doctors’ perceived satisfaction in adopting MLLM.

The influence of content quality on perceived usefulness and satisfaction is derived from the Information Quality Framework, which emphasizes the importance of accurate and relevant information in user decision-making processes.²³ In the context of healthcare, where the stakes are particularly high, content quality is considered a critical factor in the adoption of MLLMs.^24,25 Therefore, the following hypotheses are proposed:

H2a: Content quality has a positive impact on doctors’ perceived usefulness in adopting MLLM.

H2b: Content quality has a positive influence on doctors’ perceived satisfaction in adopting MLLM.

Technical support is included as a predictor of perceived usefulness, reflecting the notion that adequate support can facilitate the effective use of technology.²⁶ The critical role of technical support in addressing user concerns, facilitating the resolution of issues, and thereby contributing to a higher level of user satisfaction.^27,28 According to the existing literature, two research hypotheses are proposed:

H3a: Technical support has a positive impact on doctors’ perceived usefulness in adopting MLLM.

H3b: Technical support has a positive impact on doctors’ perceived satisfaction in adopting MLLM.

Social influence is widely recognized as a significant factor in the adoption of new technologies, emphasizing subjective norms in shaping behavioral intentions.^29,30 The difference between its impact on perceived usefulness and satisfaction is that social influence may initially drive adoption by generating positive perceived usefulness,³⁰ and it also being initial factor in determining satisfaction in the field of health information technology.³¹ Then, two hypotheses are proposed:

H4a: Social influence has a positive influence on doctors’ perceived usefulness in adopting MLLM.

H4b: Social influence has a positive impact on doctors’ perceived satisfaction in adopting MLLM.

The hypothesis linking perceived usefulness to perceived satisfaction is based on the cognitive evaluation of technology's utility leading to affective responses.^10,32 The perceived usefulness has a positive influence on usage behavior is a core proposition of the TAM, indicating that the perceived benefits of technology drive its adoption and continued use.²² The following two research hypotheses are proposed:

H5: Perceived usefulness has a positive impact on doctors’ perceived satisfaction in adopting MLLM.

H6: Perceived usefulness has a positive influence on doctors’ use behavior in adopting MLLMs.

The perceived satisfaction with technology has been shown to confirm initial expectations, ultimately leading to continued use behavior. When users’ expectations are met or exceeded by technology, they are more likely to continue using it, which aligns with the findings of numerous empirical studies that have demonstrated the positive relationship between satisfaction and continued use behavior.^33,34 It is crucial for technology designers and developers to prioritize user satisfaction to foster continued use and ultimately achieve long-term success.³³

H7: Perceived satisfaction has a positive impact on doctors’ use behavior in adopting MLLM.

The negative influence of perceived risk on satisfaction and use behavior aligns with the Protection Motivation Theory, which suggests that potential negative consequences can diminish user satisfaction and act as barriers to technology adoption.^17,18 Two research hypotheses are proposed:H8: Perceived risk has a negative impact on doctors’ perceived satisfaction in adopting MLLM.

H9: Perceived risk has a negative influence on doctors’ use behavior in adopting MLLM.

Post hoc analysis led to the examination of demographic variables as potential moderators. Specifically, we examined whether gender, age, education, and professional title might influence the relationships among perceived usefulness, perceived satisfaction, perceived risk, and usage behavior to MLLMs. Although these are not proposed hypotheses, their examination provides a deeper understanding of factors on technology adoption within different demographic subgroups.

Questionnaire design and data collection

The current study is a cross-sectional survey aimed at understanding the factors influencing Chinese doctors’ intention to use MLLMs. The questionnaire utilized in this study was developed based on established constructs from the technology acceptance model (TAM) and literature, as indicated in our theoretical model. The questionnaire consisted of two sections: demographics and 34 items measured on a 7-level Likert scale. References to the specific items of the questionnaire are provided in Table 1, ensuring transparency and replicability of our research methodology. The study received ethical approval with the approval number 2022-S313 from the Institutional Review Board (IRB) of Third Xiangya Hospital, Central South University. Before commencing the survey, written informed consent was obtained from all respondents.

Table 1.

Measurement items of the constructs.

Construct	Variable	Measuring item	References
Perceived ease of use (PEOU)	PEOU1	Medical LLM is easy to use	^9,10
	PEOU2	Is it easy to understand how medical LLM works and how to use it
	PEOU3	I can easily use medical LLM to improve diagnostic efficiency
	PEOU4	The interaction of medical LLM is flexible
	PEOU5	The information in the medical LLM is concise and clear
Content quality (CQ)	CQ1	I can find the information I need at medical LLM	^11–13
	CQ2	Medical LLM contains the functional modules I need
	CQ3	The medical LLM is well organized into logical expressions
	CQ4	Medical LLM outputs are up to date
Technical support (TS)	TS1	Medical LLM design (e.g., fonts, styles, colors, images) is friendly	^11,14
	TS2	There are clear instructions for using medical LLM services
	TS3	Medical LLM is operating stably
	TS4	Medical LLM provided me with a personalized page
	TS5	Medical LLM provides appropriate online assistance and assistance
Social influence (SI)	SI1	My colleague suggested me to use medical LLM	^15,16
	SI2	Medical association recommends the use of medical LLM
	SI3	Medical LLM accredited by influential and respected physicians
	SI4	Patients encouraged me to use medical LLM
Perceived usefulness (PU)	PU1	Using medical LLM allows me to make a more accurate diagnosis	^9,10
	PU2	The use of medical LLM has improved my work efficiency
	PU3	In general, the medical LLM is helpful
	PU4	Important and valuable communication in medical LLM interactions
Perceived satisfaction (PS)	PS1	I believe it is good to use medical LLM	^12,19,20
	PS2	I have a positive attitude toward the use of medical LLM
	PS3	I'm not afraid to use medical LLM
	PS4	I am satisfied with my experience with medical LLM
Perceived risk (PR)	PR1	I am concerned about the privacy risk of medical LLM	^17,18
	PR2	The medical LLM may contain false or misleading information
	PR3	Medical LLM may have medical ethical risks
	PR4	Medical LLM could be attacked by bad actors with technical vulnerabilities
Usage behavior (UB)	UB1	I use medical LLM to improve my medical knowledge and skills	^9,10
	UB2	I use medical LLM regularly
	UB3	I recommend medical LLM to my acquaintances
	UB4	If it is optional, I still prefer to use medical LLM as a support tool

The survey was designed to assess the perceptions of Chinese doctors toward MLLMs. Given the rapidly evolving landscape of AI in medicine, the survey did not specify particular MLLMs but rather focused on the general capabilities and features characteristic of this class of tools. This approach was intended to capture a broad perspective on the potential impact and reception of MLLMs in medical practice. However, it is acknowledged that different MLLMs may offer distinct features and capabilities, which could influence the adoption factors under investigation. For further insights into specific MLLMs and their applications, we refer to recent literature.^35–37 To prevent varying understanding of MLLMs among our respondents, we included a standardized definition and a brief educational introduction about MLLMs at the beginning of our survey. This introduction aimed to provide a common baseline of knowledge. Additionally, we assessed respondents’ baseline knowledge of MLLMs with an initial question, allowing us to consider the level of understanding when analyzing responses. Feedback from pilot testing was used to refine our approach, ensuring clarity across all respondents.

Prior to the formal survey, the questionnaire was pilot-tested with 50 doctors to ensure clarity and relevance, and language revision was conducted by an expert to enhance comprehension. The pilot test helped refine the questionnaire for better understanding and reliability. The formal survey was conducted from March to December 2023 in five major cities in China: Beijing, Shanghai, Guangzhou, Changsha, and Chengdu. These cities were chosen for their diverse medical infrastructure and representativeness of the country's medical community. Questionnaires were distributed randomly through medical institutions and online platforms, ensuring anonymity and voluntary participation. The survey was accessible on mobile phones, and respondents were typically able to complete it within 3–5 minutes.

To ensure data accuracy, the survey included trap questions and IP checks. Ultimately, 955 completed surveys were collected, with 126 partial responses excluded due to incomplete data. The study is committed to transparency and reproducibility. The raw data collected are stored securely and are available upon request for researchers wishing to conduct further analyses or reproducibility studies. Access to the data will be granted in accordance with ethical considerations and data protection standards.

Data analysis

We utilized IBM SPSS and AMOS for data analysis in this study. These are industry-standard software packages widely recognized for their role in statistical analysis. The use of these tools complies with the licensing agreements provided by IBM Corporation, ensuring that all terms and conditions related to the software's utilization in research have been adhered to.

We first performed the Harman one-factor test to check for common method bias (CMB), aiming for a CMB index below 40%.³⁸ Then, we ran descriptive statistics to understand sample characteristics. The scale's structural validity was initially evaluated using exploratory factor analysis (EFA) to explore the underlying factor structure of the items within each construct. After gaining insights from the EFA, the internal consistency of the scale was assessed using comprehensive reliability (CR) with a threshold of 0.7, ensuring that the items within each construct are reliable indicators of the underlying constructs.³⁹ The validity of convergence was further examined through Cronbach's alpha, with a minimum acceptable value of 0.7, and average variance extracted (AVE > 0.5), ensuring that each item had a substantial shared variance with its corresponding construct. Discriminant validity was evaluated by comparing item cross-loadings with the square root of AVEs, confirming that each item had a stronger association with its intended construct than with others.

The confirmatory factor analysis (CFA) was employed to evaluate the model fit to the proposed theoretical model. The model fit was evaluated using various indicators, including CHI/DF less than 5, goodness-of-fit indexes greater than 0.90, and root mean square error of approximation (RMSEA) less than 0.08, which indicates an acceptable to good fit. The hypotheses were tested by structural equation modeling for the simultaneous examination of multiple relationships within the proposed model.

The moderating effects of demographic variables were investigated by MacKinnon's nonstandardized coefficient method, which involved splitting the sample into two groups based on the demographic characteristic of interest and assessing the significance of the difference in path coefficients using a Z-test with a threshold of |Z| > 1.96.⁴⁰

Results

Demographic characteristics

Harman's test showed acceptable common method biases (33.58%<40%). Demographic characteristics of respondents are shown in Table 2. The demographic distribution of respondents closely aligned with the overall doctor population. Roughly two-thirds of survey respondents were male (65.2%) and one-third were female (34.8%). The surveyed doctors’ ages mostly ranged from 31 to 45 years (35.5%), with 20% being under 30 or over 60. Mostly, they held a doctorate degree (57.5%); others had a master's or lower degree. Among them, associate chief doctors/professors made up the largest group (37.0%), while chief doctors/professors were the least (11.0%).

Table 2.

Demographic characteristics of respondents.

Items	Frequency	Percent
Gender
Male (1)	623	65.20
Female (2)	332	34.80
Age
Under 30 years old (1)	192	20.10
31–45 years old (2)	339	35.50
46–60 years old (3)	230	24.10
61 years old and above (4)	194	20.30
Education
Master degree or below (1)	406	42.50
Doctoral degree (2)	549	57.50
Title
Physician/primary title (1)	236	24.70
Attending physician/intermediate title (2)	261	27.30
Associate chief physician/associate professor (3)	353	37.00
Chief physician/professor (4)	105	11.00

Reliability and validity analysis

The results of the reliability and validity assessments provide evidence that the research instrument is robust and suitable for the purpose of examining the factors influencing Chinese doctors’ adoption of MLLMs. The internal consistency and stability were evaluated to ensure reliability. As detailed in Table 3, all items demonstrated high reliability, with Cronbach's alpha values exceeding the threshold of 0.7, indicating strong internal consistency among the items within each construct. Convergence validity was assessed to determine the degree to which the items within a scale converge to represent a single underlying construct. The results were satisfactory, with average variance extracted (AVE) values greater than 0.5 and composite reliability (CR) values exceeding 0.7 for all factors. This suggests that the constructs are well-defined and the scales are reliable in measuring the intended constructs. Each item exhibited strong factor loadings onto its respective factor, with all values greater than the recommended threshold of 0.67. This finding indicates that the measurement instrument is effective in capturing the underlying theoretical constructs it was designed to assess.

Table 3.

Factor loadings, internal consistency, and convergent validity.

Construct	Variable	Loading^a	C.R^b	Cronbach's alpha	AVE^c
Perceived ease of use (PEOU)	PEOU1	0.85	0.88	0.87	0.59
	PEOU2	0.73
	PEOU3	0.77
	PEOU4	0.73
	PEOU5	0.74
Content quality (CQ)	CQ1	0.84	0.88	0.88	0.65
	CQ2	0.76
	CQ3	0.84
	CQ4	0.77
Technical support (TS)	TS1	0.76	0.88	0.88	0.60
	TS2	0.80
	TS3	0.80
	TS4	0.78
	TS5	0.74
Social influence (SI)	SI1	0.79	0.90	0.90	0.69
	SI2	0.86
	SI3	0.84
	SI4	0.83
Perceived usefulness (PU)	PU1	0.79	0.90	0.90	0.69
	PU2	0.78
	PU3	0.95
	PU4	0.80
Perceived satisfaction (PS)	PS1	0.83	0.91	0.91	0.72
	PS2	0.84
	PS3	0.86
	PS4	0.86
Perceived risk (PR)	PR1	0.85	0.91	0.91	0.72
	PR2	0.87
	PR3	0.86
	PR4	0.82
Usage behavior (UB)	UB1	0.84	0.92	0.92	0.73
	UB2	0.87
	UB3	0.89
	UB4	0.82

Loading: standardized factor loading; ^bC.R: composite reliability; ^cAVE: average variance extracted.

The discriminant validity of the constructs was confirmed, as evidenced by the average variance extracted (AVE) square roots exceeding the interconstruct correlations for all pairs. This finding suggests that each construct is distinct from others, reinforcing the distinctiveness of the constructs. The strong results for reliability and validity provide substantial support for the measurement instrument and the research model. The high levels of reliability and validity enhance the credibility of the conclusions drawn regarding the factors influencing the adoption of MLLMs by Chinese doctors. The fit of the theoretical model to the empirical data was evaluated using various indices, yielding satisfactory results. The CHI/DF was 2.62 (<5), NFI (0.94), IFI (0.96), TLI (0.96), CFI (0.92), GFI (0.93), and AGFI (0.92) were above 0.90. RMSEA was 0.04 and below 0.08. The indices of model-data fit collectively indicate that the theoretical model is well-aligned with the sample data, providing a solid foundation for the conclusions of the study.

Hypothesis testing

The results of the hypothesis testing are presented in Table 4. The squared multiple correlation (R²) of perceived usefulness, perceived satisfaction, and usage behavior were 0.440, 0.421, and 0.553, respectively, suggesting the adequate explanatory power of the proposed model. Key findings from the hypothesis testing are as follows: Perceived ease of use (PEOU) significantly predicted both perceived usefulness (β=.42, p < .001) and perceived satisfaction (β=.10, p = .026), supporting hypotheses H1a and H1b. Content quality (CQ) had a significant effect on perceived satisfaction (β=.46, p < .001) but not on perceived usefulness (β=.06, p = .155), thus supporting H2b but not H2a. Technical support (TS) was a significant predictor of perceived usefulness (β=.18, p < .001), supporting H3a, but did not significantly influence perceived satisfaction, failing to support H3b. Similarly, social influence (SI) significantly affected perceived usefulness (β=.17, p < .001), supporting H4a, but had no significant impact on perceived satisfaction, not supporting H4b. Perceived usefulness (PU) positively predicted perceived satisfaction (β=.10, p = .019) and usage behavior (β=.26, p < .001), supporting H5 and H6. Perceived satisfaction (PS) was a significant predictor of usage behavior (β=.38, p < .001), supporting H7. Perceived risk (PR) negatively influenced perceived satisfaction (β=−.14, p < .001) and usage behavior (β=−.34, p < .001), supporting H8 and H9. These results provide valuable insights into the factors influencing Chinese doctors’ adoption of MLLMs, offering a foundation for management strategies and interventions.

Table 4.

Hypotheses testing results based on the structural equation model.

Hypotheses	Paths			t	β	p-Value	Comments
H1a	PEOU	—>	PU	11.10	0.42	***	Supported
H1b	PEOU	—>	PS	2.23	0.10	0.026	Supported
H2a	CQ	—>	PU	1.42	0.06	0.155	Not supported
H2b	CQ	—>	PS	9.88	0.46	***	Supported
H3a	TS	—>	PU	4.53	0.18	***	Supported
H3b	TS	—>	PS	0.67	0.03	0.501	Not supported
H4a	SI	—>	PU	4.909	0.181	***	Supported
H4b	SI	—>	PS	0.01	0.00	0.989	Not supported
H5	PU	—>	PS	2.36	0.10	0.018	Supported
H6	PU	—>	UB	8.55	0.26	***	Supported
H7	PS	—>	UB	11.88	0.38	***	Supported
H8	PR	—>	PS	−3.91	−0.14	***	Supported
H9	PR	—>	UB	−11.23	−0.34	***	Supported

***p < .001, bootstrapping N = 6000.

The squared multiple correlation (R2) of perceived usefulness, perceived satisfaction, and usage behavior were 0.440, 0.421, and 0.553, respectively.

^a–h

PEOU: perceived ease of use; CQ: content quality; TS: technical support; SI: social influence; PU: perceived usefulness; PS: perceived satisfaction; PR: perceived risk; UB: usage behavior.

Moderating effects

Table 5 presented the results on the moderating effects of demographic variables. The four demographic variables gender, age, education, and title, all exerted moderating effects in the theoretical model along the path between perceived usefulness, perceived satisfaction, perceived risk, and usage behavior, but the path from perceived satisfaction to usage behavior was not affected by the above variables. Gender was found to be a significant control variable, affecting relationship strength on three paths: from perceived usefulness to perceived satisfaction (Z = −2.53), from perceived risk to perceived satisfaction (Z = 2.89), and from perceived risk to usage behavior (Z = −5.84). Age significantly moderated three paths, from perceived usefulness to perceived satisfaction (Z = −4.25), from perceived risk to perceived satisfaction (Z = −2.12), and from perceived usefulness to usage behavior (Z = 5.77). Education demonstrated significant moderating effects on three paths: from perceived risk to perceived satisfaction (Z = −2.99), and from perceived usefulness to usage behavior (Z = −4.08), and from perceived risk to usage behavior (Z = −2.66). Title significantly regulated the path from perceived usefulness to perceived satisfaction (Z = −3.72).

Table 5.

Moderating effects of control variables.

Variable	Groups	PU^a —>PS^b	PR^c —>PS	PU—>UB^d	PR—>UB	PS—>UB
Gender	Male (1)	0.33^e	−0.14	0.31	−0.33	0.30
	Female (2)	0.56	−0.04	0.10	−0.01	0.36
	Z	−2.53 ^f	−1.38	2 . 89	−5 . 84	−1.21
Age	Under 45 years old (1)	0.29	−0.15	0.39	−0.20	0.34
	46 years old and above (4)	0.66	0.02	0.01	−0.21	0.33
	Z	−4 . 25	−2 . 12	5 . 77	0.19	0.19
Education	Master degree or below (1)	0.35	−0.26	0.09	−0.38	0.32
	Doctoral degree (2)	0.45	−0.06	0.37	−0.24	0.34
	Z	−1.12	−2 . 99	−4 . 08	−2 . 66	−0.35
Title	Intermediate title or below (1)	0.27	−0.17	0.28	−0.29	0.34
	Associate professor or above (2)	0.57	−0.09	0.31	−0.29	0.33
	Z	−3 . 72	−1.24	−0.45	0.09	0.19

^a—d

PU: perceived usefulness; PS: perceived satisfaction; PR: perceived risk; UB: usage behavior. ^e Standardized path coefficients for different demographic groups; ^f Z represents the significance of the moderating effects among demographic variables. If the absolute value of Z is greater than 1.96, the moderating effects are significant, otherwise not significant.

Discussion

Perceived ease of use was a key factor in determining perceived usefulness and satisfaction, aligning with past studies. Perceived ease of use is critical for medical technology acceptance and satisfaction, as seen in electronic health record (EHR) systems, where complexity hinders adoption despite advanced features.²⁷ Similarly, an intuitive Picture Archiving and Communication System (PACS) interface boosts radiologists’ satisfaction. In mHealth apps, a simple interface is vital for quick info access in busy settings.^34,41 Doctors and healthcare workers need quick access to information in busy environments, and a simple and intuitive mobile app interface is crucial in driving user acceptance and usage behavior.⁴² This study underscores perceived ease of use's role in enhancing medical technology's perceived usefulness and satisfaction, emphasizing its importance in designing large-scale medical language models to improve service quality.

In the realm of MLLMs, content quality emerges as a pivotal factor influencing perceived satisfaction, but it does not significantly affect perceived usefulness. This distinction underscores the multifaceted nature of user experience, where satisfaction is tied to the quality of interaction and information delivery, while usefulness is more closely aligned with the model's functionality and performance in specific tasks. MLLMs are responsible for analyzing vast medical datasets, including patient records, research findings, and clinical guidelines. Their crucial role in diagnostics, treatments, and clinical decision-making is crucial and has direct implications for medical productivity and patient safety.²⁴ The statistical analysis shows that while content quality is highly valued for its contribution to user satisfaction, its impact on the perceived usefulness of MLLMs is less pronounced. This suggests that future development of MLLMs should not only focus on enhancing content quality but also on improving other aspects that contribute to their perceived usefulness.

In the medical technology field, technical support is crucial for acceptance and usage.^43,44 However, in the context of MLLMs, technical support predicts perceived usefulness but not satisfaction. The significance of technical support in digital health is recognized. For instance, it aids doctors use electronic medical records and increases satisfaction with smart hospital technologies.⁴⁵ Studies show that technical support enhances the perceived usefulness of e-visits, but its impact on satisfaction may not be as significant as other elements. When it comes to MLLMs, technical support is essential for perceived usefulness, but its impact on satisfaction is unclear. This could be due to the complexity and professionalism of these models, which require more comprehensive support than basic online assistance.⁴⁶ Doctors may prioritize model performance and output quality over technical support. Therefore, in addition to technical support, factors such as model performance, content quality, and perceived ease of use should be taken into consideration for a comprehensive user experience.

Social influence wields a significant impact on the acceptance of medical technology, particularly in the realm of MLLMs. This study echoes the established notion that the social context and peer opinions significantly shape the perceived usefulness of new technologies among medical professionals.⁴⁷ However, this influence does not extend to perceived satisfaction, which appears to be more intrinsically linked to perceived ease of use, content quality, and its capacity to fulfill specific needs.¹² The distinction between the effects of social influence on perceived usefulness versus satisfaction is not a unique phenomenon. It mirrors observations from studies on other medical technologies, such as electronic medical records and telemedicine, where social factors have been shown to play a crucial role in the initial adoption phase.⁴⁸ Understanding this multifaceted nature of social influence is vital for the strategic promotion of MLLMs. To effectively leverage social influence, several approaches can be adopted: establishing platforms for open dialogue to encourage sharing experiences and discussion of best practices;⁴⁹ presenting endorsements from respected peers to build credibility and trust in technology;⁵⁰ integrating MLLMs into professional development to offer hands-on experience, which can enhance perceived usefulness and adoption.⁵¹ These measures, while not guaranteed to significantly boost adoption, could contribute to a more informed and positive perception of MLLMs among doctors.

Doctors may be inclined to utilize an MLLM if they perceive it as a tool that can efficiently aid in processing medical data. In the context of medical information technology applications, perceived usefulness and satisfaction are often regarded as pivotal factors influencing usage behavior. These observations resonate with recent research on large language models tailored for healthcare. Within the medical technology sector, the perception of a technology's usefulness is a key determinant in doctors’ acceptance of novel technologies. For instance, doctors are more likely to adopt an online health community system if they believe it can enhance their productivity.⁵² Furthermore, perceived satisfaction plays a significant role in influencing users’ willingness to persist in using healthcare information technology. These insights provide valuable support for promoting the utilization of MLLMs.

Perceived satisfaction plays a crucial role in predicting usage behavior, especially in the context of MLLMs. This finding contrasts interestingly with other research in the medical technology field. For instance, in electronic medical record systems, system functionality, user interface design, user training, and support all significantly impact satisfaction.^20,53 This underscores the diversity of satisfaction levels and varying emphases across different technology fields within medical technology. Compared to telemedicine, MLLMs prioritize the accuracy of natural language processing and information extraction, which partially determines user satisfaction with the model's performance. Conversely, telemedicine focuses more on accessibility, convenience, and the quality of doctor-patient interaction. While perceived satisfaction is vital in these areas too, it's influenced by factors like doctors’ responsiveness and service availability.¹⁹ The perceived satisfaction in medical language models bears some similarities to other medical technology research but also has unique characteristics. These differences suggest that when studying user satisfaction in medical technologies, it's essential to consider the specificities of technical features and application scenarios.

The statistical analysis indicates that perceived risk is a significant negative predictor of both perceived satisfaction and usage behavior among Chinese doctors regarding MLLMs. This finding is consistent with previous literature that has identified perceived risk as a barrier to technology adoption in healthcare settings.^54,55 To address these perceived risks and enhance the adoption of MLLMs, several strategies can be employed. First, transparency regarding the data sources, training processes, and operational mechanisms of MLLMs can alleviate concerns about privacy and data security.⁵⁶ Second, ongoing education and training for healthcare professionals can improve their understanding of MLLMs, reduce anxiety associated with the technology, and increase confidence in its use.^57,58 Third, establishing robust legal and ethical frameworks that govern the use of MLLMs can help protect user rights and ensure the responsible application of these technologies.¹⁷ By implementing these strategies to mitigate the negative impact of perceived risks on the adoption of MLLMs and foster a more positive attitude toward their integration into clinical practice.

The statistical analysis revealed that demographic variables such as gender, age, education, and title act as moderators in the relationship between perceived usefulness, perceived satisfaction, perceived risk, and usage behavior within the theoretical model. The practical implications of these moderating effects are noteworthy and suggest that individual differences significantly influence the adoption of MLLMs. Gender was found to significantly influence multiple relationships, indicating that male and female doctors may respond to MLLMs differently, possibly due to varying levels of exposure, interest, or comfort with technology. Age plays a crucial role, with older doctors potentially requiring more support or different considerations in the adoption process compared to their younger counterparts. The level of education can affect their perceived risk and satisfaction, which in turn influences their usage behavior, suggesting that educational background may be linked to their tech-savviness and risk perception. Finally, a doctor's professional title moderates the relationship between perceived usefulness and satisfaction, indicating that the status and experience associated with their title could influence their attitudes and the value they see in MLLMs. These findings align with prior research in medical IT, emphasizing the impact of demographic factors on technology adoption.^58,59 However, it is worth noting that different studies may have varying results depending on the specific context and sample characteristics. Adoption of MLLMs by healthcare professionals such as doctors, nurses, and allied health workers may be influenced by different factors. Their professional roles and work environments may result in varying priorities and needs for MLLMs. It is crucial for future research to examine these differences to ensure that technology meets the diverse needs of the broader healthcare community. Although our study concentrated on individual perceptions and attitudes, it is acknowledged that organizational and systemic factors play a crucial role in the adoption of new technologies such as MLLMs. Organizational, legal, ethical, and technological integration issues are also likely to play a crucial role in the adoption process and warrant further investigation. Understanding these factors and their influence on individual adoption behaviors is an important area for future research.

This study provides a perspective on doctors’ adoption of MLLMs, and our model integrates individual psychological and social demographic factors, offering a deeper understanding of this complexity. Based on our findings, we recommend that medical big language model developers, hospitals, and regulators collaborate to enhance training, showcase model benefits, improve technical support, and foster a supportive culture for technological innovations, considering social and cultural factors. It is worth noting that our sample consisted of physicians who had knowledge of MLLMs, which could have influenced their adoption factors. The extent of prior experience with MLLMs among our respondents was not the primary focus of this study; however, it may be a valuable area for future research. Comparative studies that examine the impact of prior experience on the adoption of MLLMs could provide deeper insight into how familiarity with such technologies affects their integration into clinical practice.

This study has some limitations. One of the limitations of this study is the possibility of bias in sample selection, as it was exclusively focused on Chinese doctors. As a result, the generalizability of the findings to other countries or healthcare systems remains unclear. Cultural and systemic differences in healthcare practices, education, and technology adoption may significantly impact the factors that drive the adoption of MLLMs. Further research should examine these factors across diverse contexts to gain a better understanding of their global applicability. Second, the study lacks specificity in terms of the individual MLLMs referenced in the survey questions, and it potentially overlooking the distinct impacts of individual models’ features on adoption factors. Future research should conduct detailed comparative analyses and employ mixed methods to capture specific MLLMs’ reception in varied medical contexts, enhancing the understanding of their integration and adoption.

We acknowledge that different MLLMs possess unique features, capabilities, and limitations that could significantly influence the adoption factors we sought to investigate. Our decision to maintain a broad approach was driven by the desire to understand the overarching perceptions and attitudes toward MLLMs as a collective, rather than focusing on specific models. However, this approach may have implications for the generalizability of our findings to specific MLLMs. Future research should examine the adoption factors of specific MLLMs to provide a more nuanced understanding of their reception and integration into medical practice. Third, it is acknowledged that the lack of a formal validation process for the questionnaire prior to use limits the validity of this study. Although the questionnaire was developed based on well-established theoretical frameworks and a pilot test was conducted with 50 doctors to refine the instrument and ensure its clarity and relevance, the lack of extensive validation may affect the generalizability of our findings. Therefore, future research should implement a more rigorous validation process to enhance the psychometric properties of the questionnaire and broaden the scope of the results. Moreover, the reliance on self-reported survey data, which may be influenced by social desirability bias. Despite potential biases, self-report measures are a common and valuable tool in psychological research, and these measures are often used to gather diagnostic and meaningful information.^60,61 Future studies might consider combining self-reported data with objective usage data to provide a more comprehensive understanding of the adoption and usage of MLLMs. Finally, the cross-sectional design restricts our ability to infer causality and monitor changes in adoption intentions over time. The performance of large language models in China is rapidly improving due to the quality of training data, which will change doctors’ attitudes toward MLLMs. These limitations highlight the need for ongoing studies that can provide a more comprehensive understanding of MLLM adoption and performance.

Conclusion

The application of MLLMs marks a digital shift. This study aimed to assess factors influencing Chinese doctors’ adoption intentions. By constructing an extended theoretical framework, the empirical analysis showed that perceived ease of use, content quality, technical support, social influence, perceived usefulness, and satisfaction positively impact adoption, while perceived risk has a negative effect. This highlights that doctors’ positive evaluation of the MLLM and their awareness of improving work efficiency are the key factors driving their adoption behavior. Furthermore, personal traits also affect the adoption intention of doctors to varying degrees, highlighting the external conditions and personal characteristics that need to be considered when promoting the MLLM. Through ongoing research and practical experience, we strive to understand doctors’ motivations for using these models to promote their effective use worldwide.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241297237 - Supplemental material for Factors influencing Chinese doctors to use medical large language models

Supplemental material, sj-docx-1-dhj-10.1177_20552076241297237 for Factors influencing Chinese doctors to use medical large language models by Shujuan Qu, Lin Liu, Min Zhou, Chuting Zhou and Kathryn S. Campy in DIGITAL HEALTH

Footnotes

Acknowledgments

The authors are deeply grateful to the Chinese doctors who participated in our study. The authors acknowledge the utilization of AI language polishing tools (Kimi and ERNIE Bot) in the refinement of this manuscript. It is important to note that while these AI tools assisted in the editing process, the conceptualization, research content, data analysis, and interpretation of results were original contributions from the authors and were not influenced by AI tools.

Contributorship

Shujuan Qu performed conceptualization, wrote the original draft, implemented the methodology, and acquired funding. Lin Liu conducted investigations, provided resources, curated data, and wrote the original draft. Min Zhou implemented the methodology, conducted investigations, utilized software, conducted validation, and performed formal analysis. Chuting Zhou wrote the original draft and conducted writing reviews and editing. Kathryn S. Campy performed conceptualization, conducted writing reviews and editing, and supervised the process.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

Not required.

Funding

This work was supported by the National Social Science Fund of China (21BGL227), the National Natural Science Foundation of China (82102280), and the Natural Science Foundation of Hunan Province (2023JJ30839, 2024JJ5119, 2022JJ30893).

Guarantor

SQ.

ORCID iD

Min Zhou

Supplemental material

Supplementary material for this article is available online.

References

Epstein

Dexter

. Variability in large language Models’ responses to medical licensing and certification examinations. Comment on “how does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment”. Letter. Jmir Medical Education 2023; 2023: 9e48305.

Singhal

Azizi

, et al. Large language models encode clinical knowledge. Article; early access. Nature 2023; 620: 72.

Gilson

Safranek

Huang

, et al. Authors’ reply to: variability in large language Models’ responses to medical licensing and certification examinations. Letter. Jmir Medical Education 2023; 2023: 9e50336.

Jowsey

Stokes-Parish

Singleton

, et al. Medical education empowered by generative artificial intelligence large language models. Editorial material; early access. Trends Mol Med 2023; 29: 971–973.

Mesko

. The ChatGPT (generative artificial intelligence) revolution has made artificial intelligence approachable for medical professionals. Article. J Med Internet Res Jun 2023; 25: e48392.

Gao

Gurd

. Problems and challenges in managing the new generation of doctors in Chinese hospitals. Article. Asia Pacific Business Review 2018; 24: 528–542. 2018.

Wang

Shao

, et al. Job burnout on subjective well-being among Chinese female doctors: the moderating role of perceived social support. Article. Front Psychol Mar 2020; 11: 435.

Tang

, et al. Does economic support have an impact on the health Status of elderly patients with chronic diseases in China?-based on CHARLS (2018) data research. Article. Front Public Health Apr 2021; 9: 658830.

Davis

Venkatesh

. A critical assessment of potential measurement biases in the technology acceptance model: three experiments. Int J Hum Comput Stud Jul 1996; 45: 19–45.

10.

Davis

. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 09/01 1989; 13: 19.

11.

Rouidi

Elouadi

Hamdoune

. Acceptance and use of telemedicine technology by health professionals: development of a conceptual model. Digit Health Jan-Dec 2022; 8: 20552076221081693.

12.

Kissi

Dai

Dogbe

, et al. Predictive factors of physicians’ satisfaction with telemedicine services acceptance. Health Informatics J Sep 2020; 26: 1866–1880.

13.

Kalavani

Kazerani

Shekofteh

. Acceptance of evidence based medicine (EBM) databases by Iranian medical residents using unified theory of acceptance and use of technology (UTAUT). Health Policy Technol 2018; 7: 287–292.

14.

Gao

Xiaojun

, et al. An empirical study of wearable technology acceptance in healthcare. Ind Manag Data Syst 2015; 115: 1704–1723.

15.

Tsai

. Integrating social capital theory, social cognitive theory, and the technology acceptance model to explore a behavioral model of telehealth systems. Int J Environ Res Public Health May 7 2014; 11: 4905–4925.

16.

Hennemann

Witthoft

Bethge

, et al. Acceptance and barriers to access of occupational e-mental health: cross-sectional findings from a health-risk population of employees. Int Arch Occup Environ Health Apr 2018; 91: 305–316.

17.

Parimbelli

Bottalico

Losiouk

, et al. Trusting telemedicine: a discussion on risks, safety, legal implications and liability of involved stakeholders. Int J Med Inform Apr 2018; 112: 90–98.

18.

Rajak

Shaw

. An extension of technology acceptance model for mHealth user adoption. Technol Soc 2021/11/01/ 2021; 67: 101800.

19.

Liu

, et al. A longitudinal examination of tablet self-management technology acceptance by patients with chronic diseases: integrating perceived hand function, perceived visual function, and perceived home space adequacy with the TAM and TPB. Appl Ergon 2022/04/01/ 2022; 100: 103667.

20.

Min-Fang

Shin-Yuan

Wen-Ju

, et al. Understanding physicians’ adoption of electronic medical records: healthcare technology self-efficacy, service level and risk perspectives. Comput Stand Interfaces Oct 2019; 66: 1–11. 103342.

21.

Davis

. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 1989; 13: 319–340.

22.

Davis

Bagozzi

Warshaw

. User acceptance of computer technology: a comparison of two theoretical models. Manage Sci 08/01 1989; 35: 982–1003.

23.

Zhu

Wang

. Information quality framework for verifiable intelligence products. In: Chan

Talburt

Talley

(eds) Data engineering: mining, information and intelligence. Boston, MA: Springer US, 2010, pp.315–333.

24.

Chakraborty

Bhattacharya

Lee

S-S

. Need an AI-enabled, next-generation, advanced ChatGPT or large language models (LLMs) for error-free and accurate medical information. Letter; early access. Ann Biomed Eng 2024; 52: 134–135.

25.

Mesnier

Suc

Sayah

, et al. Relevance of medical information obtained from chat- GPT: are large language models friends or foes? Letter; early access. Arch Cardiovasc Dis 2023; 116: 485–486.

26.

Venkatesh

Davis

. A theoretical extension of the technology acceptance model: four longitudinal field studies. Manage Sci 02/01 2000; 46: 186–204.

27.

Zhou

Owusu-Marfo

Asante Antwi

, et al. Assessment of the social influence and facilitating conditions that support nurses’ adoption of hospital electronic information management systems (HEIMS) in Ghana using the unified theory of acceptance and use of technology (UTAUT) model. BMC Med Inform Decis Mak Nov 21 2019; 19: 30.

28.

Sung

Chen

, et al. Applying natural language processing techniques to develop a task-specific EMR interface for timely stroke thrombolysis: a feasibility study. Int J Med Inform. Apr 2018; 112: 149–157.

29.

Venkatesh

Morris

Davis

, et al. User acceptance of information technology: toward a unified view. MIS Q Sep 2003; 27: 425–478.

30.

Venkatesh

Thong

JYL

Chan

FKY

, et al. Extending the two-stage information systems continuance model: incorporating UTAUT predictors and the role of context. Inf Syst J 2011; 21: 527–555.

31.

Hudson

Kushniruk

Borycki

, et al. Physician satisfaction with a critical care clinical information system using a multimethod evaluation of usability. Int J Med Inform Apr 2018; 112: 131–136.

32.

Zhou

Zhao

Kong

, et al. Factors influencing behavior intentions to telehealth by Chinese elderly: an extended TAM model. Int J Med Inf 2019-Jun 2019; 126: 118–127.

33.

Lin

T-C

Hsu

JS-C

, et al. The integration of value-based adoption and expectation–confirmation models: an example of IPTV continuance intention. Decis Support Syst 2012; 54: 63–75.

34.

Yen

Sousa

Bakken

. Examining construct and predictive validity of the health-IT usability evaluation scale: confirmatory factor analysis and structural equation modeling results. J Am Med Inform Assoc Oct 2014; 21: 241–248.

35.

Mao

Lin

, et al. A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics. ArXiv 2023; 11: abs/2310.05694.

36.

Zhou

Zou

, et al. A survey of large language models in medicine: progress, application, and challenge. ArXiv 2023; 11: abs/2311.05112.

37.

Qiu

Sun

, et al. Large AI models in health informatics: applications, challenges, and the future. IEEE J Biomed Health Inform 2023; 27: 6074–6087.

38.

Podsakoff

MacKenzie

Lee

, et al. Common method biases in behavioral research: a critical review of the literature and recommended remedies. Review. Journal of Applied Psychology Oct 2003; 88: 879–903.

39.

Hair

Sarstedt

Ringle

, et al. An assessment of the use of partial least squares structural equation modeling in marketing research. J Acad Mark Sci May 2012; 40: 414–433.

40.

MacKinnon

Lockwood

Williams

. Confidence limits for the indirect effect: distribution of the product and resampling methods. Multivariate Behav Res 2004; 39: 99–128.

41.

Puspitasari

Firdauzy

. Characterizing consumer behavior in leveraging social Media for E-patient and health-related activities. Int J Environ Res Public Health 2019; 16: 3348.

42.

Singh

Dube

Varshney

, et al. A comprehensive mobile health intervention to prevent and manage the complexities of opioid use. Int J Med Inf Aug 2022; 164: 104792.

43.

Grout

Cheng

Carroll

, et al. A six-year repeated evaluation of computerized clinical decision support system user acceptability. Int J Med Inform Apr 2018; 112: 74–81.

44.

Harris

Shi

Brealey

, et al. Critical care health informatics collaborative (CCHIC): data, tools and methods for reproducible research: a multi-centre UK intensive care database. Int J Med Inform Apr 2018; 112: 82–89.

45.

Spatar

Kok

Basoglu

, et al. Adoption factors of electronic health record systems. Technol Soc 2019; 58: 101144.

46.

Wang

. Perceived city smartness level and technical information transparency: the acceptance intention of health information technology during a lockdown. Comput Human Behav Sep 2021; 122: 106840.

47.

Cao

Feng

Lim

, et al. How social influence promotes the adoption of Mobile health among young adults in China: a systematic analysis of trust, health consciousness, and user experience. Behavioral Sciences (Basel, Switzerland) Jun 2024; 14: 498.

48.

Bitrián

Buil

Catalán

. Enhancing user engagement: The role of gamification in mobile apps. Journal of Business Research 2021/08/01/ 2021; 132: 170–185. https://doi.org/10.1016/j.jbusres.2021.04.028

49.

Richardson

Lawrence

Schoenthaler

, etal. A framework for digital health equity. npj Digital Medicine 2022; 5: 119–125.

50.

Patrick

Venkatesh

Stukus

. Social media and its impact on health care. Ann Allergy Asthma Immunol 2022/02/01/ 2022; 128: 139–145.

51.

Weik

Fehring

Mortsiefer

, et al. Understanding inherent influencing factors to digital health adoption in general practices through a mixed-methods analysis. npj Digital Medicine 2024/02/27 2024; 7: 47.

52.

Zhang

Zhu

. An empirical study on Patients’ acceptance of physician-patient interaction in online health communities. Int J Environ Res Public Health Dec 2019; 16: 5084.

53.

Lee

Tsai

Ruangkanjanases

. The determinants for food safety push notifications on continuance intention in an e-appointment system for public health medical services: the perspectives of UTAUT and information system quality. Int J Environ Res Public Health Nov 9 2020; 17. DOI:https://doi.org/10.3390/ijerph17218287

54.

Zhao

Zhou

. What factors influence the mobile health service adoption? A meta-analysis and the moderating role of age. International Journal of Information Management 2018/12/01/ 2018; 43: 342–350. doi:https://doi.org/10.1016/j.ijinfomgt.2017.08.006

55.

Lambiase

Catolino

Pecorelli

, et al. An empirical investigation into the influence of software communities’ cultural and geographical dispersion on productivity. Journal of Systems and Software 2024/02/01/ 2024; 208: 111878. doi:https://doi.org/10.1016/j.jss.2023.111878

56.

Yang

, et al. Understanding the role of mobile internet-based health services on patient satisfaction and word-of-mouth. Int J Environ Res Public Health Sep 10 2018; 15. doi:https://doi.org/10.3390/ijerph15091972

57.

Wong

RSM

EYT

Wong

, et al. Development and pilot evaluation of a mobile app on parent-child exercises to improve physical activity and psychosocial outcomes of Hong Kong Chinese children. BMC Public Health Oct 14 2020; 20: 1544. doi:https://doi.org/10.1186/s12889-020-09655-9

58.

Zhou

Kong

, et al. Factors influencing user acceptance of weight management apps among Chinese obese individuals during the COVID-19 pandemic. Health Policy Technol 2023/06/01/ 2023; 12: 100758.

59.

Rentrop

Damerau

Schweda

, et al. Predicting acceptance of e-mental health interventions in patients with obesity by using an extended unified theory of acceptance model: cross-sectional study. JMIR Form Res Mar 17 2022; 6: e31229.

60.

Devaux

Sassi

. Social disparities in hazardous alcohol use: self-report bias may lead to incorrect estimates. Eur J Public Health 2015; 26: 129–134.

61.

Levin-Aspenson

Watson

. Mode of administration effects in psychopathology assessment: analyses of gender, age, and education differences in self-rated versus interview-based depression. Psychol Assess 2018; 30: 287–295.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB