Sage Journals: Discover world-class research

Abstract

LLM-driven healthcare chatbots for preliminary medical consultation are a promising innovation to improve healthcare accessibility and efficiency. However, public acceptance of this technology in the Chinese context, especially the impact of users’ previous experience with relevant technologies on user behavior, remains underexplored. To address this gap, we extended the classical Unified Theory of Acceptance and Use of Technology (UTAUT) framework by examining the moderating effects of users’ previous experience with telemedicine and large language models (LLMs). Using a scenario-based survey, we collected 502 valid responses from general Chinese users and analyzed the data through Structural Equation Modelling (SEM). Our results demonstrated that performance expectancy, social influence, trust, and facilitating conditions were significant contributing factors, whereas effort expectancy was not, which contradicts previous literature. Moreover, users’ previous experience with LLMs exhibited significant moderating effects whereas previous experience with telemedicine didn’t. These findings contribute to the literature by suggesting that as LLMs become more widely adopted, users’ familiarity with them may enhance trust and, consequently, increase the general acceptance of LLM-driven healthcare chatbots.

Keywords

LLM-driven healthcare chatbots public acceptance UTAUT moderating effects structural equation modeling

Introduction

Given the rapid development of Artificial Intelligence (AI) across various sectors, including healthcare, healthcare providers are increasingly incorporating AI in healthcare services, by means such as AI assisted triage, enquiry, treatment plan and CT report generation to improve efficiency and parity.^1,2 Among them, LLM-driven healthcare chatbots, the intelligent conversational agents that utilize generative artificial intelligence to provide medical information, symptom assessment, and preliminary consultations in conversations have achieved notable breakthroughs and are being promoted by healthcare providers,³ which may save them from repetitive labor and reduce users’ waiting time for consultation services.^4,5 Tech-savvy users have embraced LLM-driven chatbots as their preferred method of medical consultation. However, the general public may perceive interacting with AI as cognitively demanding or may doubt the chatbot’s diagnostic accuracy compared to human doctors.⁶ The unfamiliarity and complicated procedure may impede the users’ willingness to use such services.⁷ Moreover, the user’s trust in LLM-driven chatbots may also affect their willingness to use them for healthcare purposes.⁸ Considering these potential challenges, it is important to examine and understand the mechanism of which users choose to accept these LLM-driven healthcare chatbots.⁹

Previous research has investigated the acceptance of LLM-driven chatbots from different orientations.¹⁰ For example, researchers focused on individual-level determinants and found that perceived usefulness, subjective norms, and trust significantly influence users’ intention to use LLM-driven health chatbots.¹¹ Similarly, another study that explored seniors’ adoption of AI-driven healthcare services highlighted the importance of ease of use, perceived usefulness, and emotional support.¹² However, the mechanism of which users’ expectations and perceptions influences the adoption of LLM-driven healthcare chatbots through moderation effects of previous user experiences of related technology remains unexplored.

Behavioral studies often adopt a technology-psychology-society framework to examine the acceptance of LLM-driven chatbots in healthcare and other domains. From a technological perspective, factors such as perceived usefulness, performance expectancy, and system quality are commonly identified as key drivers of user adoption.¹³ From a psychological perspective, trust, privacy concerns, and perceived risk often shape users’ emotional and cognitive responses to chatbot use.¹³ Meanwhile, the societal dimension captures the impact of subjective norms, peer influence, and organizational endorsement, which can significantly affect behavioral intention.¹⁴ We also incorporate this framework in this study to investigate the contributing factors that affect users’ behavioral intention to adopt the LLM-driven healthcare chatbots.

Users’ previous experience has been identified as an important factor in the adoption of new technology. For instance, researchers grouped survey participants into 3 clusters based on their previous user experience to examine the factors affecting their acceptance of commercial healthcare apps.¹⁵ Another research investigated the acceptance of AI health assistants and incorporated user experience as a control variable.¹⁶ However, its moderation effects were not considered in previous studies. From our knowledge, few studies have considered the impact of previous experience with LLMs and telemedicine on the acceptance of LLM-driven healthcare chatbots. Telemedicine has been increasingly used by patients. Its functions include online appointment, follow-up care, health education, and remote consultations. For remote consultation, text and graphic communication are mostly widely used, which is similar to talking with an LLM-driven healthcare chatbots. However, remote consultations in telemedicine involve communication with actual physicians, which fundamentally differs from engaging with LLM-driven healthcare chatbots. Consequently, the impact on public acceptance of healthcare chatbots may vary. Therefore, investigating the effects of users’ previous experience on the adoption of healthcare chatbots would extend our current understanding of users’ acceptance for these chatbots, which could potentially guide the chatbot developers to fulfil the users’ expectations. Therefore, in this study, we seek to address this gap by examining the moderation effects of users’ previous experience with LLMs and telemedicine on users’ behavioral intention to adopt the healthcare chatbots. We propose the following research questions:

1. How do technological, psychological and societal contributing factors influence users’ behavioral intention to adopt the LLM-driven healthcare chatbots?

2. How does previous user experience with telemedicine moderate the relationship between contributing factors and users’ behavioral intention to adopt LLM-driven healthcare chatbots?

3. How does previous user experience with LLMs moderate the relationship between contributing factors and users’ behavioral intention to adopt LLM-driven healthcare chatbots?

The Unified Theory of Acceptance and Use of Technology (UTAUT) framework is applied for our analysis and discussion, which comprehensively captures the technological, psychological, and social dimensions of user behavior. In this study, we identified five key constructs: performance expectancy, effort expectancy, social influence, trust, and facilitating conditions, and we employed a survey-based research to address our proposed research questions. Firstly, we conducted a survey to explore users’ underlying perceptions, expectations, and concerns regarding healthcare chatbots related to the constructs, and their previous experience with telemedicine and LLM chatbots. Secondly, we performed a Structural Equation Modeling (SEM) analysis with the collected data from the questionnaire and then conducted a multi-group analysis (MGA) to identify the moderation effects of users’ previous experience with telemedicine and LLM chatbots.

Our study makes several contributions to the existing study. First, it integrates trust and previous experience with telemedicine and LLM chatbots into the well-established UTAUT framework to examine users’ acceptance of LLM-driven healthcare chatbots. To our knowledge, this is one of the first attempts to investigate the moderation effects of users’ previous experience on their behavioral intention to adopt LLM-driven healthcare chatbots. Furthermore, this research provides developers of LLM-driven healthcare chatbots with a comprehensive view of user adoption from a UTAUT perspective, which can help guide the design and implementation of LLM-based conversational agents in healthcare contexts.

Methods

LLMs and LLM-driven healthcare chatbots

Generative artificial intelligence is a class of machine learning models designed to produce content by learning patterns from large datasets.¹⁷ It has increasingly been recognized as a transformative technology with profound implications across multiple domains, including education, journalism, and financial technology.^18–20 Meanwhile, it has also been revolutionizing the way medical services are used to be delivered and becomes an integral component of the healthcare industry.²¹ For example, emerging research shows that GPT-4 has demonstrated significant potential in clinical diagnostics, outperforming simulated human readers,²² highlighting its capability as a powerful supportive tool in medical diagnostics.

Recently, LLM-driven healthcare chatbots are increasingly being adopted to enhance efficiency, accessibility, and accuracy in healthcare services.^23–25 These chatbots have been utilized for managing chronic conditions, providing health education, behavior change support, and self-care guidance globally,²⁶ which is expected to improve the efficiency of healthcare services. It is expected that soon they will also be integrated into primary care settings to assist with patient consultations, aiming to alleviate physicians’ workload and improve patient access to medical care.²⁷

In the context of China, the integration of technological advancements into healthcare services has been actively embraced by local governments, hospitals and the healthcare industry, driven by persistent systemic challenges such as the unequal distribution of medical resources, the disparities in access to high-quality care, and the growing pressure of an aging population. Despite notable progress in expanding healthcare coverage and infrastructure in recent years, these structural issues continue to necessitate innovative, technology-driven solutions. For example, telemedicine has experienced rapid development in China with the potential to provide high-quality healthcare services, especially for remote and rural populations, who has limited access to traditional healthcare services. The emergence of AI technologies and their integration with healthcare services, including chatbot doctors and others, have presented a novel and efficient solution to enhance healthcare accessibility and equity.^25,28–31 However, the user acceptance of these LLM-driven healthcare chatbots in China have yet to be comprehensively examined. In this research, we apply the Unified Theory of Acceptance and Use of Technology (UTAUT) framework to investigate the key factors influencing Chinese users’ willingness to adopt LLM-driven healthcare chatbots, using a structured survey to assess constructs such as performance expectancy, effort expectancy, social influence, trust, facilitating conditions, and behavioral intention.

Theoretical lenses on user acceptance to LLM-driven healthcare chatbots

Researchers around the world have already investigated the factors affecting AI implementation in healthcare practices.³² Table 1 included several studies which met the following criteria: published in peer-reviewed journals, employed survey-based methods, focused on chatbot adoption or user acceptance, and drew on established theoretical frameworks such as TAM, UTAUT, or trust/risk perception models. As summarized in the table, factors such as trust, perceived usefulness, and responsiveness have emerged as significant drivers of chatbot acceptance. Nevertheless, one important gap remains: the potential moderating effects of users’ previous experience with telemedicine and LLMs have not yet been systematically examined, both of which might serve as important moderators of the acceptance of new technologies.³³ In the context of LLM-driven systems, user experience could serve as a potential moderator that influences the strength or direction of relationships between key acceptance constructs and behavioral intention.³⁴ Therefore, this study contributes to the understanding of the acceptance of these chatbots, focusing on the moderation effects of users’ previous experience with telemedicine and LLMs.

Table 1.

Summary of studies on healthcare chatbots adoption.

Study source	Method	Theory/framework	Variables of interest									Moderation effects
Study source	Method	Theory/framework	Performance	Anthropomorphism	Ease of use	Facilitating conditions	Norm	Trust	Risk	Bias	Social influence	Moderation effects
This paper	Survey	Extended UTAUT	✓		✓	✓		✓			✓	✓
Pang et al.¹¹	Survey	Extanded TAM	✓		✓		✓	✓	✓
Choudhury et al.¹³	Survey	Extended TAM, risk perception theory	✓		✓			✓	✓
Zhan et al.³¹	Survey	Extended TAM, trust theory	✓	✓	✓			✓
Liou & Vo³²	Survey	Extended UTAUT	✓			✓			✓		✓
Venkatesh et al.³⁴	Survey	PPM framework, status quo bias theory	✓							✓	✓

Direct effects

This study adopts the Unified Theory of Acceptance and Use of Technology (UTAUT) to examine the factors influencing users’ intention to adopt LLM-driven healthcare chatbots, including performance expectancy, effort expectancy, social influence, and facilitating conditions.³⁵ Moreover, trust, a commonly added construct in healthcare technology research, is also included as an additional predictor.¹⁰

Performance expectancy refers to users’ belief that adopting an LLM-driven healthcare chatbot will enhance efficiency and effectiveness in healthcare-related tasks. Research indicates that it plays a crucial role in the adoption of LLM-driven healthcare chatbots. Researchers identified performance expectancy as a key factor influencing both patients’ and healthcare professionals’ willingness to adopt conversational agents.¹⁴ Moreover, another study demonstrated that performance expectancy is a significant predictor of behavioral intention toward AI applications in healthcare.³⁶ Based on these results, we proposed the following hypothesis:

Performance Expectancy positively influences users’ behavioral intention to adopt LLM-driven healthcare chatbots.

Effort expectancy refers to users’ perception of how easy it is to interact with a chatbot. Existing studies have highlighted that the ease of use is a key determinant of AI chatbot adoption. For instance, researchers found that older users’ acceptance of healthcare chatbots is driven by the ease of use, suggesting that user-friendly chatbot interfaces can facilitate users’ adoption.¹¹ In addition, others also emphasized the role of effort expectancy in chatbot acceptance, particularly among patients with limited technical skills.¹⁴ Based on these arguments, we proposed the following hypothesis:

Effort Expectancy positively influences users’ behavioral intention to adopt LLM-driven healthcare chatbots.

Social influence captures the impact of external opinions, such as those from healthcare professionals, peers, and social norms, on chatbot adoption. Previous research emphasized that social influence plays a crucial role in patients’ and healthcare professionals’ decisions to adopt chatbots.¹⁴ Similarly, a study found that subjective norms significantly influence the intention to use LLM-driven health diagnostic chatbots, suggesting that recommendations from trusted individuals could shape users’ intention to adopt new technology.¹⁰ Based on these findings, we proposed the following hypothesis:

Social Influence positively influences users’ behavioral intention to adopt LLM-driven healthcare chatbots.

Research suggests that trust in AI chatbots significantly impacts user acceptance, as concerns over accuracy, security, and reliability influence behavioral intention. Research found that trust in AI-led chatbot services is a key driver of acceptability, while concerns about cybersecurity remain a challenge³⁷ Researchers also emphasized that trust in AI-driven chatbots is crucial for medical students’ acceptance, especially regarding data security and workplace monitoring.³⁸ Additionally, another study found that the skepticism about AI chatbots’ ability to provide accurate medical advice limits the overall adoption, indicating the necessity of trust-building measures.³⁹ Researchers further highlighted that trust in AI-based home care systems directly influences users’ perceived usefulness and willingness to adopt such technology.⁴⁰ Based on the arguments above, we proposed the following hypothesis:

Trust positively influences users’ behavioral intention to adopt LLM-driven healthcare chatbots.

Facilitating conditions refer to the availability of resources, technical support, and training that are necessary for the adoption of LLM-driven healthcare chatbots. Prior research suggests that these supportive environments can enhance AI chatbot usage in healthcare. For example, researchers noted that physicians indicated that better support systems could improve adoption of these healthcare chatbots.⁴¹ Based on these findings, we proposed the following hypothesis:

Facilitating Conditions positively influence users’ behavioral intention to adopt LLM-driven healthcare chatbots.

Moderation effects

In addition to the direct effects, this study further examines how previous user experience with telemedicine and LLMs influences the strength and direction of the relationships between the five core predictors and users’ intention to adopt LLM-driven healthcare chatbots. In light of the current research gap, we propose that users’ previous experience with telemedicine and LLMs could act as moderators.

Individuals who have used online hospitals are more likely to perceive AI-based health chatbots as accessible, useful, and credible. For instance, researchers highlighted that users familiar with digital health tools, including online platforms, perceive AI chatbots as credible and effective for mental health support.⁴² and another study identified that individuals with prior experience to digital healthcare services (e.g., online hospitals) are more likely to adopt LLM-driven healthcare chatbots.⁴³ Based on these findings, we propose the following hypothesis:

Previous experience with telemedicine moderates the relationships between the contributing factors and users’ behavioral intention to adopt LLM-driven healthcare chatbots.

Also, previous experience with AI chatbots may significantly influence users’ cognitive processing, trust, and acceptance of healthcare chatbots. Previous research indicated that experience with LLMs can reduce uncertainty, increase confidence, and enhance perceived ease of use. For instance, Researchers highlighted that previous interaction experience with AI could reduce users’ uncertainty and increase their overall confidence in the technology.⁴³ and others demonstrated that familiarity with LLM tools would lower users’ risk perception in downstream applications, thereby increasing trust and behavioral intention.⁴⁴ Based on the research above, we propose the following hypothesis:

Previous experience with LLMs moderates the relationship between the contributing factors and users’ behavioral intention to adopt LLM-driven healthcare chatbots.

From the above hypotheses, we can present the conceptual model as follows (Figure 1).

Figure 1.

Conceptual model.

Research design

Our study employed a structured questionnaire survey with a vignette. The study was designed and reported in accordance with the STROBE guidelines. All relevant items for cross-sectional studies, including study design, participants selection, variables, data sources, bias, study size, statistical methods, and results, were addressed as recommended by STROBE.

Following the data collection process, reliability and validity were assessed through construct reliability, convergent reliability, discriminant validity, and confirmatory factor analysis. Subsequently, direct effects were examined using covariance-based structural equation modeling (CB-SEM), and moderation effects were analyzed through multi-group analysis (MGA). The sequential analysis of the model is depicted in Figure 2.

Figure 2.

Sequential approach for validation of the model.

Survey

At the beginning of the questionnaire, a contextual example of an LLM-driven healthcare chatbot was presented with a case of medical consultation process, as shown in Figure 3.

Figure 3.

A contextual example of AI healthcare chatbots.

In the example, the LLM-driven healthcare chatbot collected information by asking about the user’s symptoms and medical background. At the end of the dialogue, the chatbot provided an assessment report suggesting the possible causes of the symptoms and treatment.

Secondly, questions were provided to the respondents to assess the factors influencing their willingness to use such LLM-driven healthcare chatbots. A total of 18 questions related to 5 model constructs were presented, with responses measured on a five-point Likert scale ranging from “strongly disagree” to “strongly agree.” The questions were based on previous research on the adoption of healthcare chatbots using the UTAUT framework.^12,31,32

Thirdly, the survey also included questions related to respondents’ demographic characteristics, as well as measures of their previous experience with telemedicine and LLMs.

Samples and data collection

The experiment was conducted online between February 17 and February 24, 2025, yielding 502 valid responses. The participants were drawn from the general public in China, recruited online nationwidely through the Wenjuanxing survey platform. Table 2 presents the descriptive statistics of the sample.

Table 2.

Sample descriptive statistics.

Items	Category	Count	Percentage (%)
Gender	Male	289	57.57%
Gender	Female	213	42.43%
Age	<21	33	6.57%
	21-30	133	26.49%
	31-40	132	26.29%
	41-60	116	23.11%
	>60	88	17.53%
Education	Junior High School	9	1.79%
	Senior High School	35	6.97%
	College/Undergraduate	347	69.12%
	Postgraduate and above	111	22.11%
Income (in RMB per month)	<5000	132	26.29%
	5000-10000	210	41.83%
	10000-20000	118	23.51%
	>20000	42	8.37%
Used LLMs?	Used	261	51.99%
Used LLMs?	Have not Used	241	48.01%
Used Telemedicine?	Used	321	63.94%
Used Telemedicine?	Have not Used	181	36.06%

Results

Reliability and validity analysis

The reliability and validity of the measurement model are presented in Table 3. All of the Cronbach’s α values are above the cutoff value of 0.7, which displays a desirable level of internal consistency.^45,46 In terms of composite reliability (CR), all constructs achieved values above the conventional cutoff of 0.7, confirming that the items consistently represent their underlying constructs.⁴⁷ The average variance extracted (AVE) values were also examined to assess convergent validity. According to Fornell and Larcker (1981), “average variance extracted may be a more conservative measure than composite reliability”, and “the researcher may conclude that the convergent validity of the construct is adequate, even though more than 50% of the variance is due to error”.^48,49 Therefore, although some AVE values from our research fell slightly below the 0.5 threshold, they are still considered acceptable if they have a CR greater than 0.7.^48,49 Overall, the aforementioned results indicate that the model has acceptable sufficient internal consistency, construct reliability, and convergent validity.

Table 3.

Convergent validity and reliability.

Constructs	Items	Significance estimation				Question reliability		CR	AVE	Cronbach’s α
Constructs	Items	UnStd.	S.E.	z-value	p	Std.	SMC	CR	AVE	Cronbach’s α
PE	PE1	1.000				0.776	0.602	0.780	0.541	0.771
	PE2	1.098	0.071	15.400	***	0.715	0.511
	PE3	0.845	0.055	15.381	***	0.714	0.510
EE	EE1	1.000				0.705	0.497	0.732	0.476	0.723
	EE2	1.124	0.087	12.882	***	0.723	0.523
	EE3	1.213	0.103	11.825	***	0.639	0.408
SI	SI1	1.000				0.759	0.576	0.794	0.561	0.790
	SI2	0.979	0.060	16.281	***	0.761	0.579
	SI3	0.780	0.050	15.558	***	0.727	0.529
TT	TT1	1.000				0.573	0.328	0.705	0.446	0.720
	TT2	1.111	0.112	9.931	***	0.652	0.425
	TT3	1.224	0.118	10.373	***	0.765	0.585
FC	FC1	1.000				0.705	0.497	0.730	0.473	0.729
	FC2	0.961	0.082	11.655	***	0.690	0.476
	FC3	0.955	0.083	11.455	***	0.667	0.445
BI	BI1	1.000				0.732	0.536	0.733	0.479	0.722
	BI2	1.117	0.086	12.932	***	0.620	0.384
	BI3	1.074	0.072	14.954	***	0.719	0.517

Note. ***p<0.001; Unstd.=Unstandardized Loading; S.E.=Standard Error; Std.=Standardized Loading; SMC = Squared Multiple Correlation.

The Heterotrait-Monotrait Ratio (HTMT) results in Table 4 are all below the strict threshold of 0.85, indicating that the latent variables are sufficiently distinct, confirming that the measurement model meets the requirements for discriminant validity.⁵⁰

Table 4.

HTMT analysis.

	PE	EE	SI	TT
PE
EE	0.512
SI	0.625	0.517
TT	0.442	0.214	0.342
FC	0.328	0.415	0.419	0.158

The model has a GFI of 0.908, an AGFI of 0.869, a CFI of 0.912, a TLI of 0.888, a SRMR of 0.058 and a RMSEA of 0.071. Overall, these Goodness of fit statistics indicate a satisfactory level of fit of the model.^51,52

Direct effects analysis

The structural model is validated through the estimation of path coefficients using Amos 28.0. The results displayed in Table 5 provide empirical support for four out of five hypothesized direct relationships. Specifically, performance expectancy (β = 0.228, p < 0.05), social influence (β = 0.518, p < 0.001), and facilitating conditions (β = 0.131, p < 0.05) have significant and positive effects on behavioral intention, confirming H1, H3, and H5. H4 is also confirmed as the measurement items under the construct labeled “Trust” are negatively worded and capture concerns and lack of confidence toward LLM-driven healthcare chatbots. For example, TT1 (“I am worried that my personal privacy may be leaked…”) reflects privacy concern, TT2 (“I am concerned about receiving inaccurate or outdated information…”) reflects accuracy distrust, and TT3 (“I do not trust the professional competence…”) expresses lack of professional confidence. Therefore, higher scores indicate lower levels of trust. The significant negative standardized coefficient (β = –0.148, p < 0.05) suggests that higher distrust is associated with lower adoption intention, which is theoretically consistent with H4 that higher trust is associated with higher adoption intention. In contrast, effort expectancy (β = 0.064, p = 0.411) shows no significant effects, leading to the rejection of H2. Collectively, the five predictors explained 36.3% of the variance in behavioral intention (R² = 0.363), indicating a moderate level of explanatory power in the proposed model.

Table 5.

Direct effects analysis.

Paths	UnStd.	S.E.	C.R.	P	Std.	R²
PE → BI	0.211	0.104	2.030	*	0.228	0.363
EE → BI	0.068	0.083	0.823	0.411	0.064
SI → BI	0.440	0.084	5.210	***	0.518
TT → BI	-0.135	0.058	-2.307	*	-0.148
FC → BI	0.120	0.055	2.195	*	0.131

Note. ***p<0.001; **p<0.01; *p<0.05; Unstd.=Unstandardized Estimate; S.E.=Standard Error; C.R.=Critical Ratio; Std.=Standardized Estimate.

Moderation effects analysis

The moderating role of previous experiences with telemedicine was examined using nested model comparisons in Amos with 2000 bootstrapped samples, which relies on Chi-Square difference tests to determine whether a moderator produces statistically significant differences in structural relationships.

Table 6 presents the results of the multi-group moderation analysis examining the moderating effect of users’ previous experience with telemedicine on the structural relationships within the model. Although the overall model comparison indicates a significant difference between groups (CMIN = 11.317, p < 0.05), individual path comparisons did not reveal statistically significant differences. This suggests that the group differences may be distributed across multiple parameters with small effects, rather than being driven by a single dominant pathway. Therefore, we conclude that users’ previous experience with telemedicine has very limited moderating effects on users’ behavioral intention to adopt LLM-driven healthcare chatbots, and H6 is therefore rejected.

Table 6.

Multi-group moderation analysis: Previous experience with telemedicine.

Paths	Without previous experience (N=181)		With previous experience (N=321)		Group difference
Paths	Std.	p	Std.	p	CMIN	p
PE → BI	0.180	0.223	0.356	0.214	0.576	0.448
EE → BI	0.099	0.337	-0.095	0.544	1.112	0.292
SI → BI	0.522	***	0.380	0.153	0.079	0.779
TT → BI	-0.213	0.079	-0.086	0.405	1.361	0.243
FC → BI	0.094	0.188	0.305	*	1.638	0.201
Overall					11.317	*

Table 7 presents the results of the multi-group moderation analysis examining the moderating effect of users’ previous experience with LLMs. The Chi-Square difference tests reveal significant differences between the two groups in two out of the five hypothesized paths (PE → BI and TT → BI), as well as in the overall model comparison. These findings suggest that prior experience with LLMs moderates the effects of two key predictors on users’ behavioral intention. Specifically, users without previous experience with LLMs are more likely to be impacted by the level of trust they have in these chatbots, while users with prior experience using LLMs care more about the performance of these chatbots. Therefore, we conclude that users’ previous experience with LLMs plays an important moderating role in users’ adoption of LLM-driven healthcare chatbots, supporting the validity of H7.

Table 7.

Multi-group Moderation Analysis: Previous experience with LLMs.

Paths	Without previous experience (N=241)		With previous experience (N=261)		Group difference
Paths	Std.	p	Std.	p	CMIN	p
PE → BI	-0.077	0.613	0.971	***	6.254	*
EE → BI	0.119	0.276	-0.238	0.370	2.194	0.139
SI → BI	0.746	***	0.282	*	0.885	0.347
TT → BI	-0.198	*	0.291	0.130	8.464	**
FC → BI	0.111	0.150	0.278	0.241	1.081	0.298
Overall					29.522	***

Note. ***p<0.001; **p<0.01; *p<0.05; Std.=Standardized Estimate.

Discussion

Key findings

In this study, we developed a context-specific research model with five contributing factors that would potentially influence users’ behavioral intention to adopt LLM-driven healthcare chatbots based on the existing literature. Our results revealed the relationships between each contributing factor and users’ behavioral intention. The findings also highlight the importance of the moderation effects of previous user experience with LLMs on the adoption of the LLM-driven healthcare chatbots.

First, social influence emerges as the strongest predictor affecting user adoption of LLM-driven healthcare chatbots, highlighting the critical role of perceived support or endorsement from peers and healthcare professionals. This finding indicates that the peers and healthcare professionals’ endorsement would enhance the users’ acceptance of LLM-driven healthcare chatbots. This finding also shows that the “words-of-mouth” effect is valid in this context. Additionally, performance expectancy positively influences the users’ behavior intention, suggesting that efficiency, convenience, and health outcomes are vital factors when users are making their adoption decisions. Facilitating conditions such as reliable internet connectivity and accessible technical infrastructure also significantly boost users’ intention to adopt LLM-driven healthcare chatbots by easing adoption barriers.

Second, our results also underscore the significant role of trust as a distinct factor shaping users’ acceptance of LLM-driven healthcare chatbots. Greater trust in these AI systems corresponds to higher adoption intentions, reinforcing the importance of credibility, security, and transparency in technological design. Users who perceive chatbots as trustworthy are less constrained by privacy risks, thus increasing their willingness to rely on these digital health solutions. This finding also underscores the belief that trust serves as a crucial psychological factor as far as healthcare services are concerned.

However, effort expectancy did not show significant effects in the adoption process, potentially reflecting users’ growing familiarity and comfort with new APPs. As individuals become more accustomed to install new APPs on their laptops or mobile phones, the perceived effort required to use a new APP, including healthcare chatbots, may diminish. This shift suggests that perceived ease-of-use is no longer a primary barrier to user adoption. Consequently, traditional constructs like effort expectancy may play a less prominent role in shaping users’ behavioral intentions in this context.

More importantly, regarding the moderating effects, previous experience with telemedicine services did not exhibit substantial effects on the user adoption of LLM-driven healthcare chatbots. One plausible explanation is that the form of telemedicine is inherently different from LLM-driven healthcare chatbots. In particular, telemedicine platforms often involve functionalities including appointment scheduling, retrieval of medical results, and remote consultations with human doctors, aligning closely with familiar internet services. Users still interact with human service providers when using these telemedicine services, which indicates that the experience with telemedicine cannot be directly applied in their interactions with LLM-driven healthcare chatbots. Therefore, users’ previous experience with telemedicine has a limited impact due to the minimal overlap with the unique cognitive or trust-related challenges posed by generative AI.

In contrast, previous experience specifically with LLMs significantly moderate adoption intentions. Users who had previous interactions with LLMs tend to better comprehend their underlying operational principles, thus mitigating concerns regarding AI-generated content’s reliability and accuracy. Their previous engagements with LLMs could reduce trust-related barriers, allowing these users to emphasize functional benefits rather than perceived risks. Conversely, users with limited or no generative AI experience face higher uncertainty, leading to increased skepticism. Instead of performance-based evaluations, their adoption decisions are therefore influenced by their level of trust in these LLM-driven healthcare chatbots, including their risk perceptions and ethical concerns.

Implications for research

This paper makes several important contributions to the technology acceptance literature by addressing the existing gaps. Our foremost contribution is that we have identified the moderating effects of users’ previous experience with LLMs and clarified its role in the user adoption of LLM-driven healthcare chatbots. This approach was often overlooked by previous research on technological and psychological features of adoption in technology acceptance literature.^10,12 Our findings highlighted that previous experience with LLMs could decrease the impact of trust in users’ adoption decisions and make users focus more on performance expectancy. To our knowledge, this study is one of the first attempts to incorporate previous experience as a moderator to investigate user adoption behavior in LLM-driven healthcare chatbots. By doing so, our research provided a foundation for future research examining experience-based differences in technology adoption.

Moreover, our research advances technology acceptance literature by investigating users’ adoption decisions from the extended UTAUT perspective. We have systematically explored the intricate mechanisms associated with users’ adoption of LLM-driven healthcare chatbots. Previous literature has focused on the influence of trust on user intent to use LLMs for healthcare purposes but have not holistically considered the role of other technological, societal, and psychological factors in user adoption of LLM-driven healthcare chatbots.¹² We address this gap in our study from a more comprehensive perspective in terms of the user adoption process.

Implications for practice

Our findings offer valuable guidance for developers and practitioners of healthcare chatbots. First, in order to enhance user acceptance, chatbot providers should emphasize building and maintaining user trust through transparent communication, robust security protocols, and clear disclosure of data privacy policies. Offering users insights into how their data is used and protected can further bolster confidence in the service.

Second, chatbot developers should prioritize user-centric functional performance by focusing on key aspects such as information reliability and diagnostic accuracy, which ensures that the tools provide precise and dependable results. Information reliability is fundamental, which helps the chatbots to generate accurate results, ultimately improve users’ adoption of the technology. Higher diagnostic accuracy can also promote trust from users, and produce new realiable information (e.g. feedback) to optimize diagnostics, which may enhance user engagements in the long run. Moreover, user-friendly interface is essential, as a clear navigation and interface design could greatly enhance the user experience.

Additionally, given the significant positive effect of social influence, healthcare organizations should take an active role in promoting LLM-driven services through endorsements by trusted professionals and well-designed informational campaigns. Adoption is likely to increase when these LLM-driven healthcare chatbots are explicitly recommended and supported by healthcare institutions and qualified medical practitioners, as such endorsements would enhance credibility and reduce user uncertainty in the technology.

Besides, our findings also carry important practical implications for policymakers and healthcare organizations seeking to promote the responsible adoption of LLM. Given that prior user experience is positively associated with trust, targeted strategies to facilitate safe initial exposure may be particularly effective among populations who have limited familiarity with such technologies before. For example, healthcare institutions could implement pilot programs or guided demonstration sessions that allow patients to interact with LLM-based tools in supervised settings, thereby reducing uncertainty and perceived risk. Transparent communication and legislation regarding system capabilities, limitations, data usage, and privacy protections is also essential to mitigate concerns about misuse of private data. In addition, policymakers may consider strengthening regulatory oversight and establishing clear accountability frameworks to enhance institutional trust. Finally, digital literacy initiatives tailored to vulnerable or less technologically experienced groups could further support informed and confident engagement with LLM-enabled healthcare services.

Limitations and future research

Despite its contributions, this study also faces certain limitations.

First, our research utilized survey data from a single country, China. Perceptions of technology are likely to be shaped by culturally embedded values, of which may differ substantially across national contexts. Therefore, the attitudinal patterns observed in this study should not be assumed to generalize automatically to other countries. While the effects of prior user experience may exhibit a certain degree of cross-contextual consistency due to common behavioral mechanisms, the magnitude and expression of these effects may still vary depending on local norms and institutional settings. Future research employing cross-national or comparative designs would be necessary to assess the external validity of our findings more rigorously.

Second, our sample predominantly consisted of younger individuals with generally high levels of education, potentially limiting the generalizability of the findings to older and less educated populations. Previous research suggests that individuals with lower level of education may hold more skeptical attitudes toward AI in healthcare. For instance, Erul et al. found that less-educated cancer patients exhibited significantly higher discomfort and concern regarding AI-assisted diagnosis, particularly regrading to the accuracy and data privacy.⁵³ This indicates that education level can shape perceptions of AI reliability and safety. Therefore, while our results reflect strong acceptance among a more educated and digitally literate population, users that are less-educated may have opposite attitudes. However, as our findings show, previous experience with AI or LLM technologies may mitigate such skepticism, suggesting that familiarity could play a key role in fostering broader acceptance across educational groups. Future studies should consider broader, more diverse populations to better capture the full spectrum of user acceptance.

Lastly, although previous experience with LLMs was identified as a significant moderator, the mechanisms through which user experiences are translated into trust remain underexplored. Future research should delve deeper into these psychological processes using quantitative or mixted research methods to enrich theoretical and practical insights.

Conclusion

This study examined the factors influencing users’ behavioral intention to adopt LLM-driven healthcare chatbots by developing an extended UTAUT framework that incorporates users’ previous experience with LLMs and telemedicine. The results reveal that social influence, performance expectancy, trust, and facilitating conditions significantly shape users’ adoption intentions, while effort expectancy plays a limited role. These findings suggest that as users become increasingly familiar with digital technologies, traditional usability concerns may diminish, giving greater importance to social and trust-related factors.

Moreover, prior experience with LLMs enhances user acceptance by reducing uncertainty and strengthening confidence in LLM-driven healthcare chatbots. This highlights the role of familiarity in promoting positive attitudes toward emerging healthcare technologies.

From a practical perspective, building user trust, ensuring data transparency, and promoting LLM-driven services through healthcare professionals and credible institutions can effectively increase users’ adoption. Moreover, fostering digital literacy through education and training may help mitigate the skepticism among users with limited technical experience.

Supplemental material

Supplemental Material - Public acceptance of LLM-driven healthcare chatbots in China: An empirical study

Supplemental Material for Public acceptance of LLM-driven healthcare chatbots in China: An empirical study by Ying Qian, Yuxin Fu, Yanting Chen, Ke Lu, Xin Zhao in DIGITAL HEALTH.

Footnotes

Acknowledgement

We sincerely appreciate the support and assistance provided by University of Shanghai for Science and Technology.

ORCID iDs

Ying Qian

Yuxin Fu

Xin Zhao

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Social Science Fund of China (Grant Number: 20CSH092 & 22BGL240).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Contributorship

Ying Qian conceptualized the research, completed the data analysis, and wrote the original manuscript. Yuxin Fu wrote the original manuscript and completed the risk-of-bias assessment and data extraction. Yanting Chen, Ke Lu and Xin Zhao revised and edited the manuscript. Xin Zhao is the Corresponding Author. All authors have read and approved the manuscript.

Guarantor

Ying Qian

Supplemental material

Supplemental material for this article is available online.

References

Topol

. High-performance medicine: the convergence of human and artificial intelligence. Nat Med 2019; 25: 44–56. https://doi.org/10.1038/s41591-018-0300-7

Jiang

Zhi

, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol 2017; 2: 230–243. https://doi.org/10.1136/svn-2017-000101

Lee

Bubeck

Petro

. Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine. N Engl J Med 2023; 388: 1233–1239. https://doi.org/10.1056/NEJMsr2214184

Singhal

Azizi

, et al. Large language models encode clinical knowledge. Nature 2023; 620: 172–180. https://doi.org/10.1038/s41586-023-06291-2

Esmaeilzadeh

. Use of AI-based tools for healthcare purposes: a survey study from consumers’ perspectives. BMC Med Inform Decis Mak 2020; 20: 170. https://doi.org/10.1186/s12911-020-01191-1

Wutz

Hermes

Winter

, et al. Factors Influencing the Acceptability, Acceptance, and Adoption of Conversational Agents in Health Care: Integrative Review. J Med Internet Res 2023; 25: e46548. https://doi.org/10.2196/46548

Laranjo

Dunn

Tong

, et al. Conversational agents in healthcare: a systematic review. J Am Med Inform Assoc 2018; 25: 1248–1258. https://doi.org/10.1093/jamia/ocy072

Bickmore

Trinh

Olafsson

, et al. Patient and Consumer Safety Risks When Using Conversational Assistants for Medical Information: An Observational Study of Siri, Alexa, and Google Assistant. J Med Internet Res 2018; 20: e11510. https://doi.org/10.2196/11510

Milne-Ives

De Cock

Lim

, et al. The Effectiveness of Artificial Intelligence Conversational Agents in Health Care: Systematic Review. J Med Internet Res 2020; 22: e20346. https://doi.org/10.2196/20346

10.

Balaskas

Tsiantos

Chatzifotiou

, et al. Determinants of ChatGPT Adoption Intention in Higher Education: Expanding on TAM with the Mediating Roles of Trust and Risk. Information 2025; 16: 82. https://doi.org/10.3390/info16020082

11.

Pang

Liew

Tan

S-M

, et al. Analyzing Use Intentions for Health-Diagnostic Chatbots: An Extended Technology Acceptance Model Approach. In: Proceedings of the 2024 The 6th World Symposium on Software Engineering (WSSE). ACM, pp. 208–217.

12.

Mesbah

Pumplun

. ‘Hello, I’m here to help you’ - Medical care where it is needed most: Seniors’ acceptance of health chatbots. In: Rowe

Amrani

Limayem

, et al. (eds). 28th european conference on information systems - liberty, equality, and fraternity in a digitizing world, ECIS 2020, Marrakech, Morocco, June 15-17, 2020, 2020. https://aisel.aisnet.org/ecis2020_rp/209

13.

Choudhury

Shahsavar

Shamszare

. User Intent to Use DeepSeek for Health Care Purposes and Their Trust in the Large Language Model: Multinational Survey Study. JMIR Hum Factors 2025; 12: e72867. https://doi.org/10.2196/72867

14.

Ramayah

Arshad

, et al. Decoding digital dependency: Flow experience and social belonging in short video addiction among middle-aged and elderly Chinese users. Telemat Inform 2025; 96: 102222. https://doi.org/10.1016/j.tele.2024.102222

15.

Park

Y-E

Tak

Kim

, et al. User Experience and Extended Technology Acceptance Model in Commercial Health Care App Usage Among Patients With Cancer: Mixed Methods Study. J Med Internet Res 2024; 26: e55176. https://doi.org/10.2196/55176

16.

Wang

Liu

, et al. Investigating the factors influencing users’ adoption of artificial intelligence health assistants based on an extended UTAUT model. Sci Rep 2025; 15: 18215. https://doi.org/10.1038/s41598-025-01897-0

17.

Goodfellow

Pouget-Abadie

Mirza

, et al. Generative adversarial networks. Commun ACM 2020; 63: 139–144. https://doi.org/10.1145/3422622

18.

Guo

. Generative artificial intelligence empowers educational reform: current status, issues, and prospects. Front Educ 2023; 8: 1183162. https://doi.org/10.3389/feduc.2023.1183162

19.

Zellers

Holtzman

Rashkin

, et al. Defending Against Neural Fake News. Epub ahead of print 2019. https://doi.org/10.48550/ARXIV.1905.12616

20.

Noy

Zhang

. Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence. SSRN Electron J. https://doi.org/10.2139/ssrn.4375283, Epub ahead of print 2023.

21.

Eletter

Yasmin

Elrefae

, et al. The Impact of AI and the Internet of Things on Healthcare Delivery. In: Yaseen

(ed). Cutting-Edge Business Technologies in the Big Data Era. Springer Nature Switzerland, pp. 396–402.

22.

Eriksen

Möller

Ryg

. Use of GPT-4 to Diagnose Complex Clinical Cases. NEJM AI; 1: 1-3. https://doi.org/10.1056/AIp2300031, Epub ahead of print January 2024.

23.

Fan

Tang

Chen

, et al. AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator. Epub ahead of print 2024. https://doi.org/10.48550/ARXIV.2402.09742

24.

Palepu

Schaekermann

, et al. Towards Conversational Diagnostic AI. Epub ahead of print 2024. https://doi.org/10.48550/ARXIV.2401.05654

25.

Yang

, et al. Using artificial intelligence to improve medical services in China. Ann Transl Med 2020; 8: 711. https://doi.org/10.21037/atm.2019.11.108

26.

Bekbolatova

Mayer

Ong

, et al. Transformative potential of AI in healthcare: Definitions, applications, and navigating the ethical landscape and public perspectives. Healthcare; 12: 125. https://doi.org/10.3390/healthcare12020125, Epub ahead of print 2024.

27.

Lizée

Beaucoté

P-A

Whitbeck

, et al. Conversational Medical AI: Ready for Practice. Epub ahead of print 2024. https://doi.org/10.48550/ARXIV.2411.12808

28.

Perez

Wisniewski

Ari

, et al. Investigation into Application of AI and Telemedicine in Rural Communities: A Systematic Literature Review. Healthcare 2025; 13: 324. https://doi.org/10.3390/healthcare13030324

29.

Wang

Zhang

, et al. “Brilliant AI Doctor” in Rural Clinics: Challenges in AI-Powered Clinical Decision Support System Deployment. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, pp. 1–18.

30.

Nov

Singh

Mann

. Putting ChatGPT’s Medical Advice to the (Turing) Test. Epub ahead of print 2023. https://doi.org/10.48550/ARXIV.2301.10035

31.

Zhan

Abdi

Seymour

, et al. Healthcare Voice AI Assistants: Factors Influencing Trust and Intention to Use. Epub ahead of print 2024. https://doi.org/10.48550/ARXIV.2401.04543

32.

Liou

JJH

. Exploring the Relationships among Factors Influencing Healthcare Chatbot Adoption. Sustainability 2024; 16: 5050. https://doi.org/10.3390/su16125050

33.

El-Sayed

BKM

El-Sayed

AAI

Alsenany

, et al. The role of artificial intelligence literacy and innovation mindset in shaping nursing students’ career and talent self-efficacy. Nurse Educ Pract 2025; 82: 104208. https://doi.org/10.1016/j.nepr.2024.104208

34.

Venkatesh

Thong

JYL

. Consumer Acceptance and Use of Information Technology: Extending the Unified Theory of Acceptance and Use of Technology. MIS Q 2012; 36: 157–178. https://doi.org/10.2307/41410412

35.

Venkatesh

Morris

Davis

, et al. User Acceptance of Information Technology: Toward a Unified View. MIS Q 2003; 27: 425–478. https://doi.org/10.2307/30036540

36.

Kelly

Kaye

S-A

Oviedo-Trespalacios

. What factors contribute to the acceptance of artificial intelligence? A systematic review. Telemat Inform 2023; 77: 101925. https://doi.org/10.1016/j.tele.2022.101925

37.

Nadarzynski

Miles

Cowie

, et al. Acceptability of artificial intelligence (AI)-led chatbot services in healthcare: A mixed-methods study. Digit Health 2019; 5: 2055207619871808. https://doi.org/10.1177/2055207619871808

38.

Moldt

J-A

Festl-Wietek

Madany Mamlouk

, et al. Chatbots for future docs: exploring medical students’ attitudes and knowledge towards artificial intelligence and medical chatbots. Med Educ Online 2023; 28: 2182659. https://doi.org/10.1080/10872981.2023.2182659

39.

Summers

Panesar

, et al. AI Hesitancy and Acceptability—Perceptions of AI Chatbots for Chronic Health Management and Long COVID Support: Survey Study. JMIR Hum Factors 2024; 11: e51086. https://doi.org/10.2196/51086

40.

Wang

Asan

Mansouri

. Perspectives of Patients With Chronic Diseases on Future Acceptance of AI–Based Home Care Systems: Cross-Sectional Web-Based Survey Study. JMIR Hum Factors 2023; 10: e49788. https://doi.org/10.2196/49788

41.

Palanica

Flaschner

Thommandram

, et al. Physicians’ Perceptions of Chatbots in Health Care: Cross-Sectional Web-Based Survey. J Med Internet Res 2019; 21: e12887. https://doi.org/10.2196/12887

42.

Liu

Peng

Song

, et al. Using AI chatbots to provide self-help depression interventions for university students: A randomized trial of effectiveness. Internet Interv 2022; 27: 100495. https://doi.org/10.1016/j.invent.2022.100495

43.

Ameen

Tarhini

Reppel

, et al. Customer experiences in the age of artificial intelligence. Comput Hum Behav 2021; 114: 106548. https://doi.org/10.1016/j.chb.2020.106548

44.

Narayanan

Vishwakarma

. GUARD-D-LLM: An LLM-Based Risk Assessment Engine for the Downstream uses of LLMs. Epub ahead of print 2024. https://doi.org/10.48550/ARXIV.2406.11851

45.

Fornell

Larcker

. Structural Equation Models with Unobservable Variables and Measurement Error: Algebra and Statistics. J Mark Res 1981; 18: 382–388. https://doi.org/10.1177/002224378101800313

46.

Tavakol

Dennick

. Making sense of Cronbach’s alpha. Int J Med Educ 2011; 2: 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd

47.

Henseler

Sarstedt

. Goodness-of-fit indices for partial least squares path modeling. Comput Stat 2013; 28: 565–580. https://doi.org/10.1007/s00180-012-0317-1

48.

Fornell

Larcker

. Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. J Mark Res 1981; 18: 39. https://doi.org/10.2307/3151312

49.

Lam

. Impact of competitiveness on salespeople’s commitment and performance. J Bus Res 2012; 65: 1328–1334. https://doi.org/10.1016/j.jbusres.2011.10.026

50.

Henseler

Ringle

Sarstedt

. A new criterion for assessing discriminant validity in variance-based structural equation modeling. J Acad Mark Sci 2015; 43: 115–135. https://doi.org/10.1007/s11747-014-0403-8

51.

Bentler

. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model Multidiscip J 1999; 6: 1–55. https://doi.org/10.1080/10705519909540118

52.

Hair

Hollingsworth

Randolph

, et al. An updated and expanded assessment of PLS-SEM in information systems research. Ind Manag Data Syst 2017; 117: 442–458. https://doi.org/10.1108/imds-04-2016-0130

53.

Erul

Aktekin

Danışman

, et al. Perceptions, Attitudes, and Concerns on Artificial Intelligence Applications in Patients with Cancer. Cancer Control J Moffitt Cancer Cent 2025; 32: 10732748251343245. https://doi.org/10.1177/10732748251343245