Sage Journals: Discover world-class research

Abstract

Objective

There is currently a lack of in-depth understanding of patient satisfaction and usage of internet hospitals in real-world scenarios. This study aims to comprehensively collect internet hospital Applications (APPs) in China, investigate their patient satisfaction, identify influencing factors, and understand the differences in the factor attributes.

Methods

This study was a cross-sectional observational study. We collected China's internet hospital APPs and their patient reviews from eight Chinese APP stores in October 2024. First, data preprocessing was conducted through deduplication, identification of bot accounts, sentiment analysis, and manual inspection. Second, based on the Two-Factor Theory, the Latent Dirichlet Allocation topic model and Tobit model were employed to identify influencing factors. Third, the Wald test was used to examine the effect differences of these factors. Finally, the factor attributes were identified using the Kano model.

Results

A total of 148 internet hospital APPs in China and their 121,458 patient reviews were included. The number of these APPs and users showed an initial increase followed by a decrease, peaking in 2020. For influencing factors, 12 factors significantly affected patient satisfaction and dissatisfaction. The Wald test results indicated that there is a significant difference in the influencing effect between patient satisfaction and dissatisfaction. Twelve factors were further categorized into ten charm factors and two essential factors.

Conclusion

In recent years, patient satisfaction and real-world usage effectiveness of internet hospital APPs have been suboptimal. Research has shown that influencing factors exhibit asymmetry and can be further classified into charm factors and essential factors. On the one hand, reliability and customer service are basic needs of patients. On the other hand, online diagnosis and treatment functions, doctor's professional level, easy to use, and compatibility can effectively improve patient compliance.

Keywords

Internet hospital patient satisfaction user-generated content Kano model influence factor

Introduction

The global imbalance and unequal distribution of medical resources have intensified, becoming a major public health issue. Internet hospitals can address these problems by expanding the supply of medical services, improving medical efficiency, and promoting the sharing of medical resources. The imbalance in the supply and demand of global medical services is severe.¹ This is mainly reflected in the shortage of medical resources, the irrational distribution of these resources,^2,3 and the increasing demand for medical services.⁴ On the supply side, there is a significant shortage of health workers globally. As of 2020, the world was short of 15 million healthcare workers,⁵ and at least 132 countries faced serious shortages of medical personnel.^6–9 On the demand side, the aging population has led to a continuous increase in the demand for medical services.¹⁰ Multiple studies have reported that the prevalence of diseases such as cancer, chronic conditions, and cardiovascular diseases will nearly double in the coming years.^11–15 However, a large number of patient medical needs remain unmet, with at least half of the global population unable to access basic medical services.¹⁶ The imbalance in medical service supply and demand has caused significant harm to both healthcare workers and patients. On the one hand, it has increased the workload of healthcare workers, leading to frequent occurrences of physician burnout and patient safety incidents.^17,18 On the other hand, it has increased the disease burden and potential mortality risk for patients.¹⁹ Internet hospitals are a unique telemedicine model established in China. They refer to remote medical platforms that use the internet and information technology to provide diagnosis and treatment services through one or more medical entities or their own medical resources.²⁰ Compared to the patient portal, it is an online medical service provider certified by policies, which can directly provide core clinical services such as online diagnosis, online follow-ups, and electronic prescription issuance. Based on their initiators, internet hospitals are categorized into hospital-led internet hospitals, enterprise-led internet hospitals, and government-led internet hospitals.²¹ Hospital-led internet hospitals are second-named internet hospitals built independently or in cooperation with third-party institutions by physical medical institutions, such as Peking Union Medical College Internet Hospital. Enterprise-led internet hospitals are established by companies based on physical medical institutions, such as WeDoctor Internet Hospital. Government-led internet hospitals are established by government departments, integrating regional medical resources to achieve online management of residents’ health, such as Nanjing Internet Hospital. These hospitals can address the imbalance in medical resource supply and demand by increasing the accessibility and convenience of medical resources and expanding the supply of medical services.²² They can also promote the rational distribution of high-quality medical resources by advancing the medical consortium system.²³

As a novel telemedicine technology, internet hospitals have developed rapidly in China, and numerous APPs have been launched and put into use. However, patient satisfaction and utilization rates remain relatively low. The Chinese government has continuously increased policy support for internet hospitals and telemedicine. Specific policies are detailed in Table S1 in Multimedia Appendix 1 in the supplemental material. Moreover, the COVID-19 pandemic has accelerated the construction of internet hospitals in China,²⁴ and online consultation platforms have solved the problem of inaccessible offline medical services. As of February 2024, more than 2700 internet hospitals had been approved and set up, with over 26 million people receiving internet diagnosis and treatment services.^25,26 The user scale of internet medical services reached 365 million.²⁷ Internet hospitals have diverse platforms and carriers, including APPs, online websites, WeChat public accounts, WeChat mini-programs, Alipay mini-programs, and Douyin mini-programs. Among them, APPs have been widely used (among the top 200 internet hospitals in China, 121 [60.50%] use APPs as the carrier).^28–30 Meanwhile, most enterprise-led internet hospitals have higher daily active users on APPs than on mini-programs (e.g., the daily active users of Good Doctor Online's APPs are approximately 2.1 million, while the WeChat mini programs are approximately 0.6 million). However, most Chinese internet hospitals were developed without incorporating patient feedback, and they have not undergone official qualification certification before going online.³¹ This impedes accurate assessment of patient satisfaction, which may hinder effective patient use. On the one hand, a small-sample questionnaire survey showed that patient satisfaction with internet hospitals in China is relatively low (68.10%).³² On the other hand, the actual usage of internet hospitals is poor. Ninety percent of internet hospitals have a few active users,³³ and over 80% of patients stop using internet hospitals due to various issues.^34,35 Existing studies have shown that improving patient satisfaction can significantly enhance patient usage behavior and continuance intention.^36–38 Therefore, exploring patient satisfaction based on real patient feedback, identifying influencing factors, and understanding the differences in the factor attributes are of great significance for more accurately assessing the development effectiveness of internet hospitals in China, improving patient satisfaction, and improving the effectiveness of internet hospitals.

Existing studies have preliminarily explored the factors influencing patients’ acceptance and satisfaction of internet hospital using methods such as literature review, interviews, and questionnaire surveys, with most focusing on the analysis of small sample data. For example, Liu et al.³⁹ conducted a cross-sectional survey of 1653 participants and found that the willingness to visit internet hospitals was high and influenced by the internet hospital type. Zhiyang Jin⁴⁰ collected 407 questionnaires and used structural equation modeling to show that perceived interactivity, perceived value, perceived quality, perceived risk, and interface design all significantly affect patient satisfaction with internet hospitals. In addition, a few studies have used online medical platform data to explore the impact of specific factors on patient satisfaction with internet hospitals. For example, Wu et al.⁴¹ extracted different physician communication strategies from 20,000 records of online healthcare services and found that response load, detailed replies, and emotional comfort all have positive effects on patient satisfaction with internet hospitals.

However, previous studies have limitations such as small sample sizes, specific user groups, incomplete summarization of influencing factors, and superficial analysis of influence effects, resulting in high research costs and potential bias. Specifically, for data sources, most studies conducted offline surveys through interviews and questionnaires, with survey subjects typically limited to specific user groups and small sample datasets, making the results prone to sample bias. Additionally, post hoc surveys of subjects are susceptible to adverse effects from observer bias and recall bias. For example, Han et al.³² focused on the satisfaction of outpatients. By analyzing 1481 questionnaires, they found that familiarity and willingness to use were the main factors affecting outpatient satisfaction. For the summarization of influencing factors, potential factors derived through literature review and theoretical deduction may not be comprehensive and may not represent the true views of patients. It is necessary to obtain the factors that patients truly care about from their feedback. For example, Yiting Qian et al.⁴² found that social presence has a positive impact on satisfaction with online medical services based on the social presence theory. For influence effect analysis, there is a lack of in-depth analysis of influence effects, and insufficient exploration of the patient satisfaction asymmetry and the influencing factors’ attributes. For example, Jing You et al.⁴³ found that service content, convenience, and barriers are all key factors affecting residents’ satisfaction with internet hospitals through questionnaire surveys, but they did not conduct further analysis on the differences in influence effects of these factors. The Two-Factor Theory⁴⁴ and Kano model⁴⁵ indicate that factors influencing user satisfaction are asymmetric, and these factors have different attributes.

Therefore, this study aims to comprehensively collect internet hospital APPs in China, investigate their patient satisfaction, identify influencing factors, and understand the differences in the factor attributes. On the one hand, this can help the Chinese government, enterprises, and hospitals obtain a comprehensive understanding of patient satisfaction and real-world usage of internet hospital APPs, providing a data foundation for further policy-making regarding internet hospitals. On the other hand, it will assist relevant stakeholders in gaining in-depth insights into the factors influencing patient satisfaction with internet hospitals. They can then develop more precise and effective improvement measures based on the differences in the utility and attributes of these factors, thereby enhancing patient satisfaction and real-world usage effectiveness of internet hospital APPs in China.

Aim of the study

This study investigated the user satisfaction of internet hospital APPs in China, and explored its influencing factors.

Research questions

What are the prevailing level of patient satisfaction and the dominant perceptions expressed by patients regarding Chinese internet hospital APPs?

What key factors significantly influence patient satisfaction and dissatisfaction with Chinese internet hospital APPs?

Do these factors have asymmetric effects, and how to classify their attributes?

Core concepts and theoretical background

For core concepts, we define internet hospitals, outline their key characteristics, healthcare functions, and development history. According to official Chinese policy documents,⁴⁶ the internet hospital is an integrated service platform anchored to a physical medical institution. It primarily offers online follow-up consultations and routine medical advice, integrating diagnosis, prescription issuance, payment processing, and medication delivery. By merging traditional hospital services with internet technology, it connects healthcare demand (patients), service providers (medical professionals), payers, and pharmaceutical suppliers. This integration delivers layered, coordinated, joint, continuous, and comprehensive healthcare services to patients. Internet hospitals represent a specific product of Internet-based care, emerging as a novel telemedicine model pioneered through collaboration between Chinese hospitals and the government. It is a complex of patient portals and telemedicine with Chinese characteristics, and currently exists only in China. Internet hospitals combine online and offline access to medical institutions to provide a variety of telemedicine services to patients, including convenience services (online appointments, checking test results), online medical services (online consultation, electronic prescriptions), and related support (chronic disease management, patient follow-up, health education).²¹ The medical insurance coverage of internet hospitals is the same as that of offline hospitals. China's first internet hospital was established in Guangdong Province in 2014.⁴⁷ Subsequently, China began promoting this novel online healthcare model, actively developing internet-based medical services and establishing medical network information platforms. In July 2015, the Chinese government issued the “Opinions on Actively Promoting the ‘Internet Plus’ Initiative,” explicitly encouraging the development of internet hospitals and identifying them as a key component of the “Healthy China” strategy.⁴⁸ Internet hospitals began to emerge gradually across different regions of China in 2016, with 25 established that year.⁴⁹ By the end of 2018, the Chinese government introduced a series of regulations including the “Opinions on Promoting ‘Internet Plus’ Healthcare Development,” the “Administrative Regulations for Internet Diagnosis and Treatment (Trial),” the “Administrative Regulations for Internet Hospitals (Trial),” and the “Management Specifications for Telemedicine Services (Trial).” These provided specific operational and supervisory requirements for diagnosis and treatment activities within internet hospitals, marking the beginning of their standardized development.⁴⁶ Subsequently, the COVID-19 outbreak significantly accelerated the rapid expansion of internet hospitals. Their number increased rapidly from 158 in 2019 to 1700 in 2022.⁵⁰ As of 2024, China has approved and established over 2700 internet hospitals, with more than 365 million Chinese residents having received online diagnosis and treatment services through these platforms.⁵¹ On the one hand, internet hospitals enhance the efficient allocation of medical resources and significantly improve healthcare efficiency for patients. They demonstrate an online appointment booking rate exceeding 80%, a 60% reduction in average waiting time, an on-time follow-up consultation rate reaching 83%, and a treatment effectiveness rate surpassing 70%.^52–54 On the other hand, they reduce barriers to accessing quality medical resources in remote areas, promoting equitable distribution of healthcare resources.⁵⁵

For theoretical basis, this study adopted the Two-Factor Theory and the Kano model. Two-Factor Theory indicates that factors influencing user satisfaction are asymmetric and can be divided into motivators that enhance user satisfaction and hygiene factors that reduce user dissatisfaction. Noriaki Kano developed the Kano model, which further categorizes factors into five types (charm factors, essential factors, expected factors, indifference factors, and reverse factors) based on their “nonlinear impact” on satisfaction. Patient satisfaction in digital health services inherently exhibits “asymmetric drivers”—a phenomenon empirically observed in prior mobile health studies,^38,56,57 but underexplored in China's internet hospital ecosystem. If the asymmetry of user satisfaction is ignored, it may lead to significant deviations in the analysis results.⁵⁸ In order to reduce research bias and improve the credibility and interpretability of the results, we applied the Two-Factor Theory and Kano model to the user satisfaction of internet hospitals, deeply analyze the asymmetry of influencing factors and classify the attribute of factors. Specifically, we applied Two-Factor Theory and Kano model to research through the following steps. Firstly, we used Latent Dirichlet Allocation (LDA) model to mine topics from user reviews, and Positive Deviation/Negative Deviation (PD/ND) were conceptualized as proxies for satisfaction intensity and dissatisfaction intensity. Secondly, we used Tobit models to conduct regression analysis on topics and PD/ND, exploring the factors that affect user satisfaction and dissatisfaction. Then, we validated the asymmetric impact of these factors on user satisfaction and dissatisfaction through Wald test. Finally, we used the Kano Model to categorize the influencing factors into five attribute categories.

Methods

Ethical approval

This study was a cross-sectional observational analysis that exclusively examined publicly available, anonymized user-generated content from APP stores. As such, it did not involve human/animal experimentation or access to private health data. Per institutional policies and international guidelines governing non-interventional research using pre-existing public datasets, formal ethics approval or waiver documentation was not required.

Analytical framework

Referring to the user-generated content-driven methodological framework proposed by Tong Wang,⁵⁹ this study constructed an analytical framework for patient satisfaction and its influencing factors in internet hospital APPs, as shown in Figure 1. Compared with the original version, this framework focuses on patient satisfaction management in internet hospitals and extends the relationship between influencing factors and clinical practice. The analytical framework is divided into five parts:

Data gathering and processing. We systematically searched for and screened internet hospital APPs in China, and preprocessed patient ratings and reviews.

Identifying topics based on topic modeling. We used the LDA topic model to identify key topics from patient reviews.

Exploring influencing factors. We employed the Tobit model to uncover potential factors influencing patient satisfaction and dissatisfaction.

Impact effects analysis and attribute classification. We used the Wald test to verify effect differences of influencing factors and categorize them using the Kano model.

Suggestions for improvement. We proposed improvement suggestions from three aspects: clinical practice, policy recommendations, and technological optimization.

Figure 1.

Analytical framework for patient satisfaction with internet hospital APPs.

Data gathering and filtering

Data gathering

This study systematically acquired internet hospital APPs in China through two approaches. First, we searched for internet hospital APPs in eight Chinese APP stores using a set of keywords. Based on active user numbers, we selected eight Chinese APP stores from the iOS and Android platforms for a comprehensive review, including the China Apple App Store, Huawei App Store, Xiaomi App Store, OPPO App Store, VIVO App Store, Baidu App Store, 360 App Store, and Application Treasure App Store. In October 2024, we used keywords such as “internet hospital,” “online hospital,” and “cloud hospital”⁶⁰ to search these eight APP stores and identified 1058 internet hospital APPs, yielding 844 unique apps after deduplication (detailed search results are shown in Table S2 in Multimedia Appendix 1 in the supplemental material). Second, we compiled a list of major public hospitals in China and reviewed their internet hospital APPs to complement the first approach. We selected mainstream hospitals based on three dimensions: internet hospital construction level, annual hospital income, and comprehensive hospital strength. We compiled the top 100 internet hospitals,⁶¹ the top 100 hospitals by annual income,⁶² and the top 100 hospitals in the Fudan hospital comprehensive ranking (China's authoritative hospital ranking)⁶³ to form a collection of mainstream hospitals. We searched for internet hospital APPs of these representative hospitals on search engines such as Baidu, Bing, and Google, as well as in the eight APP stores. We found that the majority of APPs (92.6%, 50/54) had already been acquired through the first approach, while the remaining minority (7.4%, 4/54) were further supplemented.

Data filtering

To ensure the scientificity and accuracy of the data processing and analysis process, this study strictly adhered to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist.⁶⁴ We developed detailed inclusion and exclusion criteria based on prior research⁶⁰ (see Table S3 in Multimedia Appendix 1 in the supplemental material). Two researchers in the medical informatics field (YFH and QCF) independently screened the APPs according to these criteria. Both researchers had undergone unified training and achieved great consistency (kappa = 0.93). Disagreements between them were resolved by arbitration from another hospital expert (LY). The screening flowchart of this study is shown in Figure 2, with a total of 148 internet hospital APPs in China ultimately included.

Figure 2.

Flowchart of the internet hospital APPs screening process.

Data preprocessing

We obtained all ratings and reviews of the 148 internet hospital APPs from eight APP stores through the Qimai mobile application data analysis platform.⁶⁵ As of October 2024, a total of 310,590 patient reviews were collected, including 292,007 from enterprise-led internet hospitals, 12,819 from hospital-led internet hospitals, and 5764 from government-led internet hospitals. To eliminate false and nonsensical patient reviews, the following data preprocessing steps were conducted. First, we removed 156 reviews containing only ratings. Second, we deleted 39,695 duplicate reviews. Third, we used the tweetbotornot package in R ⁶⁶ to remove 113,909 reviews posted by bot accounts. Fourth, we removed 14,367 reviews with blank values, non-Chinese text, garbled characters, or nonsensical content. Fifth, we implemented the SKEP sentiment analysis algorithm⁶⁷ to classify sentiment polarity of each patient review, categorizing them into negative, neutral, and positive, and excluded 11,018 reviews with inconsistent sentiment polarity and patient ratings (removed reviews with ratings of 1‒2 but non-negative sentiment polarity; removed reviews with a rating of 3 but non-neutral sentiment polarity; removed reviews with ratings of 4‒5 but non-positive sentiment polarity.⁶⁸ See Figure S1 in Multimedia Appendix 1 in the supplemental material for details of the removal results). Finally, we standardized different expressions with the same concept, with detailed information in Table S4 in Multimedia Appendix 1 in the supplemental material. After data preprocessing, a total of 131,445 reviews were included in the subsequent analysis (enterprise-led internet hospitals: 117,828; hospital-led internet hospitals: 9028; government-led internet hospitals: 4589).

Additionally, this study also preprocessed patient ratings to eliminate the impact of different evaluation criteria among various APPs. We calculated the difference between individual patient ratings and the overall APP store ratings. This difference indicates the deviation of individual patient ratings from the average rating. Based on these calculations, we defined two variables: PD and ND.⁶⁹ For ease of calculation, we took the absolute value of ND. A higher ND value indicates a greater degree of dissatisfaction. Both PD and ND have a value range of [0, 4].

Identifying topics based on topic modeling

We used natural language processing (NLP) techniques and the Latent Dirichlet Allocation (LDA) topic model⁷⁰ to extract the topics from patient reviews of internet hospital APPs. On the one hand, we extracted keywords through the following NLP process. First, we tokenized the Chinese reviews using the Jieba package in Python.⁷¹ Second, we removed stop words (including numbers, punctuation, emojis, and blank values) from the tokenized reviews using a combined stop word list from Baidu, Harbin Institute of Technology, Sichuan University Machine Intelligence Laboratory, and Chinese stop words. Third, we deleted blank reviews after stop words removal. Finally, we included 121,458 reviews in the LDA topic modeling. On the other hand, we performed topic clustering on the above reviews using the LDA topic model. First, we determined the optimal number of topics to be 12 based on the perplexity curve and the clustering visualization results using PyLDAvis (Figure 3). Second, we set the hyperparameter $α$ to 0.1 based on the data characteristics and hyperparameter $β$ to 0.1 based on the number of topics and text samples. Third, we constructed the LDA topic model for the reviews using the Gensim package in Python.⁷² Two researchers (YFH and QCF) independently summarized and named the topics for each keyword set in the topic model, with disagreements resolved by a third expert (LY). Finally, we determined the best topic for each review based on the distribution probabilities of all topics generated by the LDA model for each review.

Figure 3.

Perplexity curve and clustering effect.

Data analysis

Exploring influencing factors

According to two-factor theory, factors influencing patient satisfaction and dissatisfaction are asymmetric. To further explore the factors affecting patient satisfaction and dissatisfaction with internet hospital APPs, this study selected PD and ND as dependent variables to reflect patient satisfaction and dissatisfaction. Meanwhile, clustering topics were chosen as independent variables, with the probability distribution of each topic in individual reviews serving as the values of the independent variables to represent patient experiences and needs. Additionally, to standardize the data scale, the probability distributions of the independent variables were normalized using formula 2.1. After processing, the range of values for the independent variables was [0, 1].

y_{i} = \frac{x_{i} - min_{1 \leq j \leq n} {x_{j}}}{max_{1 \leq j \leq n} {x_{j}} - min_{1 \leq j \leq n} {x_{j}}}

(2.1)

The dependent variables PD/ND are left-censored data, which can only take values of 0 or positive numbers. Therefore, this study employed the Tobit model, which effectively handles left-censored variables, to identify factors influencing patient satisfaction and dissatisfaction. The specific model definitions are presented in equations 2.2 and 2.3. In these equations,

β_{k}

represents the correlation coefficient between k topic

χ_{k i}

in review i and patient satisfaction (

P D_{i}

) or dissatisfaction (

N D_{i}

). K is the number of topics included in the model, and

δ_{i}

is the error term. To ensure no multicollinearity in the statistical analysis, the variance inflation factor (VIF) was calculated for both the positive and negative deviation models.

P D_{i} = \sum_{K = 1}^{K} β_{P D, k} χ_{k i} + δ_{P D, i}

(2.2)

N D_{i} = \sum_{K = 1}^{K} β_{N D, k} χ_{k i} + δ_{N D, i}

(2.3)

Asymmetric analysis of impact effects

To further explore the asymmetry of factors influencing patient satisfaction and dissatisfaction with internet hospital APPs, this study used the Wald test to examine the differences in the absolute values of effect coefficients for the same variables in the positive and negative deviation models. If the absolute values of the effect coefficients for the same influencing factor show a significant statistical difference between the positive and negative deviation models, it indicates that the factor has an asymmetric impact on patient satisfaction and dissatisfaction. Both the Tobit model construction and the Wald test were implemented using Stata 16.0, with the significance level for statistical testing set at a two-sided p < .05.

Attribute classification of influencing factors

This study draws on the Kano model to construct a discriminant model for categorizing the factor attributes. The discriminant model is based on the significance of positive and negative deviations, the signs of the deviation coefficients, the significance of differences between these coefficients, and the distribution of positive and negative reviews. The model includes seven criteria: $T_{i}, P S_{i}, N S_{i}, P C_{i}, N C_{i}, C D_{i}, N T_{i}$ . The codes, names, specific meanings, and calculation methods for these criteria are detailed in Table S5 in Multimedia Appendix 1 in the supplemental material.

Based on these seven criteria and referencing prior studies,⁷³ we developed specific discriminant rules (see Appendix 1 “Attribute classification of the Kano model” in the supplemental material) and a decision model diagram (Figure S2 in Multimedia Appendix 1 in the supplemental material) for attribute classification. According to these rules and the decision model diagram, the influencing factors are categorized into five attribute types: charm factors, essential factors, expected factors, indifference factors, and reverse factors.

Results

Number of internet hospital APPs and patient reviews

This study identified a total of 1058 internet hospital APPs in eight Chinese APP stores and obtained 310,590 reviews. After screening and data preprocessing, 148 internet hospital APPs with reviews were ultimately included (hospital-led internet hospitals: 83 [56.08%, 83/148], enterprise-led internet hospitals: 50 [33.78%, 50/148], government-led internet hospitals: 15 [10.14%, 15/148]) and 121,458 reviews (hospital-led internet hospitals: 8898 [7.33%, 8898/121,458], enterprise-led internet hospitals: 108,092 [89.00%, 108,092/121,458], government-led internet hospitals: 4468 [3.67%, 4468/121,458]). The number of internet hospital APPs and reviews is detailed in Table 1 (if the same APP is listed in different APP stores, it is considered a separate analytical object).

Table 1.

The number of internet hospital APPs and reviews of each APP store.

APP store	Number of APPs	Number of reviews
China Apple App Store	145	48,616
Huawei App Store	84	10,444
Xiaomi App Store	75	8973
OPPO App Store	78	20,933
VIVO App store	77	21,102
Baidu App store	0	0
360 App store	28	9353
Application treasure App store	40	2037

We created a trend chart showing the changes in the number of internet hospital APPs and reviews, as shown in Figure 4. The results revealed that, on the one hand, internet hospital APPs were first listed in 2011, and their annual listing numbers showed an initial increase followed by a decrease, peaking in 2020 (n = 37), with minimal new listings in recent years (4 in 2021, 4 in 2022, and 2 in 2023). On the other hand, the trend in the number of reviews was similar to that of the APPs, showing an initial increase followed by a decrease, also peaking in 2020 (n = 27,172). Additionally, among these reviews, a total of 86,862 reviews had a rating of four or five stars, accounting for 71.52% (86,862/121,458).

Figure 4.

Trend in the number of internet hospital APPs and reviews.

Topics distribution in reviews of internet hospital APPs

After LDA topic modeling, reviews of internet hospital APPs were clustered into 12 topics (the topic summarization results were recognized by hospital experts), as shown in Table 2. Examples of reviews for each topic are provided in Table S6 in Multimedia Appendix 1 in the supplemental material.

Table 2.

Topics and keywords of reviews formulated by Latent Dirichlet Allocation topic modeling.

Topics	Keywords	Reviews, n (%)
Topic 1: online consultation function	Consultation, online, doctor, answer, communication, voice, treatment	7066 (5.8)
Topic 2: online registration and appointment function	Appointment, registration, hospital, online, expert, register, call number	11,855 (9.8)
Topic 3: easy to use	Simple, operate, convenient, find, clear at a glance, user-friendly, easy to use	10,038 (8.3)
Topic 4: user login	Registration, login, phone number, verification code, password, loading, log in	9872 (8.1)
Topic 5: reliability	Flashback, unable to open, unusable, crashing, stuck, optimized, repaired	9008 (7.4)
Topic 6: online report query function	Search, unable to find, report, online, physical examination, website, inspection	7260 (6.0)
Topic 7: electronic health record management function	Medical records, data, management, recording, medical treatment, medical condition, examination	6172 (5.1)
Topic 8: customer service	Customer service, phone, feedback, maintenance, cancellation, testing, prompt	12,014 (9.9)
Topic 9: health management function	Patient, reminder, service, SMS, monitoring, follow up, family	10,968 (9.0)
Topic 10: compatibility	Mobile phone, version, unable to access, installation, connection, latest, update	5024 (4.1)
Topic 11: doctor's professional level	Doctor, professional, technical, patient, responsive, medical staff, responsible	23,841 (19.6)
Topic 12: fee	Payment, cost, medical insurance, refund, expense, fee, reimbursement	8340 (6.9)

The 12 topics are: “online consultation function,” “online registration and appointment function,” “easy to use,” “user login,” “reliability,” “online report query function,” “electronic health record management function,” “customer service,” “health management function,” “compatibility,” “doctor's professional level,” and “fee.” These topics can be broadly categorized into two categories: factors related to the functionality and user interface design of the APPs, and factors related to the quality of online care. Specifically, the former category includes “Reliability, Compatibility, User Login, Easy to Use, Online Report Query, Health Management, and electronic health record management.” The latter category includes “Doctor's Professional Level, Online Consultation, Online Registration and Appointment, Customer Service, and Fee.” Furthermore, these categories can be classified into seven types: usability, easy to use, online diagnosis and treatment functions, health management functions, doctor's professional level, after-sales service, and cost. Specifically, usability includes “user login,” “reliability,” and “compatibility.” User login refers to issues encountered during the APP login process, such as account registration, password settings, verification code delivery, and interface loading. These issues can affect patients’ normal login and use of the APPs. Reliability refers to the stability of the APPs’ operation, such as problems with opening, frequent crashes, system crashes, freezing, and system optimization. Compatibility refers to the match between the APPs and mobile devices or operating systems. Easy to use refers to the convenience of the APPs’ functional design. For example, simple operations, clear interfaces, and user-friendly functions. Online diagnosis and treatment functions include “online consultation function” and “online registration and appointment function.” Online consultation function refers to the online interaction between doctors and patients. Patients can inquire about their conditions through the APPs, and doctors can provide medical advice based on the descriptions and medical records. Online registration and appointment function refers to patients’ appointment services, including online registration and booking of doctors. Health management functions include “online report query function,” “electronic health record management function,” and “health management function.” Online report query function allows patients to query medical information such as examination reports and medical advice through the APPs. Electronic health record management function helps patients manage their medical records, examination reports, and prescription information. Health management function involves reminding patients of follow-up appointments and monitoring their health indicators through messages and alerts. The three categories of doctor's professional level, after-sales service, and cost correspond to the topics of “doctor's professional level,” “customer service,” and “fee,” respectively. Doctor's professional level refers to the professional competence of online doctors, including their medical knowledge, service attitude, and response speed. Customer service indicates the ability of customer support to resolve issues. Fee represents the monetary cost incurred by patients for using online medical services.

Factors influencing patient satisfaction with internet hospital APPs

This study selected PD/ND, representing patient satisfaction/dissatisfaction, as dependent variables. Twelve clustered topics representing patient views were chosen as independent variables. Tobit models for patient satisfaction and dissatisfaction were constructed to explore factors influencing patient satisfaction and their effects. To ensure the validity and accuracy of the regression models, we calculated the VIF for both satisfaction and dissatisfaction models to test multicollinearity among independent variables. Results showed that VIF values for all independent variables in both models were below 5, indicating no multicollinearity issues.⁷⁴ Detailed results are shown in Tables S7 and S8 in Multimedia Appendix 1 in the supplemental material.

Results of the Tobit models for patient satisfaction and dissatisfaction are shown in Table 3. Model 1 presents the effects of different factors on PD. Results indicate that all 12 factors significantly affect PD. Among them, online consultation function, online registration and appointment function, easy to use, user login, online report query function, electronic health record management function, doctor's professional level, and fee have positive effects, with easy to use ( $β$ = 0.341, P < .001) having the largest positive effect. Reliability, customer service, health management function, and compatibility have negative effects, with customer service ( $β$ = −0.680, P < .001) having the largest negative effect. Model 2 shows the effects of different factors on ND. Results indicate that all 12 factors significantly affect ND. The factor with the largest positive effect is customer service ( $β$ = 2.677, P < .001), while the factors with the largest negative effects are easy to use ( $β$ = −1.753, P < .001), and doctor's professional level ( $β$ = −1.753, P < .001).

Table 3.

Determinant factors for rating deviations.

	Model 1^a			Model 2^b
Influencing factor	β (95% CI)	SE	P value	β (95% CI)	SE	P value
Online consultation function	0.143 (0.125, 0.160)	0.009	<.001	−0.767 (−0.834, −0.700)	0.034	<.001
Online registration and appointment function	0.293 (0.278, 0.308)	0.008	<.001	−0.494 (−0.545, −0.443)	0.026	<.001
Easy to use	0.341 (0.325, 0.357)	0.008	<.001	−1.753 (−1.817, −1.689)	0.032	<.001
User login	0.102 (0.086, 0.119)	0.008	<.001	−0.193 (−0.246, −0.140)	0.027	<.001
Reliability	−0.481 (−0.498, −0.464)	0.009	<.001	1.680 (1.631, 1.728)	0.025	<.001
Online report query function	0.249 (0.232, 0.266)	0.009	<.001	−1.647 (−1.720, −1.575)	0.037	<.001
Electronic health record management function	0.101 (0.083, 0.118)	0.009	<.001	−0.605 (−0.669, −0.541)	0.033	<.001
Customer service	−0.680 (−0.697, −0.664)	0.008	<.001	2.677 (2.631, 2.723)	0.023	<.001
Health management function	−0.027 (−0.042, −0.012)	0.008	.001	0.955 (0.906, 1.003)	0.025	<.001
Compatibility	−0.093 (−0.112, −0.073)	0.010	<.001	0.844 (0.785, 0.903)	0.030	<.001
Doctor's professional level	0.277 (0.264, 0.290)	0.007	<.001	−1.753 (−1.804, −1.702)	0.026	<.001
Fee	0.211 (0.194, 0.229)	0.009	<.001	−0.586 (−0.648, −0.523)	0.032	<.001

Positive rating deviations. The maximum likelihood estimate of model 1 was − 134,706.45.

Negative rating deviations. The maximum likelihood estimate of model 2 was − 112,484.25.

β: Coefficient. SE: Standard error.

Differences in the effects of influencing factors and their attribute classification

Asymmetry of influence effects

To further explore the asymmetry of influence effects of each factor on patient satisfaction and dissatisfaction, we used the Wald test to examine the differences in the absolute values of parameters between Models 1 and 2. The results are shown in Table 4. The table shows that the absolute values of the effect coefficients of the 12 influencing factors on PD and ND are significantly different. Specifically, online consultation function, online registration and appointment function, easy to use, user login, online report query function, electronic health record management function, doctor's professional level, and fee have significant positive effects on PD and significant negative effects on ND, with significant differences in the absolute values of effect coefficients in the two models. Reliability, customer service, health management function, and compatibility have significant negative effects on PD and significant positive effects on ND, with significant differences in the absolute values of effect coefficients in the two models. This indicates that the impact of all factors on patient satisfaction with internet hospital APPs is asymmetric.

Table 4.

Comparison between the Wald test parameters of models 1 and 2.

Influencing factor	Wald test	P value	n in PD (%)	n in ND (%)
Online consultation function	301.85	<.001	5571 (82.12)	1213 (17.88)
Online registration and appointment function	73.49	<.001	8748 (77.53)	2536 (22.47)
Easy to use	2075.13	<.001	8465 (90.36)	903 (9.64)
User login	15.93	<.001	6256 (65.77)	3256 (34.23)
Reliability	4059.02	<.001	2647 (30.00)	6175 (70.00)
Online report query function	1332.28	<.001	6173 (90.09)	679 (9.91)
Electronic health record management function	252.90	<.001	4777 (81.73)	1068 (18.27)
Customer service	12,883.52	<.001	2470 (20.93)	9332 (79.07)
Health management function	1959.47	<.001	5950 (56.17)	4642 (43.83)
Compatibility	896.64	<.001	2639 (55.31)	2132 (44.69)
Doctor's professional level	3869.06	<.001	21,145 (91.52)	1959 (8.48)
Fee	161.88	<.001	6095 (77.79)	1740 (22.21)

Note. PD: positive deviation. ND: negative deviation.

Different attributes of influencing factors

This study calculated and summarized the following six indicators for all influencing factors using the Tobit model and Wald test: positive significance (PS), negative significance (NS), positive deviation coefficient (PC), negative deviation coefficient (NC), coefficient differences significance (CD), and the number of topics (NT). The results are shown in Table S9 in Multimedia Appendix 1 in the supplemental material.

Based on the Kano model and the six indicators mentioned above, we categorized the factor attributes, with the results shown in Figure 5. Figure 5 shows that the 12 influencing factors can be further divided into 10 charm factors and two essential factors. Charm factors include online consultation function, online registration and appointment function, easy to use, user login, online report query function, electronic health record management function, health management function, compatibility, doctor's professional level, and fee. These factors have significant effects on both PD and ND. The absolute values of the coefficients for PD and ND are significantly different, and the number of positive reviews exceeds the negative reviews (PS = 1, NS = 1, CD = 1, NT = 1). Therefore, they are classified as charm factors. Essential factors include: reliability, and customer service. These factors also have significant effects on both PD and ND. The absolute values of the coefficients for PD and ND are significantly different, but the number of negative reviews exceeds the positive reviews (PS = 1, NS = 1, CD = 1, NT = −1). Therefore, they are classified as essential factors.

Figure 5.

Attribute classification of factors influencing patient satisfaction with internet hospital APPs.

Discussion

The study included 148 internet hospital APPs and their 121,458 reviews. It identified 12 topics and uncovered 10 charm factors and two essential factors. Both the number of APPs and reviews showed an initial increase followed by a decrease, peaking in 2020. Patients were most concerned about the doctor's professional level, customer service, and online registration and appointment function. The overall patient satisfaction was 71.52% (86,862/121,458), which is consistent with previous research.^32,75 The key factors influencing patient satisfaction and dissatisfaction were easy to use and customer service, respectively. All 12 factors had asymmetric effects on patient satisfaction and dissatisfaction and could be further categorized into charm factors and essential factors.

For theoretical contribution, we extended the Kano model to digital health and validated its effectiveness. We found that the factors affecting user satisfaction with digital health technology are asymmetric and can be classified into different attribute categories. This finding challenges the conventional assumption in prior research that influencing factors are symmetric.^41,76 This reminds us to distinguish between factors that affect user satisfaction and dissatisfaction with digital health technology, and to improve digital health technology in a targeted manner.

For practical implication, the essential and charm factors excavated in this study can help stakeholders to improve the services of internet hospital APPs, improve their user satisfaction, and enhance the actual use effect.

Patient satisfaction and dominant perceptions on internet hospitals

The overall patient satisfaction with internet hospital APPs is relatively low, only 71.52%. In general, factors related to the functionality and user interface design of the APPs represent the technical infrastructure of internet hospitals. Issues here (e.g., poor reliability) directly disrupt access to care and amplify frustration. Factors related to the quality of online care reflect the core medical service experience. These factors (e.g., Doctor's Professional Level) are pivotal to perceived care quality. Their excellence drives satisfaction. Specifically, patients discussed their usage experience of APPs from seven dimensions: usability, easy to use, online diagnosis and treatment functions, health management functions, doctor's professional level, after-sales service, and cost. They focused particularly on the doctor's professional level, customer service, online registration and appointment function, and health management function. For quantitative ratings, among the 121,458 reviews, 86,862 were rated four or five stars, indicating an overall patient satisfaction of 71.52%. This is consistent with previous survey results. For example, Cui et al.⁷⁵ found that both doctors and patients have low satisfaction with telemedicine, with 60.4% (136/225) and 53.8% (121/225) respectively. Han et al.³² found that 68% (1009/1481) of patients expressed satisfaction with internet hospitals. These comparisons underscore that suboptimal satisfaction is a systemic challenge, and issues encountered by patients require ongoing attention.

Regarding qualitative reviews, on the one hand, patients focused on the doctor's professional level, customer service, online registration and appointment function, and health management function. First, as an online extension of offline hospital services, the professional level and service quality of doctors are important indicators for evaluating the quality of internet hospitals. Reliable medical diagnosis and considerate medical services are the most important aspects for patients. A competent medical team can promptly address patients’ medical inquiries online, provide professional medical diagnoses, help patients prevent or treat diseases quickly, and enhance patients’ trust. Second, customer service is an effective way for patients to communicate with the internet hospital platform, used to solve usage problems and ensure the normal operation of APP functions. Good customer service can serve as an important feedback mechanism for identifying issues and improving service levels. Ineffective customer service, however, can exacerbate patient dissatisfaction and reduce their usage intention. Third, the online registration and appointment function, and health management function are the two most discussed features by patients. Patients mainly discussed the convenience of online registration, the simplicity of the appointment process, the management of health data, and reminders and alerts for health behaviors. On the other hand, patients’ negative emotions were related to usability and after-sales service. Reviews containing these topics mostly had negative ratings. Currently, internet hospital APPs generally have issues with low reliability, poor compatibility, and inadequate customer service. For example, many patients reported frequent crashes, freezing, and system failures of the APPs, as well as incompatibility with mobile operating systems and difficulties in contacting customer service. These fundamental software issues have been widely discussed by patients and significantly affect the use of APPs, requiring focused attention and resolution.

Charm and essential influencing factors

There are 12 factors that significantly affect patients’ satisfaction and dissatisfaction with internet hospital APPs. These influencing factors exhibit asymmetry, which is consistent with previous research.^38,56,57 Reliability and customer service are essential factors, while online consultation function, online registration and appointment function, easy to use, user login, online report query function, electronic health record management function, health management function, compatibility, doctor's professional level, and fee are charm factors. Using the Tobit model, we found that all 12 factors significantly affect both PD and ND. The Wald test revealed significant differences in the effects of these factors, indicating the asymmetry, which aligns with the assumptions of Two-Factor Theory. Based on the Kano model, we further categorized these factors into charm factors and essential factors, corresponding to the primary causes of patient satisfaction and dissatisfaction. Compared with previous studies, this study considered the effect differences and categorized the factor attributes using quantitative indicators. This helps developers understand the priority of different factors and identify issues that need to be improved first.

The essential factors include reliability and customer service. These are fundamental prerequisites for patients using the APPs and should be prioritized. Enhancing the overall software quality, addressing software performance issues, fixing program errors such as freezing, crashing, and system failures, and providing effective customer service feedback channels are basic needs of patients. These measures can effectively reduce patient dissatisfaction. The higher the probability of these factors occurring, the higher the level of patient dissatisfaction. Conversely, the lower their occurrence, the higher the level of patient satisfaction. This indicates that when these factors fail, patients feel dissatisfied, but their absence does not affect patient satisfaction. Improving these factors can help eliminate patient dissatisfaction, reduce abandonment behavior, and thereby increase overall patient engagement and usage intention. This finding is consistent with Cui et al.,⁷⁶ which found that reliability is important factor affecting user satisfaction.

The charm factors include online consultation function, online registration and appointment function, easy to use, user login, online report query function, electronic health record management function, health management function, compatibility, doctor's professional level, and fee. These factors are effective means to increase patient usage frequency and should be actively improved after meeting the essential factors. Basic functions of online medical services, such as online registration, consultation, report querying, and health management, can help patients access medical care quickly and assist in self-health management, thereby improving the medical service experience and enhancing user retention. Convenience and compatibility, for example, clear interfaces, simplified operations, and multi-system compatibility, can improve the patient experience and increase continuance intention. Additionally, highly skilled doctors can effectively enhance patients’ trust in medical services, promoting repeat use. Online payment and medical insurance payment functions simplify the medical process and expand the scope of online medical services, helping to attract offline patients. The higher the probability of these factors occurring, the higher the level of patient satisfaction. Conversely, the lower their occurrence, the higher the level of patient dissatisfaction. This indicates that these factors are the main reasons for patient satisfaction, but their absence has a smaller impact on patient dissatisfaction. Providing or optimizing these factors can effectively increase patient satisfaction and thereby enhance patient compliance. Wu et al.⁴¹ further confirmed our findings, as they proposed that online consultation function and doctor's professional level can significantly improve user satisfaction.

Recommendations for improving patient satisfaction with internet hospitals

Suggestions for improvement are proposed based on the influencing factor attributes. For example, for charm factors, which mainly affect patient satisfaction, these factors can be actively provided on the basis of meeting the essential factors. For essential factors, which directly influence patient dissatisfaction, these factors must be prioritized and satisfied. For expected factors, increasing these factors will lead to higher patient satisfaction, so they should be provided as much as possible. For reverse factors, increasing these factors will lead to lower patient satisfaction, so they should be minimized or eliminated. For indifference factors, which do not affect patient satisfaction, they can be ignored. Based on the research findings, we systematically summarized recommendations for improving patient satisfaction from three aspects: policy suggestions, technical optimization, and clinical practice.

For government departments, first, they can enhance policy incentives, increase financial support and infrastructure for the construction of internet hospitals, and stimulate the willingness of hospitals and enterprises to build these platforms. Simplify the approval process for online medical insurance reimbursement to increase patient usage. Second, strengthen industry supervision and regulation, improve the approval and supervision processes for internet hospitals,⁷⁷ establish uniform construction standards, and implement annual grading assessment systems and year-end performance rankings to ensure the practical usability of internet hospitals and prevent the emergence of “shell hospitals” (platforms that exist in name only without functional services). Third, it is suggested that the government take the lead in integrating regional medical resources and coordinating the construction of internet hospital platforms to facilitate patient access. Finally, publicize successful examples of internet hospital construction and organize training sessions for builders to guide the proper development of these platforms.

For enterprises, developers are advised to pay attention to patient feedback to understand their real needs. Focus on improving essential factors, such as enhancing software quality and fixing common issues like app crashes and freezes. Ensure effective customer service channels.⁷⁸ After addressing essential factors, work on improving charm factors. For example, enhance software usability and compatibility by providing a user-friendly interface and simplifying operations.

For hospitals, first, they can use reviews to understand their needs and continuously improve online diagnosis and treatment functions. Second, gradually introduce online services in stages, starting with basic functions like registration, appointment scheduling, and consultations, then developing interactive features like health management and alerts, and finally incorporating intelligent diagnostic tools.⁷⁹ Third, implement measures to encourage doctors to provide online services, such as increasing their income for online consultations. Finally, it is recommended that hospitals actively collaborate with enterprises to build internet hospital APPs. Hospitals can provide medical expertise and processes, while enterprises can ensure the stable operation of the APPs.

Limitation

This study has several limitations. First, it focused only on internet hospital APPs and did not analyze views on other platforms. However, since APPs are the primary medium for internet hospitals, the findings of this study are still representative. Second, considering that some patients used the APPs but did not leave reviews, this study did not explore the satisfaction of these patients. However, given that this study included a total of 121,458 reviews and that the process of patient reviewing is random, the views of patients who did not review are also randomly distributed. Thus, the results of this study are still representative. Third, this study did not further analyze the pathways. Finally, due to differences in national and cultural contexts and telemedicine models, the findings of this study may not be directly applicable to other countries. However, the results provide valuable reference material for developing a framework of user satisfaction factors in telemedicine platforms internationally. Meanwhile, the analytical framework for exploring internet hospital user satisfaction and its influencing factors, as established in this study, offers important methodological support. Thus, it can enhance the analysis of user satisfaction in telemedicine platforms across other countries.

For future research, first, interview and questionnaire data could be combined to explore views on other platforms of internet hospitals and compare them with the findings of this study. Second, a temporal dimension could be incorporated into the LDA topic model to further investigate changes in views, satisfaction, and influencing factors over different time periods. Third, models such as the Technology Acceptance Model or the Information Systems Success Model could be used to deeply explore the pathways. Finally, future research can apply this methodology to analyze user satisfaction with similar telemedicine platforms in other countries. This will help explore the factors influencing user satisfaction and compare the findings with those of this study to gain cross-national comparative insights.

Conclusion

With the outbreak of the COVID-19 pandemic and the exacerbation of the imbalance in medical supply and demand, a large number of internet hospital APPs have rapidly emerged in China. However, due to the lack of effective regulation and the neglect of patient feedback, their actual satisfaction and usage effects are poor. Mining influencing factors from reviews and categorizing them can quickly identify patients’ core needs, thereby increasing APPs’ patient engagement and continuance intention. This study found that in recent years, the development and usage of internet hospital APPs in China have been declining, with overall patient satisfaction being low. Influencing factors exhibit asymmetry and can be further divided into charm and essential factors. Reliability issues such as APP crashes and freezing, as well as ineffective customer service, constitute primary drivers of patient dissatisfaction. In contrast, online diagnosis and treatment services like registration, consultation, and health management, a professional medical team, and the convenience and compatibility of the software can effectively improve patients’ usage adherence. Targeting different attribute factors, it is suggested that internet hospital builders develop personalized improvement plans. Prioritize addressing issues related to essential factors, such as software system program errors, to increase patient engagement. On this basis, actively enhance charm factors, such as adding intelligent diagnosis functions and simplifying operation, to further increase patients’ continuance intention.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076251382090 - Supplemental material for Exploring patient satisfaction and its influencing factors in Chinese internet hospitals: An analysis using two-factor theory and Kano model based on user-generated contents

Supplemental material, sj-docx-1-dhj-10.1177_20552076251382090 for Exploring patient satisfaction and its influencing factors in Chinese internet hospitals: An analysis using two-factor theory and Kano model based on user-generated contents by Yunfan He, Lei Ye, Xinran He, Jiayi Chen, Tong Wang, Lili Qiao, Hongyu Pu, Yifeng Li, Yujie Wang, Xiaoyi Jiao, Qichuan Fang, Junhao Ma, Mengyao Xing, Yue Hu, Tingting Zhou, Jun Liang, Jianbo Lei and Zhao Star X in DIGITAL HEALTH

Footnotes

Abbreviations

Acknowledgments

The authors are grateful to the National Social Science Major Fund of China “The Social Impact and Information Governance of Disruptive Applications of Artificial Intelligence” (grant number: 23&ZD224).

ORCID iDs

Yunfan He

Xinran He

Jiayi Chen

Lili Qiao

Hongyu Pu

Yifeng Li

Yujie Wang

Xiaoyi Jiao

Qichuan Fang

Junhao Ma

Mengyao Xing

Tingting Zhou

Jun Liang

Jianbo Lei

Zhao Star X

Authors’ contributions

YFH conceptualized and designed the study, drafted the manuscript, summarized the literature review, and undertook data acquisition, screening, analysis, and interpretation. Zhao Star X supervised the review method and interpretation of the data and supplied valuable improvement suggestions. QCF undertook data screening. LY supervised data filtering and analysis process. TW, JL, JBL, XRH, JYC, LLQ, HYP, YFL, YJW, XYJ, QCF, JHM, YH, MYX, and TTZ did critical revision of the manuscript for important intellectual content.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Social Science Major Fund of China “The Social Impact and Information Governance of Disruptive Applications of Artificial Intelligence” (grant number: 23&ZD224).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability

We obtained reviews from eight APP stores (China Apple APP Store, Huawei APP Store, Xiaomi APP Store, OPPO APP Store, VIVO APP Store, Baidu APP Store, 360 APP Store, and Application Treasure APP Store) through the official data website of Qimai (https://www.qimai.cn). The data analysis code for this study is available on GitHub (.

Supplemental material

Supplemental material for this article is available online.

References

Pagaiya

Phanthunane

Bamrung

, et al. Forecasting imbalances of human resources for health in the Thailand health service system: application of a health demand method. Hum Resour Health 2019; 17: 4.

Boniol

Kunjumen

Nair

, et al.

The global health workforce stock and distribution in 2020 and 2030: a threat to equity and ‘universal’ health coverage?

BMJ Glob Health 2022; 7: Article e009316.

Xiong

Lin

Yao

, et al. Comparison of the market share of public and private hospitals under different medical alliances: an interrupted time-series analysis in rural China. BMC Health Serv Res 2024; 24: 96.

Kalucy

Thomas

King

. Changing demand for mental health services in the emergency department of a public hospital. Aust N Z J Psychiatry 2005; 39: 74–80.

Liu

Goryakin

Maeda

, et al. Global health workforce labor market projections for 2030. Hum Resour Health 2017; 15: 11.

GBD 2019 Human Resources for Health Collaborators . Measuring the availability of human resources for health and its relationship to universal health coverage for 204 countries and territories from 1990 to 2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet 2022; 399: 2129–2154.

Alchemy Capital . The Chinese medical IT market and investment opportunities. https://zhuanlan.zhihu.com/p/502976644 (2022, accessed 14 November 2024).

World Health Organization . World health statistics 2023: monitoring health for the SDGs, sustainable development goals. https://www.who.int/publications/i/item/9789240074323 (2023, accessed 14 November 2024).

Jinyin

. A study on the equity of inter provincial distribution of doctors, nurses, and hospital beds in China Peking Union Medical College. 2015.

10.

Jianfang

Yan’er

Jie

. Research on the threshold effect of population aging level and health resource allocation in OECD Countries. J Nanjing Med Univ (Soc Sci Ed) 2024;24:433–439.

11.

Sung

Ferlay

Siegel

, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;71(3):209–249.

12.

Global Burden of Disease Study 2013 Collaborators . Global, regional, and national incidence, prevalence, and years lived with disability for 301 acute and chronic diseases and injuries in 188 countries, 1990-2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet 2015; 386: 743–800.

13.

GBD 2021 Causes of Death Collaborators . Global burden of 288 causes of death and life expectancy decomposition in 204 countries and territories and 811 subnational locations, 1990-2021: a systematic analysis for the Global Burden of Disease Study 2021. Lancet 2024; 403: 2100–2132.

14.

Vogel

Acevedo

Appelman

, et al. The lancet women and cardiovascular disease commission: reducing the global burden by 2030. Lancet 2021; 397: 2385–2438.

15.

Liu

Zeng

, et al. Burden of cardiovascular diseases in China, 1990–2016: findings from the 2016 global burden of disease study. JAMA Cardiol 2019; 4: 342–352.

16.

World Health Organization . Tracking of universal health coverage: Global monitoring report 2017. https://www.who.int/publications/i/item/tracking-universal-health-coverage (2017, accessed 14 November 2024).

17.

Liang

Fan

, et al. A preliminary study on the abnormal deaths and work burden of Chinese physicians: a mixed method analysis and implications for smart hospital management. Front Public Health 2022; 9: 803089.

18.

Oliveira

Garcia

Nogueira

. Nursing workload and occurrence of adverse events in intensive care: a systematic review. Rev Esc Enferm USP. 2016;50:683–694.

19.

Kassebaum

Smith

AGC

Bernabé

, et al. Global, regional, and national prevalence, incidence, and disability-adjusted life years for oral conditions for 195 countries, 1990-2015: a systematic analysis for the global burden of diseases, injuries, and risk factors. J Dent Res 2017; 96: 380–387.

20.

Yiqing

. Research on the evaluation index system of Internet hospitals based on the perspective of patients. Beijing Univ Trad Chin Med 2023; 3: 10–35.

21.

Han

Lie

Guo

. The internet hospital as a telehealth model in China: systematic search and content analysis. J Med Internet Res 2020; 22: e17995.

22.

Aidi

. Status quo and improvement of internet plus medical care and service process model – taking West China hospital of Sichuan University as an example. Mod Hosp 2024; 24: 1477–1479.

23.

Jianjun

Ting

Shiliang

. Research on the construction of regional medical union based on internet hospital platform. Health Econ Res 2024; 41: 65–68.

24.

Shujuan

Yapeng

. Public crisis and health innovation adoption change – taking internet hospitals as an example. China Administration 2021; 11: 85–92.

25.

National Health Commission . Progress and effectiveness in health and hygiene. http://www.nhc.gov.cn/wjw/tia/202309/48fa4e84e5ca423c927060ffa6b2d5a8.shtml. (2023, accessed 14 November 2024).

26.

The Central People’s Government of the People’s Republic of China . The remote medical service network covers 70% of health centers in all cities and counties, and establishes remote medical cooperation relationships with higher-level hospitals. https://www.gov.cn/lianbo/bumen/202406/content_6958103.htm (2024, accessed 14 November 2024).

27.

China Internet Network Information Center . The 54th statistical report on internet development in China. https://www.cnnic.cn/n4/2024/0829/c88-11065.html (2024, accessed 14 November 2024).

28.

Xie

Zhou

Lin

, et al. Internet hospitals in China: cross-sectional survey. J Med Internet Res 2017; 19: Article e239.

29.

Wenzhao

Xueqin

Aijing

. The development status and challenges of internet medicine in China. Chin J Med Libr Info 2016; 25: 6–9.

30.

Collaborative Industry Consulting . In depth survey and market analysis forecast report on China's medical mobile APP market from 2023 to 2029. https://baijiahao.baidu.com/s?id=1775825677387831696&wfr=spider&for=pc (2023, accessed 14 November 2024).

31.

Mengqi

. An analysis of the realistic dilemma and legislative countermeasures of the development of internet medicine in China – also on the three documents including the internet diagnosis and treatment management measures (trial). J Southwest Univ (Soc Sci Ed) 2022; 48: 37–48.

32.

Han

Wei

Wang

, et al. Service quality and patient satisfaction of internet hospitals in China: cross-sectional evaluation with the service quality questionnaire. J Med Internet Res 2024; 26: e55140.

33.

Health Sector . Report on the development of internet hospitals in China in 2021. (2021). https://zk.cn-healthcare.com/doc-show-53644.html (2021, accessed 14 November 2024).

34.

Bocheng

. Analysis on the willingness to use and influencing factors of diagnosis and treatment services in Wuhan Internet Hospital for elderly patients. Huazhong Univ Sci Technol 2023; 3: 9–14.

35.

IiMedia Research . Survey data on the operation and consumption behavior of China’s Internet medical industry. https://www.163.com/dy/article/JFO6O26R0511A1Q1.html (2024, accessed 14 November 2024).

36.

Wang

Fan

Zheng

, et al. The impact of gamification-induced users’ feelings on the continued use of mHealth apps: a structural equation model with the self-determination theory approach. J Med Internet Res 2021; 23: Article e24546.

37.

Wang

Liang

, et al. Identifying major impact factors affecting the continuance intention of mHealth: a systematic review and multi-subgroup meta-analysis. NPJ Digit Med 2022; 5: 145.

38.

Zhu

Wang

, et al. Mining user reviews from hypertension management mobile health apps to explore factors influencing user satisfaction and their asymmetry: comparative study. JMIR Mhealth Uhealth. 2024;12: Article e55199.

39.

Liu

Shi

. Chinese patients’ intention to use different types of internet hospitals: cross-sectional study on virtual visits. J Med Internet Res 2021; 23: Article e25978.

40.

Zhiyang

. Research on influencing factors of Internet hospital user satisfaction. Huazhong Univ Sci Technol 2020; 1: 4–16.

41.

Zhao

. Online physician-patient interaction and patient satisfaction: empirical study of the internet hospital service. J Med Internet Res 2023; 25: Article e39089.

42.

Yiting

Hengjiang

. Research on the impact of social presence on young people's internet satisfaction with medical care. New Media Res 2023; 9: 37–42.

43.

Jing

Huiying

. Survey on “internet plus medical health” service and user satisfaction in Sichuan and Chongqing. J Chongqing Univ Sci Technol (Soc Sci Ed) 2021; 6: 52–56.

44.

Frederick

. The Motivation to Work. Piscataway, New Jersey, USA: Transaction Publishers, 1993, pp.14–22.

45.

Kano . Attractive quality and must-be quality. J Jpn Soc Qual Control 1984; 14: 147–156.

46.

National Health Commission of the People’s Republic of China . Notice on printing and distributing three documents, including the administration regulations on internet diagnoses and treatments (Trial). http://www.gov.cn/zhengce/zhengceku/2018-12/31/content_5435436.htm (2018).

47.

Tian

. Construction of network hospital of Guangdong Province in the background of “internet + medical”. China Digital Medicine 2016; 1: 23–25.

48.

The State Council of the People’s Republic of China . Opinion on actively pushing forward the development of “Internet+” Action. http://www.gov.cn/zhengce/content/2015-07/04/content_10002.htm (2015)

49.

Tencent Research Institute . 2016 China internet hospital white paper. http://www.360doc.com/content/16/1127/01/34899478_609811863.shtml (2016).

50.

National Health Commission of the People’s Republic of China . There are 158 Internet hospitals in China. https://www.gov.cn/xinwen/2019-05/08/content_5389676.htm (2019).

51.

Qian

Lei

, et al. Experiences and challenges of emerging online health services combating COVID-19 in China: retrospective, cross-sectional study of internet hospitals. JMIR Med Inform 2022; 10: e37042.

52.

Hebei News Network . The “Internet plus” medical service model in Hebei has accelerated its upgrading. https://www.cnr.cn/hebei/yw/20250622/t20250622_527223731.shtml (2025).

53.

China Digital Medicine . Relying on the construction of internet hospitals, the First People’s Hospital of Lianyungang has created a “Gangcheng Model” for digital and intelligent post-diagnosis services. https://mp.weixin.qq.com/s?__biz=MzA5Mzg2MTMwMA==&mid=2650414757&idx=1&sn=23912805124d0d63add9df3c4a6eb130&chksm=8859da7ebf2e53685512eb7347f4a91cf4be3d49a45ee5c6487c8dccda3fb7b2ade549ec1ba9&scene=27 (2023).

54.

Zhejiang Provincial Disabled Persons’ Federation . The “Online Ward” home rehabilitation service for the disabled in Ningbo has achieved remarkable results. http://www.zjdpf.org.cn/art/2024/10/27/art_1229458591_78082.html (2024).

55.

. Internet hospital: challenges and opportunities in China. Health Info Sci 2016; 10038: 85–90.

56.

Wang

Zheng

Liang

, et al. Use of machine learning to mine user-generated content from Mobile health apps for weight loss to assess factors correlated with user satisfaction. JAMA Netw Open 2022; 5: Article e2215014. Published 2022 May 2.

57.

Nuo

Zheng

Wen

, et al. Mining the influencing factors and their asymmetrical effects of mHealth sleep app user satisfaction from real-world user-generated reviews: content analysis and topic modeling. J Med Internet Res. 2023;25: Article e42856. Published 2023 Jan 31.

58.

Wiertz

de Ruyter

Keen

, et al. Cooperating for service excellence in multichannel service systems – an empirical assessment. J Bus Res 2004; 57: 424–436.

59.

Wang

Feng

, et al. A novel user-generated content-driven and Kano model focused framework to explore the impact mechanism of continuance intention to use mobile APPs. Comput Human Behav 2024; 157: Article 108252.

60.

Xie

Zhou

Lin

, et al. Internet hospitals in China: cross-sectional survey. J Med Internet Res 2017; 19: Article e239.

61.

DBC Deben Consulting . 2024 Top 100 internet hospitals. http://www.enet16.com/article/2024/0722/A202407221875.html (2024, accessed 28 December 2024).

62.

New knowledge on hospital operation consulting . Observation of the top 100 hospital budgets in China in 2024. https://mp.weixin.qq.com/s?__biz=MzI5MjkyMjYzNA=&mid=2247657143&idx=1&sn=b12c6a6c1fe0cc78c14317a84fa8cb56&chksm=edf8ed3ca91e89313618278780f7aaae34ae5c572f98cdacc2009d292272231feaae5be33c44&scene=27 (2024, accessed 28 December 2024).

63.

Fudan University Hospital Management Research Institute . 2023 China hospital comprehensive ranking list. https://www.dxy.cn/bbs/newweb/pc/post/51167743?sr=1&nm=dahao&pt=6&pd=dxyer&dt=20241116&channel=dahao (2024, accessed 28 December 2024).

64.

Douglas

Myriam

Bruno

, et al. STROBE Statement–checklist of items that should be included in reports of observational studies. (2023). https://www.strobe-statement.org/checklists/.

65.

Qimai Data (formerly ASO100) . Professional mobile product business analysis platform. (2024).. https://www.qimai.cn.

66.

Mkearney . tweetbotornot. (2018). https://github.com/mkearney/tweetbotornot (2018, accessed 28 December 2024).

67.

Tian

Gao

Xiao

, et al. SKEP: sentiment knowledge enhanced Pre-training for sentiment analysis. In: 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020). 2020; 4067–4076.

68.

Hadji

Degoulet

. Information system end-user satisfaction and continuance intention: a unified modeling approach. J Biomed Inform 2016; 61: 185–193.

69.

Park

Lee

Nicolau

. Understanding the dynamics of the quality of airline service attributes: satisfiers and dissatisfiers. Tour Manag 2020; 81: 104163.

70.

Blei

Jordan

. Latent Dirichlet Allocation. J Mach Learn Res 2003; 3: 993–1022.

71.

Junyi

. Jieba: Chinese words segmentation utilities. https://github.com/fxsjy/jieba (2020, accessed 28 December 2024).

72.

Rehurek

Sojka

. Gensim-python framework for vector space modelling. Brno: NLP Centre in Masaryk University, 2011; 3(2): p.10–14.

73.

Tong

. Research on the impact mechanism of user generated content driven mobile health continued use intention. Changchun: Jilin University, 2023.

74.

Wang

Fan

Zheng

75.

Cui

Zhai

, et al. Application of telemedicine services based on a regional telemedicine platform in China from 2014 to 2020: longitudinal trend analysis. J Med Internet Res 2021; 23: Article e28009. Published 2021 Jul 12.

76.

Cui

Zhu

, et al. Attitudes and perspectives of 534 Chinese pediatricians toward internet hospitals. Front Pediatr 2022; 10: Article 948788. Published 2022 Sep 28.

77.

Lai

Wang

, et al. Challenges and strategies of developing internet hospital: combining qualitative interview and documentary research. Digit Health 2024; 10: Article 20552076241310075.

78.

Wang

Huang

, et al. How AI-powered consultation services in internet hospitals influence patient satisfaction: a structural analysis. Digit Health 2025; 11: Article 20552076251358673.

79.

Sun

, et al. Construction of the whole-process nursing service system for day surgery patient based on the Kano model: a pilot cluster randomized controlled trial. Digit Health 2024; 10: Article 20552076241286799.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB