Sage Journals: Discover world-class research

Abstract

Understanding online patient dissatisfaction is essential for improving the quality of healthcare services, patient satisfaction, and physician career development. This study is the first to apply the structural topic model to patient satisfaction research based on patient online reviews from a mobile health communication platform, revealing eight negative topics of patient concerns. These topics include under-explored areas such as “go to the hospital for check-ups,” “incomplete counseling,” and “language expression.” Additionally, we incorporated the doctor's title as a covariate in the model to examine how specific topics varied across different conditions. The results indicated that higher-titled doctors were more likely to receive complaints about the cost of treatment and whether the question was answered, whereas lower-titled doctors were more likely to receive complaints related to physician's knowledge, incomplete counseling, and response speed. This study not only enhances our understanding of mobile health services but also provides targeted insights for healthcare providers to improve their services, thereby contributing to the advancement of patient-centered care.

Keywords

Patient dissatisfaction doctor title structural topic model online review health communication

Introduction

The integration of mobile health (mHealth) technologies in disease management has shown significant promise in enhancing patient outcomes and facilitating efficient healthcare delivery.¹ Recent studies highlight the potential of mHealth to improve disease monitoring, patient engagement, and overall health management.^2–4 These advancements underline the crucial role of mobile health in modern healthcare systems, making it an indispensable tool in disease management. Therefore, how online doctors can use this novel approach to better serve their patients has become an important issue.

In order to achieve this, an important aspect is to collect and analyze patient feedback to understand the demands of new age patients. In earlier studies, patient feedback was usually collected through face-to-face interviews and questionnaires. For example, in a survey on mHealth services, Alam et al.⁵ found that mHealth services were perceived as trustworthy and cost-effective through interviews and content analyses of 16 households. However, in other surveys, researchers noted that patients had concerns about sharing adverse healthcare experiences.⁶ Therefore, it is worth considering more modern and efficient methods of obtaining patient feedback better to understand their needs and concerns in mHealth services.

Online reviews, as a new source of information, offer new opportunities to address this issue. The design of review mechanisms for online healthcare service platforms is inspired by the e-commerce, through which patients voluntarily share their healthcare experiences and opinions.⁷ Online reviews usually allow patients to use anonymity or pseudonyms, which provides a sense of security and allows them to freely express their opinions and grievances without fear of possible conflict or discomfort.⁸ However, the presence of these negative online comments can also be damaging. They can adversely affect the reputation of healthcare providers and reduce patient trust in healthcare.⁹ Patel et al. showed in a survey of physicians that healthcare professionals often feel criticized and frustrated when faced with negative feedback online, which reduces their job satisfaction and engagement.¹⁰ Therefore, an in-depth analysis of the factors influencing online patient dissatisfaction is necessary to improve online patient satisfaction and avoid the potential harm caused by negative online reviews.

In the realm of online healthcare, patients, through prolonged interactions and comparisons, have increasingly recognized the significance of doctors’ professional capital, such as their titles and affiliations. They tend to place greater trust and willingness to pay in doctors with higher levels of professional capital.¹¹ The recognition of professional capital holds a very high priority for patients, to some extent that they may overlook other characteristics of doctors.^12,13 Consequently, in this work, we plan to use data from patient comments following consultations on mHealth platforms to reveal antecedents of online patient dissatisfaction. Specifically, we plan to answer the following questions: what are the areas of concern for dissatisfied patients? How do patients’ dissatisfaction levels differ across physicians with different professional capitals (i.e. doctor titles)?

Although many studies have examined the influences associated with online patient satisfaction, there are significant differences between the current work and previous investigations. First, from a methodological perspective, our study is the first to apply a structural topic model (STM) to patient satisfaction. While other automated text analysis methods, such as Latent Dirichlet Allocation (LDA), are widely used in text data analysis, it is necessary to be aware of the method's limitations. In the online health domain, patients may mention aspects of dissatisfaction in addition to satisfaction, such as a positive comment, “Resolved my distress, although a bit slow to respond halfway through.” For this type of text, LDA is usually difficult to handle. If only positive review texts are modeled, the researcher may get a negative topic and vice versa. In contrast, the STM model overcomes the limitations of the LDA approach because it allows document-level variables (e.g. the polarity of the comment text, i.e. positive or negative) to be incorporated into the generation of textual data.¹⁴ Based on this feature, the STM model can more accurately reveal thematic differences between positive and negative comments in statistical tests.

Second, exploring whether patient dissatisfaction varies by doctor title goes beyond the findings of the existing literature. In the current healthcare system, a doctor's professional title is a valuable capital that healthcare practitioners have acquired through good education and hard work.¹⁵ It is well-documented that doctors’ professional titles have become a symbol of their status and professionalism and are essential for patients to choose their doctors.¹⁶ Given patients’ concerns about doctors’ professional titles, patients may have certain expectations about doctors’ titles, for example, believing that a doctor with a senior title will be better able to solve their problems. However, this expectation may lead to increased patient expectations of doctors. A survey from the UK suggests that dissatisfaction may be triggered as soon as a patient's healthcare experience does not match expectations.¹⁷ Thus, for patients, the title of the doctor may be an essential factor contributing to differences in their comment topics. Examining the impact of this individual difference in physicians about the level of patient dissatisfaction will help provide more specific and targeted recommendations for improving mHealth services.

The remainder of this study is organized as follows. In the “Literature review” section, we review existing research on online patient reviews and explain the details of the STM model. The “Data and model settings” section details the data and model setup for this study. In the “Results” section, we present the results of the investigation. In the “Discussion” section, we discuss the effects and implications of the study and point out the limitations of the current study and future research directions.

Literature review

Research related to patient dissatisfaction

“Patient dissatisfaction” refers to patients feeling dissatisfied with their online healthcare services or care experiences. Over the past two decades, online reviews have made it more straightforward and familiar for patients to express negative perceptions of healthcare services.^6,9 These negative feedbacks often reflect the quality of healthcare services and reveal patients’ emotional and psychological state, profoundly impacting healthcare experiences and outcomes.¹⁸ Given the importance of negative patient feedback in online health, a growing body of research aims to identify and understand the factors that trigger dissatisfaction to develop targeted improvements.

In the past, manual coding has been the preferred method for identifying complaints in online review texts. By developing a coding framework, Zhang et al.¹⁹ content-analyzed negative patient online review texts, showing that insufficient time for medical consultations, physician impatience, and poor treatment outcomes were the main reasons for patient complaints. However, while manual coding methods help to keep results interpretable, they do suffer from the limitations of data samples and researcher subjectivity.

With the development of information technology, automated text analysis has become a mainstream method for patient online review mining. Among them, automated tools such as the widely adopted LDA topic model can process large amounts of online review data more efficiently and extract key topics and factors from them. For example, Gao et al. analyzed 19,852 patient reviews in an online healthcare service platform based on LDA, and they outlined patient concerns from six dimensions of a service quality evaluation model.⁴ However, due to the limitations of LDA, the researchers could not directly determine the polarity of the topics or observe the influence of other variables on the topics directly from the output. Researchers have made various attempts to overcome this shortcoming. For example, Hao et al.²⁰ modeled negative patient reviews individually to tap into common dissatisfaction influences in Chinese and US patient review texts, including logistics, bedside attitudes, and medical treatment. Stokes et al.²¹ deviated from existing text analytics approaches by developing a hybrid framework combining star ratings on websites and topic discovery capabilities to reveal certain negative and positive topics for mental health care facilities. In addition, researchers are interested in other variables that may lead to differences in patients’ review perspectives, not only topic polarity. For example, through a combination of SentiNet and LDA, Shah et al. considered differences in satisfaction with physician services across patients (high-risk and low-risk disease), suggesting that treatment experience, communication, physician attitudes, and the appointment process were the main factors contributing to dissatisfaction among patients with high-risk disease, in contrast to low-risk patients whose complaints came more from the environment, unfriendly staff, physicians regarding pain management skills and cost of treatment.²²

The present study fully develops the current stream of research. We introduced a novel approach, STM. Compared to LDA, the critical feature of STM is that it presents document-level structural information to influence the prevalence of topics. This feature gives STM a unique advantage when dealing with textual data, especially in contexts where document-level variables are linked to textual content. In recent years, the application areas of STM have expanded. STM-based research has covered various fields, such as hospitality, tourism, e-commerce, and lodging.^23–26 However, to our knowledge, STM has not been adopted in the online health communication literature. Therefore, this study aims to combine the advantages of STM to analyze the critical issues in patients’ online consultation process more comprehensively and in-depth to improve the quality of healthcare services and patient satisfaction.

Structural topic model

To analyze patient satisfaction and dissatisfaction, text mining techniques are widely used. One of the prominent methods is LDA, which has been employed to identify latent topics within large sets of textual data.²⁷ However, LDA has limitations in accounting for external factors that might influence topic prevalence. To address this, our study employs the STM model, an advanced method that extends LDA by incorporating document-level covariates.²⁸ This allows us to explore how different factors, such as the doctor's title and the sentiment of reviews, influence the topics discussed by patients.

The STM model is particularly relevant to our research as it offers an effective way to analyze the thematic structure of patient reviews while considering covariates. STM allows us to understand the complex dynamics of patient dissatisfaction by revealing how specific issues vary with different covariates. For example, by including the doctor's title as a covariate, we can examine how complaints differ between higher-titled and lower-titled doctors, providing insights into the role of professional status in patient dissatisfaction.

Data and model settings

Data collection and pre-processing

We followed the following process for data collection and pre-processing for this study. First, there was data collection. The patient review data came from the Chunyu Doctor, a leading mobile health platform in China with a broad user base. The platform not only provides a convenient online consultation service but also accumulates many accurate comments left by patients after consultation, including text comments and ratings. Given the richness of the platform's data, many studies have used the platform as a data source. We wrote a Python-based program to obtain all patient review data in the platform for the period 2020.01.01 to 2023.12.31. A total of 45,002 independent online review samples were obtained by removing short review data (less than five words²⁹) and removing duplicate samples.

The second step is to select an appropriate sample dataset for the study. In e-commerce platforms, due to individual selection bias, there is usually a J-distribution in the review samples, that is, there are far more positive than negative reviews.³⁰ The Chunyu Doctor platform has a rating mechanism similar to those of these e-commerce platforms and thus suffers from the same problem. In our sample, the rating labels consisted of three types: satisfaction, general, and dissatisfaction, and their percentages in the dataset were 0.724, 0.051, and 0.225, respectively. Given that the goal of this study is to extract topics and pay special attention to the topics that appeared in negative reviews, we need to balance the number of positive and negative reviews when constructing the topic model to ensure the consistency of the sample.^31,32 To achieve this, and in line with Hao et al.,²⁰ we attributed general and dissatisfaction to negative comments and satisfaction to positive comments. However, after processing, the data still showed a significant imbalance. Therefore, referring to the study by Hu et al.,³⁰ we planned to build a corpus and use random sampling for sample selection to achieve an equal number of positive and negative comments. Specifically, we regarded each doctor as an independent service provider, and considering the seasonal characteristics of the disease, we set up a sampling procedure, that is, if a doctor receives n negative reviews in a particular quarter, we randomly select n positive reviews received by that doctor in the same quarter. Finally, we obtained 3070 review data that can be used to construct STM topic models and further analyses.

The third step is to pre-process the text data. This consists of the following steps. (1) non-Chinese characters such as URLs, spaces, numbers, and punctuation marks were removed²³ and (2) the lexical analysis model of Baidu AI (https://cloud.baidu.com/product/nlp_basic/lexical) was used to segment and annotate the comment text lexically. One advantage of choosing this model is that it not only performs well in identifying the basic structure of the core vocabulary but also in identifying new vocabulary and proper nouns in the domain. Also, based on the lexical annotations given by the model, we have eliminated personal names, conjunctions, prepositions, auxiliaries, and dummy words that have little meaning in the sentence. (3) Replace synonyms in the comment text using manual annotation with the help of Harbin Institute of Technology and HIT-CIR Tongyici Cilin (extended). This is because the same meaning can be represented by different words in the Chinese context (e.g. recover and cured), and the presence of a large number of synonyms may make the subject difficult to interpret.^4,33 (4) Remove the deactivated words (e.g. of, had, and bar) in the comment text according to the Chinese deactivation word list. In this process, we manually identified other deactivated words (e.g. doctor and patient) related to the online consultation, which occur frequently in all documents but have little effect on topic detection.^34,35 (5) Filtering some irrelevant words. Following the suggestion of Lester et al.,³⁶ we removed the lowest 5% of words in both positive and negative review texts by calculating the average tf-idf scores of the words. We used the R software's Predocument and Plotremoved to identify and remove low-frequency words appearing in fewer than three documents,³⁷ which will help improve the STM model's performance.

Model setup and selection

The goal of our study is not only to analyze what topics exist in online patient reviews but also to examine how these topics affect their satisfaction and how they change with physician titles. To achieve this goal, we need to specify a model for STM, which will be used to simulate how different factors affect the topic prevalence parameter. These factors include the polarity of the review (polarity), the doctor's title (doctor_title), and their interaction (polarity × doctor_title). Polarity is 1 if the comment is positive and 0 otherwise. doctor_title is a categorical variable, that is, chief physician, associate chief physician, attending physician, and physician are coded as 4, 3, 2, and 1, respectively. We then constructed the model to describe the relationship between topic prevalence and these factors through equation (1).

prevalence = g (polarity, doctor_title, polarity \times doctor_title)

(1)Another critical issue in STM modeling is the selection of the number of topics. In order to determine the optimal number of topics, this study constructed several STM models based on the number of topics (from 2 to 30). Following Ding et al.²⁴ and Hu et al.,³⁰ we selected exclusivity and semantic coherence as the model evaluation criteria. Exclusivity refers to the likelihood that words in a topic appear in other topics, and semantic coherence refers to the probability that common words in a topic appear in the same comment text. Usually, as the number of topics increases, exclusivity shows an increasing trend, but semantic coherence shows a decreasing trend. Therefore, selecting the number of topics is essential to trade-off based on the values of these metrics and to check the interpretability of the model at the same time.²⁴ Based on these considerations, we finally specified the number of topics for the model in this study as 14.

For the selected sample of 3070 reviews, all statistical analyses and data manipulations in this study were performed using R version 4.2.1. All tests were conducted with a significance level of P < 0.05 as the criterion for statistical significance.

Results

Topic summary and identification

To present the study results, we provide a clear summary in Table 1. Each row of Table 1 represents a topic that reflects the patient's concerns in the review. The first column of the table is a unique serial number assigned to each topic. The second column lists the labels for each topic, which the researcher manually labeled to describe the topic better. The third and fourth columns are the outputs of the model. Among them, the third column shows the frequency-exclusivity (FREX), which are keywords that are highly relevant to the topic and less relevant to other topics. The fourth column shows the proportion of occurrences of each topic in the corpus, which indicates how often the topic appears in all comments.

Table 1.

Definition of topics.

Category	No.	Topic label	Topic words (FREX)	Ratio	Reference
–	1	Finding doctors on the platform	encounter, consultation, experience, value, platform, opinion	5.2%	Hu et al.³¹
Healthcare-related topics	2	Recommendation of care	recommendation, give, useful, clear, pertinent, perspective	8.2%	Lester et al.³⁶
	3	Treatment effects	drugs, open, eat, effect, get better, recommend	10.4%	Hu et al.³¹
	4	Physician's knowledge	profession, feel, level, knowledge, inadequate, worthwhile	7.5%	Zhang et al.¹⁹ and Wei and Hsu³⁵
	5	Go to the hospital for check-ups	go, hospital, check, do, online, impatient	5.8%	–
	6	Whether answer questions	question, solution, positive, no answer, three, answer	6.3%	Lester et al.³⁶
Time-related topics	7	Waiting time	wait, hours, result, time, long, times	5.0%	Nemec et al.⁶
Time-related topics	8	Response speed	slow, fast, speed, reply, efficiency, message	10.5%	Yin et al.³⁷
–	9	Incomplete counseling	over, end, close, Baidu, substantive, short	4.4%	–
–	10	Treatment cost	money, spend, useless, might as well, cost, not worth it	6.2%	Espinel et al.³⁸ and Agarwal et al.³⁹
Service-related topics	11	Explanation	illness, analysis, explanation, guidance, treatment, effective	5.3%	Hakimi et al.⁴⁰
	12	Patience	patience, explain, carefulness, gentle, take great pains, responsive	14.0%	Wan et al.⁴¹
	13	Language expression	say, audio, listen, understanding, word, ambiguity	6.4%	–
	14	Timely help	help, timely, attentive, confuse, peaceful, serious	4.8%	–

The critical topic label inference task is achieved by analyzing FREX, representative comments, and the literature review. Firstly, FREX provides more semantically meaningful labels for topics by taking into account word frequency and exclusives. Second, considering that each document contains multiple topics representing a broad range of the entire corpus, we also better understand the content and relevance of the topics by analyzing representative documents for each topic, especially those with the most significant proportion of focal topics. Finally, it is essential to note that our study is the first attempt to introduce STM modeling into online health communication. To enable comparison with previous studies, the name of each topic was considered based on the topic labels identified in previous studies, but if no matching topic labels could be found, then these topics were identified through manual analysis and group discussion. As shown in the fifth column of Table 1, by comparing the topics extracted from the STM with previous literature, we found that four of the 14 topics had yet to be addressed in earlier literature. They were “go to the hospital for check-ups” (Topic 5), “incomplete counseling” (Topic 9), “language expression” (Topic 13), and “timely help” (Topic 14).

Negative topic analysis

The main advantage of STM is that it not only helps us discover topics, but more importantly, it allows us to quantify covariates to influence the distribution of topics in a document.⁴² This unique feature allows us to understand how different topics vary in a given context, thus allowing us to identify negative topics.

Figure 1 shows the mean of the estimated differences in topic prevalence (dots), along with their 95% confidence intervals (lines). When the confidence intervals lie precisely to the left of the vertical line, the topic is significantly more frequent in negative than in positive comments, and the difference is statistically significant. These are labeled as “negative topics” because they have a considerably higher and statistically substantial prevalence in negative comments than positive ones.³⁰ For example, “treatment cost” (Topic 10) appeared in negative comments on average 11.5% more frequently than positive comments, and this difference was statistically significant within the range of 11.0% and 11.9%. Similarly, we identified other negative topics, which included “physician's knowledge” (Topic 4), “waiting time” (Topic 7), “go to the hospital for check-ups” (Topic 5), “whether answer questions” (Topic 6), “response speed” (Topic 8), “incomplete counseling” (Topic 9), and “language expression” (Topic 13). In addition, the positive topics were “treatment effects” (Topic 3), “explanation” (Topic 11), “recommendation of care” (Topic 2), “patience” (Topic 12), and “timely help” (Topic 14). In contrast, “finding doctors in platform” (Topic 1) did not show significant thematic polarity (across the center line); a possible explanation for this is that mHealth platforms merely provide a place for patients and doctors to interact online and that the doctor is the actual service provider to the patient. Therefore, patients rarely have a positive or negative perception of the platform directly.

Figure 1.

Differences in the prevalence of patient review topics (positive or negative).

As mentioned earlier, we paid particular attention to negative topics in the text of patient comments. By analyzing Figure 1 in-depth, firstly, we found that “treatment cost” (Topic 10) was the most negative topic. It describes the mismatch between the healthcare costs paid and the value of services received. This is in line with previous studies, which have shown that the issue of treatment cost is one of the most frequently occurring topics in negative reviews.^22,43 Secondly, time-related topics, including “waiting time” (Topic 7) and “response speed” (Topic 8), were also negative. Waiting time (Topic 7) refers to the interval between when a patient asks a question and when the doctor attends. At the same time, speed of response (Topic 8) refers to the speed of the doctor's response in treating the patient's inquiry after attending the doctor's appointment. Thus, these two topics primarily reflect the efficiency of doctors’ attendance and response, negatively affecting patient experience. Healthcare-related topics, including “go to the hospital for checkups” (Topic 5), “physician's knowledge” (Topic 4), and “whether answer questions” (Topic 6), were also significant sources of patient dissatisfaction. The results suggest that online consultations offer convenience but have many uncertainties and limitations.^44–46 Due to the lack of physical conditions, it is sometimes difficult for doctors to diagnose based on limited information and ask patients to provide further test results. In this case, the effectiveness of online consultations is challenged because under conditions of insufficient data, the patient's needs for this consultation may not be met, creating questions about the doctor's professionalism. In addition, we are noting that “incomplete counseling” (Topic 9) is also a negative topic. By examining the consultation records corresponding to this topic, we determined that the main reason for such complaints came from the fact that patients continued to develop new questions and needs during the consultation, which ultimately exceeded the time limit of the consultation. The last negative topic was “language expression” (Topic 13). Representative comments on this topic indicate that doctors using too much jargon or long, vague speech can be unintelligible to patients and give them a negative view of the doctor's services.

The moderating role of doctor titles

Patient dissatisfaction with physician services may stem from their expectations. For physicians with different job titles, the topics that patients focus on in their online reviews may vary.

In Figure 2, we plot how the prevalence of the eight negative topics changes with the physician's title. In this case, the horizontal axis represents the physician's professional title (chief physician, associate chief physician, attending physician, and physician are coded as 4, 3, 2, and 1, respectively). The vertical axis represents the prevalence of the topics, with the blue line indicating the prevalence of positive topics, the red line indicating the trend in the prevalence of negative topics, and the dashed portion of the line indicating the 95% confidence intervals.

Figure 2.

Topic prevalence change with doctor titles: (a) physician's knowledge, (b) incomplete counseling, (c) response speed, (d) language expression, (e) treatment cost, (f) whether answer questions, (g) go to the hospital for check-ups and (h) waiting time.

From Figure 2(a) to (c), we find that the frequency of occurrence of (a) physician's knowledge, (b) incomplete counseling, and (c) response speed in the negative comment text shows a decreasing trend with the rise of doctor's title. In particular, the frequency of doctor knowledge in negative comments decreased sharply from 11.53% (low title) to 7.55% (high title). This result suggests that the expertise of low-title doctors is an essential aspect of patient complaints, and they should recognize the importance of continuous learning and constant improvement of their knowledge to ensure the core function of health advice.⁴⁷ In addition, incomplete counseling is another primary reason why low-titled doctors experience patient complaints. This suggests that low-titled doctors may need more ability to manage patients’ needs or that there is significant room for improvement. Finally, our results also show that low-title doctors should be more wary of the speed of response compared to high-title doctors. This finding is interesting because low-title physicians are often perceived to have more time and energy to engage in online consultation services than high-title physicians.⁴⁸ In an in-depth analysis, we found that low-title doctors may face dual pressures from increased patient expectations and capacity constraints. First, with the popularity of online consultation services, patients are demanding more immediate responses and efficient services from doctors.^49–51 This expectation may be higher for lower-titled physicians, who are perceived to be more generous with their time. Second, low-title physicians may feel challenged when faced with complex cases, and they need more time to carefully study the condition, develop a treatment plan, and communicate adequately with the patient. Therefore, although lower-titled doctors may have more time and effort in online consultation services, they still need to be vigilant about the speed of response to ensure the quality of care without sacrificing service efficiency and patient satisfaction.

Figure 2(e) and (f) shows that as the doctor's title increases, Figure 2(e) treatment cost, and Figure 2(f) whether answer questions, increase in frequency of negative comments and remain constant or decrease in frequency in positive comments. This suggests that higher treatment costs and the ability to solve the patient's problem have become crucial to online patient complaints about high-title doctors. One possible explanation is that highly titled doctors deal more with complex cases. However, online consultations often lack the physical conditions to make diagnosing complex problems challenging. In this case, overpricing exacerbates patient dissatisfaction. As a result, higher-title doctors may need to consider offering more high-quality, high-value healthcare services in the future so that patients are more willing to pay for these services.

We also found that the frequency of Figure 2(g) going to the hospital for check-ups, Figure 2(h) waiting time, and Figure 2(d) language expression in the negative reviews were not affected by the doctor's professional title. This suggests that doctors should pay adequate attention to these issues regardless of their professional title. We observe that the frequency of Figure 2(d) language expression in positive reviews increases significantly as the doctor's professional title rises. In other words, while not masking patients’ complaints about the doctor's manner of speaking, the doctor's professional title can serve as an icing on the cake. This finding emphasizes the importance of doctors’ communication skills and that a good speaking style helps patients better understand medical advice, comply with treatment plans, and feel more cared for.

Discussion

In this study, we aimed to gain insight into the critical factors of online patient dissatisfaction. To achieve this goal, we used the innovative approach of the STM model to obtain and analyze accurate patient review data on mobile healthcare service platforms. Based on the strengths of the STM model, we incorporated document-level covariates, that is, review text polarity and doctor's professional title, to identify trends among covariates for a given topic and to statistically test them. Our study not only represents a methodological breakthrough but also provides a more comprehensive and precise insight into online patient dissatisfaction complaints.

First, based on STM, we identified 14 topics in patient review texts, each articulating a specific aspect of patient concern. Notably, four of these topics, namely “go to the hospital for check-ups” (Topic 5), “incomplete counseling” (Topic 9), “language expression” (Topic 13), and “timely help” (Topic 14), have not been fully explored in the existing literature. These topics provide new insights that can guide healthcare providers in addressing unmet patient needs and improving their services.

Second, by including the polarity of the comment text as a covariate in the prior distribution of STM, our study reveals eight key factors contributing to patient dissatisfaction in the context of mHealth consultations. These factors can be regarded as the authentic voice of patient complaints because, unlike previous explorations of only negative comment texts,¹⁹ we identified them in a statistically significant way as appearing significantly more frequently in negative than in positive comments. Furthermore, among these factors, “go to the hospital for check-ups” (Topic 5), “incomplete counseling” (Topic 9), and “language expression” (Topic 13) did not in previous studies appear or be identified as negative topics. Specifically, the emergence of Topic 5 about “go to the hospital for check-ups” sparked our interest. In previous studies, medical consultations have often focused on online diagnosis and advice. In contrast, hospital check-ups may be considered a routine step unlikely to trigger complaints.²² However, our results suggest that some online patients are skeptical about the requirement to go to the hospital for check-ups, which may be related to patient's lack of understanding of the healthcare process and the limitations of online visits. Second, Topic 9 about “incomplete counseling” also raised our concerns. Platform doctors are asked to provide services within a limited timeframe. While previous studies have widely emphasized the positive effect of response timeliness on patient satisfaction,⁵⁰ the completeness of the consultation needs to be addressed. In contrast, our results emphasize the importance of managing patient demands, and online doctors should try to avoid generating too many new demands during the consultation process. Finally, Topic 13 on language expression highlights the importance of verbal communication in medical consultations. Patients can be susceptible to a doctor's language and manner of expression, and an inappropriate style of speech can lead to patient dissatisfaction. Thus, our study identifies not only new patient complaint factors but also provides new directions for improving online medical counseling services.

Finally, to the best of our knowledge, this work explores the impact of doctors’ professional titles on patient comments for the first time. Our survey results present some interesting findings revealing significant differences in patients’ concerns about higher-titled and lower-titled doctors when they complain. Specifically, our study showed that higher-titled doctors were more likely to receive complaints about the cost of treatment and whether the question was answered. This phenomenon may reflect patients’ higher expectations of the healthcare services provided by higher-titled doctors. Higher-titled doctors usually have more clinical experience and expertise, so patients may be more concerned about whether the cost of treatment is reasonable and whether their health problems are adequately addressed. This is of great significance for high-titled doctors as it emphasizes the need to improve the quality of the service and for patients to feel that they are getting value for money. At the same time, lower-titled doctors were more likely to receive complaints about expertise, consultation completeness, and response speed. This implies that lower-titled doctors may sometimes be challenged by inadequate knowledge, difficulty in managing patient needs, and inefficient responses. Therefore, lower-titled doctors may need to consider more resources and support to help them improve their medical knowledge and response efficiency to provide better care.

In terms of theoretical contributions, our study enriches the understanding of possible reasons for online patient complaints by uncovering a previously unidentified influence mechanism: the effect of physicians’ professional titles on patient comments. Furthermore, we have validated the feasibility and superiority of the STM model in the field of digital health research, providing insights for future studies that may utilize other text analysis methods to mine online patient reviews within this domain. Practically, our findings hold significant implications for healthcare providers and policymakers. Understanding the specific areas of patient dissatisfaction enables the design of targeted interventions aimed at enhancing patient satisfaction and improving service quality. For instance, training programs can be developed to enhance communication skills among junior doctors, and strategies can be implemented to address cost-related concerns raised by patients. Moreover, platforms can leverage physicians’ professional titles to emphasize specific areas of focus during registration or upon each login, offering timely prompts and examples to encourage conscious improvements in these aspects. Additionally, our research offers a framework for continually refining and strengthening mobile health services using patient feedback, enabling dynamic and timely improvements in patient satisfaction.

However, several limitations of this study deserve further exploration in future research. Firstly, this study solely selected data from a single mobile health platform. The unique characteristics of this platform's user base, as well as China's specific cultural factors, may both influence patient satisfaction to some extent, which will make the universality of this study in other populations and cultures to be verified. Secondly, to better construct the STM model, this study balanced the number of positive and negative comments, which might introduce bias and fail to capture important nuances potentially implied in patient feedback. Lastly, when considering the interactions of covariates, this study only took into account the impact of doctors’ professional titles. Yet, prior literature suggests that in China's Confucian culture, patients’ expectations regarding doctors’ gender roles may be more prominent.^52,53 Doctor characteristics like these can be a potential influence, but may be overlooked.

Future research can focus on validating the findings of this study in more different types of mobile health services. Validation in different cultural backgrounds is also worth exploring to improve the applicability of these findings in different contexts. At the same time, future research should aim to further explore more factors that may affect patient complaints and establish a more comprehensive model based on this study.

Conclusion

Our study expands the understanding of negative topics for online patients. Furthermore, we have discovered that patients hold distinct expectations towards doctors with different professional titles. They tend to be more critical of senior doctors in terms of treatment costs and problem-solving capabilities, while being particularly sensitive to the professional knowledge, consultation completeness, and response speed of junior doctors. These findings can provide valuable insights for healthcare providers and policymakers, potentially leading to significant improvements in online patient satisfaction in the future.

Footnotes

Acknowledgements

We thank the reviewers for their careful review and insightful comments.

Contributorship

JL, HJ, and PD: conceptualization; HJ: methodology; HJ: software; HJ: validation; HJ: formal analysis; HJ: investigation; HJ: resources; HJ: data curation; HJ and JL: writing–original draft preparation; HJ, PD, and JL: writing–review and editing; HJ and PD: visualization. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Guarantor

Informed consent and ethical considerations

This study did not involve human participants and did not include or analyze any individual patient data, so patient consent was not required. During the ethical review process, the ethics committee reviewed the matters related to the informed consent form for the subjects. The ethics committee agreed with our explanation, and the research team obtained ethical approval (approval number: ECSHU 2023-114).

ORCID iD

Huihong Jiang

References

Mehraeen

Safdari

SeyedAlinaghi

, et al. A mobile-based self-management application – usability evaluation from the perspective of HIV-positive people – ScienceDirect. Health Policy Technol 2020; 9: 294–301.

Mehraeen

Noori

Nazeri

, et al. Identifying features of a mobile-based application for self-care of people living with T2DM. Diabetes Res Clin Pract 2021; 171: 108544.

Mehraeen

Safdari

Mohammadzadeh

, et al. Mobile-based applications and functionalities for self-management of people living with HIV. Stud Health Technol Inform 2018; 248: 172–179.

Gao

Kou

. Research on the identification of medical service quality factors: based on a data-driven method. INTR 2022; 32: 1617–1645.

Alam

Banwell

Olsen

, et al. Patients’ and doctors’ perceptions of a mobile phone-based consultation service for maternal, neonatal, and infant health care in Bangladesh: a mixed-methods study. JMIR Mhealth Uhealth 2019; 7: e11842.

Nemec

Kolar

Rusjan

. Online communities as a new source of exploring patient dissatisfaction. JHOM 2018; 32: 962–979.

Pang

PC-I

Liu

. Why do consumers review doctors online? Topic modeling analysis of positive and negative reviews on an online health community in China. 2020 Epub ahead of print 2020. DOI: https://doi.org/10.24251/HICSS.2020.087.

Suler

. The online disinhibition effect. Cyberpsychol Behav 2004; 7: 321–326.

Khansa

Pearson

. Patient satisfaction after rhinoplasty: a social media analysis. Aesthet Surg J 2016; 36: NP1–NP5.

10.

Patel

Cain

Neailey

, et al. General practitioners’ concerns about online patient feedback: findings from a descriptive exploratory qualitative study in England. J Med Internet Res 2015; 17: e276.

11.

Guo

Fang

, et al. How doctors gain social and economic returns in online health-care communities: a professional capital perspective. J Manage Inf Syst 2017; 34: 487–519.

12.

Liu

Zhang

Gao

, et al. Physician voice characteristics and patient satisfaction in online health consultation. Inf Manage 2020; 57: 103233.

13.

Nisbett

Wilson

. The halo effect: evidence for unconscious alteration of judgments. J Pers Soc Psychol 1977; 35: 250–256.

14.

Korfiatis

Stamolampros

Kourouthanassis

, et al. Measuring service quality from unstructured data: a topic modeling application on airline passengers’ online reviews. Expert Syst Appl 2019; 116: 472–486.

15.

Ouyang

Wang

J-J

. Physician’s online image and patient’s choice in the online health community. INTR 2022; 32: 1952–1977.

16.

Huang

Duan

Yang

, et al. Online selection of a physician by patients: the impression formation perspective. BMC Med Inform Decis Mak 2022; 22: 193.

17.

Patel

Cain

Neailey

, et al. Exploring patients’ views toward giving web-based feedback and ratings to general practitioners in England: a qualitative descriptive study. J Med Internet Res 2016; 18: e217.

18.

Nathan

Mascarenhas

. Analysis of formal complaints in 1840 consecutive primary breast augmentations. Plast Reconstr Surg 2013; 132: 883e–884e.

19.

Zhang

Deng

Hong

, et al. Unhappy patients are not alike: content analysis of the negative comments from China’s good doctor website. J Med Internet Res 2018; 20: e35.

20.

Hao

Zhang

Wang

, et al. A tale of two countries: international comparison of online doctor reviews between China and the United States. Int J Med Inf 2017; 99: 37–44.

21.

Stokes

Kishton

McCalpin

, et al. Online reviews of mental health treatment facilities: narrative themes associated with positive and negative ratings. PS (Wash DC) 2021; 72: 776–783.

22.

Shah

Yan

Tariq

, et al. What patients like or dislike in physicians: analyzing drivers of patient satisfaction and dissatisfaction using a digital topic modeling approach. Inf Process Manag 2021; 58: 102516.

23.

Bai

Han

, et al. Exploring thematic influences on theme park visitors’ satisfaction: an empirical study on Disneyland China. J Consum Behav 2023; 23: 90–106.

24.

Ding

Choo

, et al. Employing structural topic modelling to explore perceived service quality attributes in Airbnb accommodation. Int J Hosp Manag 2020; 91: 102676.

25.

Han

Zhou

, et al. The voice of drug consumers: online textual review analysis using structural topic model. Int J Environ Res Public Health 2020; 17: 3648.

26.

Shi

Chen

. Listening to your employees: analyzing opinions from online reviews of hotel companies. Int J Contemp Hosp Manag 2021; 33: 2091–2116.

27.

Blei

. Probabilistic topic models. Commun ACM 2012; 55: 77–84.

28.

Roberts

Stewart

Tingley

, et al. Structural topic models for open-ended survey responses. Am J Pol Sci 2014; 58: 1064–1082.

29.

Chen

Z-S

Yang

, et al. A textual data-oriented method for doctor selection in online health communities. Sustainability 2023; 15: 1241.

30.

Zhang

Gao

, et al. What do hotel customers complain about? Text analysis using structural topic model. Tour Manag 2019; 72: 417–426.

31.

Zhang

Pavlou

. Overcoming the J-shaped distribution of product reviews. Commun ACM 2009; 52: 144–147.

32.

Tian

Yin

Gan

. Exploring public attitudes toward live-streaming fitness in China: a sentiment and content analysis of China’s social media Weibo. Front Public Health 2022; 10: 1027694.

33.

Liu

, et al. Developing embedded taxonomy and mining patients’ interests from web-based physician reviews: mixed-methods approach. J Med Internet Res 2018; 20: e254.

34.

Hao

Zhang

. The voice of Chinese health consumers: a text mining approach to web-based physician reviews. J Med Internet Res 2016; 18: e108.

35.

Wei

Hsu

Y-T

. Extracting additional influences from physician profiles with topic modeling: impact on ratings and page views in online healthcare communities. Front Psychol 2022; 13: 830841.

36.

Lester

Wang

Vydiswaran

VGV

. Describing the patient experience from Yelp reviews of community pharmacies. J Am Pharm Assoc 2019; 59: 349–355.

37.

Yin

Tian

, et al. Exploring the factors associated with mental health attitude in China: a structural topic modeling approach. Int J Environ Res Public Health 2022; 19: 12579.

38.

Espinel

Shah

Beach

, et al. What parents say about their child’s surgeon: parent-reported experiences with pediatric surgical physicians. JAMA Otolaryngol Head Neck Surg 2014; 140: 397.

39.

Agarwal

Mahoney

Lanza

, et al. Online ratings of the patient experience: emergency departments versus urgent care centers. Ann Emerg Med 2019; 73: 631–638.

40.

Hakimi

Yasmeh

Foulad

. Assessing patient satisfaction among ABHRS surgeons: opportunities to improve. J Cosmet Dermatol 2020; 19: 462–465.

41.

Wan

Peng

Wang

, et al. Influencing factors and mechanism of doctor consultation volume on online medical consultation platforms based on physician review analysis. INTR 2021; 31: 2055–2075.

42.

Kuhn

. Using structural topic modeling to identify latent topics and trends in aviation incident reports. Transp Res C, Emerg Technol 2018; 87: 105–122.

43.

Rahim

AIA

Ibrahim

Musa

, et al. Patient satisfaction and hospital quality of care evaluation in Malaysia using SERVQUAL and Facebook. Healthcare 2021; 9: 1369.

44.

Atanasova

Kamin

Petrič

. The benefits and challenges of online professional-patient interaction: comparing views between users and health professional moderators in an online health community. Comput Human Behav 2018; 83: 106–118.

45.

Briet

Hageman

Blok

, et al.

When do patients with hand illness seek online health consultations and what do they ask?

Clin Orthop Relat Res 2014; 472: 1246–1250.

46.

Islam

MRR

Islam

Ferdous

, et al. Teledentistry as an effective tool for the communication improvement between dentists and patients: an overview. Healthcare 2022; 10: 1586.

47.

Brookes

Baker

. What does patient feedback reveal about the NHS? A mixed methods study of comments posted to the NHS Choices online service. BMJ Open 2017; 7: e013821.

48.

Deng

, et al. How professional capital and team heterogeneity affect the demands of online team-based medical service. BMC Med Inform Decis Mak 2019; 19: 119.

49.

Mai

Shan

, et al. Operational efficiency and patient-centered health care: a view from online physician reviews. J Ops Management 2019; 65: 353–379.

50.

Yang

Guo

. Exploring the influence of the online physician service delivery process on patient satisfaction. Decis Support Syst 2015; 78: 113–121.

51.

Yang

Zhang

Lee

PKC

. Improving the effectiveness of online healthcare platforms: an empirical study with multi-period patient-doctor consultation data. Int J Prod Econ 2019; 207: 70–80.

52.

Schmid Mast

Hall

Roter

. Disentangling physician sex and physician communication style: their effects on patient satisfaction in a virtual medical visit. Patient Educ Couns 2007; 68: 16–22.

53.

Utych

. Speaking style and candidate evaluations. Polit Groups Identit 2021; 9: 589–607.

Exploring sources of patient dissatisfaction in mobile health communication: A text analysis based on structural topic model

Abstract

Keywords

Introduction

Literature review

Research related to patient dissatisfaction

Structural topic model

Data and model settings

Data collection and pre-processing

Model setup and selection

Results

Topic summary and identification

Negative topic analysis

The moderating role of doctor titles

Discussion

Conclusion

Footnotes

Acknowledgements

Contributorship

Declaration of conflicting interests

Funding

Guarantor

Informed consent and ethical considerations

ORCID iD

References