Tracking subjective symptom improvement from patient narratives in mobile health: An observational natural language processing study

Abstract

Background

Subjective symptom monitoring is central to patient-centered care but often relies on burdensome surveys prone to recall bias. Mobile health (mHealth) platforms increasingly collect user-generated reviews that may provide real-time insights into patient experiences. However, it remains unclear whether such unsolicited narratives can serve as valid indicators of perceived health outcomes, particularly in non-Anglophone contexts.

Objective

This study examines whether linguistic features from Chinese-language mHealth reviews can be used to identify signals related to patient satisfaction and perceived symptom improvement.

Methods

An observational study was conducted using 6,362 publicly available user-generated reviews from the WeDoctor mHealth platform. A natural language processing pipeline extracted sentiment polarity, a keyword-derived perceived improvement indicator, and TF-IDF features. Sentiment was analyzed using linear regression to predict satisfaction, while logistic regression and Random Forest models were used to identify reviews containing improvement-related expressions.

Results

Sentiment polarity significantly predicted satisfaction (β=0.351, p<0.001). Approximately 29% of reviews contained improvement-related expressions. The TF-IDF model achieved strong classification performance (F1-score = 0.945), with Random Forest showing slightly improved performance (F1-score = 0.953).

Conclusion

Patient narratives contain emotional and functional signals that support real-time, low-burden monitoring of satisfaction and perceived improvement, complementing traditional survey-based outcome measures.

Introduction

Digital technologies are transforming how healthcare systems capture, interpret, and respond to patient experiences. Beyond traditional clinical indicators, there is increasing recognition that subjective patient-reported information, including perceptions of symptom change, emotional responses, and treatment experiences, plays a central role in evaluating healthcare effectiveness. However, conventional approaches to collecting such information often rely on structured surveys or periodic assessments, which may fail to capture real-time variations in patient conditions and experiences. As healthcare delivery becomes increasingly digitized, new forms of patient-generated data are emerging that capture lived experiences in more continuous and naturalistic ways. Narrative expressions generated within digital environments provide rich, context-sensitive insights into how patients interpret their health trajectories and care experiences. These developments raise important questions about whether such unstructured data can be systematically analyzed to support scalable and low-burden outcome monitoring.

Building on these developments, the rapid growth of mobile health (mHealth) platforms has revolutionized patient engagement with healthcare services, enabling continuous interaction through remote consultations, digital prescriptions, and asynchronous symptom monitoring.^1,2 Prominent platforms such as PingAn Good Doctor, WeDoctor, Amwell, and Teladoc now offer scalable digital care solutions that extend beyond traditional clinical environments.^3,4 As part of this shift, user-generated reviews and narrative feedback have become a routine part of mHealth ecosystems, often reflecting subjective changes in symptoms, emotional responses, and satisfaction with treatment or care encounters. Unlike structured surveys or follow-up assessments, these textual inputs are generated spontaneously and in real-time, making them a rich yet underutilized source of longitudinal, patient-reported data. ⁵

Current approaches to outcome tracking in digital health still rely heavily on retrospective questionnaires or scheduled symptom check-ins, which can suffer from response fatigue, recall bias, and limited scalability, particularly in resource-constrained or high-volume settings.^3,6 These challenges are even more pronounced in regions with lower digital health infrastructure, as highlighted in comparative reviews of mHealth use in Africa and Europe. ⁷

To address this, we draw on the Service-Dominant Logic (SDL) framework,^8,9 which conceptualizes healthcare value as co-created through patient-system interactions, rather than solely measured by outcomes. From this perspective, patient feedback serves not just as a passive evaluation, but as a dynamic co-production of meaning, encompassing both emotional and functional interpretations of care quality. Linguistic features embedded in free-text reviews, such as sentiment polarity, emotion tone, and keywords related to symptom improvement, may thus serve as lightweight, actionable indicators of patient-perceived outcomes, offering a new avenue for low-burden, real-time monitoring.^10–13

Prior research has demonstrated that patient-generated reviews on digital health platforms can provide insights into user satisfaction, usability issues, and overall app evaluation. ¹⁴ However, much of this work has focused primarily on sentiment analysis or rating prediction rather than examining whether narrative feedback can function as an indicator of perceived health outcomes. ¹⁵ In addition, existing studies often rely on English-language datasets or concentrate on mental health applications, leaving non-Anglophone mHealth contexts comparatively underexplored. ¹⁶ As a result, it remains unclear whether unsolicited patient narratives in other linguistic and cultural settings can provide reliable signals of perceived symptom improvement or treatment effectiveness. This study addresses this gap by analyzing Chinese-language mHealth reviews to examine whether narrative text contains detectable emotional and functional signals associated with patient satisfaction and perceived improvement.

In this study, we examine whether unsolicited user feedback from a large Chinese mHealth platform contains detectable signals related to two subjective outcome constructs: satisfaction with care and perceived symptom improvement. Using a dataset of 6,362 user reviews, we apply natural language processing (NLP), sentiment analysis, and machine learning models to identify the emotional and outcome-related signals embedded in text. Our contributions are threefold: (1) we propose a tailored NLP pipeline for analyzing Chinese-language mHealth narratives; (2) we show that emotional and semantic cues in narrative reviews are associated with subjective health experiences and perceived improvement signals; and (3) we offer a practical framework for incorporating such feedback into mHealth systems as a complementary, low-burden source of patient-centered monitoring. This approach aligns with current calls for patient-centered, data-driven digital care systems that minimize response burden while maximizing actionable insight.

Literature review

Theoretical framework: Service-Dominant Logic (SDL)

This study is grounded in Service-Dominant Logic (SDL). This framework redefines healthcare value as a co-created, dynamic process shaped through interactions between patients and services, rather than a static output of clinical intervention.^17,18 In the context of mHealth, this perspective encourages us to view patient feedback as an expression of longitudinal experience and subjective symptom evaluation, rather than merely transactional satisfaction, which typically reflects a one-time evaluative judgement of a service encounter. In contrast, user-generated narratives in mHealth environments capture ongoing experiential value, including symptom progression, emotional responses, and perceived treatment effectiveness over time. ¹⁹

In this study, SDL is an interpretive framework for understanding the variables extracted from patient narratives. Within this perspective, sentiment polarity is treated as an indicator of emotional value, as it reflects the affective tone through which patients evaluate their care experiences. Improvement-related expressions in the narrative text are interpreted as indicators of functional value, capturing perceived symptom relief or treatment benefit resulting from healthcare interactions. Satisfaction ratings represent perceived outcomes, reflecting the patient’s overall judgment of the service encounter. This operational mapping clarifies how narrative expressions in mHealth reviews may encode multiple dimensions of value-in-use within digital healthcare interactions.

By treating reviews as free-flowing, context-rich expressions of lived health experiences, SDL enables us to reinterpret these data as part of a broader feedback loop in which patients shape care pathways through continuous digital interaction. Our study employs sentiment analysis, emotion detection, and outcome-related keyword identification to extract such signals from naturally occurring narrative data. This extends SDL into the domain of digital health informatics, offering a theoretical justification for using subjective, patient-reported data as input into dynamic outcome tracking systems. ²⁰

Textual feedback in mHealth

Patient-generated feedback has been increasingly used to examine user experience, sentiment, and outcome-related signals in mHealth environments. Prior research has explored these dimensions through sentiment analysis, topic modeling, and predictive modeling approaches.

The potential of patient-generated text, especially symptom narratives, for monitoring health outcomes is gaining scholarly interest. ²¹ Prior research has utilized sentiment and topic modeling to examine themes in app reviews, including usability and trust, but often stops at surface-level descriptions.^22,23 Few studies operationalize these narratives as predictive signals of subjective symptom trajectories, despite their regular use in real-world contexts where structured follow-ups are infeasible. Most applications of NLP in healthcare focus on structured EHRs or post-hoc surveys, with limited emphasis on capturing real-time, self-initiated expressions of health states.^24–26

Recent methodological advances highlight the importance of handling high-dimensional data in predictive modeling, particularly through feature selection and machine learning techniques that improve classification performance. In parallel, research in health outcome modeling shows that machine learning can outperform traditional statistical approaches in forecasting disease trends and patient outcomes. However, these approaches are largely applied to structured or time-series data rather than unstructured, patient-generated narratives. As a result, the applicability of such methods to narrative-based outcome monitoring in mHealth environments remains underexplored.^27,28

Some notable exceptions, particularly in mental health, have explored text as a proxy for symptom progression, ²⁹ supporting the hypothesis that free-text entries may function as longitudinal, patient-reported data streams. However, studies using Chinese-language physical health data remain scarce. Our work fills this gap by analyzing whether unstructured reviews reflect not only satisfaction but also subjective perceptions of symptom improvement, thereby enhancing both real-world outcome tracking and linguistic inclusivity. ³⁰

Sentiment analysis, emotion mining, and outcome prediction in mHealth

Sentiment analysis and emotion mining offer critical tools for unpacking subjective health experiences embedded in mHealth narratives.^31–33 These tools go beyond satisfaction ratings by uncovering the emotional inflections, relief, frustration, and hope that often accompany symptom descriptions and perceived treatment efficacy. While many studies have employed these methods retrospectively to enhance app design or triage complaints, ³⁴ few have implemented them in predictive frameworks for monitoring changes in symptom perception or emotional well-being. ³⁵

This gap is especially pronounced in Chinese-language platforms, where linguistic nuances shape how patients report symptom improvement or distress. Our study addresses this by examining whether affective signals and semantic markers in review text can help identify perceived outcome-related patterns in patient narratives. This responds directly to the special issue’s call for longitudinal symptom tracking using computational methods tailored to subjective and context-sensitive data sources. ³⁶

A major limitation in current mHealth analytics is the lack of actionable insight into perceived health changes. While user engagement and satisfaction are frequently modeled, few studies leverage textual indicators to track subjective outcomes over time.^37–39 Even where feedback is available, the distinction between dissatisfaction with service delivery and worsening symptoms is often blurred. ⁴⁰

Our contribution lies in proposing a replicable, low-burden NLP framework that converts spontaneous user narratives into signals for self-reported improvement, aligning with a longitudinal and participatory model of health tracking. Unlike traditional patient-reported outcome measures requiring structured surveys, this approach explores whether symptom-related perceptions may be inferred from language use, capturing emotional and experiential shifts as they unfold in natural language. This positions user reviews as a rich source of subjective and temporally anchored symptom data.

Methodology

Study design and conceptual framework

This study employed a quantitative, data-driven design that integrated natural language processing (NLP), supervised machine learning, and statistical modeling to examine whether patient-generated textual reviews on a Chinese mobile health (mHealth) platform can serve as scalable text-driven indicators related to patient-reported outcomes. ⁴¹ Informed by the SDL framework, we framed patient feedback as a form of co-created healthcare value, emphasizing that these narratives are not ancillary but central to understanding digital health engagement and perceptions of outcomes. Linguistic features extracted from reviews, including sentiment polarity, emotional tone, and outcome-relevant keywords, were treated as signals of emotional value and perceived health improvement. These features were examined within regression and classification analyses to assess whether patient narratives contain distinct emotional and functional signals relevant to mHealth evaluation.

Drawing on SDL, this study conceptualizes patient narratives as expressions of value-in-use within digital healthcare interactions. Specifically, the framework distinguishes between emotional value, captured through sentiment polarity, and functional value, captured through improvement-related expressions in review text. The text-derived dimensions are theorized to shape perceived outcomes, operationalized here as satisfaction ratings and perceived improvement. The conceptual framework presented in Figure 1 illustrates these relationships and informs the subsequent modeling strategy.

Figure 1.

Conceptual model of text-derived patient value in mHealth narratives. Patient narratives express emotional value through sentiment polarity and functional value through improvement-related expressions. These two dimensions contribute to perceived outcomes, including satisfaction ratings and perceived improvement.

Guided by this conceptual framework, the empirical analysis focuses on extracting linguistic indicators of emotional and functional value from patient narratives and examining their relationships with perceived outcomes.

Data source and collection

We extracted publicly available user-generated reviews from WeDoctor, a widely used Chinese mHealth platform that offers online medical consultations, prescription support, and symptom self-reporting. Data were collected over six months, from May to November 2025, using automated web crawlers designed to comply with ethical data collection standards. Each review in the dataset contained four core fields: the free-text comment (评论内容; review content), a numerical satisfaction rating on a five-point scale (评论打分; review rating), the self-reported symptom description (评论症状; symptom description), and system- or user-generated tags (评论标签; review tags). After conducting quality checks and removing incomplete or duplicate entries, a final corpus of 6,362 reviews was retained for analysis. The dataset spans multiple clinical departments and user geographies, enhancing its representativeness across diverse patient populations and health concerns. All reviews were collected from publicly accessible pages of the platform, and no login-restricted, private, or user-identifiable content was accessed.

Text processing and cleaning

Given the structural features of written Chinese, preprocessing was tailored for short-form Chinese-language text. We removed punctuation, non-Chinese characters, and redundant whitespace from each review. The cleaned text was then tokenized using the Jieba tokenizer, which segments continuous Chinese character strings into discrete lexical units necessary for linguistic analysis. Stopword filtering was optionally applied to eliminate semantically neutral tokens that may dilute the strength of sentiment or topical signals. The output of this pipeline, a cleaned and tokenized corpus, was used as input for feature extraction and model development.

Feature engineering

We derived three main categories of linguistic features to serve as predictors in our models. First, sentiment polarity scores were computed using the SnowNLP toolkit, a sentiment analysis library optimized for Chinese-language social text. ⁴² Each review was assigned a score ranging from 0 (entirely negative) to 1 (entirely positive), representing the emotional tone embedded in the narrative.

Second, a keyword-derived perceived improvement indicator was generated based on the presence of predefined expressions associated with symptom relief and recovery. Expressions such as “好转” (improved), “缓解” (relief), and “不疼了” (no longer in pain/pain-free) were used to infer whether the user perceived a positive health change. These improvement-related expressions were identified using a predefined lexical list reflecting commonly reported recovery or symptom-relief experiences in patient reviews. This rule-based approach was adopted to provide an initial exploratory indicator of perceived improvement within the narrative text.

Third, Term Frequency–Inverse Document Frequency (TF-IDF) vectorization was applied to the cleaned text to capture word-level importance across the corpus. TF-IDF is a widely used text representation technique in natural language processing that assigns higher weight to words that appear frequently within a given document but less frequently across the overall dataset. This approach helps identify terms that are particularly informative for distinguishing one review from another. Each review was transformed into a sparse vector of weighted terms, and the top 1000 most informative features were retained based on information gain. These vectors served as semantic inputs to our classification models while minimizing overfitting.

Outcome variables

Two outcome variables were defined to reflect distinct yet complementary dimensions of patient-reported experience. The first was the satisfaction score, a continuous variable ranging from 1 (least satisfied) to 5 (most satisfied), which was used as the dependent variable in the regression model. The second outcome-related variable was a keyword-derived perceived improvement indicator, defined by the presence of predefined expressions associated with symptom relief or recovery. Reviews containing such expressions were coded as 1, while all others were coded as 0. This indicator was generated using a rule-based keyword approach rather than manual annotation. Consequently, no manual labeling, multiple annotators, or inter-rater reliability measures were involved in the construction of this variable. Because this variable was derived heuristically from review text rather than from annotated or clinically validated outcomes, it is interpreted as an exploratory proxy rather than a validated outcome variable.

Modeling and analysis

To assess the predictive value of text-based features on user satisfaction, we applied Ordinary Least Squares (OLS) regression using sentiment polarity and the keyword-derived perceived improvement indicator as independent variables. Model performance was evaluated using standard metrics, including R-squared values, standardized beta coefficients, and p-values. For the classification task, we trained logistic regression models to identify reviews containing the keyword-derived perceived improvement indicator using TF-IDF vectors.

For exploratory analysis, we also examined a hybrid specification combining TF-IDF features with both sentiment polarity and the keyword-derived perceived improvement indicator. Because the improvement indicator was derived directly from narrative review text using predefined lexical patterns, no externally annotated dataset was available for independent validation. Consequently, the classification analysis was interpreted as exploratory and intended to demonstrate the feasibility of detecting outcome-related signals in patient narratives rather than an independent predictive model. Evaluation metrics for both models included accuracy, precision, recall, F1-score, and the area under the ROC curve (AUC).

To ensure model stability and generalizability, we performed five-fold stratified cross-validation during training and used an 80/20 train-test split for final validation. Results were interpreted using confusion matrices, Receiver Operating Characteristic (ROC) plots, and heatmaps to visualize model performance and misclassification trends.

Implementation environment

All analysis was conducted using Python within a JupyterLab environment. Data preprocessing and manipulation were performed using the pandas and NumPy libraries. Jieba was used for Chinese word segmentation, while SnowNLP was employed for sentiment scoring. TF-IDF vectorization and all machine learning models were implemented using the Scikit-learn library. Regression diagnostics were conducted with Statsmodels, and all visualizations, including ROC plots and heatmaps, were generated using Matplotlib and Seaborn. All scripts were version-controlled using Git to support transparency and reproducibility.

Ethical considerations

This study analyzed user-generated reviews from WeDoctor, a publicly accessible Chinese mHealth platform. All content used in the study was publicly available at the time of data collection. No identifiable personal information was collected, and the research did not involve interaction with human participants or access to private data. Because the study used publicly available anonymized secondary data, individual consent was not required. All data were accessed in accordance with the platform’s terms of service, and the analysis followed responsible research practices for the use of user-generated online content.

Results

Descriptive and correlation analyses

We began by exploring the relationships among the core textual and evaluative variables: sentiment polarity, keyword-derived perceived improvement indicator, and patient satisfaction ratings. As shown in Table 1, sentiment polarity exhibited a moderate positive correlation with satisfaction ratings (r = 0.35), indicating that emotionally positive reviews are generally associated with higher star ratings. This supports the SDL view that emotional feedback forms a core part of co-created experiential value in digital health settings. By contrast, the correlation between the keyword-derived perceived improvement indicator and satisfaction ratings was low (r = 0.04), indicating that user-reported clinical improvement is conceptually and statistically distinct from evaluative satisfaction. Sentiment polarity and improvement-related expressions were themselves only weakly correlated (r = 0.14), reinforcing their independence as affective and functional dimensions of mHealth user feedback.

Table 1.

Correlation matrix among variables.

​	Sentiment score	Improvement indicator	Ratings
Sentiment score	1.00	0.14	0.35
Improvement indicator	0.14	1.00	0.04
Ratings	0.35	0.04	1.00

The visualizations presented in Figures 2 and 3 further support these distinctions. Approximately 29% of the reviews explicitly referenced clinical improvement (Figure 2), supporting the usefulness of the keyword-derived perceived improvement indicator as an exploratory descriptive variable. The boxplot in Figure 3 illustrates a clear positive trend in sentiment scores across increasing satisfaction ratings, with 4- and 5-star reviews clustering near the upper end of the sentiment spectrum. Figure 4 presents a correlation heatmap, confirming that sentiment polarity is the most strongly associated predictor of satisfaction. At the same time, the keyword-derived perceived improvement indicator serves as an independent signal more relevant to perceived health change than to service evaluation.

Figure 2.

Distribution of improvement-related keywords in patient reviews. This bar chart illustrates the frequency of improvement-related keywords within the patient feedback dataset. The x-axis shows the keyword-derived perceived improvement indicator (No improvement vs. Improvement keywords), and the y-axis indicates the number of comments in each group. Approximately 1900 reviews contain improvement expressions, while over 4400 do not, supporting its use as an exploratory binary indicator in classification modeling.

Figure 3.

Sentiment score distribution across patient satisfaction ratings. This boxplot displays sentiment polarity scores organized by satisfaction ratings, ranging from 1 to 5 stars. Each box represents the interquartile range (IQR), with the line inside indicating the median, and the circles represent outliers. Higher satisfaction ratings (4 and 5) are associated with higher median sentiment scores (above 0.9), while lower ratings (1 and 2) exhibit lower and more variable sentiment scores.

Figure 4.

Correlation heatmap of core variables. This heatmap shows the Pearson correlation coefficient among the key variables: sentiment score, keyword-derived perceived improvement indicator, and satisfaction ratings. Cell color intensity reflects the strength of correlation, with red indicating strong positive relationships and blue indicating weak negative correlations. The sentiment-rating correlation is moderate (r = 0.35), while the improvement indicator shows a low correlation with the rating.

Regression modeling

OLS regression results (Table 2) show that sentiment polarity is a statistically significant predictor of patient satisfaction ratings (β = 0.351, p < .001), with a 95% confidence interval ([0.328, 0.374]), confirming the reliability of emotional tone as a proxy for perceived service quality. In contrast, the presence of improvement-related keywords was not a significant predictor (β = -0.011, p = .30), reinforcing the earlier observation that clinical progress and service satisfaction are perceived and expressed separately. The model accounted for 12.2% of the variance in satisfaction ratings (R² = 0.122), indicating that sentiment polarity captures only a limited component of how patients evaluate mHealth services. Although sentiment expressed in narrative reviews is associated with satisfaction ratings, a substantial proportion of variation remains unexplained by these predictors.

Table 2.

Regression results.

Relationships	β (standardized)	Std. Error	p-value	CI
Sentiment scores→satisfaction	0.351	0.012	***	[0.328, 0.374]
Improvement indicator→Satisfaction	-0.011	0.012	0.30 (NS)	[-0.034, 0.012]

Note. Standardized coefficients (β) are based on z-scores of transformed variables to allow magnitude comparison. Confidence intervals computed using 1000 bootstrapped samples. *** p < 0.001; NS not significant.

Figure 5 visualizes the model’s fit, showing more substantial predictive alignment for high satisfaction scores and greater dispersion for lower ratings, suggesting that negative experiences may involve factors not captured by sentiment alone, such as platform delays or clinical misalignment.

Figure 5.

Regression model: actual versus predicted satisfaction ratings. This scatter plot compares actual satisfaction scores (x-axis) to those predicted by the OLS regression model (y-axis). The red dashed diagonal line indicates the ideal prediction, where the actual equals the expected value. Clustering points around this line, especially in mid-to-high ranges, indicates a good model fit.

Classification modeling

To examine whether text-derived features could identify reviews containing the keyword-derived perceived improvement indicator, we developed logistic regression models using TF-IDF vectorized review content. In an additional specification, sentiment polarity was included alongside the TF-IDF features to assess whether incorporating emotional signals improved detection of improvement-related language in the narratives. The TF-IDF model (Table 3) achieved strong performance, with an F1-score of 0.945 for the “Improved” class and 0.98 for the “Not Improved” class. The overall accuracy was 97%, and the model achieved an AUC of 0.99 (Figure 6), indicating excellent discriminative ability. The confusion matrix (Figure 7) confirms the model’s effectiveness, with very few misclassifications.

Table 3.

Classification modeling report.

​	Precision	Recall	F1-score	Support
TF-IDF model
0	0.97	1.00	0.98	905.00
1	0.99	0.91	0.95	368.00
Accuracy	0.97	0.97	0.97	0.97
Macro avg	0.98	0.96	0.97	1273.00
Weighted avg	0.97	0.97	0.97	1273.00
Hybrid-Model (continued)
0	1.00	1.00	1.00	905.00
1	1.00	1.00	1.00	368.00
Accuracy	1.00	1.00	1.00	1.00
Macro avg	1.00	1.00	1.00	1273.00
Weighted avg	1.00	1.00	1.00	1273.00

Performance comparison of logistic regression and Random Forest classificiers
Model	Accuracy	Precision	Recall	F1-score
Logistic regression (TF-IDF)	0.969	0.994	0.900	0.945
Random Forest (TF-IDF)	0.974	1.000	0.911	0.953

Figure 6.

ROC Curve of TF-IDF model for outcome prediction. The receiver operating characteristic curve displays the trade-off between true positive and false positive rates for the TF-IDF-only model. The blue curve represents the model’s performance, and the dotted diagonal line indicates random guessing. The Area Under the Curve (AUC) is 0.99, indicating excellent discriminative ability.

Figure 7.

Confusion matrix for TF-IDF-Based logistic regression classification. This matrix shows the performance of the TF-IDF-only logistic regression model in identifying reviews containing the keyword-derived perceived improvement indicator. True labels are on the y-axis and predicted labels on the x-axis. Class labels include “Improved” and “Not Improved.” The color gradient reflects frequency, with most predictions correctly classified. Minor false positives and negatives are present.

The hybrid specification combining TF-IDF, sentiment polarity, and the keyword-derived improvement indicator produced near-complete separation in the classification task. However, because the keyword-derived perceived improvement indicator itself was derived from review text using predefined lexical patterns, this result likely reflects lexical overlap rather than independent predictive capability. Consequently, these results should be interpreted as exploratory evidence of alignment between improvement-related language and the extracted text features. Figures 8 and 9 show complete separation in the confusion matrix and the ROC curve, respectively.

Figure 8.

Confusion matrix for the hybrid model combining sentiment, keywords, and TF-IDF. This confusion matrix illustrates the classification performance of the hybrid model, which integrates sentiment scores, the keyword-derived perceived improvement indicator, and TF-IDF features. All 1273 predictions match the true labels, showing perfect internal separation. The deep purple shading indicates high confidence and accuracy in both “Improved” and “Not Improved” classes.

Figure 9.

ROC curve for hybrid model classification performance. This ROC curve illustrates the predictive performance of the hybrid model. The curve hugs the top-left corner, and the AUC value reaches 1.00, indicating complete separation within the dataset. The black dashed diagonal line again serves as the baseline for random performance.

To benchmark the linear classifier against a non-linear approach, a Random Forest was trained using the same TF-IDF feature set. The Random Forest classifier achieved an accuracy of 0.974, precision of 1.000, recall of 0.911, and F1-score of 0.953. Compared with the logistic regression model (accuracy = 0.969, F1-score = 0.945), the Random Forest model showed a modest improvement in performance. The confusion matrix indicated that the Random Forest model eliminated false-positive predictions and slightly reduced false negatives. Overall, the results suggest that while some non-linear interactions may exist in the narrative text, the interpretable logistic regression baseline already captures most of the predictive signal present in the TF-IDF features.

Model robustness: Cross-validation

We conducted a 5-fold stratified cross-validation to test the generalizability of both classification models. As shown in Table 4, the TF-IDF model maintained a mean F1-score of 0.94 (SD = 0.04) across folds, indicating stable performance on unseen data. The hybrid model again showed perfect internal seperation, with an F1-score of 1.00 and zero variance across folds. While these results suggest strong internal robustness, they also call for caution. The consistent performance likely reflects lexical overlap between the keyword-derived indicator and the text features used in the model.

Table 4.

Cross-validation results.

​	Fold	TF-IDF F1	Hybrid F1
0	1	0.96	1.00
1	2	0.97	1.00
2	3	0.86	1.00
3	4	0.94	1.00
4	5	0.96	1.00
Mean	-	0.94	1.00
Standard deviation	-	0.04	0.00

Taken together, the results highlight two core findings: (1) sentiment polarity is a reliable predictor of patient satisfaction, supporting its use for real-time experiential tracking; and (2) improvement-related language is useful for identifying the keyword-derived perceived improvement indicator, particularly when interpreted alongside broader semantic review content. Notably, the divergence between affective (sentiment) and clinical (improvement) signals supports the value of modeling both as independent, complementary pathways in digital health evaluation. These findings underscore the feasibility of leveraging patient-generated text for passive, scalable, and patient-centered outcome monitoring in mobile health environments.

Discussion

This study demonstrates the feasibility and value of leveraging patient-generated textual feedback as a source of patient-reported outcome signals in mobile health platforms. By applying natural language processing to unsolicited reviews, the findings show that linguistic features encode distinct yet complementary dimensions of patient experience, emotional satisfaction, and perceived health improvement, which align closely with the aims of patient-centered digital health research. The results provide empirical support for the use of passive, real-time patient narratives as scalable complementary signals for outcome monitoring in mHealth systems, particularly in contexts where structured longitudinal reporting is limited or burdensome.

First, the regression results confirm that sentiment polarity is a statistically significant predictor of patient satisfaction. This finding is consistent with prior digital health and platform research, which demonstrates that emotionally positive language aligns with higher satisfaction ratings and perceived service quality.^43,44 Notably, this study extends earlier descriptive work by showing that sentiment polarity shows predictive utility, rather than merely correlational or explanatory value. Within the context of mHealth, where user engagement and sustained use are closely tied to perceived experience, sentiment-based signals offer a practical mechanism for continuously monitoring patient perceptions without requiring additional surveys. From a Service-Dominant Logic (SDL) perspective, these emotional expressions represent value-in-use, reflecting how patients interpret and co-create healthcare value through their interactions with digital services. ⁴⁵ Sentiment polarity, a form of subjective patient-reported data, emerges as a real-time experiential indicator that can supplement conventional satisfaction measures.

The modest explanatory power of the regression model suggests that patient satisfaction in mHealth environments is influenced by multiple factors beyond the emotional tone expressed in narrative reviews. Potential contributors include waiting time for consultations, physician communication behavior, perceived treatment effectiveness, platform usability, and consultation cost, which were not directly available in the review dataset.

Second, the classification results highlight that improvement-related language serves as a strong indicator of the keyword-derived perceived improvement indicator used in the exploratory study, even though it does not significantly influence satisfaction ratings. The high performance of the TF-IDF model (F1 = 0.95) and the near-complete separation observed in the hybrid model suggests substantial lexical overlap between the constructed indicator and the extracted text features. These findings are consistent with emerging evidence from digital mental health research, which suggests that symptom-related language can reflect patient-reported outcomes. ⁴⁶ This study extends the existing literature by demonstrating similar exploratory classification potential in Chinese-language physical health contexts. This distinction between satisfaction and perceived improvement is particularly relevant for patient-reported outcome research, as satisfaction may reflect the quality of service delivery, whereas improvement language more directly captures changes in symptoms or treatment effectiveness. SDL provides a useful interpretive lens here: emotional satisfaction reflects affective value co-creation, whereas improvement-related expressions reflect functional value tied to health outcomes. ⁴⁷ Together, these results underscore the importance of modeling patient experience as multidimensional, rather than relying solely on single indicators.

Third, the observed divergence between star ratings and improvement-related language underscores the limitations of traditional evaluative metrics in mHealth. While satisfaction ratings align closely with emotional tone, they do not reliably capture the perceived change in health. This finding supports prior critiques of relying solely on numeric ratings or cross-sectional surveys to assess the effectiveness of digital health.^46,48,49 By demonstrating that unstructured feedback can capture both experiential and outcome-related dimensions, this study contributes to the growing body of work advocating for richer, patient-centered evaluation frameworks in digital health. Such frameworks are particularly important for longitudinal symptom monitoring, where patients may not continuously report structured outcomes but still communicate meaningful changes through narrative feedback.

Model robustness analyses further support the reliability of text-based outcome prediction. The TF-IDF model demonstrated stable performance across cross-validation folds, which indicates generalizability and supports its suitability within the dataset. While the hybrid model achieved perfect performance, this result should be interpreted cautiously, as similar concerns regarding overfitting and internal labeling have been raised in prior studies. ²¹ Nonetheless, these findings suggest that patient-generated text contains meaningful signals that may support automated monitoring of both satisfaction and perceived outcomes. For the special issue’s focus on methodological considerations in patient-reported longitudinal data, this highlights the potential of passive text mining approaches to complement traditional repeated-measures designs, particularly in reducing responder burden and measurement fatigue.

Theoretical implications

This study contributes to theory in digital health, service research, and computational social science by operationalizing the Service-Dominant Logic (SDL) framework in the context of mobile health (mHealth) platforms. Specifically, it provides empirical evidence that unstructured patient narratives can serve as measurable proxies for co-created value, extending SDL’s core propositions into the domain of real-time digital health analytics. Traditionally, SDL has been applied in service marketing and healthcare design contexts to explore how users contribute meaning and value through their interactions. ⁴³ This study extends beyond conceptual applications ⁵⁰ by quantifying how patient-generated text through features such as sentiment polarity and improvement-related expressions can empirically reflect dimensions of value-in-use.

The finding that sentiment polarity significantly predicts satisfaction demonstrates that emotional tone is not merely incidental or decorative in patient narratives but is a fundamental expression of user-defined value. This insight advances affective computing in healthcare by showing that emotional expressions are directly interpretable as experience metrics, particularly in digital ecosystems where verbal interactions often replace face-to-face communication. By treating satisfaction not as a static metric but as a language-mediated, context-specific construct, this study reframes patient engagement in mHealth as an active, interpretive, and co-creative process. It aligns with calls for theoretically grounded end-to-end frameworks for mHealth intervention design, ⁵¹ particularly in terms of embedding behavior change elements such as feedback, self-monitoring, and digital delivery modes.

Finally, applying SDL to Chinese-language mHealth reviews introduces necessary theoretical expansion into non-Western, linguistically unique healthcare contexts. This challenges the assumption that SDL’s value constructs are culturally universal in form while affirming their conceptual validity across sociotechnical systems. By showing how Chinese patients linguistically encode both affective and functional value in narratives, the study enriches cross-cultural theory-building in digital health. It opens a pathway for comparative SDL studies across languages and geographies.

The comparison between logistic regression and Random Forest models showed only modest performance differences, suggesting that the primary classification signal is largely driven by lexical patterns in the review text. Although non-linear ensemble models may capture additional feature interactions, the results indicate that the interpretable logistic regression baseline already provides a robust representation of the narrative signals associated with perceived improvement.

Practical implications

The practical applications of this study are significant for digital health system designers, platform developers, clinicians, and public health administrators. The findings demonstrate how user-generated textual feedback, typically treated as anecdotal or informal, can be systematically leveraged for real-time monitoring of both emotional and clinical dimensions of patient experience.

First, the integration of sentiment analysis into mHealth platforms offers a scalable method for tracking passive experiences. Platforms could deploy automated tools that flag declines in sentiment polarity, signaling patient dissatisfaction or disengagement in real time. Such tools could be used for dynamic triage, service recovery, or personalized interface adjustments, reducing user churn and increasing retention. These findings suggest that narrative-based indicators derived from patient reviews may provide a low-burden complementary signal for monitoring patient experience in digital health platforms, potentially supplementing traditional patient-reported outcome surveys.

Second, the use of improvement-related keywords for outcome prediction offers a lightweight complement to traditional symptom tracking. mHealth platforms could flag users who mention feeling better, worse, or unchanged in their feedback, triggering tailored follow-up pathways. This complements conventional patient-reported outcome measures by offering a passive, organic feedback loop that functions between formal clinical assessments. In large-scale deployments, such systems can be integrated into AI-driven case management pipelines, facilitating longitudinal monitoring without additional staffing demands.

Third, the hybrid model developed in this study (combining sentiment, keywords, and TF-IDF vectors) offers an exploratory framework for integrating narrative-derived indicators into clinical or administrative dashboards. Health administrators and platform managers can visualize both satisfaction trends and text-derived outcome-related signals at the department, region, or physician level, allowing for system-level quality monitoring. This type of dual-channel insight, which combines experiential and functional perspectives, can inform training needs, system redesign, or broader quality-improvement initiatives in value-based care models.

Finally, the study’s culturally grounded methodology, built on Chinese-language data, supports the development of localized NLP models for underrepresented populations in digital health research. This aligns with broader calls for equitable mHealth development and evaluation frameworks across global regions, particularly in settings where mHealth infrastructure and service delivery remain uneven. ⁷ As health systems and app providers expand globally, models trained on non-English corpora will be essential for delivering equitable care. This work demonstrates how tailored linguistic pipelines can empower personalized, culturally aware digital health analytics, advancing the agenda of global health equity and inclusive innovation.

Limitations and future research

Despite its contributions, the study has some limitations that must be acknowledged and addressed in future work. First, the dataset was sourced from a single Chinese-language mHealth platform and covered a six-month observation period. As a result, the linguistic patterns observed in the review narratives may reflect platform-specific user behavior, moderation practices, and regional healthcare expectations. Patient feedback in other languages, healthcare systems, or digital platforms may exhibit different characteristics. Future research should analyze multiple platforms, longer time periods, and cross-cultural datasets to assess the robustness and generalizability of narrative-based outcome monitoring in digital health environments.

Second, the keyword-derived perceived improvement indicator was constructed using predefined improvement-related expressions rather than human annotation or externally validated labels. Accordingly, it should be interpreted as an exploratory proxy rather than a validated outcome measure, and the classification analysis may partially reflect lexical overlap between predictors and labels. Future research should employ externally annotated datasets or human-coded labels to provide stronger validation of narrative-based outcome detection.

Third, the models relied on lexicon-based sentiment scoring (SnowNLP) and TF-IDF vectorization. While interpretable and practical for short-form content, these approaches may miss contextual subtleties and compound linguistic patterns. The integration of deep learning models, particularly transformer architectures like BERT, could offer deeper semantic insight and capture emotion dynamics or temporal shifts across sequences of user interactions.

Finally, the study focused exclusively on textual content, omitting multimodal data streams, such as in-app behavior, response latency, consultation duration, and physiological inputs (for instance, heart rate from wearables). Future research should investigate how combining textual, behavioral, and biometric data can yield more comprehensive and real-time models of patient-reported outcomes. This is especially relevant for longitudinal tracking and early warning systems in chronic disease management, remote monitoring, and telepsychiatry.

Conclusion

This study demonstrates the feasibility of using patient-generated narratives as a complementary, low-burden source for monitoring patient-reported outcomes in mHealth environments. By applying natural language processing and machine learning techniques, we show that unstructured review data can capture signals related to both patient satisfaction and perceived symptom improvement.

Empirically, sentiment polarity significantly predicted satisfaction (β = 0.351, p < 0.001), while the TF-IDF classification model achieved strong performance in identifying improvement-related expressions (F1 = 0.945), with Random Forest showing slightly improved results (F1 = 0.953). These findings indicate that narrative text contains detectable patterns associated with both experiential and outcome-related dimensions of care.

The study contributes by operationalizing Service-Dominant Logic within a text-based framework and demonstrates how narrative analytics may complement traditional survey-based approaches for scalable, real-time monitoring in mHealth systems. Future research should validate these findings using externally annotated data and more advanced modeling techniques.