Sage Journals: Discover world-class research

Abstract

Background

Glucolipid metabolic disorders is a disorder characterized by derangement of glucose and lipid metabolism, which is involved in multiple factors. Since the emergence of accelerated technological evolution, it has progressively evolved into a significant concern in contemporary medicine. Therefore, early screening and diagnosis are crucial. This study aims to explore the possibility of early noninvasive diagnosis of glucolipid metabolic disorders using facial and tongue image indicators.

Method

In this study, we constructed a tongue-face segmentation model based on Deeplabv3 + for extracting tongue and facial indicators. The study collected information of 614 participants, including 296 patients with GLMD and 318 healthy controls. After baseline comparison, we respectively conducted intergroup comparison of laboratory biochemical indicators and correlation analysis of facial indicators and tongue image indicators for two groups. We also attempted to build machine learning diagnostic models for glycolipid metabolic diseases based on SVM, Random Forest, KNN, Naive Bayes, XGBoost, and AdaBoost by separately applying facial images and tongue images, and used Shapley to evaluate the contribution of each indicator in the model.

Result

The results show that there is a statistically significant difference in the facial and lip color indicators and tongue color indicators. The facial, lip and tongue brightness indicators have a higher correlation coefficient with LDL-C, TG, and CHO, among which F-L is most correlated with LDL-C. Then, six classical machine learning models for predicting GLMD were constructed based on facial and tongue image indicators, and XGBoost performed the best with an AUC of 0.946, accuracy of 0.861, among which the color indicators TB-Y, TB-S, and TB-G are the top three indicators in terms of contribution.

Conclusion

The GLMD diagnostic model combined with tongue-facial indicators can achieve disease classification, and through modern information-based TCM diagnosis technology, the accuracy of noninvasive diagnosis of glucose-lipid metabolism diseases can be further improved.

Keywords

Tongue image facial image machine learning glucolipid metabolic disorders noninvasive diagnosis model

Background

Metabolic problems, which impose billions of dollars in medical costs each year, have become a major global threat to human health.¹ Lipids and glucose, the key substrates of life activities,² are involved in most metabolic processes in the human body. Under physiological conditions, these two components are tightly coordinate and jointly regulate human metabolic activities.³ Glucolipid metabolic disorders represent a complex group of diseases associated with glycolipid dysregulation.^4,5 Influenced by environmental and lifestyle factors, the number of diabetic patients in China resulting from abnormal glycolipid metabolism has increased from 90 million to 140 million over the past decade. Meanwhile, the number of adults with hypertension in China has reached 245 million, 81.2% of whom have at least one type of dyslipidemia. Currently, atherosclerotic cardiovascular disease (ASCVD) driven by dyslipidemia has become the leading cause of death among urban and rural residents in China.^6,7 Therefore, early intervention, timely diagnosis and treatment, and long-term health management are crucial for patients with glucose and lipid metabolism diseases as well as high-risk populations.^8,9

Traditional Chinese medicine (TCM) is widely recognized as a practical and effective approach for disease diagnosis and treatment worldwide.^10–12 Inspection, as a representative noninvasive diagnostic method, plays an essential role in TCM diagnosis. This approach provides pathological information from multiple aspects, including the face, tongue coating, and tongue texture. With the integration of modern information technology, computer vision and image processing techniques have made it possible to quantitatively analyze tongue image features.¹³ Recent studies have demonstrated the diagnostic potential of this approach. For example, researchers¹⁴ combined tongue image observation and biological information to build a deep learning model for the diagnosis of gastrointestinal cancer. Similarly, tongue image features can provide more objective reference for the diagnosis of diabetes.^15,16 Doctors can also assess patients’ health status by observing their facial features.¹⁷ Besides, our team previously developed a tongue image classification system based on deep learning, which has achieved classifications for diabetes and NAFLD.¹⁸ Our preliminary researches further demonstrated that incorporating TCM visual diagnostic images into disease assessment can enhance diagnostic accuracy.^19,20 An increasing body of evidence suggests that TCM-based visual diagnostic methods provide promising new directions for the noninvasive diagnosis of glucolipid metabolic disorders.²¹

With the application of artificial intelligence technologies, particularly machine learning,^22,23 in the medical field, the precise segmentation of medical images has significantly enhanced the accuracy of disease diagnosis. Consequently, diagnostic techniques based on tongue image segmentation have also undergone substantial development.^24–26 However, comprehensive studies on the application of TCM inspection diagnosis to glycolipid metabolism are still lacking. This study utilizes digitalized TCM visual diagnosis technology to investigate the pathogenesis and characteristic patterns of tongue and facial manifestations, aiming to explore noninvasive diagnostic approaches for glucolipid metabolic disorders.

Materials and method

Patients

This was a single-center retrospective case-control study conducted among individuals undergoing routine health examinations at the Physical Examination Center of the Shanghai University of Traditional Chinese Medicine Affiliated Baoshan Hospital of Integrated Traditional Chinese and Western Medicine from 3 January 2020 to 28 December 2021. A total of 615 patients were included in this experiment, with 318 in the control group and 297 in the disease group. The case process can be shown as Figure 1. All participants are required to fill out an informed consent form approved by the Ethics Committee of Shuguang Hospital Affiliated to Shanghai University of Traditional Chinese Medicine and all research was performed in accordance with relevant regulations. Besides, this research is performed in accordance with the Declaration of Helsinki.Health status is defined as the absence of acute or chronic diseases, and the diagnostic criteria for glucose and lipid metabolism diseases are based on “The International Clinical Practice Guidelines for Traditional Chinese Medicine.”²⁷ It mainly includes: (1) Type 2 diabetes: fasting blood glucose ≥ 7.0 mmol/L, or blood glucose ≥ 11.0 mmol/L 2 h after meal, or glycosylated hemoglobin ≥ 65%; (2) Dyslipidemia: Total cholesterol ≥ 6.21 mol/L, low-density lipoprotein cholesterol ≥ 4.1 mmol/L, triglycerides ≥ 2.3 mmol/L, and nonhigh density lipoprotein cholesterol ≥ 4.9 mmol/L; and (3) MASLD: Upper abdominal ultrasound suggests enhanced liver anterior echogenicity (“bright liver”), far-field echo attenuation, and unclear display of intrahepatic duct structures.

Figure 1.

Case collection process.

Clinical data

Collect clinical data and calculate body mass index (BMI) and waist-to-hip ratio (WHR) based on the name, age, height, weight, waist circumference, and hip circumference of the research subjects. Laboratory biochemical indicators include Alanine aminotransferase (ALT), Aspartate aminotransferase (AST), Gamma-glutamyl transferase (γ-GT), Alkaline phosphatase (ALP), fasting blood glucose (FBG), Blood urea nitrogen (BUN), Creatinine (CREA), Uric acid (UA), Cholesterol (CHO), Triglyceride (TG), High-density lipoprotein (HDL), and Low-density lipoprotein (LDL).

Tongue and facial image sampling

All participants were required to fast for 30 min before filming to avoid staining their tongues with colors of food, drinks, etc. If there are food residues in the subjects’ mouths, they are required to rinse with water for 3–5 min before collection. The collection device is TFDA-1 Instrument developed by the Intelligent Diagnostic Laboratory of Shanghai University of Traditional Chinese Medicine. Set parameters such as shutter speed, aperture, sensitivity, metering, white, etc.; Instruct the subject to sit upright, closely adhere to the collection ring, look straight ahead, open their mouth and tongue, relax and flatten their tongue, and let it naturally droop. Click on the center of the screen to complete the image collection. The image is in JPG format with a size of 5568 × 3712 pixel(px).^28–30 The brightness parameters can be found in the Supplementary Material. All images were collected in accordance with the collection standards and under a unified equipment and standard environment.

Tongue and face image feature extraction

Quality control of tongue and facial images is an important prerequisite for tongue surface feature analysis. In our preliminary research, we established a high-quality, standardized clinical database of tongue and facial images. Precise tongue/facial image segmentation is the fundamental work for obtaining facial and lip-tongue features. Unlike previous work based on Mask R-CNN, generative adversarial network Tongue-GAN,³¹ we pay more attention to the accurate segmentation of the blurred areas at the connection of the lips and tongue as well as the accuracy of facial indicators.

Deeplabv3+ is a further extension of the DeepLab series models,^32,33 aimed at improving the accuracy and efficiency of semantic segmentation by combining encoder decoder structures and dilated convolutions.³⁴

The specific steps are as follows: Firstly, we randomly selected 1150 tongue and facial images from the database. We annotated the fine-grained tongue and lip regions by using Labelme, with 805 tongue images used as training data and 345 tongue images used as testing data. The results showed that the segmentation model based on DeepLabv3+ achieved a mean Intersection over Union (mIoU) of 87.5% and a mean accuracy (mean_ACC) of 0.994 on the test set. An example of the segmentation output can be found in Supplementary Figure A. This indicates that the DeepLabv3+ model is capable of performing accurate segmentation of the lips and tongue with high precision. Finally, the region segmentation of the tongue body and tongue coating is completed through the “segmentation and merging algorithm” and “color threshold method.” And the color parameters of Lab color space, texture indicators, and tongue coating indicators are extracted in sequence according to the previous pattern recognition method.

The extraction of facial indicators involves two parts: skin segmentation and color texture calculation, including the use of facial keypoint detector and facial recognition model in Dlib toolbox,³⁵ to determine the geometric position relationship of the face and the position information of 68 facial keypoints, which can determine the position information of 8 ROIs, and automatically select skin color boxes for segmentation based on the above position information. Next, based on the elliptical skin color clustering model in YCbCr color space, skin color and background can be separated, and the average value of skin blocks can be calculated. The Gray-Level Co-occurrence Matrix (GLCM) method was used to calculate the relative frequencies of reference pixel pairs and their adjacent pixels within the skin block, in order to extract texture features. The lip features have good clustering characteristics in the YIQ color space, where the color changes of the Q component precisely cover the color range of the lips. Figure 2 shows the entire process of extracting tongue and facial indicators.

Figure 2.

Tongue and face features extraction process.

Data analysis methods

Statistical analysis was conducted using IBM SPSS (Version 26.0). If the measured data follows a normal distribution or an approximately normal distribution, it is represented by X ± SD, minimum value, and maximum value; if the data does not follow a normal distribution, it is described by M, min, max, and quartiles (Q1 and Q3). Count data is represented by frequency, composition ratio, ratio, and relative ratio, etc. If the data follows a normal distribution or an approximately normal distribution and the variances are homogeneous, a two-independent sample t-test is used. Applying Bonferroni correction in multiple comparisons to control false positive rates.

Development of a diagnostic model for glucolipid metabolic disorders

We use Python 3.10.9 for machine learning, including Support Vector Machine (SVM), Random Forest (RF), K-nearest neighbor classifier, Gaussian Naive Bayes classifier (GaussianNB), XGBoost classifier, AdaBoost classifier. We use the skLearning (Version 1.3.1) library to compute machine learning classification results. The implementation of other deep learning networks is based on PyTorch 2.0. The specific parameter settings for these machine learning methods are summarized in Supplementary. Besides, we use an NVIDIA RTX 3090 with 24GB VRAM for model training and inference, a total of 120 training epochs incorporating cosine annealing learning rate scheduling for the last 20 epochs, the AdamW optimizer with a weight decay of 0.0001, and an initial learning rate of 0.0001 that decays by a factor of 0.9 every 20 epochs.

In machine learning, the prediction results of diseases are usually divided into true positive (TP), true negative (TN), false positive (FP), and false negative (FN). TP is the number of cases correctly identified as positive, FP is the number of cases incorrectly marked as positive, TN is the number of cases correctly identified as negative, and FN is the number of cases incorrectly marked as negative. By analyzing the above categories, metrics such as accuracy (ACC), precision (Precision), recall (Recall), and F1 score can be calculated to evaluate model performance. ACC reflects the proportion of all correct predictions in the total sample, precision represents the model's ability to correctly predict positive numbers in all positive predictions, recall represents the model's ability to correctly predict positives from actual positive samples, and F1 score is the harmonic average of precision and recall. By considering precision and recall comprehensively, a more comprehensive evaluation perspective is provided for model performance.^30,36 Besides, We re-divided the data into training, validation, and testing sets using a 7:1:2 ratio, ensuring comparable distributions of gender, age, and disease categories. This approach can reduce the chance errors caused by data partitioning, making the performance evaluation more stable.

Results

Description of basic information and biochemical experimental indicators between disease group and control group

This study included 297 GLMD patients and 318 nondisease control members. All subject information is shown in Table 1. There is no statistically significant difference in gender and age between the two groups, and they are comparable. The clinical indicators of ALT, AST, γ - GT, ALP, GLU, BUN, CREA, UA, CHO, TG, HDL, LDL, etc., in the disease group were significantly higher than those in the control group (P < 0.05).

Table 1.

Basic information and biochemical experimental indicators between-group variation.

Index		Non-GLMD (n = 318)	GLMD (n = 297)	t/χ²	Cohen d/Phi	P
Age		40.19 ± 0.50	41.49 ± 0.49	−1.857	−0.150 (−0.308, 0.009)	0.064
Gender (%)	male	30 (9.43)	37 (12.50)	1.447	0.229	−0.048
	female	288 (90.57)	260 (86.50)
ALT/U·L⁻¹		13.94 ± 7.43	22.46 ± 16.15**	−8.30	−0.685 (−0.847, −0.522)	<0.01
AST/U·L⁻¹		18.00 ± 4.72	21.22 ± 8.52**	−5.73	−0.471 (−0.631, −0.310)	<0.01
γ-GT/U·L⁻¹		15.23 ± 6.90	26.17 ± 24.21**	−7.51	−0.624 (−0.786, −0.462)	<0.01
ALP/U·L⁻¹		56.53 ± 18.42	64.95 ± 19.10**	−5.56	−0.449 (−0.609, −0.289)	<0.01
FBG/mmol·L⁻¹		4.98 ± 0.41	5.26 ± 0.88**	−4.93	−0.407 (−0.566, −0.247)	<0.01
BUN/mmol·L⁻¹		4.28 ± 1.00	4.69 ± 1.12**	−4.81	−0.389 (−0.549, −0.230)	<0.01
CREA/μmol·L⁻¹		56.90 ± 9.78	62.89 ± 13.48**	−6.27	−0.511 (−0.671, −0.350)	<0.01
UA/μmol·L⁻¹		263.12 ± 56.09	311.06 ± 82.15**	−8.40	−0.686 (−0.848, −0.523)	<0.01
CHO/mmol·L⁻¹		4.38 ± 0.48	5.31 ± 0.83**	−16.71	−1.372 (−1.547, −1.196)	<0.01
TG/mmol·L⁻¹		0.75 ± 0.27	1.43 ± 1.04**	−10.85	−0.902 (−1.067, −0.735)	<0.01
HDL/mmol·L⁻¹		1.46 ± 0.25	1.37 ± 0.31**	3.88	0.316 (0.156,0.475)	<0.01
LDL/mmol·L⁻¹		2.63 ± 0.36	3.40 ± 0.60**	−19.10	−1.566 (−1.747, −1.385)	<0.01

Note. Compared with the control group: *P < 0.05,**P < 0.01.

Analysis of face and tongue image feature differences between disease group and control group

All subjects’ facial image indicators can be shown in Table 2 and the tongue image indicators can be shown in Table 3, In terms of facial indicators, the F-L and F-Y indicators of facial brightness in the glucose and lipid metabolism disease group were significantly lower than those in the control group, F-b was significantly higher than that in the control group, and F-Cr was lower than that in the control group. The brightness indicators L-L and L-Y in the disease group were significantly lower than those in the control group, and L-a and L-b, L-Cr was higher. In terms of tongue image indicators, the glucose and lipid metabolism disease group had lower levels of TB-L, TB-Y, and TC-L, TC-Y, which represent the brightness of tongue texture, compared to the control group. The disease group had significantly higher levels of TB-a, TC-a, TB-b, and TC-a compared to the control group. The perAll in the disease group was significantly reduced compared to the control group.

Table 2.

Facial indicators between-group variation.

Index	Non-GLMD (n = 318)	GLMD (n = 297)	t	Cohen d	P
F-R	166.36 ± 10.72	155.02 ± 8.35**	14.685	1.175 (1.003, 1.346)	<0.01
F-G	94.71 ± 10.02	84.37 ± 6.66**	15.165	1.208 (1.035, 1.379)	<0.01
F-B	91.60 ± 11.25	80.54 ± 7.60**	14.371	1.145 (0.974, 1.315)	<0.01
F-L	48.65 ± 3.87	44.52 ± 2.73**	15.407	1.229 (1.056, 1.041)	<0.01
F-a	29.60 ± 1.95	29.61 ± 1.87	−0.078	−0.006 (−0.164, 0.152)	0.938
F-b	15.23 ± 1.87	15.94 ± 1.61**	−5.018	−0.403 (−0.562, −0.243)	<0.01
F-Y	115.44 ± 8.71	106.23 ± 6.00**	15.355	1.224 (1.051, 1.396)	<0.01
F-Cb	116.01 ± 1.24	115.85 ± 1.12	1.758	0.142 (−0.017, 0.300)	0.079
F-Cr	159.69 ± 1.96	159.30 ± 1.94*	2.447	0.197 (0.039, 0.356)	0.015
L-R	175.79 ± 12.98	164.76 ± 10.02**	11.843	0.947 (0.780, 1.114)	<0.01
L-G	123.17 ± 15.12	110.68 ± 12.18**	11.317	0.907 (0.740, 1.072)	<0.01
L-B	108.77 ± 16.66	96.13 ± 13.39**	10.398	0.833 (0.668, 0.997)	<0.01
L-L	56.86 ± 5.46	52.26 ± 4.36**	11.577	0.927 (0.760, 1.093)	<0.01
L-a	19.80 ± 2.40	20.82 ± 2.35**	−5.306	−0.428 (−0.588, −0.268)	<0.01
L-b	16.61 ± 2.84	17.38 ± 2.76**	−3.410	−0.275 (−0.434, −0.116)	<0.01
L-Y	133.88 ± 12.40	123.52 ± 9.81**	11.541	0.924 (0.757, 1.090)	<0.01
L-Cb	113.87 ± 2.08	113.59 ± 2.03	1.684	0.136 (−0.023, 0.294)	0.093
L-Cr	152.14 ± 2.43	152.79 ± 2.29**	−3.417	−0.276 (−0.435, −0.117)	<0.01

Note. “F” stands for “facial,” while the preceding “L” refers to the “lip.” Compared with the control group,*P < 0.05, **P < 0.01.

Table 3.

Tongue indicators between-group variation.

Index	Non-GLMD (n = 318)	GLMD (n = 297)	t	Cohen d	P
perAll	0.30 ± 0.10	0.24 ± 0.12**	6.680	0.542 (0.381, 0.703)	<0.01
perPart	0.87 ± 0.50	0.85 ± 0.55	0.658	0.053 (−0.105, 0.211)	0.511
TB-CON	107.48 ± 34.48	96.13 ± 27.52**	4.527	0.363 (0.203, 0.522)	<0.01
TC-CON	152.22 ± 53.68	133.11 ± 44.80**	4.803	0.385 (0.225, 0.545)	<0.01
TB-ASM	0.06 ± 0.01	0.07 ± 0.01**	−3.900	−0.315 (−0.474, −0.155)	<0.01
TB-ENT	1.29 ± 0.08	1.27 ± 0.07**	4.099	0.331 (0.171, 0.490)	<0.01
TB-MEAN	0.03 ± 0.01	0.03 ± 0.00**	4.572	0.367 (0.207, 0.526)	<0.01
TC-ASM	0.05 ± 0.01	0.06 ± 0.01**	−4.716	−0.381 (−0.540, −0.221)	<0.01
TC-ENT	1.36 ± 0.08	1.33 ± 0.09**	5.235	0.423 (0.263, 0.583)	<0.01
TC-MEAN	0.04 ± 0.01	0.04 ± 0.01**	4.654	0.376 (0.216, 0.535)	<0.01
TB-B	107.19 ± 8.78	93.35 ± 6.27**	22.615	1.805 (1.617, 1992)	<0.01
TB-R	168.45 ± 8.47	156.79 ± 6.18**	19.591	1.564 (1.383, 1.745)	<0.01
TB-G	102.54 ± 7.88	88.19 ± 5.79**	25.844	2.064 (1.868, 2.260)	<0.01
TC-R	157.70 ± 13.03	140.02 ± 14.88**	15.631	1.267 (1.093, 1.440)	<0.01
TC-G	110.00 ± 12.67	92.55 ± 14.06**	16.134	1.307 (1.132, 1.480)	<0.01
TC-B	113.19 ± 13.68	95.74 ± 14.89**	15.109	1.223 (1.050, 1.393)	<0.01
TB-L	50.74 ± 2.99	45.58 ± 2.05**	25.096	2.000 (1.806, 2.193)	<0.01
TB-a	27.08 ± 2.08	28.79 ± 2.48**	−9.246	−0.751 (−0.914, −0.587)	<0.01
TB-b	8.72 ± 1.81	9.50 ± 1.95**	−5.159	−0.417 (−0.577, −0.257)	<0.01
TC-L	51.34 ± 4.96	44.43 ± 5.67**	16.033	1.300 (1.125, 1.473)	<0.01
TC-a	19.45 ± 1.91	19.94 ± 2.36**	−2.834	−0.230 (−0.389, −0.072)	<0.01
TC-b	5.88 ± 1.76	6.21 ± 1.87*	−2.239	−0.181 (−0.339, −0.022)	0.026
TB-Y	121.44 ± 6.71	109.86 ± 4.55**	25.206	2.008 (1.814, 2.202)	<0.01
TB-Cr	156.62 ± 2.10	157.76 ± 2.47**	−6.168	−0.501 (−0.661, −0.340)	<0.01
TB-Cb	120.27 ± 1.36	120.10 ± 1.44	1.57	0.127 (−0.032, 0.285)	0.117
TC-Y	123.03 ± 10.92	107.98 ± 12.18**	16.094	1.304 (1.192, 1.477)	<0.01
TC-Cr	148.72 ± 1.87	148.63 ± 2.22	0.591	0.048 (−0.110, 0.206)	0.555
TC-Cb	122.33 ± 1.34	122.36 ± 1.40	−0.3	−0.024 (−0.182, 0.134)	0.764

Note. “TB” refers to the tongue body, and “TC” refers to the tongue coating. Compared with the control group: *P < 0.05, **P < 0.01.

Analysis of the correlation between tongue surface indicators and laboratory examination of diseases

This study analyzed tongue and facial indicators of 297 GLMD patients and 318 nondisease control group members (Figures 3–5). The results indicate that facial indicators are correlated to varying degrees with various biochemical indicators, which can be seen in Figure 6. LDL is most correlated with F-L and F-Y, with correlation coefficients of −0.30 (P < 0.05) and the correlation coefficients between LDL with L-L and L-Y is −0.24. That means with the increase of LDL, the gloss of face and lips will decrease the correlation coefficient between CHO and F-L, F-Y is −0.27, and the correlation coefficient with L-L, L-Y is −0.21. The correlation coefficient between TG and F-Y is −0.25, and the correlation coefficient with L-L, L-Y is −0.22. So, the gloss of face and lips is negatively correlated with TG, CHO.

Figure 3.

Performance of face and tongue indicators in lab color space.

Figure 4.

Performance of face and tongue indicators in YCrCb color space.

Figure 5.

Tongue image texture expression.

Figure 6.

Heat map of correlation between facial indicators and laboratory parameters.

As shown in Figure 7, tongue image indicators are correlated with most biochemical indicators. LDL has the highest correlation with TB-L and TB-Y, with correlation coefficients of −0.44 (P < 0.05), followed by TC-L (−0.36) and TC-Y (−0.36); CHO is highly correlated with TB-L (−0.40) and TB-Y (−0.40), followed by TC-L (−0.33) and TC-Y (−0.33). TG is negatively correlated with TB-L (−0.31), TB-Y (−0.31), and TC-L (−0.24). In summary, as the increase of LDL, CHO, TG, the luster of tongue darkens. Besides, we conducted logistic regression analyses accounting for the indicators, as shown in Supplementary Table B.

Figure 7.

Heat map of correlation between tongue indicators and laboratory parameters.

We separately incorporated facial and tongue indicators to construct a machine learning diagnostic model for glucose and lipid metabolism diseases based on tongue indicators. By comparing the performance of each model and the contribution of each indicator included in the model, we further explored the deep connection between tongue indicators and glucose and lipid metabolism diseases. Model 1 incorporates facial indicators based on all color space, while Model 2 incorporates facial and tongue indicators based on all color space.

In Model 2, XGBoost performed the best with an AUC of 0.946, accuracy of 0.861, and sensitivity of 0.864, followed by Random Forest with an AUC of 0.933, ACC of 0.852, and sensitivity of 0.898. SVM had an AUC of 0.919, accuracy of 0.819, and sensitivity of 0.864; Adaboost had an AUC of 0.895, accuracy of 0.803, and sensitivity of 0.813; NaiveBayes had an AUC of 0.886, accuracy of 0.761, and sensitivity of 0.882; KNN had an AUC of 0.876, accuracy of 0.803, and sensitivity of 0.848. The ROC curves of all models and the confusion matrices based on tongue surface index parameters are shown in Figure 8 and Table 4.

Figure 8.

GLMD classification model based on face and tongue image indicators. (a) Diagnosis model of GLMD based on facial parameters and (b) diagnosis model of GLMD based on facial and tongue parameters.

Table 4.

Performance of GLMD diagnostic model based on tongue and face indicators.

Models	AUC	Accuracy	Sensitivity	Specificity	Precision	F1-Score	MCC
SVM	0.919 ± 0.045	0.819 ± 0.070	0.864 ± 0.088	0.776 ± 0.103	0.783 ± 0.103	0.820 ± 0.076	0.641 ± 0.133
Random Forest	0.933 ± 0.042	0.852 ± 0.061	0.898 ± 0.075	0.809 ± 0.099	0.815 ± 0.097	0.853 ± 0.068	0.708 ± 0.117
KNN	0.876 ± 0.058	0.803 ± 0.066	0.848 ± 0.087	0.761 ± 0.105	0.768 ± 0.105	0.805 ± 0.073	0.610 ± 0.131
Naive Bayes	0.886 ± 0.052	0.761 ± 0.074	0.882 ± 0.078	0.649 ± 0.122	0.701 ± 0.107	0.780 ± 0.078	0.543 ± 0.136
XGboost	0.946 ± 0.035	0.861 ± 0.061	0.864 ± 0.086	0.858 ± 0.085	0.850 ± 0.095	0.856 ± 0.067	0.721 ± 0.122
Adaboost	0.895 ± 0.052	0.803 ± 0.070	0.813 ± 0.097	0.793 ± 0.100	0.786 ± 0.104	0.798 ± 0.078	0.606 ± 0.138

Besides, given the sample size constraints, we additionally report the counts and model performance for two major subtypes, dyslipidemia-only (subtype 1) and MASLD-only (subtype 2). The results can be found in Supplementary Tables C and D.

In the subsequent diagnostic model, the addition of tongue image indicators greatly improved the diagnostic accuracy of the model. In the SHAP model interpreter (Table 5 and Figure 9),³⁷ TB-Y showed the largest mean absolute SHAP value, indicating that tongue brightness was the most influential feature for the model's predictions. In our study, TB-Y was associated with lipid-related biochemical indicators (LDL, CHO, and TG), suggesting that the model may capture metabolic-risk signals reflected in these measures. TB-S and TB-G also made meaningful contributions to the prediction.

Figure 9.

SHAP chart of model contribution. (a) Visualization of model indicator contribution and (b) distribution of SHAP values for model indicators.

Table 5.

Top10 SHAP variance of main features.

Feature	SHAP mean	SHAP variance	SHAP Std
TB-Y	2.2319	0.3153	0.5616
TB-S	0.9878	0.0272	0.1650
TB-G	0.7510	0.0268	0.1637
TB-B	0.5588	0.0109	0.1044
TB-L	0.4176	0.2578	0.5078
TB-H	0.2995	0.0099	0.0994
F-G	0.2941	0.0317	0.1780
TC-a	0.2941	0.0069	0.0829
F-S	0.2529	0.0014	0.0377
TC-Cr	0.2335	0.0055	0.0738

Discussion

The development and prosperity of human society have been accompanied by an increasing prevalence of metabolic disorders. In recent years, the incidence of glucolipid metabolic diseases has risen sharply, gradually becoming a major public health problem concern that threatens human health. Modern lifestyle characterized by high sugar and fat intake, coupled with physical inactivity, have led to a doubling of obesity rates.⁹ Consequently, glucolipid metabolic diseases and their associated complications have emerged as serious health challenges for society. The diagnosis of glucolipid metabolic diseases³⁸ typically relies on laboratory biochemical tests, imaging examinations, pathological examinations, and other procedures. However, the rapidly increasing incidence of such diseases has imposed a substantial burden on healthcare systems, while invasive diagnostic methods also carry a risk of infection. Therefore, there is an urgent need to develop convenient, effective, and noninvasive diagnostic approaches.³⁹

In medical image processing, the advantage of DeepLabv3+ demonstrates strong adaptability and robustness to complex medical images, effectively handling challenges such as noise, low contrast, and irregular shapes in images.⁴⁰ Its depth wise separable convolution design significantly reduces computational complexity, allowing the model to achieve efficient inference speeds without compromising accuracy, which makes it particularly suitable for clinical deployment.⁴¹ Furthermore, with its unique encoder-decoder structure and the Atrous Spatial Pyramid Pooling (ASPP) module, DeepLabv3+ can effectively capture multiscale contextual information in medical images while preserving fine spatial details, enabling high-precision segmentation of target areas. By integrating a powerful backbone network with other backbone networks and flexible model structures, DeepLabv3+ provides an efficient and reliable solution for medical image segmentation.

This study developed a new method based on DeepLabV3+ to extract facial and tongue image parameters and conducted a correlation analysis between tongue surface involvement and clinical examination. We found that the F-L, F-Y, and F-Cr in the disease group were lower than those in the control group, whereas the F-b was significantly higher. These results indicate that the proportion of yellow color in the facial color of the disease group was higher, while the proportion of red color was lower. For the lip image indicators, the brightness parameters L-L and L-Y in the disease group were significantly lower than those in the control group, whereas the color indicators L-a, L-Cr, and L-b was higher. This suggests that the lip brightness in the disease group was reduced, accompanied by increased red and yellow color components. Regarding tongue image indicators, the disease group showed lower values than the control group for parameters representing tongue texture brightness such as TB-L and TB-Y, as well as tongue coating brightness such as TC-L and TC-Y. In contrast, the TB-a, TC-a, TB-b, and TC-a were significantly higher in the disease group, indicating that the tongue texture and coating were brighter and richer in red and yellow tones compared with the control group. These findings are consistent with the results reported by Li et al.¹⁶ In addition, we found that the brightness parameters of the face and tongue were closely correlated with blood components such as LDL, CHO, and TG, which is consistent with the research showing that increased adipose tissue significantly affect plasma and blood viscosity.⁴² Our study also demonstrated that metabolic status in GLMD was associated with facial and tongue features, consistent with the findings of Wen et al.⁴³ who reported that these features were related to metabolic status, renal and hepatic, cardiac, and thyroid function, as well as to blood cell proportions and Hb levels in patients with T2DM.

In the diagnosis model of GLMD based on facial and tongue parameters, the XGBoost model hat incorporated both sets of indicators achieved a classification accuracy of 86.13%. Among all features, the brightness indicators TB-Y, TB-S, and TB-G ranked among the top three in terms of model contribution, indicating that tongue surface parameters can serve as effective indicators for noninvasive diagnosis of diseases. As a traditional diagnostic basis in traditional Chinese medicine, tongue images can express various pathological information and are less susceptible to external environmental influences compared to facial expressions. Therefore, the inclusion of tongue image parameters significantly improves the diagnostic accuracy of the model. In modern medicine, machine learning has been widely applied to disease diagnosis and prediction. Its ability to detect subtle variations in human facial expressions enables it to support the diagnosis and clinical evaluation of a wide range of neurological disorders.⁴⁴ Among various algorithms, XGBoost stands out for its capacity to model complex nonlinear relationships and handle high-dimensional data, making it a preferred choice for prediction tasks. Combining strong accuracy with good interpretability, it serves as a valuable decision-support tool for clinicians. In disease diagnosis, XGBoost can build risk prediction models from clinical and laboratory data, offering efficiency and robustness in managing missing values and noise common in medical datasets. Furthermore, its interpretability provides transparent decision support for doctors and the algorithm has been successfully applied in various fields such as disease prediction,⁴⁵ cancer classification,⁴⁶ heart disease risk assessment,⁴⁷ and disease prognosis analysis.⁴⁸ Notably, Nguyen Hai Minh et al.⁴⁹ found that XGBoost was the most suitable model for assisting in the diagnosis of colorectal cancer.

This study still has certain limitations. Glucolipid metabolic diseases are involving multiple factors. Therefore, regional, environmental, dietary, and individual differences should be taken into account. Future studies should include a broader sample collection from both northern and southern regions to enhance the representativeness of the data.²³ Since the current dataset was obtained from physical examination units, there was a significant difference in the male-to-female ratio. Hence, future research should aim to achieve a more balanced gender distribution. Moreover, because the model is trained and validated using single-center data, its generalizability remains limited. Therefore, the current findings should be interpreted as an internal proof-of-concept, and real-world clinical utility will require prospective evaluation in routine workflows and independent external validation across different centers, populations, and devices. Besides, a limitation is that GLMD was defined as a composite outcome and pooling these clinically distinct conditions may mask disease-specific patterns and reduce interpretability. Nevertheless, we have done our best to include variables and to evaluate performance using a rigorous internal validation strategy to minimize the risk of overfitting. Future studies will conduct multicenter, multidevice testing and prospective validation across diverse populations and real-world acquisition settings to confirm robustness and generalizability. In addition, epigenetic mechanisms³⁶ have been proven to play a role in the occurrence of glucolipid metabolic diseases. As suggested by a previous study,⁵⁰ tongue characteristics are related to microbial metabolism. Therefore, future research on multimodal diagnostic models for glycolipid metabolic diseases can consider incorporating analyses at the micro genetic level and integrating multiomics data.^51,52 Future studies may integrate tongue and facial indicators with oral and gut microbiome profiles to jointly train a more accurate and multidimensional model for assisting in the diagnosis of glucolipid metabolic diseases.

Conclusions

This study shows that the new method of extracting facial and tongue image parameters based on deeplabv3+ can help to establish the diagnosis model of glucolipid metabolic disorders. The diagnosis model of glucolipid metabolic disorders based on XGBoost, which includes the tongue indicators, can effectively classify the disease. In addition, we can further improve the accuracy of noninvasive diagnosis of glucose and lipid metabolism diseases by combining the traditional Chinese medicine diagnosis technology of modern informatics.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076261435866 - Supplemental material for Intelligent tongue and facial image analysis for noninvasive prediction of glucolipid metabolic disorders

Supplemental material, sj-docx-1-dhj-10.1177_20552076261435866 for Intelligent tongue and facial image analysis for noninvasive prediction of glucolipid metabolic disorders by Shi Liu, Zhanhong Chen, Yang Gao, Jialin Deng, Ruomeng Hu, Xin Tan, Tao Jiang and Jiatuo Xu in DIGITAL HEALTH

Footnotes

Acknowledgments

Not applicable.

ORCID iD

Tao Jiang

Ethical considerations

This study has been approved by the ethics committee of Shuguang Hospital Affiliated to Shanghai University of TCM. The ethics committee number is 2018-599-28-01.

Consent to participate

All participants signed a written informed consent form that was approved by the Ethics Committee of Shuguang Hospital Affiliated to Shanghai University of Traditional Chinese Medicine. All subjects agree for publication of identifying information and images in an online open-access publication.

Consent for publication

Not applicable.

Author contributions statement

Tao Jiang and Jiatuo Xu designed the experiments and reviewed and edited the manuscript. Zhanhong Chen participated in the experiments, provided visualization, and reviewed and edited the manuscript. Shi Liu participated in the experiments, analyzed the data, and wrote original manuscript. Jialin Deng was responsible for data management. Ruomeng Hu and Yang Gao assisted in collecting clinical samples. Xin Tan provided the technical support and advices for the study. Zhanhong Chen and Shi Liu contributed equally to this work. All authors read and approved the final manuscript.

Guarantor

Tao Jiang and Jiatuo Xu are the guarantors.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research, authorship, and publication of this article were financially supported by National Science and Technology Major Project(2025ZD0545001),the National Natural Science Foundation of China (82104738), the General Project of China Postdoctoral Science Foundation (2023M732337), the High-level Key Discipline Construction Project of Traditional Chinese Medicine by the National Administration of Traditional Chinese Medicine (ZYYZDXK-2023069), Shanghai Municipal Health Commission (20244Y0129), and the “Youth Talent Support Project” by the China Association of Chinese Medicine (2025-QNRC2-B15).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

The publicly available research data generated by this study are included in this article. The rest of the data are from the national key R&D projects funded by science and technology of the China and are temporarily closed to the public according to the project regulations. The data can be obtained with reasonable requests by contacting the corresponding author.

Supplemental material

Supplemental material for this article is available online.

References

Chong

Kong

Shankar

, et al. The global syndemic of metabolic diseases in the young adult population: a consortium of trends and projections from the global burden of disease 2000–2019. Metabolism 2023; 141: 155402.

Dong

Sun

Nie

, et al. Metabolic memory: mechanisms and diseases. Signal Transduct Target Ther 2024; 9: 38.

Hornburg

Moqri

, et al. Dynamic lipidome alterations associated with human health, disease and ageing. Nat Metab 2023; 5: 1578–1594.

Yang

Huang

, et al. The epigenetic regulation of RNA N6-methyladenosine methylation in glycolipid metabolism. Biomolecules 2023; 13: 273.

Guo

. Research progress on prevention and treatment of glucolipid metabolic disease with integrated traditional Chinese and Western medicine. Chin J Integr Med 2017; 23: 403–409.

Rong

, et al. Liver-adipose tissue crosstalk: a key player in the pathogenesis of glucolipid metabolic disease. Chin J Integr Med 2017; 23: 410–414.

De Lima

Moretti

RCJR

Torres Pomini

, et al. Glycolipid metabolic disorders, metainflammation, oxidative stress, and cardiovascular diseases: unraveling pathways. Biology (Basel) 2024; 13: 519.

Liang

Ren

Liang

, et al. Relationship between quality of life and adolescent glycolipid metabolism disorder: a cohort study. World J Diabetes 2022; 13: 566–580.

Xiong

Zhang

Liu

, et al. Metaflammation in glucolipid metabolic disorders: pathogenesis and treatment. Biomed Pharmacother 2023; 161: 114545.

10.

Yang

Meng

Liu

, et al. Distinct metabonomic signatures of Polygoni multiflori radix praeparata against glucolipid metabolic disorders. J Pharm Pharmacol 2021; 73: 796–807.

11.

Morita

Murakami

Noguchi

, et al. Combination image analysis of tongue color and sublingual vein improves the diagnostic accuracy of Oketsu (blood stasis) in Kampo medicine. Front Med (Lausanne) 2021; 8: 790542.

12.

Balasubramaniyan

Jeyakumar

Nachimuthu

. Panoramic tongue imaging and deep convolutional machine learning model for diabetes diagnosis in humans. Sci Rep 2022; 12: 186.

13.

Zhuang

Gan

Zhang

. Human-computer interaction based health diagnostics using ResNet34 for tongue image classification. Comput Methods Programs Biomed 2022; 226: 107096.

14.

Yuan

Yang

Zhang

, et al. Development of a tongue image-based machine learning tool for the diagnosis of gastric cancer: a prospective multicentre clinical cohort study. EClinicalMedicine 2023; 57: 101834.

15.

Jiang

, et al. Deep learning multi-label tongue image analysis and its application in a population undergoing routine medical checkup. Evid Based Complement Alternat Med 2022; 2022: 3384209.

16.

Chen

, et al. Establishment of noninvasive diabetes risk prediction model based on tongue features and machine learning techniques. Int J Med Inform 2021; 149: 104429.

17.

Cheng

Chen

, et al. Objective study of the facial parameters ofobservations in patients with type 2 diabetes mellitus by machine learning. AnnTransl Med 2022; 10: 960.

18.

Jiang

Guo

, et al. Application of computer tongue image analysis technology in the diagnosis of NAFLD. Comput Biol Med 2021; 135: 104622.

19.

Huang

Jiang

, et al. A multi-step approach for tongue image classification in patients with diabetes. Comput Biol Med 2022; 149: 105935.

20.

Jiang

Liang

Chen

, et al. Integrating next-generation sequencing and traditional tongue diagnosis to determine tongue coating microbiome. Sci Rep 2012; 2: 936.

21.

Wang

Chen

Duan

, et al. Noninvasive diagnostic technique for nonalcoholic fatty liver disease based on features of tongue images. Chin J Integr Med 2024; 30: 203–212.

22.

Zhang

Wen

Zhong

, et al. MLP-Like Model with convolution Complex transformation for auxiliary diagnosis through medical images. IEEE J Biomed Health Inform 2023; 27: 4385–4396.

23.

Jiang

Liu

. Research status and prospect of tongue image diagnosis analysis based on machinelearning. Digit Chin Med 2024; 7: 3–12.

24.

Zhou

Wang

, et al. A transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics. Nat Biomed Eng 2023; 7: 743–755.

25.

Mei

Liu

Singh

, et al. Interstitial lung disease diagnosis and prognosis using an AI system integrating longitudinal data. Nat Commun 2023; 14: 2272.

26.

Wang

Gao

Wang

, et al. Accurate estimation of biological age and its application in disease prediction using a multimodal image transformer system. Proc Natl Acad Sci U S A 2024; 121: e2308812120.

27.

SOCIETIES W F O C M. International clinical practice guideline of Chinese medicine: SCM. World Chin Med 2021; 16: 3278–3284.

28.

Zhang

Wang

, et al. An assessment method of tongue image quality in traditional Chinese medicine. In: Proceedings of the 2016 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), F 15-17 Oct. 2016, 2016 [C]. Year: 640–644.

29.

Jiang

Yao

, et al. Tongue image quality assessment based on a deep convolutional neural network. BMC Med Inform Decis Mak 2021; 21: 147.

30.

Shanbhog

Harjai

Tiwari

, et al. Quantification of tongue coating using quantitative light-induced fluorescence-digital image analysis and its correlation with visual evaluation. J Indian Soc Pedod Prev Dent 2023; 41: 297–301.

31.

Dai

Guo

Liu

, et al. Application of intelligent tongue image analysis in conjunction with microbiomes in the diagnosis of MAFLD. Heliyon 2024; 10: e29269.

32.

CHEN L-C. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:170605587, 2017.

33.

Wang

Zhu

, et al. ECA-Net: Efficient channel attention for deep convolutional neural networks, F 2020. Year: 11534–42.

34.

Tang

Tan

, et al. RTC_Tonguenet: an improved tongue image segmentation model based on DeepLabV3. Digit Health 2024; 10: 20552076241242773.

35.

Lin

Wang

, et al. Facial color classification of traditional Chinese medicine inspection based on fusion of facial image features. J Clin Med Pract 2020; 24: 1–5.

36.

Thirunavukkarasu

Umapathy

Ravi

, et al. Tongue image fusion and analysis of thermal and visible images in diabetes mellitus using machine learning techniques. Sci Rep 2024; 14: 14571.

37.

Xia

Gao

Wang

, et al. Modeling of diagnosis for metabolic syndrome by integrating symptoms into physiochemical indexes. Biomed Pharmacother 2021; 137: 111367.

38.

Niu

Feng

Peng

, et al. A narrative review on the mechanism of natural flavonoids in improving glucolipid metabolism disorders. Phytother Res 2024; 38: 4202–4229.

39.

Qiao

Duan

, et al. Intelligent tongue diagnosis model for gastrointestinal diseases based on tongue images. Biomed Signal Process Control 2024; 96: 106643.

40.

Ullah

Khan

Albarakati

, et al. Multimodal brain tumor segmentation and classification from MRI scans based on optimized DeepLabV3+ and interpreted networks information fusion empowered with explainable AI. Comput Biol Med 2024; 182: 109183.

41.

Xie

Zhang

, et al. SESV: accurate medical image segmentation by predicting and correcting errors. IEEE Trans Med Imaging 2021; 40: 286–296.

42.

Cui

Liu

, et al. Oral, tongue-coating Microbiota, and metabolic disorders: a novel area of interactive research. Front Cardiovasc Med 2021; 8: 730203.

43.

Wen

, et al. The relationship between computerized face and tongue image segmentation and metabolic parameters in patients with type 2 diabetes based on machine learning. Diabetes Metab Syndr Obes 2024; 17: 4049–4068.

44.

Yoonesi

Abedi

Arab

, et al. Facial expression deep learning algorithms in the detection of neurological disorders: a systematic review and meta-analysis. BioMed Eng OnLine 2025; 24: 64.

45.

Chen

Yin

Wang

, et al. XGBoost-based machine learning test improves the accuracy of hemorrhage prediction among geriatric patients with long-term administration of rivaroxaban. BMC Geriatr 2023; 23: 418.

46.

Duan

Zhang

Qiu

, et al. Machine learning-based prediction model for distant metastasis of breast cancer. Comput Biol Med 2024; 169: 107943.

47.

Liu

, et al. Predicting mortality in intensive care unit patients with heart failure using an interpretable machine learning model: retrospective cohort study. J Med Internet Res 2022; 24: e38082.

48.

Zhang

Hong

. Machine learning for the prediction of volume responsiveness in patients with oliguric acute kidney injury in critical care. Crit Care 2019; 23: 112.

49.

Hai Minh

Quy

Tam

, et al. A practical approach for colorectal cancer diagnosis based on machine learning. PLoS One 2025; 20: e0321009.

50.

Deng

Dai

Liu

, et al. Clinical study of intelligent tongue diagnosis and oral microbiome for classifying TCM syndromes in MASLD. Chin Med 2025; 20: 78.

51.

Deng

Dai

Liu

, et al. Application of tongue image characteristics and oral-gut microbiota in predicting pre-diabetes and type 2 diabetes with machine learning. Front Cell Infect Microbiol 2024; 14: 1477638.

52.

Cheng

Chen

, et al. Objective study of the facial parameters of observations in patients with type 2 diabetes mellitus by machine learning. Ann Transl Med 2022; 10: 960.