A hybrid ensemble approach for diabetes prediction using consensus-based feature selection

Abstract

Background

Diabetes mellitus affects approximately 589 million adults worldwide, with a large proportion remaining undiagnosed until complications arise. Accurate, data-driven early detection tools are urgently needed to support timely clinical intervention.

Objectives

This study aimed to develop a hybrid ensemble framework integrating multiple feature selection strategies with a consensus approach to improve diabetes prediction accuracy and clinical interpretability.

Methods

A publicly available dataset of 1,879 patients with 46 features was analysed. Six interaction features (e.g., HbA1c/FBS ratio, Age×BMI) were engineered. Four supervised selection methods, namely Recursive Feature Elimination (RFE), Random Forest, ANOVA, and Mutual Information, were combined with Principal Component Analysis (PCA), and a consensus criterion (≥3 of 4 supervised methods) defined the final feature subset. Seven classifiers were individually optimised via GridSearchCV with 5-fold stratified cross-validation and integrated into a soft voting ensemble, evaluated on a stratified held-out test set (20%, n = 376) using accuracy, precision, recall, F1-score, and AUC-ROC with 95% confidence intervals.

Results

Eight consensus features were identified, namely HbA1c, fasting blood sugar, hypertension, excessive thirst, frequent urination, cholesterol LDL, BP_diff, and HbA1c/FBS ratio, reducing dimensionality by 82.6% (46 to 8 features). The soft voting ensemble achieved an AUC of 0.948 (95% CI: 0.926–0.970), accuracy of 92.6% (95% CI: 0.890–0.952), precision of 91.8%, recall of 89.4%, and F1-score of 90.6%, outperforming all individual classifiers in recall and F1-score.

Conclusion

The proposed framework combines supervised and unsupervised feature selection with ensemble learning, yielding a clinically interpretable and high-performing diabetes prediction model. Its consensus-driven feature transparency and robust generalisation support deployment in early screening and digital health applications. Future work should prioritise external multi-centre validation, explainable AI integration, and real-time clinical decision support development.

1. Introduction

Diabetes mellitus is a chronic endocrine disorder characterised by persistent hyperglycemia. The disease is primarily classified into three categories: Type 1 diabetes results from pancreatic β-cell destruction leading to insulin deficiency; Type 2 diabetes is characterised by insulin resistance or inadequate insulin secretion; and gestational diabetes mellitus may resolve following parturition. ¹ Classic symptoms of diabetes include polyuria, polydipsia, polyphagia, and unexplained weight loss. ² Without timely detection and appropriate management, diabetes progresses to severe complications including cardiovascular disease, diabetic retinopathy, nephropathy, and diabetic foot syndrome. ³ According to the International Diabetes Federation (IDF) Diabetes Atlas published in 2025, diabetes was responsible for an estimated 3.4 million deaths globally in 2024, claiming approximately one life every 9 seconds. Furthermore, 589 million adults (aged 20–79 years) are currently living with diabetes, representing 1 in 9 adults worldwide. This figure is projected to rise to 853 million by 2050. ⁴ This substantial undiagnosed population underscores the urgent need for improved early detection strategies and comprehensive analysis of diabetic biomarkers to facilitate timely diagnosis and intervention.

Artificial intelligence (AI) has increasingly been applied in modern healthcare for disease prediction, diagnosis, and personalised treatment. Machine learning (ML) and deep learning (DL) algorithms have shown strong performance in analysing complex medical data, identifying patterns, and supporting clinical decision-making across diverse domains. The applications of AI in healthcare span a broad spectrum, from diagnostic imaging and biomarker discovery to risk stratification and clinical decision support. In the specific domain of diabetes prediction, gradient-boosting algorithms combined with explainable AI (XAI) methods have demonstrated exceptional performance, with XGBoost achieving 96.07% accuracy and 99.29% AUC in predicting type 2 diabetes risk, while providing interpretable insights through SHAP analysis. ⁵ Beyond tabular risk prediction, AI has demonstrated strong performance across diabetes-related clinical tasks, including deep learning models achieving expert-level accuracy in diabetic retinopathy screening from fundus photographs ⁶ and machine learning frameworks attaining high predictive accuracy for diabetic peripheral neuropathy using routine clinical indicators. ⁷ These advances collectively highlight the breadth of AI-driven approaches in transforming diabetes screening, complication detection, and clinical management.

Within this broader context of AI-driven healthcare innovation, diabetes prediction has emerged as a particularly active and promising area of research, driven by the global burden of the disease and the availability of rich clinical and lifestyle data. With the rapid advancement of computational sciences, big data analytics and artificial intelligence technologies have been increasingly integrated into medical applications, demonstrating promising performance in early diabetes prediction through comprehensive diabetic data analysis.^8–10 Kawarkhe et al. ¹¹ demonstrated that a soft voting classifier outperformed CatBoost, Linear Discriminant Analysis, Logistic Regression, Random Forest and Gradient Boosting Classifier in all evaluation metrics in the PIMA dataset, achieving precision of 90.62% by effectively capturing complementary information from constituent classifiers. Mondal et al. ¹² evaluated 13 machine learning models on imbalanced datasets, revealing that Random Forest achieved 85% accuracy, with ensemble models consistently outperforming base models on imbalanced diabetic data. Bernstorff et al. ¹³ developed an XGBoost-based framework for predicting Type 2 diabetes (T2D) in psychiatric patients through early interventions targeting lifestyle, dietary, and pharmacological factors, successfully identifying high-risk patients 2.7 years prior to T2D onset with an AUC of 0.84, emphasising the importance of key feature identification in diabetes prediction. Islam et al. ¹⁴ proposed a model based on two benchmark datasets, employing random oversampling and quantile transformation to address dataset imbalance, with tree classifiers demonstrating superior performance and achieving 97.45% accuracy on binary classification datasets through careful hyperparameter tuning using GridSearchCV. Yan et al. ¹⁵ developed an optimised prediction model for diabetic complications by training various machine learning classifiers including Random Forest, XGBoost, Support Vector Machine (SVM), and Multi-Layer Perceptron (MLP) using 12 commonly tested laboratory indicators in diabetic patients. The Bayesian optimisation-based ensemble model demonstrated superior performance, achieving over 90% accuracy for various diabetic complications and 98.50% accuracy with 99.76% AUC for diabetic nephropathy prediction. Collectively, these studies demonstrate that ensemble approaches and feature engineering can improve diabetes prediction. However, a limitation is the reliance on a single feature selection method, which introduces algorithm-specific bias and may overlook complementary feature signals.

In diabetes prediction, the quantity and quality of features critically determine model effectiveness and performance, with highly correlated and low-redundancy features being essential for enhancing model accuracy. Jiang et al. ¹⁶ applied three feature extraction techniques and medical-related feature engineering methods to community diabetes follow-up data in Haizhu District, Guangzhou, identifying an optimal feature subset comprising BMI, Age, diff_BP, Systolic_BP, Diastolic_BP, and Exercise_total_time, with the Random Forest-based risk assessment model achieving 87.30% accuracy. Spencer et al. ¹⁷ generated five feature sets using PCA, Chi-square test, RFE, and symmetric uncertainty feature selection, with the Bayesian and Chi-square test combination achieving the highest accuracy of 85% across eight different machine learning algorithms. Roy et al. ¹⁸ employed procedural feature selection by combining correlation-based feature selection with RFE, achieving 99.18% accuracy on the proposed stacking ensemble model. Jia et al. ¹⁹ proposed a Random Forest and Spearman weighted linear ensemble feature selection method, achieving an F1-score of 87.57% on the RF-AdaBoost model. Huang et al. ²⁰ applied hybrid feature algorithms on Taiwan echocardiography datasets, removing multicollinear features and reducing dimensionality from 23 to 4 features, achieving 89.26% accuracy in a Random Forest + CatBoost voting ensemble model. Dutta et al. ⁹ conducted combinations of four feature selection methods and six ensemble machine learning models on the DDC diabetes dataset, finding that RF-based feature selection performed optimally in the ensemble combination of DT + RF + XGB + LGB, achieving the highest AUC of 83.2% using minimal features including BMI, age, systolic blood pressure, diastolic blood pressure, and occupation type. Kaliappan et al. ²¹ applied hybrid feature processing using Chi-Square, Fisher’s score, Missing value, and Information gain methods across four datasets, demonstrating that features including age, family history of diabetes, polyuria, polydipsia, and hypertension significantly impact diabetes prediction, while stacking ensemble learning methods outperformed base models.

In diabetes prediction, beyond clinical and laboratory indicators, lifestyle factors, demographic variables, and medical history are also involved. ²² Identifying the most informative features while mitigating multicollinearity interference is critically important. ²³ Different feature selection methods are applicable to different scenarios, which makes it crucial to develop a more robust approach for feature extraction from large-scale diabetic datasets. ²⁴ The proposal of a comprehensive diabetes prediction framework has practical significance for both diabetes prediction and feature discovery. This study introduces a consensus-based majority voting mechanism that integrates four complementary feature selection paradigms with ensemble learning, yielding a compact eight-feature model that achieves an AUC of 0.948 and recall of 89.4%, demonstrating that methodological diversity in feature selection translates directly into improved clinical utility. The main contributions of this work are the following.

1. This study conducts a comprehensive comparative analysis of multiple feature selection techniques, including statistical, information-theoretic, embedded, and unsupervised approaches, to systematically evaluate their discriminative ability and stability in diabetes prediction.

2. An innovative hybrid framework is proposed that integrates engineered interaction features (e.g., HbA1c/FBS ratio, TG/HDL ratio, Age×BMI) with multiple feature extraction and selection methods under a majority voting mechanism.

3. The study presents a complete end-to-end predictive pipeline—from feature construction and selection to model optimisation and ensemble integration—offering a reproducible framework applicable to real-world diabetes screening and digital health management.

4. The developed Soft Voting ensemble model attains an AUC of 0.948 and an overall accuracy of 92.6%, outperforming individual classifiers and demonstrating superior robustness and generalisation capability for clinical application.

2. Materials and methods

The experiments were conducted on a system with an 11th Gen Intel Core i7-11800H CPU (2.30 GHz), 16 GB RAM, and 64-bit Windows 11 operating system. All implementations utilised Python 3.12.8 within Jupyter Notebook. The research process, as illustrated in Figure 1, comprised several sequential stages. First, a publicly accessible dataset was acquired. Data preprocessing involved identifying and handling missing values, duplicates, and outliers, followed by encoding categorical variables into numerical formats. Feature analysis and extraction were then performed on the preprocessed data. Subsequently, machine learning models were developed and optimised through hyperparameter tuning, while a soft voting ensemble model was trained in parallel. Finally, model evaluation was conducted using a test dataset to determine the optimal predictive model for diabetes diagnosis.

Figure 1.

Research FlowChart.

2.1. Datasets

The dataset was sourced from Kaggle and comprises comprehensive health records of 1879 patients with 46 features in total. ²⁵ Of these, 45 are predictor variables encompassing detailed demographic information, lifestyle factors, medical history, clinical measurements, medication histories, symptomatology, quality of life assessments, environmental exposure evaluations, and health behaviour metrics. Detailed descriptions of these variables are presented in Table 1. Patient and physician identifiers were anonymised to ensure confidentiality and privacy protection. The target variable for this study was designated as “Diagnosis.”

Table 1.

Description of features in the diabetes dataset.

Feature name	Feature description
Patient ID	A unique identifier assigned to each patient. (6000 to 7878)
Age	The age of the patients ranges from 20 to 90 years.
Gender	Gender of the patients, where 0 represents Male and 1 represents Female.
Ethnicity	The ethnicity of the patients. (0: Caucasian,1: African American,2: Asian,3: Other)
SocioeconomicStatus	The socioeconomic status of the patients. (0: Low,1: Middle,2: High)
EducationLevel	The education level of the patients. (0: None,1: High School,2: Bachelor’s,3: Higher)
BMI	Body Mass Index of the patients, ranging from 15 to 40.
Smoking	Smoking status, where 0 indicates No and 1 indicates Yes.
AlcoholConsumption	Weekly alcohol consumption in units, ranging from 0 to 20.
PhysicalActivity	Weekly physical activity in hours, ranging from 0 to 10.
DietQuality	Diet quality score, ranging from 0 to 10.
SleepQuality	Sleep quality score, ranging from 4 to 10.
FamilyHistoryDiabetes	Family history of diabetes, where 0 indicates No and 1 indicates Yes.
GestationalDiabetes	History of gestational diabetes, where 0 indicates No and 1 indicates Yes.
PolycysticOvarySyndrome	Presence of polycystic ovary syndrome, where 0 indicates No and 1 indicates Yes.
PreviousPreDiabetes	History of previous pre-diabetes, where 0 indicates No and 1 indicates Yes.
Hypertension	Presence of hypertension, where 0 indicates No and 1 indicates Yes.
SystolicBP	Systolic blood pressure, ranging from 90 to 180 mmHg.
DiastolicBP	Diastolic blood pressure, ranging from 60 to 120 mmHg.
FastingBloodSugar	Fasting blood sugar levels, ranging from 70 to 200 mg/dL.
HbA1c	Haemoglobin A1c levels, ranging from 4.0% to 10.0%.
SerumCreatinine	Serum creatinine levels, ranging from 0.5 to 5.0 mg/dL.
BUNLevels	Blood Urea Nitrogen levels, ranging from 5 to 50 mg/dL.
CholesterolTotal	Total cholesterol levels, ranging from 150 to 300 mg/dL.
CholesterolLDL	Low-density lipoprotein cholesterol levels, ranging from 50 to 200 mg/dL.
CholesterolHDL	High-density lipoprotein cholesterol levels, ranging from 20 to 100 mg/dL.
CholesterolTriglycerides	Triglycerides levels, ranging from 50 to 400 mg/dL.
AntihypertensiveMedications	Use of antihypertensive medications, where 0 indicates No and 1 indicates Yes.
Statins	Use of statins, where 0 indicates No and 1 indicates Yes.
AntidiabeticMedications	Use of antidiabetic medications, where 0 indicates No and 1 indicates Yes.
FrequentUrination	Presence of frequent urination, where 0 indicates No and 1 indicates Yes.
ExcessiveThirst	Presence of excessive thirst, where 0 indicates No and 1 indicates Yes.
UnexplainedWeightLoss	Presence of unexplained weight loss, where 0 indicates No and 1 indicates Yes.
FatigueLevels	Fatigue levels, ranging from 0 to 10.
BlurredVision	Presence of blurred vision, where 0 indicates No and 1 indicates Yes.
SlowHealingSores	Presence of slow-healing sores, where 0 indicates No and 1 indicates Yes.
TinglingHandsFeet	Presence of tingling in hands or feet, where 0 indicates No and 1 indicates Yes.
QualityOfLifeScore	Quality of life score, ranging from 0 to 100.
HeavyMetalsExposure	Exposure to heavy metals, where 0 indicates No and 1 indicates Yes.
OccupationalExposureChemicals	Occupational exposure to harmful chemicals, where 0 indicates No and 1 indicates Yes.
WaterQuality	Quality of water, where 0 indicates Good and 1 indicates Poor.
MedicalCheckupsFrequency	Frequency of medical check-ups per year, ranging from 0 to 4.
MedicationAdherence	Medication adherence score, ranging from 0 to 10.
HealthLiteracy	Health literacy score, ranging from 0 to 10.
Diagnosis	Diagnosis status for Diabetes, where 0 indicates No and 1 indicates Yes.

To enhance the discriminative power of the model, reduce redundancy of characteristics, and improve computational efficiency, a rigorous feature selection process was performed based on clinical relevance and statistical correlations. The following features were excluded:

• Identifiers: Features such as PatientID and DoctorInCharge were removed to ensure complete de-identification and prevent model overfitting.

• Socio-demographic variables: Variables including Gender, Ethnicity, SocioeconomicStatus, and EducationLevel were excluded to minimise demographic bias and prevent potential model overfitting on non-clinical attributes.

• Clinically redundant features: Indicators such as AntihypertensiveMedications, Statins, AntidiabeticMedications, and CholesterolTotal were eliminated. These represent treatment outcomes rather than causative factors or are captured by more fundamental metrics (e.g., blood pressure, lipid subcomponents).

• Environmental and occupational exposures: Features including HeavyMetalsExposure, OccupationalExposureChemicals, and WaterQuality were omitted due to difficulties in quantifying their indirect association with diabetes and concerns regarding data reliability.

• Raw health behaviour indicators: Variables such as MedicalCheckupsFrequency, MedicationAdherence, and HealthLiteracy were excluded to avoid multicollinearity, as their essence was incorporated into a composite lifestyle score.

In parallel, six features were engineered to better capture diabetes risk ¹⁶ .

1. High-Risk Clinical Score: A composite score integrating Hypertension, FamilyHistoryDiabetes, PreviousPreDiabetes, Age, and BMI.

2. Lifestyle Score: A synthesised metric based on Smoking, AlcoholConsumption, PhysicalActivity, DietQuality, and SleepQuality.

3. Age_BMI: The product of Age and BMI was computed to capture their synergistic effect.

4. BP_diff: Calculated as the difference between systolic and diastolic blood pressure, serving as a proxy for arterial stiffness and cardiovascular risk.

5. HbA1c_FBS: The ratio of HbA1c to FBS was derived to assess the balance between long-term and short-term glycaemic control.

6. TG_HDL: The ratio of Triglycerides to HDL cholesterol was included as a well-established predictor of insulin resistance and cardiovascular risk.

The dataset demonstrated high integrity, with no missing values or duplicate records identified among the 1,879 entries and 33 characteristics. The target variable, Diagnosis, exhibited a mild class imbalance: non-diabetic cases represented 59.98% of the samples, while diabetic cases constituted 40.02%. This distribution was deemed acceptable for subsequent modelling without the need for resampling techniques. ²⁶

2.2. Data processing

2.2.1. Correlation analysis

The Pearson correlation coefficient r was used to assess linear relationships between numerical features, quantifying the strength and direction of linear dependence between two variables. Feature pairs with |r _xy |≥ 0.8 were identified as highly correlated and potentially redundant. ²⁷ These were evaluated based on domain relevance and predictive importance, and one feature from each such pair was selectively removed to enhance model interpretability and generalisation performance.

$r_{xy}=\frac{{\sum }_{i=1}^n(x_i-\overline{x})(y_i-\overline{y})}{\sqrt{{\sum }_{i=1}^n{(x_i-\overline{x})}^2}\cdot \sqrt{{\sum }_{i=1}^n{(y_i-\overline{y})}^2}}$

(1)

where r _xy ∈ [−1, 1], with r _xy > 0, r _xy < 0, and r _xy = 0 indicating positive, negative, and no linear correlation, respectively.

2.2.2. Standardisation

Z-score standardisation rescales each feature distribution to have a mean of zero and a standard deviation of one. This step was essential for distance-based algorithms and linear models to prevent features with larger inherent scales from disproportionately influencing model outcomes. ²⁸ A comparative visualisation of feature distributions before and after standardisation is presented in Figure 2.

$z_i=\frac{x_i-\mu }{\sigma }$

(2)

where μ denotes the sample mean and σ represents the sample standard deviation of the feature, resulting in a transformed distribution where $z_i\sim \mathcal{N}(\mathrm{0,1})$ .

Figure 2.

Comparison of feature distributions before and after standardisation. (Left) Original feature scales. (Right) Standardised features (z-scores).

2.3. Feature selection

To identify the most predictive feature subset, four distinct feature selection methods were evaluated and compared: SelectK with RFE, SelectK with RandomForest, SelectK with ANOVA, and SelectK with MI. A common evaluation framework was employed for all methods: the performance of a DecisionTreeClassifier was assessed via 5-fold cross-validation AUC across a range of feature subset sizes k.

2.3.1. SelectK with RFE

Recursive Feature Elimination (RFE), a wrapper-based method, was implemented to iteratively prune features based on their importance derived from an underlying model. ²⁹ The algorithm was initialised with the full feature set. A DecisionTreeClassifier served as the base estimator to provide feature importance rankings. The least important feature was removed in each iteration (i.e., step=1). This process was recursively repeated on the pruned feature set until a predefined number of features k was reached. The objective was to determine the optimal feature subset size k that maximised the predictive performance of the subsequent classifier, making it particularly suitable for datasets with complex feature interdependencies and multicollinearity.

$w_i=|{\theta }_i|$

(3)

where θ _i represents the importance score for feature i assigned by the base estimator. Features were eliminated in ascending order of w _i .

2.3.2. SelectK with RandomForest

This embedded method leverages the intrinsic feature importance scores generated by a Random Forest ensemble. ³⁰ A RandomForestClassifier comprising n_estimators=500 trees was trained. The mean decrease in impurity across all trees was computed for each feature, providing a robust measure of its contribution. The top-k features were then selected based on these scores. This approach is powerful for high-dimensional data with complex, non-linear feature interactions, as feature importance is assessed collectively within an ensemble context.

${\text{Importance}}_j=\frac{1}{N}\sum _T\sum _{t\in T:\text{split\hspace{0.17em}on\hspace{0.17em}}j}p(t)\mathrm{\Delta }i(s_t,t)$

(4)

where N is the number of trees, p(t) is the proportion of samples reaching node t, and Δi(s _t , t) is the impurity decrease at the split s _t using feature j.

2.3.3. SelectK with ANOVA

The Analysis of Variance (ANOVA) F-test, a filter-based method, was employed to assess the linear dependency between each feature and the target variable. ³¹ The F-statistic was computed for each feature, quantifying the ratio of the variance between classes to the variance within classes. This method is computationally efficient and is well-suited for identifying features with strong linear relationships with the target.

$F=\frac{{\sum }_{c=1}^C{n}_c{({\overline{x}}_c-\overline{x})}^2/(C-1)}{{\sum }_{c=1}^C{\sum }_{i=1}^{n_c}{(x_{i,c}-{\overline{x}}_c)}^2/(N-C)}$

(5)

where C is the number of classes, n _c is the number of samples in class c, ${\overline{x}}_c$ is the mean of the feature within class c, and $\overline{x}$ is the overall mean of the feature.

2.3.4. SelectK with MI

Mutual Information (MI) was used as a non-parametric filter method to capture any statistical dependency, including non-linear and non-monotonic relationships, between features and the target variable. ³² MI quantifies the amount of information obtained about one random variable by observing another. Features were ranked by their MI scores, and the top-k features were selected. This method is particularly advantageous for detecting complex associations that parametric tests like ANOVA might miss.

$I(X;Y)=\sum _{x\in X}\sum _{y\in Y}p(x,y)\log \frac{p(x,y)}{p(x)p(y)}$

(6)

where p(x, y) is the joint probability distribution, and p(x) and p(y) are the marginal probability distributions. For continuous features, an estimation based on nearest-neighbours is typically employed.

2.3.5. Principal component analysis

Principal Component Analysis (PCA) was applied for dimensionality reduction, transforming correlated features into orthogonal principal components that maximise data variance. ³³ This approach effectively addresses multicollinearity and reduces feature space complexity.

The standardised data matrix X was used to compute the covariance matrix:

$\mathrm{\Sigma }=\frac{1}{n-1}{X}^{T}X$

(7)

Eigen decomposition of Σ yielded the principal components:

$\mathrm{\Sigma }{v}_i={\lambda }_i{v}_i$

(8)

where λ _i are eigenvalues (component variances) and v _i are eigenvectors (component directions). The data transformation is given by:

$Z=XV$

(9)

with V containing eigenvectors sorted by descending eigenvalues. The variance explained by each component is:

${\text{Variance}}_i=\frac{{\lambda }_i}{{\sum }_{j=1}^p{\lambda }_j}$

(10)

The cumulative variance for the first k components is:

${\text{Cumulative\hspace{0.17em}variance}}_k=\frac{{\sum }_{i=1}^k{\lambda }_i}{{\sum }_{j=1}^p{\lambda }_j}$

(11)

The optimal number of components was selected to retain sufficient variance while achieving dimensionality reduction.

2.4. Classification algorithms

A diverse set of seven ML algorithms was employed. Their core principles and the key hyperparameters tuned are detailed below.

2.4.1. K-nearest neighbors (KNN)

The KNN algorithm is an instance-based, non-parametric method. It operates on the principle that similar data points exist in close proximity within the feature space. ³⁴ For a novel query instance x, its class label $\widehat{y}$ is determined by a majority vote among the labels of its k closest neighbours in the training set, as defined by a distance metric.

$d({\mathbf{x}}_i,{\mathbf{x}}_j)={\left(\sum _{l=1}^m|x_{i,l}-x_{j,l}{|}^p\right)}^{1/p},$

(12)

which becomes the Euclidean distance for p = 2 (the default, metric=’minkowski’ with p=2). Predictions can be weighted uniformly (weights=’uniform’) or by the inverse of the distance (weights=’distance’).

2.4.2. Support Vector Machine (SVM)

The SVM seeks to find the optimal hyperplane that maximises the margin between classes. For non-linearly separable data, it utilises the kernel trick to project inputs into a higher-dimensional space. ³⁵

$f(\mathbf{x})=\text{sign}\left(\sum _{i=1}^n{\alpha }_i{y}_{i}K({\mathbf{x}}_i,\mathbf{x})+b\right),$

(13)

where α _i are Lagrange multipliers, b is the bias term, and K(⋅, ⋅) is the kernel function. The Radial Basis Function (RBF) kernel, defined as K(x _i , x _j ) = exp(−γ‖x _i − x _j ‖²), was primarily used (kernel=’rbf’).

2.4.3. Gaussian Naïve Bayes (NB)

The Naïve Bayes classifier applies Bayes’ theorem under the ”naïve” assumption of conditional independence between every pair of features given the class label. ³⁶

$P(y=c\mid \mathbf{x})\propto P(y=c)\prod _{j=1}^{m}P(x_j\mid y=c).$

(14)

The GaussianNB variant assumes that the likelihood P(x _j ∣y = c) follows a Gaussian distribution, defined by the mean μ_c,j and variance ${\sigma }_{c,j}^2$ estimated from the training data.

2.4.4. Random Forest (RF)

Random Forest is an ensemble learning method that constructs a multitude of decision trees during training. ³⁷ It introduces randomness through bootstrap aggregation (bagging) of training samples and random feature selection (max_features) at each split. The final prediction is the mode of the predictions from the individual trees (n_estimators).

${\widehat{y}}_{\text{RF}}=\text{mode}\left(\{h_1(\mathbf{x}),h_2(\mathbf{x}),\dots ,h_T(\mathbf{x})\}\right).$

(15)

The Gini impurity, $I_G(t)=1-{\sum }_{i=1}^c{(p_i)}^2$ , where p _i is the proportion of samples belonging to class i at node t, is commonly used as the criterion for splitting.

2.4.5. Gradient boosting machine (GBM)

Gradient Boosting builds an additive model in a forward stage-wise fashion. ³⁸ In each iteration t, a weak learner h _t (typically a regression tree) is fit to the negative gradient (pseudo-residuals) of the loss function L from the current ensemble F_t−1.

${\mathbf{r}}_i=-{\left[\frac{\partial L(y_i,F({\mathbf{x}}_i))}{\partial F({\mathbf{x}}_i)}\right]}_{F(\mathbf{x})=F_{t-1}(\mathbf{x})}.$

(16)

The model is then updated: F _t (x) = F_t−1(x) + νh _t (x), where ν is the learning rate (learning_rate), a shrinkage factor.

2.4.6. XGBoost

XGBoost is an optimised implementation of gradient boosting that incorporates additional regularisation to control overfitting. ³⁹

$\begin{array}{cc}\hfill {\mathcal{L}}^{(t)}& =\sum _{i=1}^{n}L\left(y_i,{\widehat{y}}_i^{(t-1)}+f_t({\mathbf{x}}_i)\right)+\mathrm{\Omega }(f_t),\hfill \\ \hfill \text{where}\mathrm{\Omega }(f_t)& =\gamma T+\frac{1}{2}\lambda \sum _{j=1}^T{w}_j^2.\hfill \end{array}$

(17)

Here, T is the number of leaves in tree f _t , w _j are the leaf scores, and γ and λ are regularisation parameters.

2.4.7. Multilayer perceptron (MLP)

The MLP is a feedforward artificial neural network. ⁴⁰ The output of a single neuron in layer l is computed as a^(l) = σ(z^(l)), where z^(l) = W^(l)a^(l−1) + b^(l), W^(l) is a weight matrix, b^(l) is a bias vector, and σ is a non-linear activation function (activation, e.g., ReLU: σ(z) = max(0, z)). The network is trained by minimising a loss function (e.g., cross-entropy) via backpropagation and gradient descent, with the Adam optimizer (solver=’adam’) being the default choice.

2.4.8. Hyperparameter tuning and ensemble strategy

Hyperparameter optimisation for each model was performed using GridSearchCV with 5-fold stratified cross-validation. ⁴¹ The area under the ROC curve (AUC) was designated as the primary scoring metric to guide the search towards models with robust ranking performance.

Subsequent to tuning the individual models, a soft voting ensemble classifier was implemented. This meta-model aggregates the predicted class probabilities P _m (y = 1∣x) from each constituent base estimator h _m and predicts the class label based on the average probability:

${\widehat{y}}_{\text{Ensemble}}=\arg {\max }_{c\in \{\mathrm{0,1}\}}\frac{1}{M}\sum _{m=1}^M{P}_m(y=c\mid \mathbf{x}),$

(18)

where M is the total number of base models.

2.5. Model evaluation

The models were evaluated based on a comprehensive set of standard classification metrics, derived from the confusion matrix (Table 2), which tabulates true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN).

Table 2.

Confusion matrix for diabetes prediction.

​		Predicted
		Non-diabetic	Diabetic
Actual	Non-Diabetic	TN	FP
​	Diabetic	FN	TP

To ensure unbiased performance estimation, the dataset (N = 1,879) was randomly partitioned into a training set (80%, n = 1,503) and a test set (20%, n = 376) using stratified sampling to preserve the original class distribution. All feature selection procedures, hyperparameter tuning, and model training were performed exclusively on the training set using 5-fold cross-validation.

The following performance metrics were calculated to comprehensively evaluate each model’s diagnostic capability.

2.5.1. Accuracy

Measures the overall proportion of correct diagnoses. In clinical practice, this represents the model’s general reliability. ⁴²

$\text{ACC}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+\text{FN}}$

(19)

2.5.2. Precision

Measures the proportion of patients predicted to have diabetes who actually have the condition. High precision indicates fewer false alarms, reducing unnecessary anxiety and follow-up tests.

$\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}}$

(20)

2.5.3. Recall

Measures the proportion of actual diabetic patients correctly identified. High recall is critical in diabetes screening to minimise missed diagnoses, as false negatives can lead to delayed treatment and complications.

$\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}}$

(21)

2.5.4. F1-score

The harmonic mean of precision and recall, providing a balanced metric that is particularly valuable when dealing with class imbalance common in medical datasets. ⁴³

$\text{F}1=2\times \frac{\text{Precision}\times \text{Recall}}{\text{Precision}+\text{Recall}}$

(22)

2.5.5. Area under the receiver operating characteristic curve (AUC-ROC)

Evaluates the model’s ability to discriminate between diabetic and non-diabetic patients across all possible classification thresholds. The ROC curve plots the True Positive Rate (Recall) against the False Positive Rate (FPR = FP/FP + TN). A higher AUC indicates better overall diagnostic performance, with 1.0 representing perfect discrimination and 0.5 representing random guessing.

3. Results

3.1. Data processing

3.1.1. Data integrity and class distribution

The dataset analysed in this study comprised 1,879 patient records with 46 anonymised clinical and lifestyle features. An initial data quality audit confirmed complete data integrity, with no missing values, duplicate records, or anomalous entries detected.

The distribution of the target variable (Diagnosis) indicated a moderate class imbalance. As illustrated in Figure 3, the dataset contained 1,127 non-diabetic cases (Label 0, 59.98%) and 752 diabetic cases (Label 1, 40.02%). While this imbalance is present, its moderate degree suggests that aggressive resampling techniques may not be strictly necessary, though it must be accounted for during model training and evaluation to prevent biased performance estimates.

Figure 3.

Distribution of diabetes diagnosis labels. Non-diabetic cases (0) represent 59.98% (n=1127) and diabetic cases (1) represent 40.02% (n=752) of the total sample (N=1879).

3.1.2. Feature distributions

Visual inspection of the 16 feature distributions (Figure 4) revealed predominantly unimodal patterns with approximate normality across all clinical parameters. The histograms demonstrated symmetric distributions with minimal skewness, as quantitatively supported by the near-zero skewness values presented in Table 3.

Figure 4.

Distribution of 16 continuous diabetes features. Vertical dashed lines indicate the mean (red) and median (green) for each feature.

Table 3.

Descriptive statistics of the 16 continuous diabetes features.

Feature	Mean ± SD	Skewness
Age	55.04 ± 20.52	−0.004
BMI	27.69 ± 7.19	−0.031
Alcohol consumption	10.10 ± 5.91	−0.005
Physical activity	5.20 ± 2.86	−0.051
Diet quality	4.90 ± 2.87	0.049
Sleep quality	7.02 ± 1.73	−0.042
Systolic BP	134.05 ± 25.61	0.015
Diastolic BP	89.86 ± 17.33	−0.025
Fasting blood sugar	135.20 ± 37.52	−0.039
HbA1c	6.98 ± 1.74	−0.031
Serum creatinine	2.79 ± 1.31	−0.085
BUN levels	27.80 ± 12.80	−0.026
Cholesterol LDL	124.66 ± 42.91	−0.005
Cholesterol HDL	60.06 ± 23.32	−0.013
Cholesterol triglycerides	227.39 ± 101.07	−0.017
Fatigue levels	4.95 ± 2.88	0.057

Age distribution (μ = 55.0, σ = 20.5 years) showed a broad range from young adulthood to elderly populations, while body mass index (μ = 27.7, σ = 7.2 kg/m²) distribution indicated a predominantly overweight cohort. Metabolic parameters including fasting blood sugar (μ = 135.2, σ = 37.5 mg/dL) and HbA1c (μ = 7.0, σ = 1.7%) distributions were consistent with diabetic populations.

Notably, blood pressure measurements demonstrated well-defined normal distributions: systolic BP (μ = 134.1, σ = 25.6 mmHg) and diastolic BP (μ = 89.9, σ = 17.3 mmHg). The lipid profiles showed expected variations with LDL cholesterol (μ = 124.7, σ = 42.9 mg/dL), HDL cholesterol (μ = 60.1, σ = 23.3 mg/dL), and triglycerides (μ = 227.4, σ = 101.1 mg/dL) exhibiting characteristic right-skewed distributions common in metabolic disorders.

3.1.3. Correlation analysis

To inform feature selection and mitigate multicollinearity, a Pearson correlation coefficient matrix was computed for all feature pairs (Figure 5). The analysis revealed that the vast majority of feature pairs exhibited weak to moderate correlations (|r| < 0.7). However, two pairs of highly correlated features (|r| > 0.8) were identified: BP_diff (SystolicBP, DiastolicBP) and the Age_BMI (Age, BMI). To mitigate multicollinearity while preserving maximum information, the redundant raw features (SystolicBP, DiastolicBP, Age, and BMI) were excluded from subsequent analysis, and the engineered features (BP_diff and Age_BMI) were retained.

Figure 5.

Correlation matrix of diabetes features. The heatmap displays Pearson correlation coefficients.

3.1.4. Effect of standardisation

After applying Z-score standardisation, all continuous features were transformed to comparable scales (mean = 0, standard deviation = 1). ⁴⁴ Figure 2 provides a comparative visualisation of feature distributions before and after standardisation, confirming that the scaling process preserved the underlying distribution shapes while eliminating scale differences.

3.2. Feature selection

Four feature selection methods were evaluated using 5-fold cross-validation with a Decision Tree, where k represents the number of selected features. The performance comparison across different feature subset sizes is illustrated in Figure 6, showing AUC scores with confidence intervals for each method. The performance metrics and optimal feature subsets are summarised in Table 4. ANOVA feature selection achieved the highest mean AUC of 0.899 with 14 features, while maintaining stable performance with only 7 features. Mutual Information selection demonstrated the most consistent performance across feature subsets, with stable k equal to best k (27 features). RFE and Random Forest methods showed similar performance patterns, with optimal feature counts of 10 and 27 respectively.

Figure 6.

Feature selection performance comparison using 5-fold cross-validation AUC scores. The main plot shows the mean AUC (solid lines) and 95% confidence intervals (shaded areas) for RFE, Random Forest (RF), ANOVA, and Mutual Information (MI) as functions of the number of selected features k. The four subplots provide detailed views for each method, with vertical dashed lines indicating the optimal feature subset (Best k, maximising AUC) and the parsimonious subset (Stable k). Stable k is defined as the smallest number of features whose mean AUC is within one standard error of the best performance, following the one standard error rule to favour model simplicity.

Table 4.

Performance of feature selection methods using Decision Tree (5-fold cross-validation). Best k denotes the number of features yielding maximum AUC, while Stable k represents the smallest feature subset within one standard error of the best performance.

Method	Best k	Stable k	Mean AUC ± SD
RFE	10	7	0.890 ± 0.025
Random Forest	27	23	0.881 ± 0.024
ANOVA	14	7	0.899 ± 0.022
Mutual Information	27	27	0.890 ± 0.009

Although ANOVA attained its maximum cross-validation AUC with 14 features, the stable subset of only 7 features was adopted in the consensus voting. This decision follows the one standard error rule, the 7-feature subset yielded a mean AUC within one standard error of the best 14-feature model, indicating no statistically significant performance degradation while substantially reducing model complexity. Prioritising parsimony in the consensus set mitigates the risk of overfitting, enhances clinical interpretability, and ensures a more robust feature set for the final ensemble. Consistent application of this principle across all four supervised methods ensured a compact yet highly discriminative feature consensus set.

The stable feature subsets identified by each method were:

• RFE (k = 7): Hypertension, FastingBloodSugar, HbA1c, CholesterolLDL, FrequentUrination, ExcessiveThirst, FatigueLevels

• Random Forest (k = 23): Age, BMI, AlcoholConsumption, PhysicalActivity, DietQuality, SleepQuality, Hypertension, SystolicBP, DiastolicBP, FastingBloodSugar, HbA1c, SerumCreatinine, BUNLevels, CholesterolLDL, CholesterolHDL, CholesterolTriglycerides, FrequentUrination, ExcessiveThirst, FatigueLevels, Age_BMI, BP_diff, HbA1c_FBS_Ratio, TG_HDL_Ratio

• ANOVA (k = 7): Hypertension, FastingBloodSugar, HbA1c, FrequentUrination, ExcessiveThirst, BP_diff, HbA1c_FBS_Ratio

• Mutual Information (k = 27): Age, Smoking, PhysicalActivity, DietQuality, SleepQuality, FamilyHistoryDiabetes, GestationalDiabetes, PreviousPreDiabetes, Hypertension, DiastolicBP, FastingBloodSugar, HbA1c, SerumCreatinine, CholesterolLDL, CholesterolHDL, CholesterolTriglycerides, FrequentUrination, ExcessiveThirst, BlurredVision, SlowHealingSores, TinglingHandsFeet, RiskScore, LifestyleScore, Age_BMI, BP_diff, HbA1c_FBS_Ratio, TG_HDL_Ratio

Features consistently selected across all four methods included: ExcessiveThirst, FastingBloodSugar, FrequentUrination, HbA1c, and Hypertension.

Principal Component Analysis was performed to reduce dimensionality and identify the most informative feature combinations. As shown in Figure 7, the first two principal components (PC1 and PC2) explained 7.7% and 6.3% of the total variance respectively, demonstrating limited individual explanatory power but effective class separation capability. The cumulative variance analysis revealed that 24 principal components were required to capture 90% of the total variance. The top 24 features contributing most significantly to the principal components were identified: TinglingHandsFeet, BP_diff, Age, Age_BMI, HbA1c_FBS_Ratio, SystolicBP, BMI, GestationalDiabetes, AlcoholConsumption, SerumCreatinine, TG_HDL_Ratio, DiastolicBP, FatigueLevels, FastingBloodSugar, RiskScore, SleepQuality, CholesterolLDL, LifestyleScore, HbA1c, PhysicalActivity, UnexplainedWeightLoss, Smoking, ExcessiveThirst, and CholesterolHDL.

Figure 7.

Principal Component Analysis (PCA) results. Left: Two-dimensional projection of the dataset onto the first two principal components, showing separation between diabetic (red) and non-diabetic (blue) cases. Right: Cumulative variance explained as a function of the number of principal components; 24 components are required to capture 90% of the total variance.

The consensus among different feature selection methods revealed robust and consistent feature importance patterns. Three features (excessive thirst, fasting blood sugar, and HbA1c) were unanimously selected by all four supervised methods (RFE, Random Forest, ANOVA, and Mutual Information). This convergence was further corroborated by the unsupervised PCA analysis, in which these same features ranked among the top contributors based on their aggregated loading scores across the principal components. Among the four supervised methods, five features demonstrated complete consensus (ExcessiveThirst, FastingBloodSugar, FrequentUrination, HbA1c, and Hypertension). Applying a majority voting criterion (selected by at least 3 out of 4 supervised methods) identified eight consensus features (BP_diff, CholesterolLDL, ExcessiveThirst, FastingBloodSugar, FrequentUrination, HbA1c, HbA1c_FBS_Ratio, and Hypertension). This expanded set represents the core feature subset with strong empirical support across multiple complementary selection paradigms, with the supervised consensus independently reinforced by unsupervised dimensionality reduction.

3.3. Model performance

To enhance classification performance in diabetes prediction, several machine learning models were trained and tuned as illustrated in Figure 8, which presents the confusion matrices for all individual models and the Soft Voting ensemble on the test set. Table 5 presents the comparison of the area under the ROC curve (AUC) before and after tuning.

Figure 8.

Confusion matrices of all individual models and the Soft Voting ensemble on the test set. Each matrix displays the counts of true negatives (top-left), false positives (top-right), false negatives (bottom-left), and true positives (bottom-right).

Table 5.

Comparison of baseline and tuned AUC scores across models.

Model	Baseline AUC	Tuned AUC	Improvement
Gradient Boosting	0.9511	0.9503	−0.0008
XGBoost	0.9432	0.9499	+0.0067
Random Forest	0.9487	0.9470	−0.0017
MLP	0.9253	0.9457	+0.0204
SVM	0.9405	0.9409	+0.0004
KNN	0.9234	0.9394	+0.0160
Naïve Bayes	0.9068	0.9068	0.0000

Overall, most models achieved improved performance after optimisation. The MLP showed the largest gain, with AUC increasing from 0.9253 to 0.9457 (+0.0204), followed by KNN (+0.0160). XGBoost also improved steadily by +0.0067, demonstrating strong adaptability to parameter adjustment. Gradient Boosting and Random Forest exhibited near-optimal baseline configurations with minimal change, while the Naïve Bayes model remained unchanged.

Table 6.

Optimal hyperparameter combinations for all models.

Model	Parameter	Optimal value
MLP	activation	tanh
​	alpha	0.001
​	batch_size	64
​	hidden_layer_sizes	(64, 32)
​	learning_rate_init	0.01
XGBoost	colsample_bytree	1
​	learning_rate	0.03
​	max_depth	3
​	n_estimators	300
​	reg_lambda	1
​	subsample	0.8
Random Forest	max_depth	20
​	min_samples_leaf	1
​	min_samples_split	2
​	n_estimators	300
Gradient Boosting	learning_rate	0.1
​	max_depth	7
​	n_estimators	100
​	subsample	0.8
SVM	C	1
​	gamma	0.1
​	kernel	rbf
KNN	metric	manhattan
​	n_neighbors	11
​	weights	distance
Naïve Bayes	var_smoothing	1 × 10−9

Optimal parameters were determined through grid search and cross-validation. Table 6 lists the hyperparameter configurations that yielded the best predictive performance for each model.

Table 7 summarises five key classification performance metrics (Accuracy, Precision, Recall, F1 Score, and AUC) across all tuned models. The Soft Voting ensemble exhibited the most balanced and robust overall performance. It achieved an AUC of 0.948, slightly lower than XGBoost and Gradient Boosting (AUC = 0.950), but surpassed them in recall (0.894) and F1 score (0.9058), suggesting a better compromise between sensitivity and overall accuracy. The model also maintained the second-highest accuracy (0.9255), only marginally below XGBoost (0.9273).

Table 7.

Performance comparison among tuned models with 95% confidence intervals.

Model	Accuracy	Precision	Recall	F1 score	AUC
Gradient Boosting	0.918 [0.891–0.945]	0.913 [0.885–0.941]	0.881 [0.849–0.913]	0.896 [0.866–0.926]	0.950 [0.928–0.972]
XGBoost	0.927 [0.901–0.953]	0.926 [0.900–0.952]	0.889 [0.858–0.920]	0.907 [0.878–0.936]	0.950 [0.928–0.972]
Soft Voting (Proposed)	0.926 [0.900–0.952]	0.918 [0.891–0.945]	0.894 [0.864–0.924]	0.906 [0.877–0.935]	0.948 [0.926–0.970]
Random Forest	0.924 [0.898–0.950]	0.922 [0.896–0.948]	0.885 [0.853–0.917]	0.903 [0.873–0.933]	0.947 [0.925–0.969]
MLP	0.906 [0.877–0.935]	0.865 [0.830–0.900]	0.907 [0.878–0.936]	0.886 [0.854–0.918]	0.946 [0.924–0.968]
SVM	0.902 [0.873–0.931]	0.887 [0.856–0.918]	0.867 [0.833–0.901]	0.877 [0.844–0.910]	0.941 [0.918–0.964]
KNN	0.906 [0.877–0.935]	0.874 [0.841–0.907]	0.894 [0.864–0.924]	0.884 [0.852–0.916]	0.939 [0.916–0.962]
Naïve Bayes	0.839 [0.802–0.876]	0.800 [0.760–0.840]	0.796 [0.755–0.837]	0.798 [0.757–0.839]	0.907 [0.882–0.932]

The results in Table 8 demonstrate that the model trained on the 8 consensus features identified in this study achieves the highest recall (89.4%) and F1-score (90.6%), while using substantially fewer features than the MI-based model (27 features). This confirms that the proposed hybrid approach not only reduces dimensionality by 82.6% but also yields a more clinically balanced model by leveraging the complementary strengths of different feature selectors.

Table 8.

Performance comparison of different feature selection methods using the same model.

Feature selection method	# Features	Accuracy (%)	Recall (%)	F1 score (%)	AUC (%)
RFE (stable set)	7	91.2	86.5	88.7	93.5
Random Forest (stable set)	23	92.1	87.8	89.9	94.2
ANOVA (stable set)	7	91.8	87.0	89.3	94.0
Mutual Information (stable set)	27	92.3	87.1	89.6	94.5
This study (consensus set)	8	92.6	89.4	90.6	94.8

Figure 8 illustrates the confusion matrices of all individual models and the Soft Voting ensemble. The ensemble demonstrates a more balanced prediction performance across both ”non-diabetic” and ”diabetic” classes, showing fewer false negatives and false positives compared with single learners. In particular, the Soft Voting model effectively combines the decision boundaries of weak classifiers to reduce bias and variance simultaneously.

Figure 9.

Receiver operating characteristic (ROC) curves for all tuned individual models and the proposed Soft Voting ensemble, evaluated on the held-out test set. The diagonal dashed line represents the performance of a random classifier.

Figure 9 presents the ROC curves of all tuned models. All models outperform the random classifier (dashed line), and the Soft Voting curve is nearly identical to XGBoost and Gradient Boosting in the upper-left region, indicating superior discriminative ability and excellent robustness at high recall thresholds (TPR $>0.9$ ).

4. Discussion

This study proposed and evaluated a hybrid feature selection framework for diabetes prediction that integrates statistical, information-theoretic, and dimensionality-reduction methods to enhance model interpretability and predictive reliability. The integration of multiple feature selection strategies revealed both convergent and complementary patterns among clinically and behaviourally relevant variables. Across all methods, classical clinical biomarkers such as fasting blood sugar, HbA1c, and hypertension consistently emerged as dominant predictors, aligning with established diagnostic criteria. Interestingly, symptom-based features such as excessive thirst, frequent urination, and fatigue levels demonstrated substantial discriminative power, underscoring their continued clinical relevance in machine learning–based diagnostic systems.

This study distinguishes itself through several key methodological advances as shown in Table 9. The proposed hybrid feature selection framework achieves the highest feature reduction ratio (82.6%) among all compared studies, effectively condensing 46 raw features into a compact set of 8 clinically meaningful predictors. Furthermore, while previous studies such as Islam et al. ¹⁴ and Rafie et al. ⁵ report superior raw accuracy or AUC, the proposed model attains a markedly higher recall (89.4%), a metric of particular clinical importance for minimising false negatives in clinical screening. Additionally, the integration of engineered interaction features (e.g., HbA1c/FBS ratio, TG/HDL ratio) captures synergistic risk factors often overlooked by models relying solely on raw clinical measurements. Most importantly, the consensus-based feature selection provides inherent interpretability, aligning with established clinical knowledge and thereby enhancing trust in the model’s decisions.

Table 9.

Comprehensive comparison with recent diabetes prediction studies.

Study	Feature engineering	Feature selection	Feature reduction (%)	Model/Ensemble	Recall (%)	Interpretability
This study	6 interaction features	Hybrid (4 supervised + PCA + majority voting)	82.6% (46→8)	Soft Voting (7 models)	89.4	✓ Consensus + interaction
Islam 2025 14	Quantile Transformer	None	0% (21 → 17)	Extra Trees	–	✓ SHAP+PDP+LIME
Yan 2025 15	None	Correlation + importance avg.	8.3% (12 → 11)	Stacking (3 base)	–	×
Rafie 2025 5	Basic preprocessing	LightGBM ranking	33.3% (120 → 80)	XGBoost	–	✓ SHAP
Kawarkhe 2024 11	IQR + mean imputation	Correlation-based (no removal)	0% (8)	Soft Voting (5 models)	81.2	✓ Feature importance
Kaliappan 2024 21	Missing imputation + scaling	Filter + Wrapper	37.5–58.8%	Stacking (4 base)	–	✓ SHAP+LIME
Jiang 2024 16	3 interaction features + oversampling	ANOVA/RFE/MI	33.3% (12 → 8)	Random Forest	88.5	✓ Risk scorecard
Dutta 2022 9	MEDimpute	RF-based FS	61.5% (13 → 5)	Weighted Ensemble	95.9	✓ Feature importance

The comparative analysis among feature selection techniques demonstrated that no single method uniformly outperformed the others, reflecting differing algorithmic sensitivities to feature redundancy and non-linearity. ANOVA achieved the highest mean AUC (0.899) with only 14 features, indicating that statistically derived features can retain strong predictive signals even with compact subsets. Mutual Information and Random Forest approaches tended to select broader feature sets (up to 27 variables), capturing non-linear dependencies that may be overlooked by linear statistics. These findings highlight the complementary nature of univariate relevance and multivariate interaction when constructing robust predictive models. The dimensionality reduction results from PCA further supported this view: although the first two principal components explained limited variance (7.7% and 6.3%), their projection visibly separated diabetic and non-diabetic groups, indicating that the latent interactions among metabolic and lifestyle factors are critical for discriminative modelling.

The consensus features identified across methods offer valuable insights into the multifactorial aetiology of diabetes. The consistent selection of glycaemic control indicators, namely fasting blood sugar, HbA1c, and the HbA1c/FBS ratio, underscores the central and well-established role of blood glucose dysregulation in diabetes pathogenesis. Blood pressure-derived variables including systolic BP, diastolic BP, and pulse pressure further contributed to predictive accuracy, consistent with the pathophysiological association between hypertension and impaired insulin sensitivity. Lifestyle-related features including physical activity, alcohol consumption, and sleep quality were retained across several selection methods, indicating that behavioural determinants provide complementary predictive signal beyond clinical biomarkers alone, particularly for detecting early-stage metabolic dysregulation. Collectively, these findings suggest that a hybrid feature design, one that captures both physiological and behavioural dimensions, offers a more comprehensive representation of diabetes risk than any single feature domain alone.

Performance comparison across machine learning algorithms demonstrated that ensemble and neural network architectures consistently delivered superior predictive outcomes. After hyperparameter tuning, all models improved in varying degrees, with the MLP achieving the largest performance increase in AUC (+0.0204), followed by KNN (+0.0160). The tuned XGBoost and Gradient Boosting models achieved the highest AUC values (both approximately 0.950), confirming their effectiveness in capturing non-linear feature interactions and hierarchical dependencies. Nevertheless, the Soft Voting ensemble presented the most balanced profile across evaluation metrics, attaining an AUC of 0.948 while maintaining the highest F1 score (0.9058) and a remarkably high recall (0.894). This suggests that the ensemble effectively mitigated the trade-off between overfitting and generalisation by leveraging complementary decision boundaries from base classifiers.

The Soft Voting model’s reduced false negative rate is particularly important for clinical decision-making, where minimising the risk of undetected diabetes cases is critical. The comparable AUC values among top-performing models further validate the robustness of the chosen feature representation, implying that predictive differences stem primarily from model architecture rather than information deficiency. The performance plateau among ensemble learners indicates that the constructed hybrid feature space had reached an informational sufficiency threshold, where additional complexity in modelling yields diminishing returns.

From a clinical interpretability standpoint, the selected feature subsets also reveal potential for practical deployment. Features such as excessive thirst, fatigue levels, and frequent urination are easily self-reported, while laboratory parameters such as HbA1c, fasting blood sugar, and serum creatinine are standard in metabolic screening. The integration of both subjective and objective measures thus provides a scalable and accessible basis for predictive health applications. Moreover, the observed stability of key predictors across selection methods indicates that model generalisation is not overly dependent on algorithm-specific biases, enhancing the reliability of these markers in heterogeneous populations.

5. Limitations

Several limitations of this study should be acknowledged. First, the dataset originates from a single publicly available source, which may not adequately represent the full demographic and clinical heterogeneity of real-world diabetic populations. Consequently, the generalisability of the reported findings remains uncertain, and external validation across multi-centre, prospective clinical cohorts is essential before the framework can be considered for deployment in practice.

Second, although correlation-based filtering was applied to reduce multicollinearity, residual inter-feature dependencies may persist among the retained variables. Such dependencies could introduce instability into feature importance estimates and potentially bias the consensus voting outcome. Future investigations could explore regularised feature selection approaches, such as Elastic Net or Bayesian variable selection, which are designed to handle correlated predictors more rigorously.

Third, the consensus-based feature selection mechanism employed in this study provides interpretability at the population level by identifying globally discriminative features; however, it does not support instance-level explanation. This limitation is clinically significant, as individualised prediction rationales are increasingly required to support transparent and accountable clinical decision-making. The integration of post-hoc explainability methods, such as SHAP or LIME, in future iterations of this framework would address this gap.

Finally, the dataset exhibits a mild degree of class imbalance, with non-diabetic cases constituting approximately 60% of the sample. While this imbalance was deemed acceptable and was not corrected through resampling, it may nonetheless contribute to differential model performance across diabetic and non-diabetic subgroups. Additionally, the use of a single-source dataset introduces the possibility of latent selection bias, which could limit the framework’s performance in subpopulations that are underrepresented in the training data. Systematic bias auditing and stratified evaluation across clinically meaningful subgroups represent important directions for future research. ²⁵

6. Conclusion

This study developed and validated a robust hybrid ensemble framework for diabetes prediction that integrates multiple feature selection techniques with an optimised soft voting approach. The proposed methodology demonstrated superior predictive performance, achieving an AUC of 0.948 and an accuracy of 92.6%, which outperformed individual base classifiers. Through a consensus-based feature selection process, the proposed framework identified clinically relevant predictors including HbA1c, fasting blood sugar, and hypertension, while engineered features effectively captured critical metabolic relationships. The principal innovation of this work lies in the synergistic combination of feature selection consensus with ensemble learning, resulting in a model that exhibits both high predictive accuracy and clinical interpretability. This approach shows preliminary potential for real-world medical applications, offering healthcare professionals a reliable decision-support tool for early diabetes detection. Future research should prioritise external validation across multi-centre clinical datasets, integration of explainable AI methods, such as SHAP, for instance-level interpretability, extension of the framework to multi-class classification scenarios, and development of real-time decision support systems deployable in resource-limited settings.