Machine learning-based DNA microarray analysis for disease detection using the MICRO-AI framework

Abstract

DNA microarray is a transformative technique in genomics, enabling simultaneous examination of thousands of gene expression levels. However, noise, high dimensionality (typically 12,000–22,000 genes), small sample sizes (155–1097 samples) and class imbalance complicate the extraction of meaningful diagnostic patterns. This paper presents MICRO-AI (Microarray Classification and Recognition using Artificial Intelligence), a comprehensive machine learning framework for DNA microarray analysis and automated disease diagnosis. The framework integrates advanced preprocessing (quantile normalisation, ComBat batch correction, KNN imputation), attention-weighted adaptive feature selection using recursive feature elimination with cross-validation, and heterogeneous ensemble classification combining gradient boosting machines, random forests and support vector machines with adaptive weight optimisation. A novel attention-based feature fusion mechanism dynamically prioritises discriminative gene expression signatures, reducing dimensionality by over 99% (from ∼20,000 to ∼127 genes) without loss of biological significance. MICRO-AI is validated on six benchmark datasets from three repositories: Gene Expression Omnibus (GEO), The Cancer Genome Atlas (TCGA) and ArrayExpress, spanning breast cancer, gastric cancer, ovarian cancer and leukaemia across 2321 total samples. Experimental results demonstrate average classification accuracy of 96.8%, sensitivity of 95.2%, specificity of 97.4%, F1-score of 96.0%, Matthews correlation coefficient of 0.928, and area under the receiver operating characteristic curve of 0.983. Comparative benchmarking against 10 state-of-the-art methods shows that MICRO-AI achieves 1.2–7.5% higher accuracy with an average training time of 52.3 s, representing 2.4–6.0× faster execution than deep learning alternatives. The modular architecture enables seamless integration with medical informatics systems for scalable clinical diagnostic deployment.

Introduction

The introduction of high-throughput genomic technologies in biomedical research and clinical diagnostics has radically changed the study of the human body. DNA microarray technology enables scientists to measure the expression of thousands of genes simultaneously, providing a broad molecular view of cellular conditions. This has been especially useful in oncology, where gene expression signatures may be used to differentiate cancer subtypes, predict treatment response or assess prognosis. Nevertheless, microarray data analysis poses major computational challenges, including high dimensionality, small sample sizes, and batch and biological noise.

It is important to clarify the terminological relationship between artificial intelligence (AI) and machine learning (ML) in this work. AI refers to the broad discipline of creating computational systems capable of performing tasks that require human intelligence (perception, reasoning, learning and decision-making). At the same time, ML is a core subset of AI focusing on algorithms that improve through experience and data-driven learning without explicit programming for each task. The title emphasises ‘Machine Learning’ because core technical contributions – feature selection, ensemble classification, attention-based scoring – are grounded in ML algorithms. However, the framework is named MICRO-AI (Microarray Classification and Recognition using Artificial Intelligence) to reflect the broader vision of an intelligent, end-to-end diagnostic system encompassing ML classification, automated preprocessing, adaptive decision-making and clinical integration capabilities representing an AI-driven diagnostic workflow. Thus, ML describes the specific algorithmic methodology, while AI describes the holistic philosophy of intelligent system design.

Recent advances in AI and ML have demonstrated significant potential for addressing these challenges, with deep learning architectures, ensemble techniques and advanced feature selection procedures achieving promising results in microarray classification tasks. However, characterising and identifying complex diseases from genomic data requires integrated computational frameworks that can handle inherent data heterogeneity across platforms, batch effects and limited sample sizes. A detailed review of existing approaches is provided in the section ‘Related work’.

Beyond genomic data analysis, ML-assisted intelligent diagnosis has gained remarkable traction across diverse analytical platforms. Recent advances in spectroscopy-integrated ML have demonstrated that computational intelligence combined with techniques such as Raman spectroscopy and electrochemical biosensing can substantially enhance the sensitivity and specificity of diagnostic assays, achieving classification accuracies exceeding 95% for cancer detection across chemical and bioelectronic detection modalities. These developments underscore the translational potential of ML-driven frameworks like MICRO-AI, which extend the paradigm of intelligent diagnosis from analytical chemistry to high-dimensional genomic profiling.

Figure 1 also depicts the expanding field of applications of DNA microarray analysis and the application of ML methodologies in bypassing clinical areas.

Figure 1.

Convergence of machine learning, genomics and clinical diagnostics in disease detection: an overview of the landscape of DNA microarray analysis.

The principal contributions of this paper are as follows:

Attention-weighted feature fusion mechanism: Novel attention-based feature scoring (equations (8) and (9)) computing gene-level discriminative importance from statistical moment embeddings (mean, variance, skewness, kurtosis) and mutual information with class labels, achieving >99% dimensionality reduction (∼20,000→∼127 genes) while preserving biologically relevant signatures validated by Gene Ontology enrichment.

Heterogeneous ensemble classification with adaptive weighting: Ensemble framework combining gradient boosting machines (GBM), random forests (RF), and support vector machines (SVM) through constrained adaptive weight optimisation using Sequential Least Squares Programming (SLSQP), achieving 5–7% higher accuracy than individual classifiers and 0.8% improvement over equal weighting.

Comprehensive end-to-end preprocessing pipeline: Integrated pipeline sequentially applying quantile normalisation (cross-sample comparability), ComBat empirical Bayes correction (batch effect removal) and weighted KNN imputation (missing values), contributing 1.7% accuracy improvement over uncorrected data (Table 6).

Extensive multi-repository, multi-disease validation: Rigorous evaluation on six datasets from three independent repositories (GEO, TCGA, ArrayExpress) across four cancer types (breast, gastric, ovarian, leukaemia), totalling 2321 samples, consistently outperforming ten state-of-the-art methods across seven evaluation metrics.

Clinical informatics integration architecture: Modular architecture producing calibrated probability estimates (isotonic regression) with expected calibration error (ECE) minimisation, enabling integration with medical informatics workflows for interpretable and scalable clinical decision support.

Related work

This section reviews the literature on DNA microarray analysis, ML approaches for disease detection and integrated diagnostic frameworks. The review is organised into four subsections covering preprocessing techniques, feature selection methods, classification algorithms and clinical integration approaches.

DNA microarray data preprocessing

Sound microarray analysis relies on proper preprocessing. Raw microarray data are subject to systematic biases, including background noise, dye effects and batch effects. Qvick et al. ¹ successfully reached pan-cancer detection by performing DNA methylation profiling via enzymatic conversion library preparation and targeted sequencing, and proved that epigenetic biomarkers have clinical application. Yoon et al. ² surveyed bioinformatic and monitoring technologies, eDNA analysis, and emphasised the importance of computational methods for handling the complex streams of biological information. To examine the molecular signatures of related metabolic disorders, Sultan ³ conducted microarray analysis of differentially expressed genes in peripheral blood samples from individuals with gestational diabetes mellitus and type 2 diabetes, identifying molecular biomarkers of glycogen metabolic disorders. Li et al. ⁴ comprehensively reexamined copy number variants of uncertain significance using existing guidelines and future genome sequencing, demonstrating the relevance of consistent analytical frameworks. Developed microarray analysis has especially helped in cancer research. Ben Ali et al. ⁵ discovered a new lung cancer biomarker signature via data mining. They performed initial validation in an in vitro experiment, highlighting the importance of data preprocessing in biomarker discovery. Tselios et al. ⁶ used geometric methods from common transcriptomics in acute lymphoblastic leukaemia and rhabdomyosarcoma and extended pathway simulation using new preprocessing methods.

Normalisation has evolved from simple scaling techniques to complex quantile-based methods and powerful multi-array averaging (RMA). On the one hand, Yuan et al. ⁷ developed MambaYOLO-ML, a state-space-based model for mulberry leaf disease detection, demonstrating the applicability of preprocessing principles in biological contexts. Atesoglu and Bingol ⁸ enhanced hybrid models for grape leaf disease detection through feature engineering and AI-based fusion.

However, existing preprocessing methods suffer from several limitations: (1) most normalisation techniques assume homogeneous data distributions, which may not hold across diverse microarray platforms; (2) batch correction methods like ComBat require prior knowledge of batch assignments; and (3) imputation strategies rarely account for the structured missingness patterns common in multi-centre genomic studies.

Feature selection and dimensionality reduction

High-dimensional microarray data often involves limited sample sizes for thousands of features (genes), leading to the curse of dimensionality. Takou et al. ⁹ modelled gene expression in response to environmental stressors using natural variation in DNA sequences and ML to decode genotype–phenotype interactions. Shao ¹⁰ surveyed the use of ML in microwave medical imaging and lesion detection and noted that selecting features is not straightforward, as in microarray analysis. To categorise disease diagnoses via breath analysis, Kokkotis et al. ¹¹ used AI and ML on high-dimensional sensor data and employed sophisticated feature extraction. Surimova et al. ¹² identified PSG and candidate genes as potential biomarkers of therapy resistance in B-ALL using chromosomal microarray analysis and ML.

The use of ML in disease detection spans many fields.

Filter techniques such as variance thresholding, mutual information and ANOVA have been widely used to screen features at first instance. Feature-slicing algorithms, such as recursive feature elimination (RFE) and wrapper selection methods, deliver finer gene groupings by comparing feature groups against the classifiers’ accuracy. Iftikhar et al. ¹³ demonstrated the clinical implementation of ML frameworks to identify chronic kidney disease at early stages by carefully selecting features from clinical parameters.

Key limitations of current feature selection approaches include: (1) filter methods (variance, ANOVA) ignore feature interactions and classifier-specific relevance; (2) wrapper methods are computationally expensive and prone to overfitting on small sample sizes; (3) embedded methods are tied to specific classifier architectures and lack transferability; and (4) none of the existing methods incorporate attention-based dynamic weighting that considers both statistical and biological relevance simultaneously.

Classification algorithms for disease detection

Ghosh and Ura ¹⁴ proved the combination of principles of DNA-based computing and artificial neural networks to improve pattern recognition of smart manufacturing. To emphasise the significance of physicochemical property representation in molecular ML, Zhao et al. ¹⁵ have developed EM-DeepSD, a deep neural network model based on cell-free DNA end-motif signal splitting for cancer diagnosis. ML classifiers have been widely utilised for disease detection using microarrays. Hernandez Toledo et al. ¹⁶ developed extreme ML-based computational tasks for the agricultural disease detection experiment and demonstrated that ensemble methods can be quite versatile. Salaris et al. ¹⁷ used ML on social media foodborne event detection, which exemplifies methods of classifying health surveillance. Microarray analysis by Chen et al. ¹⁸ revealed that sepsis is characterised by hyperactivity of TH17 immunity, with Treg cell cytokine TGF- $\beta$ overexpressed. Abroudi et al. ¹⁹ compared microarray data and single-cell RNA-seq and identified a relationship between the tumour environment and the extracellular matrix during epithelial-mesenchymal transition in prostate cancer.

SVMs remain widely used in microarray classification because of their efficiency in high-dimensional spaces with few samples. Vrbaski et al. ²⁰ provided an overview of ML for detecting chronic kidney disease using scintigraphy and compared various classifier architectures. Dzermeikait et al. ²¹ used ML models to identify metritis early in dairy cows, demonstrating the range of applications of biological classification. In their study, Liu et al. ²² computationally examined smooth muscle cell plasticity in atherosclerosis and vascular calcification by analysing the differential gene expression.

Recently, deep learning has become increasingly popular in microarray analysis. The researchers of Lazcano-García et al. ²³ deployed deep learning-based grapevine disease early symptom detection systems on edge computing devices. Tao et al. ²⁴ reviewed multi-omics methods for predicting cancer immunotherapy treatments, emphasising the integration of ML with complex biological information. Ge et al. ²⁵ further contributed to the field through bibliometric analysis of cDNA-based surveys, highlighting evolving research trends in molecular diagnostics. Wu et al. ²⁶ further demonstrated the utility of machine learning in constructing pan-cancer prognostic models based on immunogenic cell death genes, showcasing the broad applicability of ML approaches in oncogenomics.

Despite promising results, classification methods face notable limitations: (1) single classifiers (SVM, RF) cannot capture the full spectrum of decision boundaries in heterogeneous cancer data; (2) deep learning models (CNN, Transformer) require large training datasets and lack interpretability; (3) ensemble methods with fixed equal weighting fail to exploit classifier complementarity; and (4) most methods do not provide calibrated probability estimates necessary for clinical decision support.

Integrated diagnostic frameworks

ML diagnostic tools need to be carefully clinically integrated, with attention to interpretability, reliability and workflow compatibility. The OncoOrigin, an integrative AI tool for predicting the primary cancer site that includes a graphical user interface, was validated by Brlek et al. ²⁷ and demonstrated the practical viability of ML in oncology. Arakelyan et al. ²⁸ assigned transcriptomic subtypes to chronic lymphocytic leukaemia samples using Nanopore RNA-sequencing and self-organising maps. The classifications of prognostic subtypes and treatment strategies in the soft tissue sarcomas were developed using transcriptomic-based classification in Esperança-Martins et al. ²⁹ The article by Christodoulou et al. ³⁰ used data-driven and structure-based modelling to help identify human DNMT1 inhibitors by integrating pathways for structure-activity relationships into drug discovery cascades.

Cabello-Lima et al. ³¹ established interpolation-based encoding schemes to classify protein-DNA/RNA interactions. Vaccine development and AI-based frameworks for disease prevention have become significant applications. Goud et al. ³² developed an AI-guided platform for the design and development of next-generation avian viral vaccines. In AI for risk-stratification in diffuse large B-cell lymphoma, Popescu and Gaman ³³ performed a systematic review of articles and compared models and the predictive performance of age-relevant classification.

Existing integrated frameworks are limited by: (1) the lack of end-to-end optimisation across preprocessing, feature selection and classification stages; (2) insufficient cross-platform validation; (3) absence of attention mechanisms for biologically-informed feature prioritisation; and (4) inadequate consideration of clinical deployment requirements such as calibrated confidence estimates and interpretable outputs.

Research gap analysis

However, despite major achievements in individual components, current methods have several limitations. First, most approaches can be used independently for preprocessing, feature selection and classification, and they are not end-to-end optimised. Second, deep learning methods are low in interpretability, making them difficult to adopt in clinical settings. Third, validation can be performed on single datasets, and they are not generally tested cross-platform. Fourth, it is often disregarded that it should be integrated with clinical informatics systems.

To fill these gaps, MICRO-AI has a single framework that optimises preprocessing with attention-weighted feature selection and ensemble classification with both biological explanations and clinical patients.

Proposed methodology

This section introduces the MICRO-AI model, including system architecture, mathematical models and algorithmic implementation. Figure 2 shows the entire system architecture.

Figure 2.

MICRO-AI system architecture reveals the preprocessing, attention-focused feature selection, ensemble classification and clinical integration modules.

System overview

Let $\mathbf{X}\in {\mathbb{R}}^{N\times G}$ denote the microarray expression matrix, where N represents the number of samples and G represents the number of genes. The corresponding label vector is $\mathbf{y}\in \{0,1,\dots ,C-1{\}}^N$ for C disease classes. The objective is to learn a mapping function $f:{\mathbb{R}}^G\to \{0,1,\dots ,C-1\}$ that accurately predicts disease class from gene expression profiles.

Data preprocessing module

Quantile normalisation

Raw microarray intensities must be normalised to ensure comparability across samples. Let $x_{ij}$ denote the expression value of gene j in sample i. Quantile normalisation transforms each sample to have an identical distribution:

$${\tilde{x}}_{ij}=Q_j(F_i(x_{ij}))$$

(1)

where $F_i(\cdot )$ is the empirical cumulative distribution function of sample i, and $Q_j(\cdot )$ is the quantile function of the reference distribution for gene j.

The reference distribution is computed as the mean across all samples:

$$Q_j(p)={\displaystyle \frac{1}{N}\sum _{i=1}^N{x}_{i(j)}^{(p)}}$$

(2)

where $x_{i(j)}^{(p)}$ denotes the $p$ -th quantile of gene j expression values across all samples.

Batch effect correction

MICRO-AI consists of four interdependent modules, namely: (1) Data Preprocessing Module used to perform the process of normalisation and the correction of the batch effect; (2) Feature Selection Module, which implements the process of attention-weighted gene prioritisation; (3) Ensemble Classification Module, which involves the combination of multiple learners with adaptive weighting; and (4) Clinical Integration Module, allowing the deployment of MICRO-AI in the medical informatics workflows.

$$Y_{ijg}={\alpha }_g+X{\beta }_g+{\gamma }_{ig}+{\delta }_{ig}{ϵ}_{ijg}$$

(3)

where ${\alpha }_g$ is the overall gene mean, $X{\beta }_g$ represents biological covariates, ${\gamma }_{ig}$ is the additive batch effect and ${\delta }_{ig}$ is the multiplicative batch effect.

The corrected expression values are obtained by:

$$Y_{ijg}^{*}={\displaystyle \frac{Y_{ijg}-{\hat{\gamma }}_{ig}^{*}}{{\hat{\delta }}_{ig}^{*}}+{\hat{\alpha }}_g+X{\hat{\beta }}_g}$$

(4)

Missing value imputation

Missing values are imputed using k-nearest neighbours (KNN) with weighted averaging:

$${\hat{x}}_{ij}={\displaystyle \frac{{\sum }_{k\in {\mathcal{N}}_K(i)}{w}_{ik}\cdot x_{kj}}{{\sum }_{k\in {\mathcal{N}}_K(i)}{w}_{ik}}}$$

(5)

where ${\mathcal{N}}_K(i)$ denotes the KNN of sample i, and weights are computed as:

$$w_{ik}=\exp \left(-{\displaystyle \frac{d{(i,k)}^2}{2{\sigma }^2}}\right)$$

(6)

with $d(i,k)$ representing Euclidean distance and $\sigma$ as the bandwidth parameter.

Attention-weighted feature selection

Initial filtering

Low-variance genes are removed using median absolute deviation (MAD) filtering:

$${\mathrm{MAD}}_j=\mathrm{median}(|x_{ij}-\mathrm{median}(x_{\cdot j})|)$$

(7)

Genes with ${\mathrm{MAD}}_j<{\tau }_{MAD}$ are excluded, where ${\tau }_{MAD}$ is an adaptive threshold determined by the overall MAD distribution.

Attention score computation

We compute attention scores that quantify each gene's discriminative importance. Let ${\mathbf{h}}_j\in {\mathbb{R}}^d$ denote the embedding of gene j. The attention score is computed as:

$$a_j={\displaystyle \frac{\exp ({\mathbf{w}}^T\tanh ({\mathbf{W}}_h{\mathbf{h}}_j+{\mathbf{b}}_h))}{{\sum }_{k=1}^G\exp ({\mathbf{w}}^T\tanh ({\mathbf{W}}_h{\mathbf{h}}_k+{\mathbf{b}}_h))}}$$

(8)

where ${\mathbf{W}}_h\in {\mathbb{R}}^{d\times d}$ , ${\mathbf{b}}_h\in {\mathbb{R}}^d$ and $\mathbf{w}\in {\mathbb{R}}^d$ are learnable parameters.

The gene embedding is computed from expression statistics:

$${\mathbf{h}}_j=\sigma ({\mathbf{W}}_e{[{\mu }_j,{\sigma }_j,{\mathrm{skew}}_j,{\mathrm{kurt}}_j,{\mathrm{MI}}_j]}^T+{\mathbf{b}}_e)$$

(9)

where ${\mu }_j$ , ${\sigma }_j$ , ${\mathrm{skew}}_j$ , ${\mathrm{kurt}}_j$ are statistical moments, and ${\mathrm{MI}}_j$ is the mutual information between gene j and class labels.

Clarification on gene embedding computation and integration: The gene embedding defined in equation (9) is computed in a deterministic manner and does not involve end-to-end gradient-based learning. Specifically, the embedding vector ${\mathrm{h}}_j$ is derived from fixed statistical descriptors of gene expression, including first- and higher-order moments and mutual information with class labels. The parameters ${\mathrm{W}}_e$ and ${\mathrm{b}}_e$ are learned only within the lightweight attention scoring module and are not jointly optimised with the downstream classifiers.

Notably, the resulting gene embeddings and joint attention scores are used solely to prioritise features and select them before classifier training. After identifying the best gene subsets using the attention-weighted recursive feature elimination with cross-validation (RFECV) process, the ensemble classifiers (GBM, RF and SVM) will be trained separately on the feature space obtained by the modifications. As such, the attention mechanism does not affect the optimisation of classifier parameters or the decision boundary, thereby guaranteeing modularity, interpretability and compatibility with classical ML models.

Recursive feature elimination with cross-validation

The RFECV process removes features of minimal significance repeatedly based on the classifier scores. At iteration t the final feature set is:

$${\mathcal{G}}^{(t+1)}={\mathcal{G}}^{(t)}\mathrm{\setminus }\{j:r_j^{(t)}\le {\theta }_t\}$$

(10)

where $r_j^{(t)}$ is the importance ranking at iteration t and ${\theta }_t$ is the elimination threshold.

Cross-validation performance is monitored:

$${\mathrm{CV}}^{(t)}={\displaystyle \frac{1}{K}\sum _{k=1}^K\mathrm{Accuracy}(f^{(t)},{\mathcal{D}}_k^{val})}$$

(11)

The optimal feature subset ${\mathcal{G}}^{*}$ maximises cross-validation accuracy:

$${\mathcal{G}}^{*}=\arg \underset{{\mathcal{G}}^{(t)}}{\max }{\mathrm{CV}}^{(t)}$$

(12)

Ensemble classification module

Base classifiers

MICRO-AI employs three base classifiers with complementary strengths.

GBM: The GBM sequentially fits decision trees to residuals:

$$F_m(\mathbf{x})=F_{m-1}(\mathbf{x})+\eta \cdot h_m(\mathbf{x})$$

(13)

where $\eta$ is the learning rate and $h_m$ minimises:

$$h_m=\arg \underset{h}{\min }\sum _{i=1}^{N}L(y_i,F_{m-1}({\mathbf{x}}_i)+h({\mathbf{x}}_i))$$

(14)

The selection of the GBM over alternative boosting algorithms (AdaBoost, XGBoost) was motivated by several considerations. AdaBoost is more sensitive to noisy data and outliers than GBM, as it assigns exponentially increasing weights to misclassified samples, which is problematic in microarray data with inherent biological noise. While XGBoost offers computational advantages through parallel tree construction and regularised learning objectives, a comparative analysis (Table 5) shows that, when used as a standalone classifier, XGBoost achieves 93.4% accuracy, 3.4% lower than MICRO-AI's ensemble. GBM's key advantage in the MICRO-AI ensemble lies in complementarity with RF and SVM: GBM's sequential residual learning captures complex nonlinear feature interactions, RF's bagging provides variance reduction and stability, and SVM's kernel mapping excels in high-dimensional boundary estimation. When combined through adaptive weighting (equations (19) to (21)), this heterogeneous ensemble outperforms any homogeneous boosting approach.

Supplementary experiments replacing GBM with XGBoost yielded an XGBoost-RF-SVM ensemble achieving 96.3% average accuracy versus 96.8% for the GBM-RF-SVM configuration, with the difference attributable to GBM's slightly better complementarity with RF and SVM in adaptive weighting. The AdaBoost replacement yielded 95.1% accuracy due to its higher sensitivity to noisy microarray features. These results confirm that GBM provides an optimal balance of performance, robustness and ensemble complementarity for the MICRO-AI framework.

RF: RF aggregates predictions from B bootstrap-trained decision trees:

$$\hat{y}=\mathrm{mode}\{T_b(\mathbf{x}){\}}_{b=1}^B$$

(15)

where each tree $T_b$ is trained on a bootstrap sample ${\mathcal{D}}_b$ with random feature subsets of size $\sqrt{G}$ .

SVM: SVM solves the optimisation problem:

$$\underset{\mathbf{w},b,\xi }{\min }{\displaystyle \frac{1}{2}\parallel \mathbf{w}{\parallel }^2+C\sum _{i=1}^N{\xi }_i}$$

(16)

subject to:

$$y_i({\mathbf{w}}^T\varphi ({\mathbf{x}}_i)+b)\ge 1-{\xi }_i,{\xi }_i\ge 0$$

(17)

where $\varphi (\cdot )$ is the kernel mapping, and C controls regularisation.

We employ the radial basis function (RBF) kernel:

$$K({\mathbf{x}}_i,{\mathbf{x}}_j)=\exp (-\gamma \parallel {\mathbf{x}}_i-{\mathbf{x}}_j{\parallel }^2)$$

(18)

Adaptive weight optimisation

Ensemble predictions are computed as weighted combinations:

$${\hat{p}}_c(\mathbf{x})=\sum _{m=1}^M{w}_m\cdot p_c^{(m)}(\mathbf{x})$$

(19)

where $p_c^{(m)}(\mathbf{x})$ is the probability of class c from classifier m, and weights satisfy ${\sum }_m{w}_m=1$ , $w_m\ge 0$ .

The best weights can be achieved by minimising loss on validation data based on cross-entropy:

$$\mathcal{L}(\mathbf{w})=-\sum _{i=1}^{N_{val}}\sum _{c=0}^{C-1}{y}_{ic}\log ({\hat{p}}_c({\mathbf{x}}_i))$$

(20)

The constraints are solved by means of SLSQP:

$${\mathbf{w}}^{*}=\arg \underset{\mathbf{w}}{\min }\mathcal{L}(\mathbf{w})\mathrm{s}.\mathrm{t}.\mathbf{w}\ge 0,1^T\mathbf{w}=1$$

(21)

Confidence calibration

Probability estimates are calibrated using isotonic regression:

$$p_{cal}(c|\mathbf{x})=g_c({\hat{p}}_c(\mathbf{x}))$$

(22)

where $g_c$ is a monotonically increasing function fitted on calibration data to minimise ECE:

$$\mathrm{ECE}=\sum _{b=1}^B{\displaystyle \frac{|B_b|}{N}|\mathrm{acc}(B_b)-\mathrm{conf}(B_b)|}$$

(23)

Algorithmic implementation

Algorithm 1 presents the complete MICRO-AI training procedure.

Algorithm 1: MICRO-AI training algorithm.

Expression matrix $\mathbf{X}\in {\mathbb{R}}^{\mathit{N}\times \mathit{G}}$ , Labels $\mathbf{y}$ Trained ensemble model $\mathcal{E}$ , Selected genes $\mathcal{G}*$ // Preprocessing $\tilde{\mathbf{X}}\leftarrow$ QuantileNormalize( $\mathbf{X}$ ) via Eq. 1 $\tilde{\mathbf{X}}\leftarrow$ ComBatCorrection( $\tilde{\mathbf{X}}$ ) via Eq. 3 $\tilde{\mathbf{X}}\leftarrow$ KNNImpute( $\tilde{\mathbf{X}}$ ) via Eq. 5 // Feature Selection Filter genes: ${\mathcal{G}}_0\leftarrow \{\mathit{j}:{\mathrm{MAD}}_{\mathit{j}}>{\mathit{\tau }}_{\mathit{M}\mathit{A}\mathit{D}}\}$ Compute embedding ${\mathbf{h}}_{\mathit{j}}$ via Eq. 9 Compute attention score ${\mathit{a}}_{\mathit{j}}$ via Eq. 8 $\mathcal{G}*\leftarrow$ RFECV( $\tilde{\mathbf{X}}[:,{\mathcal{G}}_0]$ , $\mathbf{y}$ , attention scores) via Eq. 10 $\mathbf{X}*\leftarrow \tilde{\mathbf{X}}[:,\mathcal{G}*]$ // Ensemble Training Train base classifier ${\mathit{f}}_{\mathit{m}}$ on $(\mathbf{X}*,\mathbf{y})$ Optimise weights $\mathbf{w}*$ via Eq. 21 Calibrate probabilities via Eq. 22 Return $\mathcal{E}=\{({\mathit{f}}_{\mathit{m}},{\mathit{w}}_{\mathit{m}}*){\}}_{\mathit{m}=1}^{\mathit{M}}$ , $\mathcal{G}*$

Algorithm 2 details the inference procedure for new samples.

Algorithm 2: MICRO-AI inference algorithm.

Test sample $\mathbf{x}\in {\mathbb{R}}^{\mathit{G}}$ , Trained ensemble $\mathcal{E}$ , Genes $\mathcal{G}*$ Predicted class $\hat{\mathit{y}}$ , Confidence $\mathit{p}*$ , $\tilde{\mathbf{x}}\leftarrow$ ApplyNormalization( $\mathbf{x}$ ) $\mathbf{x}*\leftarrow \tilde{\mathbf{x}}[\mathcal{G}*]$ ${\mathit{p}}^{(\mathit{m})}\leftarrow {\mathit{f}}_{\mathit{m}}.\mathrm{predict}\mathrm{\_}\mathrm{proba}(\mathbf{x}*)$ $\hat{\mathit{p}}\leftarrow \sum _{\mathit{m}}{\mathit{w}}_{\mathit{m}}\cdot {\mathit{p}}^{(\mathit{m})}$ via Eq. [eq:ensemble] ${\hat{\mathit{p}}}_{\mathit{c}\mathit{a}\mathit{l}}\leftarrow$ Calibrate( $\hat{\mathit{p}}$ ) via Eq. [eq:calibration] $\hat{\mathit{y}}\leftarrow \mathbf{a}\mathbf{r}\mathbf{g}\mathbf{m}\mathbf{a}{\mathbf{x}}_{\mathit{c}}{\hat{\mathit{p}}}_{\mathit{c}\mathit{a}\mathit{l}}(\mathit{c})$ $\mathit{p}*\leftarrow \mathbf{m}\mathbf{a}{\mathbf{x}}_{\mathit{c}}{\hat{\mathit{p}}}_{\mathit{c}\mathit{a}\mathit{l}}(\mathit{c})$ Return $\hat{\mathit{y}}$ , $\mathit{p}*$

Complexity analysis

The computational complexity of MICRO-AI is analysed as follows.

Preprocessing: Quantile normalisation requires $O(NG\log G)$ for sorting. ComBat correction is $O(NG)$ . KNN imputation is $O(N^{2}G)$ .

Feature selection: MAD filtering is $O(NG)$ . Attention computation is $O(G{d}^2)$ , where d is the embedding dimension. RFECV with T iterations require $O(T\cdot C_{base}\cdot K)$ where $C_{base}$ is the complexity of the base classifier and K is the number of CV folds.

Ensemble training: GBM training is $O(M_{trees}\cdot N\cdot G^{*}\cdot \log N)$ . RF training is $O\left(B\cdot N\cdot \sqrt{G^{*}}\cdot \log N\right)$ . SVM training is $O(N^2{G}^{*}+N^3)$ .

Inference: Ensemble inference is $O(M\cdot C_{inf})$ , where $C_{inf}$ is the inference complexity of individual classifiers.

SVM dominates the overall training complexity: $O(N^3)$ for small datasets or $O(N^2{G}^{*})$ for larger datasets with kernel approximation.

To provide hardware-independent computational cost measures, we report floating-point operations (FLOPs) for each MICRO-AI component. For single-sample inference with G* = 127 selected features, the total computational cost is approximately 2.1 × 10⁴ FLOPs, dominated by SVM kernel evaluations (∼1.6 × 10⁴ FLOPs) (Table 1). During training, total FLOPs scale to approximately 4.7 × 10⁹ for the GSE2034 dataset (N = 286, G = 22,283), with SVM kernel matrix construction accounting for the largest share (∼3.2 × 10⁹ FLOPs). For comparison, deep learning methods such as GeneFormer require approximately 2.8 × 10¹⁰ FLOPs for equivalent training, resulting in 6× the computational overhead of MICRO-AI. These FLOP estimates confirm that MICRO-AI achieves superior classification performance at substantially lower computational cost.

Table 1.

Computational complexity in FLOPs.

Component	Operation	FLOPs per sample	FLOPs (GSE20347, G* = 127)
Quantile normalisation	Sort + rank mapping	O(G log G)	∼2.3 × 103
ComBat correction	Linear transformation	O(G)	∼5.1 × 102
KNN imputation	Distance computation + averaging	O(N × G)	∼3.6 × 104
MAD filtering	Median computation	O(N × G)	∼6.4 × 106
Attention score	Matrix multiplication + softmax	O(G × d2) = O(G × 642)	∼5.2 × 105
GBM inference	Tree traversal (200 trees, depth 6)	O(Mtrees × depth)	∼1.2 × 103
RF inference	Tree traversal (500 trees)	O(B × depth)	∼3.0 × 103
SVM inference	Kernel evaluation (support vectors)	O(nsv × G*)	∼1.6 × 104
Ensemble aggregation	Weighted sum	O(M × C)	∼15
Total inference	–	–	∼2.1 × 104

Results and evaluation

This section provides a detailed experimental analysis of MICRO-AI, including descriptions of the datasets, the experimental setup, performance and comparisons.

Datasets

Table 2 summarises the benchmark datasets used for evaluation.

Table 2.

Summary of benchmark datasets.

Dataset	Samples	Genes	Classes	Platform	Source
GSE2034	286	22,283	2	U133A	Gene Expression Omnibus
GSE7390	198	22,283	2	U133A	GEO
TCGA-BRCA	1097	20,531	5	RNA-Seq	The Cancer Genome Atlas
GSE62254	300	20,155	4	U133Plus2	GEO
E-MTAB-365	155	18,943	3	IlluminaHT12	ArrayExpress
GSE9891	285	12,625	2	U95Av2	GEO

Dataset sources:

Gene Expression Omnibus (GEO): https://www.ncbi.nlm.nih.gov/geo/

GSE2034: https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc = GSE2034

GSE7390: https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc = GSE7390

GSE62254: https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc = GSE62254

GSE9891: https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc = GSE9891

The Cancer Genome Atlas (TCGA): https://portal.gdc.cancer.gov/projects/TCGA-BRCA

ArrayExpress: https://www.ebi.ac.uk/arrayexpress/experiments/E-MTAB-365/

Experimental setup

Experiments were conducted on a workstation with an Intel Xeon E5-2680 v4 CPU (2.4 GHz, 28 cores), 128 GB of RAM and an NVIDIA Tesla V100 GPU (32 GB). The framework was implemented in Python 3.9 using scikit-learn 1.0.2, XGBoost 1.5.0 and NumPy 1.21.5.

Hyperparameters: GBM: learning rate 0.1, max depth 6, 200 estimators. RF: 500 trees, max features $\sqrt{G}$ . SVM: RBF kernel, $C=1.0$ , $\gamma =1/G$ . Attention dimension $d=64$ . KNN imputation $K=10$ .

Evaluation protocol: 5-fold stratified cross-validation with inner 3-fold CV for hyperparameter tuning. Performance averaged over 10 random seeds.

Optimal hyperparameters were determined via nested cross-validation: 3-fold inner cross-validation for hyperparameter tuning and 5-fold outer cross-validation for performance evaluation. ComBat parameters α_g and β_g were estimated using an empirical Bayes framework (Johnson et al. ³⁴ ). Optimised ensemble weights from SLSQP minimisation of cross-entropy loss (equation (21)) consistently assigned the highest weight to GBM (w_1 = 0.42 ± 0.03), followed by RF (w_2 = 0.35 ± 0.04) and SVM (w_3 = 0.23 ± 0.03), reflecting GBM's superior individual classification performance. Table 3 summarises all optimal parameter values and selection methods.

Table 3.

Model and preprocessing hyperparameter settings.

Parameter	Symbol	Optimal value	Selection method
GBM learning rate	η	0.1	Grid search (3-fold inner CV)
GBM max depth	–	6	Grid search
GBM number of estimators	Mtrees	200	Early stopping on validation loss
RF number of trees	B	500	OOB error stabilisation
RF max features	–	√G*	Default (Breiman's recommendation)
SVM regularisation	C	1.0	Grid search {0.01, 0.1, 1, 10, 100}
SVM kernel parameter	γ	1 / G*	Grid search {1/G, 0.01, 0.001}
Attention embedding dimension	d	64	Validated over {32, 64, 128}
KNN neighbours	K	10	Validated over {5, 10, 15, 20}
MAD threshold	τMAD	Adaptive (top 40% percentile)	Distribution-based
ComBat gene mean	αg	Estimated empirically per gene	Empirical Bayes estimation
ComBat covariate effect	βg	Estimated empirically per gene	Linear regression on covariates
Ensemble weight (GBM)	w₁	0.42 ± 0.03	SLSQP optimisation (equation (21))
Ensemble weight (RF)	w₂	0.35 ± 0.04	SLSQP optimisation (equation (21))
Ensemble weight (SVM)	w₃	0.23 ± 0.03	SLSQP optimisation (equation (21))

Performance metrics

Table 4 defines the evaluation metrics.

Table 4.

Performance metrics definitions.

Metric	Formula
Accuracy	$(TP+TN)/(TP+TN+FP+FN)$
Sensitivity (Recall)	$TP/(TP+FN)$
Specificity	$TN/(TN+FP)$
Precision	$TP/(TP+FP)$
F1-Score	$2\times (Precision\times Recall)/(Precision+Recall)$
AUC-ROC	$AreaUndertheROCCurve$
MCC	$(TP\times TN-FP\times FN)/\sqrt{((TP+FP)(TP+FN)(TN+FP)(TN+FN))}$

Classification results

Table 5 presents the classification performance across all datasets.

Table 5.

Classification performance of MICRO-AI across datasets.

Dataset	Acc (%)	Sen (%)	Spec (%)	Prec (%)	F1 (%)	AUC	MCC
GSE2034	95.8	94.2	97.1	96.8	95.5	0.978	0.914
GSE7390	97.5	96.8	98.2	97.9	97.3	0.986	0.949
TCGA-BRCA	96.2	94.8	97.4	95.6	95.2	0.981	0.921
GSE62254	94.7	93.5	95.8	94.2	93.8	0.972	0.893
E-MTAB-365	98.1	97.4	98.7	98.3	97.8	0.992	0.961
GSE9891	96.5	95.6	97.3	96.9	96.2	0.984	0.929
Average	96.8	95.2	97.4	96.6	96.0	0.983	0.928

Figure 3 shows the convergence of the ensemble components’ training losses over epochs. It illustrates the convergence behaviour of the three base classifiers during training on the TCGA-BRCA dataset. Both the training and validation loss curves are presented to demonstrate that the models converge without significant overfitting. The GBM component shows stable convergence after approximately 150 iterations with a training-validation loss gap of less than 0.02. The RF out-of-bag error stabilises around 350 trees with minimal divergence from validation error. The SVM hinge loss converges within 80 iterations, with the validation loss closely tracking the training loss. These results confirm that the ensemble components are well-regularised and generalise effectively to unseen data.

Figure 3.

Loss convergence curves training of the GBM, RF (out-of-bag error) and SVM (hinge loss) components through training to convergence on the TCGA-brca dataset.

Figure 4 reveals the training accuracy.

Figure 4.

Classification accuracy versus training epochs of individual classifiers and the ensemble on the GSE2034 dataset.

Figure 5 shows the confusion error of multi-class classification on TCGA-BRCA.

Figure 5.

Confusion matrix of the TCGA-BRCA 5-classifier that depicts the prediction distribution between Luminal A, Luminal B, HER2-enriched, Basal-like and Normal-like subtypes.

Figure 6 draws the ROC curves of binary classification problems.

Figure 6.

ROC curves between the MICRO-AI ensemble and individual classifiers on the GSE2034 breast cancer dataset.

Feature selection analysis

Table 6 illustrates the frequency and functional categories of the selected genes.

Table 6.

Feature selection results and gene functional categories.

Dataset	Original features	After MAD	Final selected	Reduction (%)	Top functional category
GSE2034	22,283	8456	127	99.4	Cell Cycle
GSE7390	22,283	7892	98	99.6	Apoptosis
TCGA-BRCA	20,531	9234	156	99.2	Signaling
GSE62254	20,155	6723	142	99.3	Metabolism
E-MTAB-365	18,943	5678	89	99.5	Immune
GSE9891	12,625	4567	112	99.1	Proliferation

Figure 7 shows the distribution of attention weight across genes.

Figure 7.

Weight distribution of the top 50 genes in GSE 2034. A decrease in the weights gives increased discrimination significance when performing classification.

Comparative analysis

Table 7 compares MICRO-AI with ten state-of-the-art processes. Compared with state-of-the-art methods, including classical ML approaches (SVM-RFE ³⁵ , Random Forest ³⁶ , XGBoost ³⁷ ), deep learning architectures ³⁸ , and contemporary transformer-based models (CNN-Gene ³⁹ , Attention-Net ⁴⁰ , GeneFormer ⁴¹ , scBERT ⁴² ), and LASSO-SVM ⁴³ (Table 7), MICRO-AI demonstrates superior accuracy (96.8%) while maintaining competitive computational efficiency (52.3 s average training time).

Table 7.

Comparative analysis of MICRO-AI against state-of-the-art methods.

Method	Reference	Year	Acc (%)	Sen (%)	Spec (%)	F1 (%)	AUC	Time (s)	Features
SVM-RFE	Guyon et al.	2002	89.3	87.5	91.2	88.4	0.924	45.2	150
Random Forest	Breiman et al.	2001	91.7	90.2	93.1	91.0	0.945	28.7	200
XGBoost	Chen & Guestrin	2016	93.4	92.1	94.6	93.0	0.961	32.4	175
Deep Learning MLP	Krizhevsky et al.	2015	92.8	91.5	94.0	92.3	0.953	124.5	All
CNN-Gene	Ching et al.	2018	94.1	93.2	95.0	93.8	0.968	156.8	All
LASSO-SVM	Zhang et al.	2023	90.6	88.9	92.3	89.8	0.937	38.9	120
Attention-Net	Vaswani et al.	2017	93.9	92.7	95.1	93.5	0.964	89.3	All
GeneFormer	Theodoris et al.	2023	95.2	94.1	96.2	94.8	0.974	245.6	All
Ensemble-FS	Li et al.	2024	94.5	93.6	95.4	94.2	0.970	67.8	180
scBERT	Yang et al.	2022	95.6	94.8	96.4	95.3	0.978	312.4	All
MICRO-AI	Proposed	2025	96.8	95.2	97.4	96.0	0.983	52.3	127

Figure 8 gives a visual performance of the methods.

Figure 8.

Measurement comparison of MICRO-AI and best top-5 competing methods in six performance dimensions: accuracy, sensitivity, specificity, F1-score, AUC and computational efficiency.

Ablation study

Table 8 presents ablation study results analysing component contributions.

Table 8.

Ablation study results on TCGA-BRCA dataset.

Configuration	Acc (%)	AUC	F1 (%)	Δ Acc
Full MICRO-AI	96.2	0.981	95.2	–
w/o Attention	93.8	0.962	92.9	−2.4
w/o ComBat	94.5	0.968	93.6	−1.7
w/o RFECV	92.1	0.951	91.3	−4.1
w/o Calibration	95.8	0.979	94.8	−0.4
GBM only	91.2	0.943	90.4	−5.0
RF only	90.8	0.938	89.9	−5.4
SVM only	89.5	0.926	88.7	−6.7
Equal weights	95.4	0.975	94.5	−0.8

Computational efficiency

Table 9 gives training and inference times.

Table 9.

Computational time analysis (seconds).

Dataset	Preprocess	Feature selection	Training	Total	Inference per sample
GSE2034	2.3	18.5	31.2	52.0	0.012
GSE7390	1.8	14.2	24.6	40.6	0.010
TCGA-BRCA	8.7	42.3	78.5	129.5	0.018
GSE62254	3.1	22.8	38.4	64.3	0.014
E-MTAB-365	1.5	11.6	19.2	32.3	0.009
GSE9891	2.6	16.4	28.7	47.7	0.011

Figure 9 shows that the system scales with dataset size.

Figure 9.

Scalability comparison of training time/sample count (left) and gene count (right) with 95% confidence of the trends.

Discussion

The experimental findings show that, even compared with existing techniques, MICRO-AI consistently delivers better results across various evaluation metrics. There are a few observations that are worth mentioning.

The attention-weighted feature selection mechanism also significantly improves classification performance, as shown by the 2.4% increase in accuracy when it is omitted (Table 6). The attention mechanism can detect biologically significant expression signatures in an agnostic manner, which could otherwise be missed by traditional univariate methods, by dynamically prioritising genes based on discriminative significance. Trimming the feature count from 20,000 to about 100–150 (a 99%+ reduction) significantly reduces the computational burden while retaining predictive value.

RFECV is also crucial for maximising the identification of gene subsets, and when it is removed, the performance degradation is the most significant (4.1%). It is achieved by using an iterative elimination based on cross-validated performance on the training data, thereby avoiding overfitting to the training data and yielding strong gene subsets that do not overfit individual samples.

The ensemble method achieves a higher 5–7% accuracy than individual classifiers, demonstrating the benefits of GBM, RF and SVM. GBM is best at finding nonlinear relationships, RF variance reduction uses bagging, and SVM is effective in high-dimensional spaces. The adaptive weight optimisation also improves by 0.8 over equal weighting.

Compared with state-of-the-art methods (Table 7), MICRO-AI demonstrates superior accuracy (96.8%) while maintaining competitive computational efficiency (52.3 s average training time). Deep learning models such as GeneFormer and scBERT achieve MICRO-AI's performance but require 4–6× longer training times and lack interpretability for feature selection.

The evidence of performance stability across different types of cancer (breast, gastric, ovarian) and platforms (Affymetrix, Illumina, RNA-seq) indicated the strong generalisability of the framework. ComBat provides cross-platform batch-effect correction, leading to a 1.7% increase in accuracy, underscoring the importance of incorporating ComBat into the integration of multi-source data.

MICRO-AI's outputs may diverge from original benchmark dataset publications due to unified preprocessing (quantile normalisation, ComBat batch correction, KNN imputation) rather than dataset-specific protocols. For instance, GSE2034's original 76-gene prognostic signature (Wang et al., univariate Cox regression) versus MICRO-AI's 127 genes (attention-weighted RFECV) shows approximately 68% overlap with established breast cancer pathways (cell cycle, apoptosis, oestrogen signalling). Similarly, TCGA-BRCA: MICRO-AI selected 156 genes compared to the PAM50 panel, and these genes showed significant oncogenic pathway enrichment (FDR<0.05, Gene Ontology).

Differences arise from: (1) uniform normalisation optimised for cross-dataset comparability versus dataset-specific protocols; (2) multi-class classification accuracy optimisation versus survival analysis or single-biomarker discovery; and (3) attention mechanism prioritising discriminative capacity across all classes simultaneously, identifying combinatorial signatures that univariate methods may miss.

While MICRO-AI may not replicate exact gene lists from original publications, its strength lies in discovering complementary and potentially novel biomarker combinations, achieving superior classification performance. MICRO-AI is a classification-oriented diagnostic tool, not a biomarker discovery platform; identified gene sets should be interpreted accordingly. For clinical translation, cross-reference MICRO-AI's selected genes with established biomarker databases (COSMIC, OncoKB) to ensure biological plausibility.

Although it accounts for only a small portion of the overall classification boost (around 0.4%), probability calibration becomes important in clinical decision support environments where the values of predicted confidence directly affect referral decisions and treatment planning. Probability calibration is applied in MICRO-AI via isotonic regression (equation (22)), which learns a nonparametric, monotonic map from raw ensemble output probabilities to empirically observed outcome frequencies on a held-out (validation) set.

The ECE, a diagnostic metric that quantitatively measures the quality of probability calibration, is defined in equation (23). ECE measures the discrepancy between predicted confidence and observed accuracy across probability bins, providing an intuitive measure of probabilistic reliability. Lower ECE values indicate stronger correspondence between predicted probabilities and actual outcome frequencies. This calibration process ensures that MICRO-AI's confidence estimates are both statistically meaningful and clinically interpretable, enabling robust uncertainty quantification and reducing risk in medical decision-making.

The use of labelled training data, high sensitivity to extreme class imbalance and the stationary expression pattern are the drawbacks. Future directions will focus on semi-supervised learning with limited labels, classification under cost-sensitive criteria in imbalanced datasets and temporal modelling of disease progression.

Consistency of gene discovery with original publications

MICRO-AI's outputs may differ from the original benchmark publications due to the use of unified preprocessing (quantile normalisation, ComBat batch correction, KNN imputation) rather than dataset-specific protocols. For GSE2034, MICRO-AI selected 127 genes (attention-weighted RFECV for multi-class classification) versus the original 76-gene signature (Wang et al. ⁴⁴ ) with approximately 68% pathway overlap (cell cycle hsa04110, apoptosis hsa04210, oestrogen signalling hsa04915). For TCGA-BRCA, MICRO-AI selected 156 genes versus PAM50's 50-gene panel, with significant oncogenic pathway enrichment (Gene Ontology FDR<0.05) confirming biological plausibility despite incomplete overlap.

Differences arise from: (1) uniform normalisation for cross-dataset comparability versus dataset-specific signal maximisation; (2) multi-class classification optimisation versus survival/single-biomarker discovery; (3) attention mechanism (equations (8) and (9)) prioritising discriminative capacity across all classes, identifying combinatorial signatures that univariate methods may miss. While MICRO-AI may not replicate exact gene lists, its strength lies in discovering complementary biomarker combinations achieving superior classification (96.8% accuracy). MICRO-AI is a classification-oriented diagnostic tool, not a biomarker discovery platform; interpret gene sets accordingly. For clinical translation, cross-reference selected genes with established databases (COSMIC, OncoKB) to ensure biological plausibility and clinical relevance (Table 10).

Table 10.

Comparison of gene selection, optimisation objectives and preprocessing strategies between original studies and the proposed MICRO-AI framework.

Aspect	Original study	MICRO-AI	Overlap / notes
GSE20347 gene set	76 genes (Wang et al.; Cox regression; survival-based selection)	127 genes (Attention-RFECV; classification-oriented selection)	∼68% pathway overlap (cell cycle, apoptosis, oestrogen signalling)
TCGA-BRCA gene set	50 genes (PAM50 panel; subtype classification)	156 genes (Attention-RFECV; multi-class classification)	Significant GO enrichment (FDR < 0.05)
Optimisation target	Survival prediction / single-biomarker emphasis	Multi-class accuracy optimisation	–
Preprocessing strategy	Dataset-specific preprocessing pipelines	Unified pipeline (quantile normalisation + ComBat + KNN imputation)	–

Limitations

Although the proposed MICRO-AI framework has performed well across the benchmark datasets, there are a few limitations to consider. To begin with, the model is trained on labelled data, and thus its computational efficiency might be limited in clinical settings where annotated genomic datasets are scarce or costly to acquire. This dependence can be a weakness when performance must be conveyed on only a limited number of labelled samples.

Second, because stratified cross-validation is used, the framework can be affected by extreme class imbalance, especially in multi-class cancer subtype prediction. Imbalanced class distributions can influence classifier bias and confidence behaviour, underscoring the need for balance-aware learning methods.

Third, the ensemble uses a SVM component, and its computational complexity can become very large as the sample size increases. Although this is not a constraint of standard microarray data, future large-scale transcriptomic cohorts may need to be kernel approximated or use alternative classifiers.

Lastly, the existing assessment is based on retrospective, publicly available datasets and is not in prospective clinical validation or real-time implementation in hospital processes. In this sense, clinical utility has been determined by intuition rather than substantiated evidence, and additional research is needed before translational adoption.

We acknowledge that all current evaluations employ cross-validation within individual dataset cohorts, and no fully independent external validation set was reserved. While 5-fold stratified cross-validation with repeated random seeds provides robust performance estimates, the absence of a completely held-out test set limits the strength of generalisability claims. Future work will incorporate a leave-one-dataset-out (LODO) validation protocol, in which each dataset is systematically held out as an independent test set. In contrast, the remaining datasets are used for training and hyperparameter selection. This approach will provide a more rigorous assessment of cross-platform and cross-disease generalisability.

Conclusion

This paper introduced MICRO-AI, a comprehensive ML framework for DNA microarray analysis and automated disease detection. The framework addresses the key challenges of high dimensionality, batch effects and limited sample sizes through four integrated modules: robust preprocessing with quantile normalisation and ComBat correction, attention-weighted feature selection that reduces dimensionality by over 99% (from ∼20,000 to ∼127 genes), adaptive ensemble classification combining GBM, RF and SVM with optimised weighting, and clinical integration with calibrated probability outputs.

Extensive validation across six benchmark datasets from three independent repositories (GEO, TCGA, ArrayExpress), encompassing 2321 samples across four cancer types, demonstrated state-of-the-art performance: 96.8% average accuracy, 95.2% sensitivity, 97.4% specificity, 0.983 area under the receiver operating characteristic curve (AUC-ROC), and 0.928 Matthews correlation coefficient (MCC). Comparative analysis against ten competing methods confirmed that MICRO-AI achieves 1.2–7.5% accuracy improvement while maintaining competitive computational efficiency (52.3 s average training time), representing 2.4–6.0× faster execution than deep learning alternatives such as GeneFormer and scBERT. The ablation study demonstrated that each component contributes meaningfully, with RFECV (−4.1%), attention mechanism (−2.4%) and ComBat correction (−1.7%) providing the most significant contributions.

Despite these achievements, several limitations remain. The framework relies on labelled training data, may be sensitive to extreme class imbalance, and has been validated only on retrospective public datasets, not on prospective clinical trials. The SVM component's quadratic complexity may pose scalability challenges for very large transcriptomic cohorts.

Future research directions include: (1) multi-omics integration incorporating transcriptomic, epigenomic and proteomic data for comprehensive molecular profiling; (2) federated learning adaptation for privacy-preserving collaborative genomic analysis across clinical institutions; (3) prospective clinical validation through pilot studies in hospital environments to assess real-world robustness, calibration reliability and clinical utility; (4) cost-sensitive and class-balanced learning strategies for improved performance on rare disease subtypes; and (5) external independent validation using LODO protocols to further strengthen generalisability claims.