Comparison of an Attention-Based Multiple Instance Learning (MIL) With a Visual Transformer Model: Two Weakly Supervised Deep Learning (DL) Algorithms for the Detection of Histopathologic Lesions in the Rat Liver to Distinguish Normal From Abnormal
Restricted accessResearch articleFirst published online July, 2025
Comparison of an Attention-Based Multiple Instance Learning (MIL) With a Visual Transformer Model: Two Weakly Supervised Deep Learning (DL) Algorithms for the Detection of Histopathologic Lesions in the Rat Liver to Distinguish Normal From Abnormal
The histopathologic evaluation of regulatory toxicity studies using artificial intelligence (AI) has the potential to increase study efficiency. For example, AI could initially identify and exclude all organs without histopathologic lesions, allowing pathologists to focus solely on evaluating organs with identified lesions. In this study, whole slide images (WSIs) of liver sections from 58 different rat toxicity studies were collected, along with their corresponding histopathologic lesion diagnoses. Each WSI was labeled as either “lesion” or “no lesion” based on the presence or absence of reported histopathologic lesions. Multiple instance learning (MIL) approaches, including a transformer variant, were tested to predict lesions within a weakly supervised framework. Both methods achieved acceptable to excellent area under the receiver operating characteristic curve (AUROC) scores. Heatmap overlays were employed to visually assess the MIL model’s effectiveness in detecting lesions, confirming the accuracy of targeted areas on the WSIs. In addition, using transfer learning principles, the MIL model initially developed for liver WSIs was adapted to kidney WSIs, demonstrating the model’s versatility. This study showcases the application of weakly supervised learning for lesion detection in rat WSIs from toxicity studies, with the potential to significantly enhance the efficiency of the histopathologic evaluation process.
AdnanMKalraSTizhooshHR. Representation learning of histopathology images using graph neural networks. Paper presented at the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW); June 2020; Seattle, WA. doi:10.48550/arXiv.2004.07399.
2.
BertoliniMLeVKPencharzJ, et al. From slides (through tiles) to pixels: an explainability framework for weakly supervised models in preclinical pathology. arXiv. 2023; arXiv:2302.01653. doi:10.48550/arXiv.2302.01653.
3.
BozinovskiSFulgosiA. The influence of pattern similarity and transfer of learning upon training of a base perceptron B2 (original in Croatian: Utjecaj slicnosti likova i transfera ucenja na obucavanje baznog perceptrona B2). Proc Symp Informatica. 1976;5:3-121.
4.
BussolaNXuJWuL, et al. A weakly supervised deep learning framework for whole slide classification to facilitate digital pathology in animal study. Chem Res Toxicol. 2023;36(8):1321-1331.
5.
CampanellaGHannaMGGeneslawL, et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat Med. 2019;25(8):1301-1309.
6.
ChlipalaEAButtersMBrousM, et al. Impact of preanalytical factors during histology processing on section suitability for digital image analysis. Toxicol Pathol. 2021;49(4):755-772.
7.
DengJDongWSocherRLiLJLiKFei-FeiL. ImageNet: a large-scale hierarchical image database. Paper presented at the 2009 IEEE Conference on Computer Vision and Pattern Recognition; June 2009; Miami, FL. doi:10.1109/CVPR.2009.5206848.
8.
DosovitskiyABeyerLKolesnikovA, et al. An image is worth 16×16 words: transformers for image recognition at scale. arXiv. 2020; abs/2010.11929. doi:10.48550/arXiv.2010.11929.
9.
Ehteshami BejnordiBVetaMvan DiestJP, et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA. 2017;318(22):2199-2210.
10.
FatimaSAliSKimHC. A comprehensive review on multiple instance learning. Electronics. 2023;12:4323. doi:10.3390/electronics12204323.
11.
Gámez SernaCRomero-PalomoFArcaduFFunkJSchumacherVJanowczykA. MMO-net (multi-magnification organ network). A use case for organ identification using multiple magnifications in preclinical pathology studies. J Pathol Inform. 2022;13:100126. doi:10.1016/j.jpi.2022.100126.
12.
GeXYFunkJAlbrechtT, et al. Toxicologic pathology forum: a roadmap for building state-of-the-art digital image data resources for toxicologic pathology in the pharmaceutical industry. Toxicol Pathol. 2022;50(8):942-949.
13.
GreffKKleinAChovanecMHutterFSchmidhuberJ. The sacred infrastructure for computational research. Paper presented at the proceedings of the python in science conferences (SciPy2017); July 2017; Austin, TX. doi:10.6084/m9.figshare.5813412.v1.
14.
HeKZhangXRenSSunJ. Deep residual learning for image recognition. Paper presented at the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); June 2015; Las Vegas, NV. doi:10.48550/arXiv.1512.03385.
15.
IlseMTomczakJMWellingM. Attention-based deep multiple instance learning. Paper presented at the International Conference on Machine Learning; July 2018; Stockholm, Sweden. https://arxiv.org/abs/1802.04712
16.
JanowczykAZuoRGilmoreHFeldmanMMadabhushiA. HistoQC: an open-source quality control tool for digital pathology slides. JCO Clin Cancer Inform. 2019;3:1-7.
17.
JaumeGde BrotSSongAH, et al. Deep learning-based modeling for preclinical drug safety assessment. Biorxiv. 2024.07.20.604430. doi:10.1101/2024.07.20.604430.
18.
JavedSMahmoodAQaiserTWerghiNRajpootN. Unsupervised mutual transformer learning for multi-gigapixel Whole Slide Image classification. Med Image Anal. 2024;96:103203. doi:10.1016/j.media.2024.103203.
19.
KuklyteJFitzgeraldJNelissenS, et al. Evaluation of the use of single and multi-magnification convolutional neural networks for the determination and quantitation of lesions in nonclinical pathology studies. Toxicol Pathol. 2021;49(4):815-842.
20.
LiBLiYEliceiriKW. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. Conf Comput Vis Pattern Recognit Workshops. 2021;17:14318-14328. doi:10.1109/CVPR46437.2021.01409.
21.
MehrvarSHimmelLEBabburiP, et al. Deep learning approaches and applications in toxicologic histopathology: current status and future perspectives. J Pathol Inform. 2021;12:42.
22.
MoulinPGrünbergKBarale-ThomasEder LaakJV. IMI-Bigpicture: a central repository for digital pathology. Toxicol Pathol. 2021;49(4):711-713.
23.
PopelMBojarO. Training tips for the transformer model. Prague Bull Math Linguist. 2018;110:43-70.
24.
QuellecGCazuguelGCochenerBLamardM. Multiple-instance learning for medical image and video analysis. IEEE Rev Biomed Eng. 2017;10:213-234.
25.
RudmannDAlbretsenJDoolanC, et al. Using deep learning artificial intelligence algorithms to verify N-nitroso-N-methylurea and urethane positive control proliferative changes in Tg-RasH2 mouse carcinogenicity studies. Toxicol Pathol. 2021;49(4):938-949.
26.
ShamshadFKhanSHZamirSW, et al. Transformers in medical imaging: a survey. Med Image Anal. 2022;88:102802. doi:10.1016/j.media.2023.102802.
27.
ShaoZBianHChenYWangYZhangJJiX. TransMIL: transformer based correlated multiple instance learning for whole slide image classification. Adv Neural Inf Process Syst. 2021;34:2136-2147.
28.
ShimazakiTDeshpandeAHajraA, et al. Deep learning-based image-analysis algorithm for classification and quantification of multiple histopathological lesions in rat liver. J Toxicol Pathol. 2022;35(2):135-147.
29.
ShortenCKhoshgoftaarTM. A survey on image data augmentation for deep learning. J Big Data. 2019;6:60. doi:10.1186/s40537-019-0197-0.
30.
SrivastavaNHintonGKrizhevskyASutskeverISalakhutdinovR. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15(1):1929-1958.
31.
TanMLeQV. EfficientNet: rethinking model scaling for convolutional neural networks. arXiv. 2019; abs/1905.11946. doi:10.48550/arXiv.1905.11946.
32.
TavolaraTEGurcanMNNiaziMKK. Contrastive multiple instance learning: an unsupervised framework for learning slide-level representations of whole slide histopathology images without labels. Cancers. 2022;14:5778. doi:10.3390/cancers14235778.
33.
TellezDLitjensGBándiP, et al. Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology. Med Image Anal. 2019;58:101544. doi:10.1016/j.media.2019.101544.
TurnerOCAeffnerFBangariDS, et al. Society of toxicologic pathology digital pathology and image analysis special interest group article: opinion on the application of artificial intelligence and machine learning to digital toxicologic pathology. Toxicol Pathol. 2020;48(2):277-294.
36.
VaswaniAShazeerNMParmarN, et al. Attention is all you need. Neural Inf Process Syst. 2017; CoRRabs/1706.03762. doi:10.48550/arXiv.1706.03762.
37.
VaswaniAShazeerNParmarN, et al. Attention is all you need. Paper presented at the Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS2017); December 2017, Long Beach, CA. https://arxiv.org/pdf/1706.03762.pdf
38.
ZaheerMGuruganeshGDubeyA, et al. Big bird: transformers for longer sequences. Paper presented at the NIPS’20: Proceedings of the 34th International Conference on Neural Information Processing Systems; December 2020, Vancouver, BC. doi:10.48550/arXiv.2007.14062.
39.
ZehnderPFengJFujiRNSullivanRHuF. Multiscale generative model using regularized skip-connections and perceptual loss for anomaly detection in toxicologic histopathology. J Pathol Inform. 2022;13:100102. doi:10.1016/j.jpi.2022.100102.
