Intratumor heterogeneity (ITH) impacts cancer progression, and its characterization is crucial. Clustering algorithms applied to the variant allele frequency (VAF) of mutations can facilitate the exploratory analysis of ITH. This study comparatively evaluated six clustering algorithms to characterize ITH by clustering mutations based on their VAFs. We utilized data from The Cancer Genome Atlas to analyze three cancer types by examining the distribution of clusters in the results from various methods and four internal validation metrics. The results indicated that the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) and Variational Bayesian Gaussian Mixture Model methods identified an insufficient number of clusters in most tumor samples. The Hierarchical DBSCAN (HDBSCAN) and Ordering Points to Identify the Clustering Structure (OPTICS) algorithms exhibited greater variability in the number of clusters, while Affinity Propagation (AP) showed controlled behavior, and Mean-Shift demonstrated greater consistency. The Mean-Shift and AP methods were consistently superior in the validation metrics, in contrast to HDBSCAN and OPTICS, which had inferior performance. We conclude that Mean-Shift and AP are promising and accessible alternatives for the initial exploratory analysis of ITH by VAFs. A computational pipeline is provided on the Google Colab platform to facilitate future studies.
AbécassisJ, HamyA-S, LaurentC, et al.Assessing reliability of intra-tumor heterogeneity estimates from single sample whole exome sequencing data. PLoS One2019;14(11):e0224143; doi: 10.1371/journal.pone.0224143
2.
AlexandrovLB, Nik-ZainalS, WedgeDC, et al.Deciphering signatures of mutational processes operative in human cancer. Cell Rep2013;3(1):246–259.
3.
AnkerstM, BreunigMM, KriegelH-P, et al.Optics: Ordering points to identify the clustering structure. SIGMOD Rec1999;28(2):49–60; doi: 10.1145/304181.304187
4.
AntonelloA, BergaminR, CalonaciN, et al.Computational validation of clonal and subclonal copy number alterations from bulk tumor sequencing using cnaqc. Genome Biol2024;25(1):38.
BlackJR, McGranahanN. Genetic and non-genetic clonal diversity in cancer evolution. Nat Rev Cancer2021;21(6):379–392.
7.
CalińskiT, HarabaszJ. A dendrite method for cluster analysis. Commun Stat-Theory Methods1974;3(1):1–27.
8.
CampelloRJ, MoulaviD, SanderJ. Density-based clustering based on hierarchical density estimates. In: Advances in Knowledge Discovery and Data Mining. PAKDD 2013. Lecture Notes in Computer Science. Springer; 2013; pp. 160–172; doi: 10.1007/978-3-642-37456-2_14
9.
CirielloG, MagnaniL, AitkenSJ, et al.Cancer evolution: A multifaceted affair. Cancer Discov2024;14(1):36–48.
10.
ComaniciuD, MeerP. Mean shift: A robust approach toward feature space analysis. IEEE Trans Pattern Anal Machine Intell2002;24(5):603–619; doi: 10.1109/34.1000236
11.
Cortés-CirianoI, GulhanDC, LeeJJ-K, et al.Computational analysis of cancer genome sequencing data. Nat Rev Genet2022;23(5):298–314.
12.
CutigiJF, EvangelistaAF, ReisRM, et al.A computational approach for the discovery of significant cancer genes by weighted mutation and asymmetric spreading strength in networks. Sci Rep2021;11(1):23551.
13.
DaviesDL, BouldinDW. A cluster separation measure. IEEE Trans Pattern Anal Mach Intell1979;PAMI-1(2):224–227; doi: 10.1109/TPAMI.1979.4766909
14.
De SoutoMC, CostaIG, De AraujoDS, et al.Clustering cancer gene expression data: A comparative study. BMC Bioinformatics2008;9(1):497; doi: 10.1186/1471-2105-9-497
15.
DeshwarAG, VembuS, YungCK, et al.Phylowgs: Reconstructing subclonal composition and evolution from whole-genome sequencing of tumors. Genome Biol2015;16(1):35; doi: 10.1186/s13059-015-0602-8
EichlerEE, TuzunE, SharpAJ, et al.Fine-scale structural variation of the human genome. Nat Genet2005;37(7):727–732.
18.
EsterM, KriegelH-P, SanderJ, et al.A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining. AAAI Press; 1996; pp. 226–231.
19.
FreyBJ, DueckD. Clustering by passing messages between data points. Science2007;315(5814):972–976; doi: 10.1126/science.1136800
20.
GaoJ, ChangMT, JohnsenHC, et al.3d clusters of somatic mutations in cancer reveal numerous rare mutations as functional targets. Genome Med2017;9(1):4–13.
21.
GerlingerM, RowanAJ, HorswellS, et al.Intratumor heterogeneity and branched evolution revealed by multiregion sequencing. N Engl J Med2012;366(10):883–892.
22.
GilliesRJ, VerduzcoD, GatenbyRA. Evolutionary dynamics of carcinogenesis and why targeted therapy does not work. Nat Rev Cancer2012;12(7):487–493.
GreenmanCD, PleasanceED, NewmanS, et al.Estimation of rearrangement phylogeny for cancer genomes. Genome Res2012;22(2):346–361.
25.
MartincorenaI, CampbellPJ. Somatic mutation in cancer and normal cells. Science2015;349(6255):1483–1489.
26.
MasoodD, RenL, NguyenC, et al.Evaluation of somatic copy number variation detection by ngs technologies and bioinformatics tools on a hyper-diploid cancer genome. Genome Biol2024;25(1):163.
27.
McGranahanN, SwantonC. Clonal heterogeneity and tumor evolution: Past, present, and the future. Cell2017;168(4):613–628.
28.
MillerCA, WhiteBS, DeesND, et al.Sciclone: Inferring clonal architecture and tracking the spatial and temporal patterns of tumor evolution. PLoS Comput Biol2014;10(8):e1003665.
29.
MiuraS, VuT, DengJ, et al.Power and pitfalls of computational methods for inferring clone phylogenies and mutation orders from bulk sequencing data. bioRxiv2020. Available from: https://www.biorxiv.org/content/early/2019/07/19/697318
Nik-ZainalS, Van LooP, WedgeDC, Breast Cancer Working Group of the International Cancer Genome Consortium. et al.; The life history of 21 breast cancers. Cell (Cambridge)2012;149(5):994–1007.
32.
OesperL, MahmoodyA, RaphaelBJ. Theta: Inferring intra-tumor heterogeneity from high-throughput DNA sequencing data. Genome Biol2013;14(7):R80–R21.
33.
OstroverkhovaD, PrzytyckaTM, PanchenkoAR. Cancer driver mutations: Predictions and reality. Trends Mol Med2023;29(7):554–566.
34.
PecorinoL. Molecular biology of cancer: mechanisms, targets, and therapeutics, 5th ed. Oxford University Press: Oxford, UK, 2021.
35.
PedregosaF, VaroquauxG, GramfortA, et al.Scikit-learn: Machine learning in python. J Mach Learn Res2011;12(85):2825–2830. Available from: https://jmlr.org/papers/v12/pedregosa11a.html
36.
RibeiroPH, SimaoA. Analysis of clonal evolution in cancer: A computational perspective. J Bioinform Comput Biol2025;23(2):2531001; doi: 10.1142/S0219720025310018
37.
RibeiroPH, CutigiJF, RamosRH, et al.Exploring the influence of gene networks on driver gene classification. J Comput Biol2025;32(6):615–625; doi: 10.1089/cmb.2025.0043
38.
RodriguezMZ, CominCH, CasanovaD, et al.Clustering algorithms: A comparative approach. PLoS One2019;14(1):e0210236–e34; doi: 10.1371/journal.pone.0210236
39.
RothA, KhattraJ, YapD, et al.Pyclone: Statistical inference of clonal population structure in cancer. Nat Methods2014;11(4):396–398.
40.
RousseeuwPJ. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math1987;20:53–65; doi: 10.1016/0377-0427(87)90125-7
41.
SchererSW, FeukL, CarsonAR. Structural variation in the human genome. Nat Rev Genet2006;7(2):85–97. ISSN 1471-0056.
42.
StrattonMR, CampbellPJ, FutrealPA. The cancer genome. Nature2009;458(7239):719–724.
43.
TamboreroD, Gonzalez-PerezA, Perez-LlamasC, et al.Comprehensive identification of mutational cancer driver genes across 12 tumor types. Sci Rep2013;3(1):2650; doi: 10.1038/srep02650
44.
ThalamuthuA, MukhopadhyayI, ZhengX, et al.Evaluation and comparison of gene clustering methods in microarray analysis. Bioinformatics2006;22(19):2405–2412; doi: 10.1093/bioinformatics/btl406
45.
The ICGC/TCGA Pan-Cancer Analysis of Whole Genomes Consortium. Pan-cancer analysis of whole genomes. Nature2020;578(7793):82–93; doi: 10.1038/s41586-020-1969-6
46.
TufailM, HuJ-J, LiangJ, et al.Hallmarks of cancer resistance. iScience2024;27(6):109979; doi: 10.1016/j.isci.2024.109979
47.
WangN, TaoZ-Y, WuT, et al.Benchmarking copy number variation detection with low-coverage whole-genome sequencing. Brief Bioinform2025;26(5):bbaf514.
YuanX, LiZ, ZhaoH, et al.Accurate inference of tumor purity and absolute copy numbers from high-throughput sequencing data. Front Genet2020;11:458.
50.
ZaccariaS, RaphaelBJ. Accurate quantification of copy-number aberrations and whole-genome duplications in multi-sample tumor sequencing data. Nat Commun2020;11(1):4301.
51.
ZhangJZ, WangC. A comparative study of clustering methods on gene expression data for lung cancer prognosis. BMC Res Notes2023;16(1):319; doi: 10.1186/s13104-023-06604-8
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
