Restricted accessResearch articleFirst published online 2023-07
Comparison of the Strengths and Weaknesses of Machine Learning Algorithms and Feature Selection on KEGG Database Microbial Gene Pathway Annotation and Its Effects on Reconstructed Network Topology
The development of tools for the annotation of genes from newly sequenced species has not evolved much from homologous alignment to prior annotated species. While the quality of gene annotations continues to decline as we sequence and assemble more evolutionary distant gut microbiome species, machine learning presents a high quality alternative to traditional techniques. In this study, we investigate the relative performance of common classical and nonclassical machine learning algorithms in the problem of gene annotation using human microbiome-associated species genes from the KEGG database. The majority of the ensemble, clustering, and deep learning algorithms that we investigated showed higher prediction accuracy than CD-Hit in predicting partial KEGG function. Motif-based, machine-learning methods of annotation in new species were faster and had higher precision–recall than methods of homologous alignment or orthologous gene clustering. Gradient boosted ensemble methods and neural networks also predicted higher connectivity in reconstructed KEGG pathways, finding twice as many new pathway interactions than blast alignment. The use of motif-based, machine-learning algorithms in annotation software will allow researchers to develop powerful tools to interact with bacterial microbiomes in ways previously unachievable through homologous sequence alignment alone.
Get full access to this article
View all access options for this article.
References
1.
BaiX, NarayananA, NowakP, et al.Whole-Genome metagenomic analysis of the gut microbiome in HIV-1-Infected individuals on antiretroviral therapy. Front Microbiol, 2021; 12:667718.
BuchfinkB, ReuterK, DrostH-G. Sensitive protein alignments at tree-of-life scale using diamond. Nat Methods, 2021; 18(4):366–368.
7.
BuchfinkB, XieC, HusonDH. Fast and sensitive protein alignment using DIAMOND. Nat Methods, 2015; 12(1):59–60.
8.
CamachoC, CoulourisG, AvagyanV, et al.Blast+: Architecture and applications. BMC Bioinformatics, 2009; 10(1):1–9.
9.
CsardiG, NepuszT. The igraph software package for complex network research. Inter J Compl Syst, 2006; 1695:1–9.
10.
EdgarRC.Search and clustering orders of magnitude faster than blast. Bioinformatics, 2010; 26(19):2460–2461.
11.
FanY, PedersenO. Gut microbiota in human metabolic health and disease. Nat Rev Microbiol, 2021; 19(1):55–71.
12.
FinnRD, ClementsJ, EddySR. HMMER web server: Interactive sequence similarity searching. Nucleic Acids Res, 2011; 39(Web Server Issue):W29–W37.
13.
FortelnyN, PavlidisP, OverallCM. The path of no return—Truncated protein n-termini and current ignorance of their genesis. Proteomics, 2015; 15(14):2547–2552.
14.
FranzosaEA, McIverLJ, RahnavardG, et al.Species-level functional profiling of metagenomes and metatranscriptomes. Nat Methods, 2018; 15(11):962–968.
15.
FreundY, SchapireRE. A decision-theoretic generalization of On-Line learning and an application to boosting. J Comput Syst Sci, 1997; 55(1):119–139.
16.
FriedmanJH.Stochastic gradient boosting. Comput Stat Data Anal, 2002; 38(4):367–378.
17.
GeorgalakiMD, Van den BergheE, KritikosD, et al.Macedocin, a food-grade lantibiotic produced by Streptococcus macedonicus ACA-DC 198. Appl Environ Microbiol, 2002; 68(12):5891–5903.
18.
GosalbesMJ, DurbánA, PignatelliM, et al.Metatranscriptomic approach to analyze the functional human gut microbiota. PLoS One, 2011; 6(3):e17447.
19.
GrantCE, BaileyTL, NobleWS. FIMO: Scanning for occurrences of a given motif. Bioinformatics, 2011; 27(7):1017–1018.
20.
GreenML, KarpPD. Genome annotation errors in pathway databases due to semantic ambiguity in partial EC numbers. Nucleic Acids Res, 2005; 33(13):4035–4039.
21.
GuestJR, RussellGC. Complexes and complexities of the citric acid cycle in Escherichia coli. Curr Top Cell Regul, 1992; 33:231–247.
22.
GuoMG, SosaDN, AltmanRB. Challenges and opportunities in network-based solutions for biological questions. Brief Bioinform, 2022; 23(1):bbab437.
23.
HelminkBA, KhanMAW, HermannA, et al.The microbiome, cancer, and cancer therapy. Nat Med, 2019; 25(3):377–388.
24.
Human Microbiome Project Consortium. Structure, function and diversity of the healthy human microbiome. Nature, 2012; 486(7402):207–214.
25.
JangiS, GandhiR, CoxLM, et al.Alterations of the human gut microbiome in multiple sclerosis. Nat Commun, 2016; 7:12015.
26.
JonesP, BinnsD, ChangH-Y, et al.Interproscan 5: Genome-scale protein function classification. Bioinformatics, 2014; 30(9):1236–1240.
27.
KanehisaM, FurumichiM, SatoY, et al.KEGG for taxonomy-based analysis of pathways and genomes. Nucleic Acids Res, 2023; 51(D1):D587–D592.
28.
KatohK, MisawaK, KumaK-I, et al.MAFFT: A novel method for rapid multiple sequence alignment based on fast fourier transform. Nucleic Acids Res, 2002; 30(14):3059–3066.
29.
KimD, ZengMY, NúñezG. The interplay between host immune cells and gut microbiota in chronic inflammatory diseases. Exp Mol Med, 2017; 49(5):e339.
30.
KingmaDP, BaJ. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 2014.
31.
KishikawaT, MaedaY, NiiT, et al. Metagenome-wide association study of gut microbiome revealed novel aetiology of rheumatoid arthritis in the Japanese population. Ann Rheum Dis, 2020a;79(1):103–111.
32.
KishikawaT, OgawaK, MotookaD, et al.A Metagenome-Wide association study of gut microbiome in patients with multiple sclerosis revealed novel disease pathology. Front Cell Infect Microbiol, 2020b;10:585973.
33.
LiW.Analysis and comparison of very large metagenomes with fast clustering and functional annotation. BMC Bioinformatics, 2009; 10:359.
34.
LiW, FuL, NiuB, et al.Ultrafast clustering algorithms for metagenomic sequence analysis. Brief Bioinformatics, 2012; 13(6):656–668.
35.
LiW, GodzikA. Cd-hit: A fast program for clustering and comparing large sets of protein or nucleotide sequences. Bioinformatics, 2006; 22(13):1658–1659.
36.
LiangQ, BiblePW, LiuY, et al.DeepMicrobes: Taxonomic classification for metagenomics with deep learning. NAR Genom Bioinform, 2020; 2(1):lqaa009.
37.
Lloyd-PriceJ, MahurkarA, RahnavardG, et al.Strains, functions and dynamics in the expanded human microbiome project. Nature, 2017; 550(7674):61–66.
38.
LobbB, TremblayBJ-M, Moreno-HagelsiebG, et al.An assessment of genome annotation coverage across the bacterial tree of life. Microb Genom, 2020; 6(3):e000341.
39.
MistryJ, ChuguranskyS, WilliamsL, et al.Pfam: The protein families database in 2021. Nucleic Acids Res, 2021; 49(D1):D412–D419.
40.
MoriyaY, ItohM, OkudaS, et al.Kaas: An automatic genome annotation and pathway reconstruction server. Nucleic Acids Res, 2007; 35(suppl_2):W182–W185.
41.
NobreT, CamposMD, Lucic-MercyE, et al.Misannotation awareness: A tale of two gene-groups. Front Plant Sci, 2016; 7:868.
42.
O'LearyNA, WrightMW, BristerJR, et al.Reference sequence (RefSeq) database at NCBI: Current status, taxonomic expansion, and functional annotation. Nucleic Acids Res, 2016; 44(D1):D733–D745.
43.
OverbeekR, BegleyT, ButlerRM, et al.The subsystems approach to genome annotation and its use in the project to annotate 1000 genomes. Nucleic Acids Res, 2005; 33(17):5691–5702.
44.
PaisFS-M, RuyPdC, OliveiraG, et al.Assessing the efficiency of multiple sequence alignment programs. Algorithms Mol Biol, 2014; 9(1):1–8.
45.
PapadimitriouK, PratsinisH, Nebe-von CaronG, et al.Rapid assessment of the physiological status of Streptococcus macedonicus by flow cytometry and fluorescence probes. Int J Food Microbiol, 2006; 111(3):197–205.
46.
PedregosaF, VaroquauxG, GramfortA, et al.Scikit-learn: Machine learning in Python. J Mach Learn Res, 2011; 12:2825–2830.
QinJ, LiR, RaesJ, et al.A human gut microbial gene catalogue established by metagenomic sequencing. Nature, 2010; 464(7285):59–65.
49.
R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing: Vienna, Austria; 2013.
50.
RoussetF, BikardD. CRISPR screens in the era of microbiomes. Curr Opin Microbiol, 2020; 57:70–77.
51.
SalzbergSL.Next-generation genome annotation: We still struggle to get it right. Genome Biol, 2019; 20(1):92.
52.
SandbergR, WinbergG, BrändenCI, et al.Capturing whole-genome characteristics in short sequences using a Naïve Bayesian Classifier. Genome Res, 2001; 11(8):1404–1409.
53.
SayersEW, BoltonEE, BristerJR, et al.Database resources of the national center for biotechnology information. Nucleic Acids Res, 2022; 50(D1):D20–D26.
54.
SchliepKP.phangorn: Phylogenetic analysis in R. Bioinformatics, 2011; 27(4):592–593.
55.
SchnoesAM, BrownSD, DodevskiI, et al.Annotation error in public databases: Misannotation of molecular function in enzyme superfamilies. PLoS Comput Biol, 2009; 5(12):e1000605.
TierneyBT, SzymanskiE, HenriksenJR, et al.Using Cartesian doubt to build a sequencing-based view of microbiology. Msystems, 2021; 6(5):e00574-21.
60.
TierneyBT, YangZ, LuberJM, et al.The landscape of genetic content in the gut and oral human microbiome. Cell Host Microbe, 2019; 26(2):283–295.e8.
61.
UddinS, KhanA, HossainME, et al.Comparing different supervised machine learning algorithms for disease prediction. BMC Med Inform Decis Mak, 2019; 19(1):281.
62.
UniProt Consortium. UniProt: The universal protein knowledgebase in 2021. Nucleic Acids Res, 2021; 49(D1):D480–D489.
63.
VervierK, MahéP, TournoudM, et al.Large-scale machine learning for metagenomics sequence classification. Bioinformatics, 2016; 32(7):1023–1032.
64.
WangX, NingY, LiC, et al.Alterations in the gut microbiota and metabolite profiles of patients with Kashin-Beck disease, an endemic osteoarthritis in china. Cell Death Dis, 2021; 12(11):1015.
65.
WhiteS, MedvedovicM.Kegglincs Design and Application: An R Package for Exploring Relationships in Biological Pathways [Version 1; Not Peer Reviewed]; 2016.
YuG. tidytree: A Tidy Tool for Phylogenetic Tree Data Manipulation. R Package Version 0.3.9; 2022. Available from: https://yulab-smu.top/treedata-book/
68.
ZamkovayaT, FosterJS, de Crécy-LagardV, et al.A network approach to elucidate and prioritize microbial dark matter in microbial communities. ISME J, 2021; 15(1):228–244.
69.
ZhaoD, BadurMG, LuebeckJ, et al.Combinatorial CRISPR-Cas9 metabolic screens reveal critical redox control points dependent on the KEAP1-NRF2 regulatory axis. Mol Cell, 2018; 69(4):699–708.e7.
70.
ZitvogelL, DaillèreR, RobertiMP, et al.Anticancer effects of the microbiome and its products. Nat Rev Microbiol, 2017; 15(8):465–478.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
