With the rapid advancement of new technologies, the number of academic papers in various fields has grown exponentially, making traditional keyword-based search methods insufficient to capture semantic information comprehensively. This article presents a method called SETM, which combines Text-embedding-ada-002, BERTopic and PageRank models to perform semantic extraction, topic modelling and ranking on large-scale academic papers. UMAP is used for dimensionality reduction and visualisation, revealing semantic relationships between papers. The system employs a tree-like layout combined with multi-level interactive features, allowing users to conveniently explore and retrieve papers, from a coarse-grained ‘paper galaxy’ view to fine-grained individual papers. Experimental results demonstrate that the proposed approach achieves high accuracy in semantic mining and clustering, providing an effective tool for dynamic visualisation and analysis of large-scale academic papers.
Alonso-MencíaJ. Seeking clustering excellence: unleashing the power of sentence transformers and preprocessing techniques. In: IberLEF@ SEPLN. Transformers in review analysis, Jaén, 26 September 2023.
2.
GalliCCusanoCMeletiM, et al. Topic modeling for faster literature screening using transformer-based embeddings. Metrics2024; 1(1): 2.
3.
MershaMAGemeda YigezuMKalitaJ.Semantic-driven topic modeling using transformer-based embeddings and clustering algorithms. Procedia Comput Sci2024; 244: 121–132.
4.
YangWWangXLuJ, et al. Interactive steering of hierarchical clustering. IEEE Trans Vis Comput Graph2020; 27(10): 3953–3967.
5.
WijantoMCWidiastutiIYongHS.Topic modeling for scientific articles: exploring optimal hyperparameter tuning in BERT. Int J Adv Sci Eng Inf Technol2024; 14(3): 912–919.
6.
KoroteevMV.BERT: a review of applications in natural language processing and understanding. arXiv:2103.11943 2021.
7.
MaYJiJSunX, et al. Knowing what it is: semantic-enhanced dual attention transformer. IEEE Trans Multimedia2022; 25: 3723–3736.
8.
ReimersNGurevychI.Sentence-BERT: sentence embeddings using Siamese BERT-networks. arXiv:1908.10084 2019.
9.
AperdannierRKoeppelMUngerT, et al. Systematic evaluation of different approaches on embedding search. In: AraiK (ed.) Future of information and communication conference. Cham: Springer, 2024, pp. 526–536.
10.
TaoYKongLKanA, et al. Textual dataset distillation via language model embedding. In: Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 2024, pp. 12557–12569. Association for Computational Linguistics.
11.
BleiDMNgAYJordanMILatent dirichlet allocation. In: Journal of Machine Learning Research, 3 Januray 2003, pp. 993–1022.
12.
GrootendorstM.BERTopic: neural topic modeling with a class-based TF-IDF procedure. arXiv:2203.057942022.
13.
MassucciFADocampoD.Measuring the academic reputation through citation networks via PageRank. J Informetr2019; 13(1): 185–201.
14.
LuYMaKDuanJ. Influence model of paper citation networks with integrated PageRank and HITS. In: 2021 IEEE 24th international conference on computer supported cooperative work in design (CSCWD), Dalian, China, 5–7 May 2021, pp. 1081–1086. New York: IEEE.
15.
WangRLiSYinQ, et al. Improved pagerank and new indices for academic impact evaluation using AI papers as case studies. J Inf Sci2024; 50(3): 690–702.
16.
BelkinaACCiccolellaCOAnnoR, et al. Automated optimized parameters for T-distributed stochastic neighbor embedding improve visualization and analysis of large datasets. Nat Commun2019; 10(1): 5415.
17.
SoniJPrabakarNUpadhyayH.Visualizing high-dimensional data using t-distributed stochastic neighbor embedding algorithm. In: ArabniaHRDaimiKStahlbockR, et al. (eds) Principles of data science. Cham: Springer, 2020, pp. 189–206.
18.
McInnesLHealyJMelvilleJ.UMAP: uniform manifold approximation and projection for dimension reduction. arXiv:1802.034262018.
19.
BoyapatiMAygunR.Semanformer: semantics-aware embedding dimensionality reduction using transformer-based models. In: 2024 IEEE 18th international conference on semantic computing (ICSC), Laguna Hills, CA, USA, 5–7 February 2024, pp. 134–141. New York: IEEE.
20.
McAllisterJTLennertzLMojicaZA.Mapping a discipline: a guide to using VOSviewer for bibliometric and visual analysis. Sci Technol Libr2022; 41(3): 319–348.
21.
ArrudaHSilvaERLessaM, et al. VOSviewer and bibliometrix. J Med Libr Assoc2022; 110(3): 392–395.
22.
DingXYangZ.Knowledge mapping of platform research: a visual analysis using VOSviewer and CiteSpace. Electron Commer Res2022; 22: 1–23.
LiuYSchickTSchützeH.Semantic-oriented unlabeled priming for large-scale language models. arXiv:2202.061332022.
25.
BafnaPPramodDVaidyaA.Document clustering: TF-IDF approach. In: 2016 international conference on electrical, electronics, and optimization techniques (ICEEOT), Chennai, India, 3–5 March 2016, pp. 61–66. New York: IEEE.
26.
ChurchKW. Word2Vec. Nat Lang Eng2017; 23(1): 155–162.
27.
Nałecz-CharkiewiczKCharkiewiczKNowakRM.Quantum computing in bioinformatics: a systematic review mapping. Brief Bioinform2024; 25(5): bbae391.
28.
StrubellEGaneshAMcCallumA.Energy and policy considerations for modern deep learning research. Proc AAAI Conf Artif Intell2020; 34(09): 13693–13696.
29.
AugerTSaroyanE. Overview of the OpenAI APIs. In: Generative AI for web development: building web applications powered by OpenAI APIs and Next.js. Berkeley, CA, 2024, pp. 87–116. Apress (Springer Nature).
