Topic modelling is an important technique for extracting meaningful insights from large volumes of unstructured text data. The paper presents a federated technique for topic modelling that is based on a novel approach of the Latent Dirichlet Allocation (LDA) method for topic model generation. The proposed approach enables a topic model to be developed in a distributed environment using data continually generated from multiple sources without the need for sharing actual data. The first iteration of the topic modelling uses unsupervised LDA at each device generating the data. The results of each device are aggregated at a central server to generate a set of seed words that are used for guided LDA by the subsequent iterations of topic modelling. The proposed work, Federated LDA (F-LDA) has been evaluated using two datasets: a text dataset of dialogues between patients and doctors based on factual conversations and another comprising tweets related to depression. Comparing the performance of F-LDA with that of a centralized LDA, it was observed that F-LDA results in improved coherence score as well as diversity score in comparison to centralized LDA. This indicates that F-LDA achieves better interpretable topics covering a wide range of themes without redundancy.
QiuJWuQDingG, et al.A survey of machine learning for big data processing. EURASIP J Adv Signal Process2016; 2016: 1–16.
2.
WangHWuFLuW, et al.Identifying objective and subjective words via topic modeling. IEEE Trans Neural Netw Learn Syst2018; 29: 718–730.
3.
BleiDNgAJordanM. Latent dirichlet allocation. J. Mach Learn Res2003; 3: 993–1022.
4.
PapadiaGPacellaMPerroneM,et al.A comparison of different topic modelling methods through a real case study of Italian customer care. MDPI, 8 February 2023.
5.
JelodarHWangYYuanC, et al.Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey. Multimed Tools Appl2019; 78: 15169–15211.
6.
AngelovD. Top2vec: Distributed representations of topics. arXiv preprint arXiv:2008.09470. (2020).
7.
YochumPNisamaneewongT. Automated disease detection based on clinical text using topic modeling. December 2022, pp.74–79: ICIT.
8.
JelodarHWangYYuanC. Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey. (13 November 2018).
9.
OliveiraFBHaqueAMougoueiD, et al.Investigating the emotional response to COVID-19 news on Twitter: a topic modellingand emotion classification approach. IEEE Access2022; 10: 16883–16816.
10.
KushwahaRKaurP. Depression detection on social Media. In: 2022 1st International Conference on Informatics (ICI), 2022, pp.153–158.
11.
KorencicDRistovSReparJ,et al.A topic coverage approach to evaluation of topic models. IEEE Access2021; 9: 123280–123123.
12.
ZhangWZhouTLuQ, et al.Dynamic-Fusion-Based federated learning for COVID-19 detection. IEEE Internet of Things Journal2021; 8: 15884–15891.
13.
ZhaoFRenXYangS, et al.Latent Dirichlet allocation model training with differential privacy. IEEE Transactions on Information Forensics and Security2020; 16: 1290–1305.
14.
AledhariMRazzakRPariziRM,et al.Federated learning: a survey on enabling technologies, protocols, and applications. IEEE Access2020; 8: 140699–140725.
15.
GuoYZhaoZHeK, et al.Efficient and flexible management for industrial internet of things: a federated learning approach. Computer Networks2021; 192: 108122.
16.
WuWHeLLinW,et al.Accelerating federated learning over reliability-agnostic clients in mobile edge computing systems. IEEE Trans Parallel Distrib Syst2020; 32: 1539–1155.
17.
VerbraekenJWoltingMKatzyJ, et al.A survey on distributed machine learning. ACM Comput Surveys (CSUR)2020; 53: 1–33.
18.
XuJGlicksbergBSSuC, et al.Federated learning for healthcare informatics. J Healthcare Inform Res2021; 5: 1–19.
19.
SadilekALiuLNguyenD, et al.Privacy-first health research with federated learning. NPJ Digital Med2021; 4: 1–8.
20.
CuiJZhuHDengH, et al.FeARH: federated machine learning with anonymous random hybridization on electronic medical records. J Biomed Inform2021; 117: 103735.
21.
ChamikaraMAPBertokPKhalilI, et al.Privacy preserving distributed machine learning with federated learning. Comput Commun2021; 171: 112–125.
22.
GeSWuFWuC, et al.Fedner: Privacy-preserving medical named entity recognition with federated learning. arXiv preprint arXiv:2003.09288. (2020).
23.
ChalamalaSRKummariNKSinghAK, et al.Federated learning to comply with data protection regulations. CSI Trans ICT2022; 10: 47–60.
24.
OrkpholK. Word sense disambiguation using cosine similarity collaborates withWord2vec and WordNet. MDPI, 12 May 2019.
25.
OrkpholKYangW. Word sense disambiguation using cosine similarity collaborates withWord2vec and WordNet. MDPI, 12 May 2019.
26.
BlairSJBiY. Aggregated topic models for increasing social media topic coherence. Appl Intell2022: 138–156.
MedianoJMQuinteroJAC. A comparison study between coherence and perplexity for determining the number of topics in practitioners’ interviews analysis”. 13 April 2022, pp.225–234.
32.
SyedSSpruitM. Full-text or abstract? Examining topic coherence scores using latent dirichlet allocation. In: 2017 IEEE International conference on data science and advanced analytics (DSAA), 2017, pp.165–174: IEEE.
33.
TranNKZerrSBischoffK, et al.Topic cropping: leveraging latent topics for the analysis of small corpora. In: AalbergTPapatheodorouCDobrevaMTsakonasGFarrugiaCJ (eds) Research and advanced technology for digital libraries. 2013, pp.297–308.
MagattiDCalegariSCiucciD,et al.Automatic labeling of topics. In: 2009 Ninth International Conference on Intelligent Systems Design and Applications, Pisa, Italy, 2009, pp.1227–1232.
36.
LauJHGrieserKNewmanD,et al.Automatic labelling of topic models. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, Portland, Oregon, June 19-24, 2011, pp.1536–1545.
37.
KimHHRheeHY. An ontology-based labeling of influential topics using topic network analysis. J Inform Proces Syst2019; 15: 1096–1107.
