Transformer-based language models such as BERT and its variants have found widespread use in natural language processing (NLP). A common way of using these models is to fine-tune them to improve their performance on a specific task. However, it is currently unclear how the fine-tuning process affects the underlying structure of the word embeddings from these models. We present TopoBERT, a visual analytics system for interactively exploring the fine-tuning process of various transformer-based models – across multiple fine-tuning batch updates, subsequent layers of the model, and different NLP tasks – from a topological perspective. The system uses the mapper algorithm from topological data analysis (TDA) to generate a graph that approximates the shape of a model’s embedding space for an input dataset. TopoBERT enables its users (e.g. experts in NLP and linguistics) to (1) interactively explore the fine-tuning process across different model-task pairs, (2) visualize the shape of embedding spaces at multiple scales and layers, and (3) connect linguistic and contextual information about the input dataset with the topology of the embedding space. Using TopoBERT, we provide various use cases to exemplify its applications in exploring fine-tuned word embeddings. We further demonstrate the utility of TopoBERT, which enables users to generate insights about the fine-tuning process and provides support for empirical validation of these insights.
DevlinJChangMWLeeK, et al. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the conference of the North American Chapter of the Association for computational linguistics: human language technologies, 2019, pp.4171–4186. Association for Computational Linguistics.
2.
LiuYOttMGoyalN, et al. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint, arXiv:1907.11692, 2019.
3.
VaswaniAShazeerNParmarN, et al. Attention is all you need. Adv Neural Inf Process Syst2017; 30: 5998–6008.
4.
HowardJRuderS. Universal language model fine-tuning for text classification. In Proceedings of the 56th annual meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia, 2018, pp.328–339. Association for Computational Linguistics.
TenneyIXiaPChenB, et al. What do you learn from context? probing for sentence structure in contextualized word representations. In: International conference on learning representations (ICLR), 2019.
7.
ImmerATorroba HennigenLFortuinV, et al. Probing as quantifying inductive bias. In Proceedings of the 60th annual meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp.1839–1851. Dublin, Ireland: Association for Computational Linguistics.
8.
LasriKPimentelTLenciA, et al. Probing for the usage of grammatical number. In Proceedings of the 60th annual meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 8818–8831. Dublin, Ireland: Association for Computational Linguistics.
9.
AghazadehEFayyazMYaghoobzadehY. Metaphors in pre-trained language models: Probing and generalization across datasets and languages. In Proceedings of the 60th annual meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 2037–2050. Dublin, Ireland: Association for Computational Linguistics.
10.
ZhouYSrikumarV.A closer look at how fine-tuning changes BERT. In: Proceedings of the 60th annual meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.1046–1061. Dublin, Ireland: Association for Computational Linguistics.
11.
SinghGḾemoliFCarlssonG. Topological methods for the analysis of high dimensional data sets and 3D object recognition. Eurographics Symposium on Point-Based Graphics2007; 91–100.
12.
HoferCKwittRNiethammerM, et al. Deep learning with topological signatures. Adv Neural Inf Process Syst2017; 30: 1634–1644.
13.
RathoreAChalapathiNPalandeS, et al. TopoAct: visually exploring the shape of activations in deep learning. Comput Graph Forum2021; 40(1): 382–397.
14.
GabrielssonRBCarlssonGE. A look at the topology of convolutional neural networks. arXiv preprint, arXiv:1810.03234, 2018.
15.
CloughJRByrneNOksuzI, et al. A topological loss function for deep-learning based image segmentation using persistent homology. IEEE Trans Pattern Anal Mach Intell2022; 44: 8766–8778.
16.
HuXLiFSamarasD, et al. Topology-preserving deep image segmentation. Adv Neural Inf Process Syst2019; 32: 5657–5668.
17.
ChenCNiXBaiQ, et al. A topological regularizer for classifiers via persistent homology. In: Proceedings of the 22nd international conference on artificial intelligence and statistics, proceedings of machine learning research, pp.2573–2582, vol. 89. PMLR.
18.
RieckBTogninalliMBockC, et al. Neural persistence: A complexity measure for deep neural networks using algebraic topology. arXiv preprint, arXiv:1812.09764, 2018.
19.
CorneanuCAEscaleraSMartinezAM.Computing the testing error without a testing set. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.2677–2685. New York: ASME.
20.
NaitzatGZhitnikovALimLH. Topology of deep neural networks. J Mach Learn Res2020; 21(184): 1–40.
21.
BarannikovSSotnikovGTrofimovI, et al. Topological obstructions in neural networks learning. arXiv preprint, arXiv:2012.15834, 2020.
22.
DoshiPZadroznyW.Movie genre detection using topological data analysis, 2018. In: International conference on statistical language and speech processing, pp.117–128. Springer.
23.
SavleKZadroznyWLeeM. Topological data analysis for discourse semantics? In: Proceedings of the 13th international conference on computational semantics-student papers, 2019, pp.34–43. Association for Computational Linguistics.
24.
GuanHTangWKrimH, et al. A topological collapse for document summarization. In: IEEE 17th international workshop on signal processing advances in wireless communications (SPAWC), pp.1–5. New York: IEEE.
25.
DasSHaqueSATanveerM, et al. Persistence homology of TEDtalk: Do sentence embeddings have a topological shape?arXiv preprint, arXiv:2103.14131, 2021.
26.
KushnarevaLPiontkovskiDPiontkovskayaI. Betti numbers of attention graphs is all you really need. arXiv preprint, arXiv:2207.01903, 2022.
27.
KushnarevaLCherniavskiiDMikhailovV, et al. Artificial text detection via examining the topology of attention maps. arXiv preprint, arXiv:2109.04825, 2021.
28.
CherniavskiiDTulchinskiiEMikhailovV, et al. Acceptability judgements via examining the topology of attention maps. arXiv preprint, arXiv:2205.09630, 2022.
29.
PerezIReinauerR. The topological BERT: Transforming attention into topology for natural language processing. arXiv preprint, arXiv:2206.15195, 2022.
30.
RibeiroMTSinghSGuestrinC. “Why should I trust you?” explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016, pp.1135–1144. ACM.
31.
LiuMShiJLiZ, et al. Towards better analysis of deep convolutional neural networks. IEEE Trans Vis Comput Graph2017; 23(1): 91–100.
32.
PezzottiNHolltTVan GemertJ, et al. DeepEyes: Progressive visual analytics for designing deep neural networks. IEEE Trans Vis Comput Graph2018; 24(1): 98–108.
33.
WongsuphasawatKSmilkovDWexlerJ, et al. Visualizing dataflow graphs of deep learning models in TensorFlow. IEEE Trans Vis Comput Graph2018; 24(1): 1–12.
34.
ChooJLiuS. Visual analytics for explainable deep learning. IEEE Trans Vis Comput Graph2018; 38(4): 84–92.
35.
LiuSWangXLiuM, et al. Towards better analysis of machine learning models: A visual analytics perspective. Vis Inform2017; 1(1): 48–56.
36.
WangZJTurkoRShaikhO, et al. CNN explainer: Learning convolutional neural networks with interactive visualization. IEEE Trans Vis Comput Graph2021; 27(2): 1396–1406.
WexlerJPushkarnaMBolukbasiT, et al. The What-If Tool: interactive probing of machine learning models. IEEE Trans Vis Comput Graph2020; 26: 56–65.
39.
ChenCYuanJLuY, et al. OoDAnalyzer: interactive analysis of out-of-distribution samples. IEEE Trans Vis Comput Graph2021; 27(7): 3335–3349.
40.
YangWLiZLiuM, et al. Diagnosing concept drift with visual analytics. In: IEEE Conference on Visual Analytics Science and Technology (VAST), 2020, pp.12–23. New York: IEEE.
41.
KrauseJPererABertiniE. INFUSE: interactive feature selection for predictive modeling of high dimensional data. IEEE Trans Vis Comput Graph2014; 20(12): 1614–1623.
42.
MayTBannachADaveyJ, et al. Guiding feature subset selection with an interactive visualization. In: IEEE Conference on visual analytics science and technology (VAST), 2011, pp.111–120. New York: IEEE.
43.
SpringenbergJTDosovitskiyABroxT, et al. Striving for simplicity: the all convolutional Net. In: Workshop track proceedings of the 3rd international conference on learning representations (ICLR), San Diego, CA.
44.
RauberPEFadelSGFalcaoAX, et al. Visualizing the hidden activity of artificial neural networks. IEEE Trans Vis Comput Graph2017; 23(1): 101–110.
45.
WangJGouLYangH, et al. GANViz: A visual analytics approach to understand the adversarial game. IEEE Trans Vis Comput Graph2018; 24(6): 1905–1917.
46.
KahngMAndrewsPYKalroA, et al. ActiVis: Visual exploration of industry-scale deep neural network models. IEEE Trans Vis Comput Graph2018; 24: 88–97.
47.
HohmanFParkHRobinsonC, et al. Summit: Scaling deep learning interpretability by visualizing activation and attribution summarizations. IEEE Trans Vis Comput Graph2020; 26(1): 1096–1106.
48.
AmershiSChickeringMDruckerSM, et al. ModelTracker: Redesigning performance analysis tools for machine learning. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, 2015, pp.337–346. ACM.
49.
RenDAmershiSLeeB, et al. Squares: Supporting interactive performance analysis for multiclass classifiers. IEEE Trans Vis Comput Graph2017; 23(1): 61–70.
50.
ZhangJWangYMolinoP, et al. Manifold: A model-agnostic framework for interpretation and diagnosis of machine learning models. IEEE Trans Vis Comput Graph2019; 25: 364–373.
51.
WangQAlexanderWPeggJ, et al. HypoML: Visual analysis for hypothesis-based evaluation of machine learning models. IEEE Trans Vis Comput Graph2021; 27: 1417–1426.
52.
GarceaFFamouriSValentinoD, et al. iNNvestigate-GUI -explaining neural networks through an interactive visualization tool. In: Schilling F and Stadelmann T (eds) Artificial neural networks in pattern recognition. Cham: Springer International Publishing, 2020, pp.291–303.
53.
YuanJChenCYangW, et al. A survey of visual analytics techniques for machine learning. Comput Vis Media2021; 7(1): 3–36.
54.
ChatzimparmpasAMartinsRMJusufiI, et al. A survey of surveys on the use of visualization for interpreting machine learning models. Inf Vis2020; 19: 207–233.
55.
IslamMRAkterSRatanMR, et al. Deep visual analytics (DVA): applications, challenges and future directions. Human-Centric Intell Syst2021; 1(1-2): 3–17.
56.
HohmanFMKahngMPientaR, et al. Visual analytics in deep learning: an interrogative survey for the next frontiers. IEEE Trans Vis Comput Graph2019; 25: 2674–2693.
57.
LiuSBremerPTThiagarajanJJ, et al. Visual exploration of semantic relationships in neural word embeddings. IEEE Trans Vis Comput Graph2018; 24(1): 553–562.
58.
SpinnerTSchlegelUSchaferH, et al. ExplAIner: A visual analytics framework for interactive and explainable machine learning. IEEE Trans Vis Comput Graph2020; 26(1): 1064–1074.
59.
RibeiroMTSinghSGuestrinC.Anchors: High-precision model-agnostic explanations. In: Proceedings of the AAAI conference on artificial intelligence, 2018, vol. 32.
60.
LiuSLiZLiT, et al. NLIZE: A perturbation-driven visual interrogation tool for analyzing and interpreting natural language inference models. IEEE Trans Vis Comput Graph2019; 25(1): 651–660.
61.
ChanGYYBertiniENonatoLG, et al. Melody: Generating and visualizing machine learning model summary to understand data and classifiers together. arXiv preprint, arXiv:2007.10614, 2020.
62.
MingYCaoSZhangR, et al. Understanding hidden memories of recurrent neural networks. In IEEE conference on visual analytics science and technology (VAST), 2017, pp.13–24. New York: IEEE.
63.
StrobeltHGehrmannSPfisterH, et al. LSTMVis: A tool for visual analysis of hidden state dynamics in recurrent neural networks. IEEE Trans Vis Comput Graph2018; 24(1): 667–676.
JiXShenHWRitterA, et al. Visual exploration of neural document embedding in information retrieval: semantics and feature selection. IEEE Trans Vis Comput Graph2019; 25: 2181–2192.
66.
VigJ.A multiscale visualization of attention in the transformer model. In: Proceedings of the 57th Annual meeting of the Association for Computational Linguistics: system demonstrations, 2019, pp.37–42. Florence, Italy: Association for Computational Linguistics.
67.
PearsonK.LIII. On lines and planes of closest fit to systems of points in space. Lond Edinb Dubl Philos Mag J Sci1901; 2(11): 559–572.
68.
Van der MaatenLHintonG. Visualizing data using t-SNE. J Mach Learn Res2008; 9(11): 2579–2605.
69.
McInnesLHealyJMelvilleJ. UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv preprint, arXiv:1802.03426, 2018.
70.
LiuSMaljovecDWangB, et al. Visualizing high-dimensional data: advances in the past decade. IEEE Trans Vis Comput Graph2017; 23(3): 1249–1268.
71.
RogersAKovalevaORumshiskyA. A primer in BERTology: What we know about how BERT works. Trans Assoc Comput Linguist2021; 8: 842–866.
72.
WhitneyWFSongMJBrandfonbrenerD, et al. Evaluating representations by the complexity of learning low-loss predictors. In: Neural compression: from information theory to applications, workshop at ICLR, 2021.
73.
VoitaETitovI.Information-theoretic probing with minimum description length. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), 2020, pp.183–196. Association for Computational Linguistics.
74.
LimisiewiczTMarečekD.Introducing orthogonal constraint in structural probes. In: Proceedings of the 59th annual meeting of the Association for Computational Linguistics and the 11th international joint conference on natural language processing (Volume 1: Long Papers), pp.428–442, 2021. Association for Computational Linguistics.
75.
ZhouYSrikumarV.DirectProbe: Studying representations without classifiers. In: Proceedings of the 2021 conference of the North American Chapter of the Association for Computational Linguistics: human language technologies, 2021, pp.5070–5083. Association for Computational Linguistics.
76.
KaridiTZhouYSchneiderN, et al. Putting words in BERT’s mouth: Navigating contextualized vector spaces with pseudowords. In: Proceedings of the 2021 conference on empirical methods in natural language processing, 2021, pp.10300–10313. Association for Computational Linguistics.
77.
HewittJManningC D.A structural probe for finding syntax in word representations. In: Proceedings of the 2019 conference of the North American Chapter of the Association for Computational Linguistics: human language technologies, 2019, vol. 1 (Long and Short Papers), pp.4129–4138. Association for Computational Linguistics.
78.
EthayarajhK.How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), 2019, pp. 55–65. Association for Computational Linguistics.
79.
PetersMERuderSSmithNA.To tune or not to tune? adapting pretrained representations to diverse tasks. In: Proceedings of the 4th workshop on representation learning for NLP (RepL4NLP-2019), 2019, pp.7–14. Association for Computational Linguistics.
80.
MerchantARahimtoroghiEPavlickE, et al. What happens to BERT embeddings during fine-tuning? In: Proceedings of the third BlackboxNLP workshop on analyzing and interpreting neural networks for NLP, pp.33–44, 2020. Association for Computational Linguistics.
81.
MosbachMKhokhlovaAHedderichMA, et al. On the interplay between fine-tuning and sentence-level probing for linguistic knowledge in pre-trained transformers. In Proceedings of the third BlackboxNLP workshop on analyzing and interpreting neural networks for NLP, 2020, pp.68–82. Association for Computational Linguistics.
82.
HaoYDongLWeiF, et al. Investigating learning dynamics of BERT fine-tuning. In: Proceedings of the 1st conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th international joint conference on natural language processing, 2020, pp.87–92. Suzhou, China: Association for Computational Linguistics.
83.
AleksandroffPS. Über den allgemeinen dimensionsbegriff und seine beziehungen zur elementaren geometrischen anschauung. Math Ann1928; 98(1): 617–635.
84.
BiasottiSGiorgiDSpagnuoloM, et al. Reeb graphs for shape analysis and applications. Theoretical Computer Science2008; 392: 5–22.
85.
BiasottiSMariniSMortaraM, et al. An overview on properties and efficacy of topological skeletons in shape modelling. In: 2003 Shape Modeling International, 12–15May2003. New York: IEEE.
86.
CarrièreMMichelBOudotS. Statistical analysis and parameter selection for mapper. J Mach Learn Res2018; 19(12): 1–39.
87.
ChalapathiNZhouYWangB. Adaptive covers for mapper graphs using information criteria. In: IEEE international conference on Big Data (Big Data). New York: IEEE.
88.
EsterMKriegelHPSanderJ, et al. A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the 2nd International conference on knowledge discovery and data mining, 1996, pp.226–231. ACM.
89.
ZhouYChalapathiNRathoreA, et al. Mapper Interactive: A scalable, extendable, and interactive toolbox for the visual exploration of high-dimensional data. In: Proceedings of the IEEE 14th Pacific visualization symposium (PacificVis), pp.101–110. New York: IEEE.
90.
PurvineEBrownDJeffersonB, et al. Experimental observations of the topology of convolutional neural network activations. In: Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI), 2021.
91.
ChowdhurySNeedhamTSemradE, et al. Hypergraph cooptimal transport: Metric and categorical properties. arXiv preprint, arXiv:2112.03904, 2021.
92.
SchneiderNSrikumarVHwangJD, et al. A hierarchy with, of, and for preposition supersenses. In: Proceedings of the 9th Linguistic annotation workshop, 2015, pp.112–123. Denver, CO: Association for Computational Linguistics. DOI: 10.3115/v1/W15-1612. https://aclanthology.org/W15-1612
93.
SchneiderNHwangJDSrikumarV, et al. Comprehensive supersense disambiguation of English prepositions and possessives. In: Proceedings of the 56th Annual meeting of the Association for Computational Linguistics, 2018, vol. 1: Long Papers, pp.185–196. Melbourne: Association for Computational Linguistics. DOI: 10.18653/ v1/P18-1018. https://aclanthology.org/P18-1018
94.
HwangJDBhatiaAHanNR, et al. Double trouble: The problem of construal in semantic annotation of adpositions. In: Proceedings of the 6th joint conference on lexical and computational semantics (*SEM 2017), 2017, pp. 178–188. Vancouver: Association for Computational Linguistics. DOI: 10.18653/v1/S17-1022. https://aclanthology.org/S17-1022
95.
LiuNFGardnerMBelinkovY, et al. Linguistic knowledge and transferability of contextual representations. In: Proceedings of the 2019 conference of the North American Chapter of the Association for Computational Linguistics: human language technologies, 2019, vol. 1 (Long and Short Papers), pp.1073–1094. Minneapolis, MN: Association for Computational Linguistics.
96.
NivreJde MarneffeMCGinterF, et al. Universal Dependencies v1: A multilingual treebank collection. In: Proceedings of the tenth international conference on language resources and evaluation (LREC), 2016, pp.1659–1666. Portorož, Slovenia: European Language Resources Association (ELRA).
97.
TurcIChangMWLeeK, et al. Well-read students learn better: On the importance of pre-training compact models. arXiv preprint, arXiv:1908.08962, 2019.
98.
WolfTDebutLSanhV, et al. Transformers: State-ofthe-art natural language processing. In: Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, 2020, pp.38–45. Association for Computational Linguistics.
99.
LoshchilovIHutterF.Decoupled weight decay regularization. In: International conference on learning representations (ICLR), 2017.
100.
LloydS. Least squares quantization in PCM. IEEE Trans Inf Theory1982; 28(2): 129–137.
101.
ShneidermanB.The eyes have it: A task by data type taxonomy for information visualizations. In: Proceedings of IEEE symposium on visual languages, pp.336–343. New York: IEEE.
102.
AbadiMAgarwalABarhamP, et al. TensorFlow: Large-scale machine learning on heterogeneous systems. https://www.tensorflow.org/ (2015).
103.
MikolovTSutskeverIChenK, et al. Distributed representations of words and phrases and their compositionality. Adv Neural Inf Process Syst2013; 26: 3111–3119.
104.
de MarneffeMCDozatTSilveiraN, et al. Universal Stanford dependencies: A cross-linguistic typology. In: Proceedings of the Ninth international conference on language resources and evaluation (LREC), 2014, pp.4585–4592. Reykjavik, Iceland: European Language Resources Association (ELRA).
105.
KovalevaORomanovARogersA, et al. Revealing the dark secrets of BERT. In: Proceedings of the conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), 2019, pp.4365–4374. Hong Kong, China: Association for Computational Linguistics.
106.
PetersMENeumannMIyyerM, et al. Deep contextualized word representations. In: Proceedings of the conference of the North American Chapter of the Association for Computational Linguistics: human language technologies, 2018, vol. 1, Long Papers, pp.2227–2237. New Orleans, Louisiana: Association for Computational Linguistics.
107.
ConneauAKhandelwalKGoyalN, et al. Unsupervised cross-lingual representation learning at scale. In: Proceedings of the 58th annual meeting of the Association for Computational Linguistics, 2020, pp.8440–8451. Association for Computational Linguistics.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
