Identifying and extracting valuable information from textual documents in the form of cohesively and appropriately developed summaries is one of the most challenging tasks in text mining and natural language processing. In this article, we present a sequential Markov model, equipped with Bayesian inference, to estimate the degree of importance of sentences in a document and thereby address the text summarisation problem. The proposed methodology models the extractive sentence summarisation as a Bayesian state estimation problem, where the system state is the importance degree of each sentence in a document. The transition and observation models are derived using a nonlinear dynamical system identification based on a recurrent feedback neural model that predicts the sentence observation using the sentence input data. In the end, the transition and observation probability density functions are modelled using a mixture density network. The performance assessment of the system has been carried out by investigating the optimal feature dimensionality and the impact of the model parameters on the system accuracy, using entropy-based risk and loss-based risk measures. Finally, the superiority of the proposed methodology over the state of the art in extractive summarisation is discussed and verified by reporting the recall, precision and accuracy on the real-world benchmark data sets.
ZhouQYangNWeiFet al. Neural document summarization by jointly learning to score and select sentences. arXiv Preprint2018: 180702305, https://arxiv.org/abs/1807.02305
9.
NallapatiRZhaiFZhouB. SummaRuNNer: a recurrent neural network based sequence model for extractive summarization of documents. In: Thirty-first AAAI conference on artificial intelligence, https://dl.acm.org/doi/10.5555/3298483.3298681
10.
DongYShenYCrawfordEet al. BanditSum: extractive summarization as a contextual bandit. arXiv Preprint2018: 180909672, https://arxiv.org/abs//1809.09672
11.
XuJDurrettG.Neural extractive text summarization with syntactic compression. arXiv Preprint2019: 190200863, https://arxiv.org/abs/1902.00863
12.
ShiJLiangCHouLet al. DeepChannel: salience estimation by contrastive learning for extractive document summarization. In: Proceedings of the AAAI conference on artificial intelligence, Honolulu, HI, 27 January–1 February 2019, vol. 33, pp. 6999–7006. Palo Alto, CA: AAAI Press
13.
LewisMLiuYGoyalNet al. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv Preprint2019: 191013461, https://arxiv.org/abs/1910.13461
ZhangHXuJWangJ.Pretraining-based natural language generation for text summarization. arXiv preprint2019: 190209243, https://arxiv.org/abs/1902.09243
16.
OuSKhooCSGGohDH.Design and development of a concept-based multi-document summarization system for research abstracts. J Inf Sci2008; 34(3): 308–326.
17.
KryścińskiWKeskarNSMcCannBet al. Neural text summarization: a critical evaluation. arXiv Preprint2019: 190808960, https://arxiv.org/abs/1908.08960
18.
KryścińskiWMcCannBXiongCet al. Evaluating the factual consistency of abstractive text summarization. arXiv Preprint2019: 191012840, https://arxiv.org/abs/1910.12840
19.
DuXShaoJCardieC.Learning to ask: neural question generation for reading comprehension. arXiv Preprint2017: 170500106, https://arxiv.org/abs/1705.00106
20.
DuXCardieC. Identifying where to focus in reading comprehension for neural question generation. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp. 2067–2073, https://aclanthology.org/D17-1219/
21.
KumarVBoorlaKMeenaYet al. Automating reading comprehension by generating question and answer pairs. In: Pacific-Asia conference on knowledge discovery and data mining, Melbourne, VIC, Australia, 3–6 June 2018, pp. 335–348. New York: Springer.
22.
LiuJChenYLiuKet al. Event extraction as machine reading comprehension. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 16–20 November 2020, pp. 1641–1651, https://aclanthology.org/2020.emnlp-main.128.pdf
23.
AullsMW.Developmental and remedial reading in the middle grades. Boston, MA: Allyn & Bacon1978.
24.
BraddockR.The frequency and placement of topic sentences in expository prose. Res Teach Engl1974; 8(3): 287–302.
25.
AshtonPJO’HearMFPhersonVE.The value of main idea clues for use with college textbooks. J Coll Read Learn1985; 18(1): 59–67.
26.
O’HearMFRamseyRNPhersonVE.Location of main ideas in English composition texts. Res Teach Engl1987; 21: 318–326.
27.
HareVCRabinowitzMSchiebleKM. Text effects on main idea comprehension. Read Res Quart1989: 72–88.
28.
ConroyJMO’LearyDP.Text summarization via hidden Markov models. In: Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, New Orleans, LA, September 2001, pp. 406–407. New York: ACM.
29.
BelangerDKakadeS.A linear dynamical system model for text. PMLR2015; 37: 833–842.
30.
GruskyMNaamanMArtziY.Newsroom: a dataset of 1.3 million summaries with diverse extractive strategies. arXiv Preprint2018: 180411283, https://arxiv.org/abs/1804.11283
NapolesCGormleyMRVan DurmeB. Annotated Gigaword. In: Proceedings of the joint workshop on automatic knowledge base construction and web-scale knowledge extraction (AKBC-WEKEX), pp. 95–100, https://aclanthology.org/W12-3018.pdf
33.
SandhausE.The New York Times annotated corpus. Philadelphia, PA: Linguistic Data Consortium, 2008.
34.
RushAMChopraSWestonJ.A neural attention model for abstractive sentence summarization. arXiv Preprint2015: 150900685, https://arxiv.org/abs/1509.00685
35.
HermannKMKociskyTGrefenstetteEet al. Teaching machines to read and comprehend. Adv Neural Inf Process Syst2015; 28: 1–9.
36.
NorvigPRIntelligenceSA.A modern approach. Upper Saddle River, NJ: Prentice Hall, 2002.
37.
KoenigSSimmonsRG.Xavier: a robot navigation architecture based on partially observable Markov decision process models. In: KortenkampDPeter BonassoRMurphyR (eds) Artificial intelligence based mobile robotics: case studies of successful robot systems. Cambridge, MA: MIT Press, 1998, pp. 91–122.
38.
ElliottRJAggounLMooreJB.Hidden Markov models: estimation and control, vol. 29. New York: Springer Science & Business Media, 2008.
39.
FoxDBurgardWThrunS.Markov localization for mobile robots in dynamic environments. J Artif Intell Res1999; 11: 391–427.
KlenkeA.Probability theory: a comprehensive course. New York: Springer Science & Business Media, 2013.
42.
RauchHETungFStriebelCT.Maximum likelihood estimates of linear dynamic systems. AIAA J1965; 3(8): 1445–1450.
43.
PanYSungSWLeeJH.Data-based construction of feedback-corrected nonlinear prediction model using feedback neural networks. Control Eng Pract2001; 9(8): 859–867.
44.
MoschittiABasiliR. Complex linguistic features for text classification: a comprehensive study. In: McDonaldSTaitJ (eds) European conference on information retrieval. New York: Springer, pp. 181–196.
45.
BiberDConradS.Register, genre, and style. Cambridge: Cambridge University Press, 2019.
46.
HiraoTIsozakiHMaedaEet al. Extracting important sentences with support vector machines. In: COLING 2002: the 19th international conference on computational linguistics, pp. 1–7, https://aclanthology.org/C02-1053.pdf
47.
KorenYBellRVolinskyC.Matrix factorization techniques for recommender systems. Computer2009; 42(8): 30–37.
48.
ChenHH.Weighted-SVD: matrix factorization with weights on the latent factors. arXiv Preprint2017: 171000482, https://arxiv.org/abs/1710.00482
49.
BishopCM.Mixture density networks. Birmingham: Aston University, 1994.
ZellnerA.Bayesian estimation and prediction using asymmetric loss functions. J Am Stat Assoc1986; 81(394): 446–451.
52.
HendersonLVedralV.Information, relative entropy of entanglement, and irreversibility. Phys Rev Lett2000; 84(10): 2263.
53.
LinCY. Rouge: a package for automatic evaluation of summaries. In: Text summarization branches out, pp. 74–81, https://aclanthology.org/W04-1013.pdf
54.
ChristianHAgusMPSuhartonoD.Single document automatic text summarization using term frequency-inverse document frequency (TF-IDF). ComTech: Comput Math Eng Appl2016; 7(4): 285–294.
55.
YehJYKeHRYangWPet al. Text summarization using a trainable summarizer and latent semantic analysis. Inf Process Manag2005; 41(1): 75–95.
56.
BarreraAVermaR. Combining syntax and semantics for automatic extractive single-document summarization. In: GelbukhA (ed.) International conference on intelligent text processing and computational linguistics. New York: Springer, 2012, pp. 366–377.
57.
MallickCDasAKDuttaMet al. Graph-based text summarization using modified textrank. In: NayakJAbrahamAKrishnaBet al. (eds) Soft computing in data analytics. Singapore: Springer, 2019, pp. 137–146.
58.
HsuWTLinCKLeeMYet al. A unified model for extractive and abstractive summarization using inconsistency loss. arXiv Preprint2018: 180506266, https://arxiv.org/abs/1805.06266
59.
ZhongMLiuPChenYet al. Extractive summarization as text matching. arXiv Preprint2020: 200408795, https://arxiv.org/abs/2004.08795
