Text-based reading comprehension models have great research significance and market value and are one of the main directions of natural language processing. Reading comprehension models of single-span answers have recently attracted more attention and achieved significant results. In contrast, multi-span answer models for reading comprehension have been less investigated and their performances need improvement. To address this issue, in this paper, we propose a text-based multi-span network for reading comprehension, ALBERT_SBoundary, and build a multi-span answer corpus, MultiSpan_NMU. We also conduct extensive experiments on the public multi-span corpus, MultiSpan_DROP, and our multi-span answer corpus, MultiSpan_NMU, and compare the proposed method with the state-of-the-art. The experimental results show that our proposed method achieves F1 scores of 84.10 and 92.88 on MultiSpan_DROP and MultiSpan_NMU datasets, respectively, while it also has fewer parameters and a shorter training time.
ElnagarA., Al-DebsiR. and EineaO., Arabic text classification using deep learning models, Information Processing & Management57(1) (2020), 102121.
2.
WeiD., TangN. and WenS., The deep learning word vector model using part of speech and sentiment, Journal of Intelligent & Fuzzy Systems38(1) (2020), 427–440.
3.
HorstH.T., HartungM., CimianoP., BrazdaN., MüllerH.W. and KlingerR., Learning soft domain constraints in a factor graph model for template-based information extraction, Data & Knowledge Engineering125 (2020), 101764.
4.
BudihartoW., AndreasV. and GunawanA.A.S., Deep learning-based question answering system for intelligent humanoid robot, Journal of Big Data7(1) (2020), 1–10.
5.
ChenD., MaZ., WeiL., MaJ. and Zhu.Y., MTQA: Text-based multitype question and answer reading comprehension model, Computational Intelligence and Neuroscience2021 (2021), 1–12.
6.
NieL., WeiX., ZhangD., WangX., GaoZ. and YangY., Data-driven answer selection in community QA systems, IEEE Transactions on Knowledge and Data Engineering29(6) (2017), 1186–1198.
7.
NieL.Q., ZhaoY.L., WangX., ShenJ. and ChuaT.S., Learning to recommend descriptive tags for questions in social forums, ACM Transactions on Information Systems (TOIS)32(1) (2014), 1–23.
8.
ChengY., Semi-supervised learning for neural machine translation, in: Joint Training for Neural Machine Translation, Springer, Singapore, 2019, pp. 25–40.
LanZ.Z., ChenM., GoodmanS., GimpleK., SharmaP. and SoricutR., ALBERT:ALite Bert for Self-supervised Learning of Language Representations, International Conference on Learning Representations, 2020, 1-17.
11.
SuL.X., GuoJ.F., FanY.X., LanY.Y., XuJ. and ChengX.Q., Extractive reading comprehension model for multi-segment answers, Chinese Journal of Computers43(05) (2020), 856–867.
12.
DuaD., WangY., DasigiP., StanovskyG., SinghS. and GardnerM., DROP:AReading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs, The North American Chapter of the Association for Computational Linguistics, 2019, 2368-2378.
13.
EfratA., SegalE. and ShohamM., Tag-based multi-span extraction in reading comprehension, ArXiv Preprint ArXiv (2019).
14.
RajpurkarP., ZhangJ., LopyrevK. and LiangP., SQuAD: 100,000+Questions for Machine Comprehension of Text, Conference on Empirical Methods in Natural Language Processing, 2016, 2383-2392.
15.
RajpurkarP., JiaR. and LiangP., Know What You Don’t Know: Unanswerable Questions for SQuAD, Annual Meeting of the Association for Computational Linguistics, 2018, 784-789.
16.
JoshiM., ChoiE., WeldD.S. and ZettlemoyerL., TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension, Annual Meeting of the Association for Computational Linguistics; Conference of the European Chapter of the Association for Computational Linguistics, 2017, 1601-1611.
17.
KočiskýT., SchwarzJ., BlunsomP., DyerC., HermannK.M., Melis and E. Grefenstette,G. and Grefenstette,E., The narrativeqa reading comprehension challenge, Transactions of the Association for Computational Linguistics6 (2018), 317–328.
18.
DunnM., SagunL., HigginsM., GuneyV.U., CirikV. and ChoK., Searchqa: A new q&a dataset augmented with context from a search engine, ArXiv Preprint ArXiv (2017).
19.
HermannK.M., KociskyT., GrefenstetteE., EspeholtL., KayW., SuleymanM. and BlunsomP., Teaching machines to read and comprehend, , Advances in Neural Information Processing Systems28 (2015), 1693–1701.
20.
DevlinJ., ChangM.W., LeeK. and ToutanovaK., Bert: Pretrained of Deep Bidirectional Transformers for Language Understanding, Conference on the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019, 4171-4186.
21.
PetersM.E., NeumannM., lyyerM., GardnerM., ClarkC., LeeK. and ZettlemoyerL., Deep ContextualizedWord Representations, Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018, 2227-2237.
22.
VaswaniA., ShazeerN., ParmarN., UszkoreitJ., JonesL., GomezA.N., KaiserL. and PolosukhinI., Attention is all you need, Advances in Neural Information Processing Systems (2017), 5998-6008.
YangZ., DaiZ, YangY., CarbonellJ., SalakhutdinovR.R. and Le,Q.V., XLNet: Generalized autoregressive pretraining for language understanding, , Advances in Neural Information Processing Systems32 (2019).
ClarkK., LuongM.T., LeQ.V. and ManningC.D., Electra: Pre-training Text Encoders As Discriminators Rather Than Generators, The 2020 Conference on Empirical Methods in Natural Language Processing, 2020, 1562-1578.
27.
JiaoX., YinY., ShangL., JiangX., ChenX., LiL., WangF. and LiuQ., TinyBERT: Distilling BERT for Natural Language Understanding, The 2020 Conference on Empirical Methods in Natural Language Processing, 2020, 2563-2576.
28.
LiP., LiW., HeZ, WangX., CaoY., ZhouJ. and XuW., Dataset and neural recurrent sequence labeling model for open-domain factoid question answering, ArXiv Preprint ArXiv (2016).
29.
HuM., PengY., HuangZ. and LiD., A Multi-type Multispan Network for Reading Comprehension That Requires Discrete Reasoning, International Joint Conference on Natural Language Processing; Conference on Empirical Methods in Natural Language Processing, 2019, 1596-1606.
