Distant supervision is a widely applied approach in field of relation extraction, which could automatically generate large amounts of labeled training corpus with minimal manual effort. However, the labeled training corpus may have many false positive instances, which would hurt the performance of relation extraction. Moreover, in traditional distant supervised approaches, extraction models adopt human-design features with complicated natural language processing (NLP) preprocessing. It may cause poor performance either. To address these two shortcomings, in this work, we propose a novel Long Short Term Memory (LSTM) network integrated with multi-instance learning. Our approach is supposed to learn and extract features automatically from the data itself and treats distant supervision as a multi-instance learning problem to settle the problem of false positive instances. Experimental results demonstrate that our proposed approach is effective and achieve better performance than traditional methods.
ParikhA.P.CohenS.B. and XingE.P., Spectral unsupervised parsing with additive tree metrics, In: Proceedings of ACL, 2014, pages 1062–1072.
2.
Dos SantosC.N.XiangB. and ZhouB., Classifying relations by ranking with convolutional neural networks, In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics, 2015.
3.
WilsonD. and MartinezT., The general inefficiency of batch training for gradient descent learning, Neural Networks16(10) (2003), 1429–1451.
4.
ZelenkoD.AoneC. and RichardellaA., Kernel methods for relation extraction, J. Mach. Learn. Res.3 (2003), 1083–1106.
5.
ZengD.LiuK.LaiS. et al., Relation Classification via Convolutional Deep Neural Network, In: COLING, 2014, pages 2335–2344.
6.
ZengD.LiuK.ChenY. et al., Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks, In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2015, pp. 17–21.
7.
HuangF.AhujaA.DowneyD.YangY. et al., Learning representations for weakly supervised natural language processing tasks, Journal of Computational Linguistics40(1) (2014), 85–120.
8.
SuchanekF.IfrimG. and WeikumG., Combining linguistic and statistical analysis to extract relations from web documents, In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2006, pp. 712–717.
9.
SuchanekF.FanJ.HoffmannR. et al., Advances in automated knowledge base construction, SIGMOD Records Journal, March 2013.
10.
WuF. and WeldD., Open information extraction using Wikipedia, In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, 2010.
11.
HintonG. et al., Improving Neural Networks by Preventing Co-adaptation of Feature Detectors, Computer Science3(4) (2012), 212–223.
12.
ZhouG.D.ZhangM.JiD.H. et al., Tree kernel-based relation extraction with context-sensitive structured parse tree information, In: Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, 2007.
13.
PalangiH.DengL.ShenY. et al., Deep sentence embedding using the long short term memory network: Analysis and application to information retrieval, IEEE/ACM Transactions on Audio, Speech, and Language Processing24(4) (2016), 694–707.
14.
EbrahimiJ. and DouD.J., Chain based RNN for relation classification, In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2015, pp. 1244–1249.
15.
PenningtonJ.SocherR. and ManningC.D., Glove: Global Vectors for Word Representation, In: EMNLP14 (2014), 1532–1543.
16.
FundelK.KüffnerR. and ZimmerR., RelEx – Relation extraction using dependency parse trees, Bioinformatics23(3) (2007), 365–371.
17.
HashimotoK.MiwaM.TsuruokaY. et al., Simple Customization of Recursive Neural Networks for Semantic Relation Classification, In: EMNLP, 2013, 1372–1376.
18.
XuK.FengY.HuangS. et al., Semantic relation classification via convolutional neural networks with simple negative sampling, Computer Science71(7) (2015), 941–949.
19.
QianL.H.ZhouG.D.KongF. et al., Exploiting constituent dependencies for tree kernel-based semantic relation extraction, In: Proceedings of COLING, 2008, pp. 697–704.
20.
LyyerM. et al., A neural network for factoid question answering over paragraphs, In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014, pp. 633–644.
21.
MintzM.BillsS.SnowR. et al., Distant supervision for relation extraction without labeled data, In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume 2. Association for Computational Linguistics, 2009, pp. 1003–1011.
22.
MiwaM. and BansalM., End-to-end Relation Extraction using LSTMs on Sequences and Tree Structures[J], arXiv, 1601.00770, 2016.
23.
SurdeanuM.TibshiraniJ.NallapatiR. et al., Multi-instance multi-label learning for relation extraction, In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Association for Computational Linguistics, 2012, pp. 455–465.
24.
ZeilerM., Adadelta: An adaptive learning rate method, Computer Science, 2012.
25.
ZhangM.L. and ZhouZ.H., Adapting RBF neural networks to multi-instance learning, Neural Processing Letters23(1) (2006), 1–26.
26.
KambhatlaN., Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations, In: Proceedings of the ACL 2004 on Interactive Poster and Demonstration Sessions, 2004.
27.
KalchbrennerN.GrefenstetteE. and BlunsomP., A convolutional neural network for modelling sentences, arXiv, 1404.2188, 2014.
28.
LeQ.V. and MikolovT., Distributed representations of sentences and documents, In: ICML14 (2014), 1188–1196.
29.
BunescuR. and MooneyR., A shortest path dependency kernel for relation extraction, In: Proceedings of HLT/EMNLP, 2005, pp. 724–731.
30.
BunescuR. and MooneyR., Subsequence kernels for relation extraction, In: Proceedings of NIPS18 (2006), 171–178.
31.
CollobertR.WestonJ.BottouL. et al., Natural language processing (almost) from scratch, The Journal of Machine Learning Research12 (2011), 2493–2537.
32.
GrishmanR., Information Extraction: Capabilities and Challenges, Oxford University Press, August 2011.
33.
HochreiterS., The vanishing gradient problem during learning recurrent neural nets and problem solutions, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems6(2) (1998), 107–116.
34.
HoffmannR.ZhangC.LingX. et al., Knowledge-based weak supervision for information extraction of overlapping relations, In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Association for Computational Linguistics, 2011, pp. 541–550.
35.
McDonaldR. and NivreJ., Characterizing the Errors of Data-Driven Dependency Parsing Models, In: EMNLP-CoNLL, 2007, 122–131.
36.
RiedelS.YaoL. and McCallumA., Modeling relations and their mentions without labeled text, Machine Learning and Knowledge Discovery in Databases, Springer Berlin Heidelberg, 2010, 148–163.
37.
TakamatsuS.SatoI. and NakagawaH., Reducing wrong labels in distant supervision for relation extraction, In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1. Association for Computational Linguistics, 2012, pp. 721–729.
38.
SocherR.PenningtonJ.HuangE.H. et al., Semi-supervised recursive autoencoders for predicting sentiment distributions, In: Proceedings of the Conference on Empirical Methods in Natural Language Association for Computational Linguistics, 2011, pp. 151–161.
39.
DietterichT.LathropR.H. and Lozano-PérezT., Solving the multiple instance problem with axis-parallel rectangles, Artificial Intelligence89(1) (1997), 31–71.
40.
MikolovT.ChenK.CorradoG. et al., Efficient estimation of word representations in vector space, Computer Science, 2013.
41.
WongW.LiuW. and BennamounM., Ontology learning from text: A look back and into the future, ACM Computing Surveys (CSUR)44(4) (2012), 20.
42.
XuW.ZhaoR.L. and GrishmanR., Filling Knowledge Base Gaps for Distant Supervision of Relation Extraction, In: Proceedings of Association for Computational Linguistics, 2013.
43.
ZarembaW. and SutskeverI., Learning to execute, arXiv, 1410.4615, 2014.
44.
XuY.MouL.LiG. et al., Classifying relations via long short term memory networks along shortest dependency paths, In: Proceedings of Conference on Empirical Methods in Natural Language Processing, 2015.
45.
KimY., Convolutional neural networks for sentence classification, arXiv, 1408.5882, 2014.
