Prototype Network for Text Entity Relationship Recognition in Metallurgical Field Based on Integrated Multi-class Loss Functions

Abstract

It is of great significance to recognize the metallurgical entity relations in order to construct the Knowledge graph of Metallurgical Literature and to further understand the metallurgical literature. However, there are few researches on the textual entity relations in metallurgical fields either few marked Corpora. The syntactic structure of the same entity relationship category is relatively simple and has strong domain characteristics. The traditional entity relationship model can not identify the domain entity relationship well. Meanwhile the syntactic structure of the same entity relations class is relatively simple, and the syntactic structure is relatively simple in the recognition of entity relations in metallurgy field. Furthermore, the entities with similar syntactic structure often have the same entity relations and the different words in the sentence have different contribution to the entity relations. In order to solve the mentioned problems, this paper will combine the algorithm that can highlight the syntactic structure in sentences and improve the accuracy of the model with the Algorithm that can highlight the contribution of words in sentences and the loss function level integration is carried out in the framework of small sample prototype network, so as to maximize the advantages of each algorithm and improve the accuracy –firstly, in the coding layer of the prototype network, we use the CNN algorithm which can highlight the important words in the sentences and the TreeLSTM algorithm which can parse the sentences in the text so that the syntactic relations between the words in the sentences can be acted on in the relation recognition, the sentences are coded together by two algorithms, then, the EUCLIDEAN distance loss is calculated by using this high quality coding and the prototype coding, finally, the traditional entity relation recognition model with Attention Mechanism is integrated into the loss function, further highlighting the decisive role of important words in text sentences in relation recognition and improving the generalization of the model. The results showed that compared with the traditional methods such as CNN, RNN, PCNN and Bi-LSTM, the proposed method in this paper has better performance in the case of small sample data set.

Keywords

Syntactic analysis integration learning prototype network entity relationship recognition

Get full access to this article

View all access options for this article.

References

Zeng

, Liu

, Lai

, Zhou

and Zhao

, Relation classification via convolutional deep neural network, In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, (2014), 2335–2344.

Zhang

and Wang

, Relation classification via recurrent neural network, Computer Science (2015), 16–20.

Gormley

M.R.

, Yu

and Dredze

, Improved relation extractionwith feature-rich compositional embedding models, In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages (2015), 1774–1784.

Zhou

, Shi

, Tian

, Qi

, Li

, Hao

and Xu

, Attention-based bidirectional long shortterm memory networks for relation classification, In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), (2016), 207–212.

Zeng

, Liu

, Chen

and Zhao

, Distant supervision for relation extraction via piecewise convolutional neural networks, In Proceedings of the 2015 conference on empirical methods in natural language processing, (2015), 1753–1762.

Jiang

, Wang

, Li

and Wang

, Relation extraction with multi-instance multi-label convolutional neural networks, In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers (2016), 1471–1480.

, Liu

, He

, Zhao

, et al., Distant supervision for relation extraction with sentence-level attention and entity descriptions, In, AAAI 3060 (2017).

Feng

, Guo

, Qin

, Liu

and Liu

, Effective deep memory networks for distant supervised relation extraction, In IJCAI, (2017), 4002–4008.

Snell

, Swersky

and Zemel

, Prototypical networks for few-shot learning. In Advances in neural information processing systems (2017), 4077–4087.

10.

Gao

, Han

, Liu

and Sun

, Hybrid attention-based prototypical networks for noisy few-shot relation classification, In, Proceedings of the AAAI Conference on Artificial Intelligence 33 (2019), 6407–6414.

11.

Gao

, Han

, Xie

, Liu

, Lin

and Sun

, Neural snowball for few-shot relation learning, In AAAI (2020), 7772–7779.

12.

dos Santos

C.N.

, Xiang

and Zhou

, Classifying relations by ranking with convolutional neural networks, In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), (2015), 626–634.

13.

Nguyen

T.H.

and Grishman

, Relation extraction: Perspective from convolutional neural networks, In Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, (2015), 39–48.

14.

Jin

, Yang

, Qiu

and Zhang

, Relation of the relations: A new paradigm of the relation extraction problem, In The 2020 Conference on Empirical Methods in Natural Language Processing(EMNLP 2020), (2020).

15.

Liu

, Zhang

, Zhou

and Jia

, Neural relation extraction via inner-sentence noise reduction and transfer learning, In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2195–2204, Brussels, Belgium, October-November 2018. Association for Computational Linguistics.

16.

, Mou

, Li

, Chen

, Peng

and Jin

, Classifying relations via long short term memory networks along shortest dependency paths, In Proceedings of the 2015 conference on empirical methods in natural language processing, (2015), 1785–1794.

17.

Liu

, Wei

, Li

, Ji

, Zhou

and Wang

, A dependency-based neural network for relation classification, (2015), 285–290.

18.

Mintz

, Bills

, Snow

and Jurafsky

, Distant supervision for relation extraction without labeled data, In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, (2009), 1003–1011.

19.

Dietterich

T.G.

, Lathrop

R.H.

and Lozano-Perez

, Solving the multiple instance problem with axis-parallel rectangles, Artificial Intelligence 89(1-2) (1997), 31–71.

20.

Hoffmann

, Zhang

, Ling

, Zettlemoyer

and Weld

D.S.

, Knowledge-based weak supervision for information extraction of overlapping relations, In Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, (2011), 541–550.

21.

Surdeanu

, Tibshirani

, Nallapati

and Manning

C.D.

, Multi-instance multi-label learning for relation extraction, In Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, (2012), 455–465.

22.

Huang

Y.Y.

and Wang

W.Y.

, Deep residual learning for weakly-supervised relation extraction, In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (2017), 1803–1807.

23.

Verga

, Belanger

, Strubell

, Roth

and McCallum

, Multilingual relation extraction using compositional universal schema, In The 2016 Conference of the North American Chapter of the Association for Computational Linguistics, (2015), 886–896.

24.

Feng

, Huang

, Zhao

, Yang

and Zhu

, Reinforcement learning for relation classification from noisy data, In The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18), (2018), 5779–5786.

25.

Zeng

, He

, Liu

and Zhao

, Large scaled relation extraction with reinforcement learning, In AAAI, (2018), 5658–5665.

26.

Lake

B.M.

, Salakhutdinov

and Tenenbaum

J.B.

, Human-level concept learning through probabilistic program induction, Science 350(6266) (2015), 1332–1338.

27.

Vinyals

, Blundell

, Lillicrap

, Wierstra

, et al., Matching networks for one shot learning, In Advances in neural information processing systems (2016), 3630–3638.

28.

Guo

, Ma

, Li

and Dong

, Atrm: Attention-based task-level relation module for gnn-based fewshot learning, arXiv preprint arXiv:2101.09840, (2021).

29.

Xiao

, Jin

and Hao

, Adaptive prototypical networks with label words and joint representation learning for few-shot relation classification, arXiv preprint arXiv:2101.03526, (2021).

30.

Yan

, Yantuan

, Zhengtao

, Yonghua

, Hongbin

and Yafei

, Mean prototypical networks for text classification, In The 18th Chinese National Conference on Computational Linguistics (CCL 2019), (2019).

31.

and Zuidema

, Compositional distributional semantics with long short term memory, In Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics, (2015), 10–19.

32.

Tai

K.S.

, Socher

and Manning

C.D.

, Improved semantic representations from tree-structured long short-term memory networks, In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (2015), 1556–1566.

33.

Zhu

, Sobihani

and Guo

, Long shortterm memory over recursive structures, In International Conference on Machine Learning (2015), 1604–1612.