The article presents an approach to recognise formula entailment, which concerns finding entailment relationships between pairs of math formulae. As the current formula-similarity-detection approaches fail to account for broader relationships between pairs of math formulae, recognising formula entailment becomes paramount. To this end, a long short-term memory (LSTM) neural network using symbol-by-symbol attention for recognising formula entailment is implemented. However, owing to the unavailability of relevant training and validation corpora, the first and foremost step is to create a sufficiently large-sized symbol-level MATHENTAIL data set in an automated fashion. Depending on the extent of similarity between the corresponding symbol embeddings, the symbol pairs in the MATHENTAIL data set are assigned ‘entailment’ or ‘neutral’ labels. An improved symbol-to-vector (isymbol2vec) method generates mathematical symbols (in LATEX) and their embeddings using the Wikipedia corpus of scientific documents and Continuous Bag of Words (CBOW) architecture. Eventually, the LSTM network, trained and validated using the MATHENTAIL data set, predicts formulae entailment for test formulae pairs with a reasonable accuracy of 62.2%.
KumarPPAgarwalABhagvatiC. A structure based approach for mathematical expression retrieval. In: Proceedings of the international workshop on multidisciplinary trends in artificial intelligence, Ho Chi Minh City, Vietnam, 26–28 December 2012, pp. 23–34. Berlin: Springer.
7.
KamaliSTompaFW. Retrieving documents with mathematical content. In: Proceedings of the 36th international ACM SIGIR conference on research and development in information retrieval, Dublin, 28 July–1 August 2013, pp. 353–362. New York: ACM.
YasunagaMLaffertyJD. TopicEq: a joint topic and mathematical equation model for scientific texts. In: Proceedings of the AAAI conference on artificial intelligence, 2019, vol. 33, pp. 7394–7401, https://ojs.aaai.org/index.php/AAAI/article/view/4728
10.
MansouriBRohatgiSOardDW, et al. Tangent-CFT: an embedding model for mathematical formulas. In: Proceedings of the 2019 ACM SIGIR international conference on theory of information retrieval, 2019, pp. 11–18, https://www.cs.rit.edu/~rlaz/files/Mansouri_ICTIR2019.pdf
11.
Greiner-PetteAYoussefARuasT, et al. Math-word embedding in math search and semantic extraction. Scientometrics2020; 125(3): 3017–3046.
12.
GaoLJiangZYinY, et al. Preliminary exploration of formula embedding for mathematical information retrieval: can mathematical formulae be embedded like a natural language?https://arxiv.org/abs/1707.05154
LeQMikolovT. Distributed representations of sentences and documents. In: Proceedings of the international conference on machine learning, Beijing, China, 2014, pp. 1188–1196, https://cs.stanford.edu/~quocle/paragraph_vector.pdf
EvansRSaxtonDAmosD, et al. Can neural networks understand logical entailment? In: Proceedings of the sixth international conference on learning representations (ICLR), Vancouver, BC, Canada, 2018, pp. 1–15, https://openreview.net/pdf?id=SkZxCk-0Z
18.
KhotTSabharwalAClarkP. SciTaiL: a textual entailment dataset from science question answering. In: Proceedings of the AAAI conference on artificial intelligence, 2018, pp. 1–9, https://ojs.aaai.org/index.php/AAAI/article/view/12022
19.
YinWRothDSchützeH. End-task oriented textual entailment via deep explorations of inter-sentence interactions. In: Proceedings of the 56th annual meeting of the association for computational linguistics, Melbourne, VIC, Australia, 2018, vol. 2, pp. 540–545, https://aclanthology.org/P18-2086/
AllamanisMChanthirasegaranPKohliP, et al. Learning continuous semantic representations of symbolic expressions, https://arxiv.org/abs/1611.01423
22.
MeuschkeNSchubotzMHamborgF, et al. Analyzing mathematical content to detect academic plagiarism. In: Proceedings of the 2017 ACM conference on information and knowledge management, Singapore, 2017, pp. 2211–2214, https://d-nb.info/1163536059/34
PathakAPakrayPSarkarS, et al. MathIRs: retrieval system for scientific documents. Comput Sist2017; 21(2): 253–265.
25.
KristiantoGYTopićGAizawaA. Utilizing dependency relationships between math expressions in math IR. Inform Retriev J2017; 20(2): 132–167.
26.
SchubotzMGrigorevALeichM, et al. Semantification of identifiers in mathematics for better math information retrieval. In: Proceedings of the 39th international ACM SIGIR conference on research and development in information retrieval, Pisa, 2016, pp. 135–144, https://d-nb.info/1163536121/34
27.
SchubotzMKrämerLMeuschkeN, et al. Evaluating and improving the extraction of mathematical identifier definitions. In: Proceedings of the international conference of the cross-language evaluation forum for European languages, Dublin, 2017, pp. 82–94, https://d-nb.info/1163534730/34
28.
ElizarovAKirillovichALipachevE, et al. Semantic formula search in digital mathematical libraries. In: Proceedings of the second Russia and Pacific conference on computer technology and applications, Vladivostok, 25–29 September 2017, pp. 39–43. New York: IEEE.
29.
WangYGaoLWangS, et al. WikiMirs 3.0: a hybrid MIR system based on the context, structure and importance of formulae in a document. In: Proceedings of the 15th ACM/IEEE-CS joint conference on digital libraries, Knoxville, TN, 21–25 June 2015, pp. 173–182. New York: ACM.
30.
YuanK. Multi-dimensional formula feature modeling for mathematical information retrieval. In: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, Tokyo, Japan, 7–11 August 2017, p. 1381. New York: ACM.
MunavalliRMinerR. MathFind: a math-aware search engine. In: Proceedings of the 29th annual international ACM SIGIR conference on research and development in information retrieval, Seattle, WA, 6–11 August 2006, p. 735. New York: ACM.
ChiccoDJurmanG. The advantages of the Matthews correlation 609 coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genom2020; 21: 6.
