DELE: Deductive E L + + Embeddings for Knowledge Base Completion

Abstract

Ontology embeddings map classes, roles, and individuals in ontologies into ${\mathbb{R}}^n$ , and within ${\mathbb{R}}^n$ similarity between entities can be computed or new axioms inferred. For ontologies in the Description Logic ${\mathcal{E}\mathcal{L}}^{++}$ , several optimization-based embedding methods have been developed that explicitly generate models of an ontology. However, these methods suffer from some limitations; they do not distinguish between statements that are unprovable and provably false, and therefore they may use entailed statements as negatives. Furthermore, they do not utilize the deductive closure of an ontology to identify statements that are inferred but not asserted. We evaluated a set of embedding methods for ${\mathcal{E}\mathcal{L}}^{++}$ ontologies, incorporating several modifications that aim to make use of the ontology deductive closure. In particular, we designed novel negative losses that account both for the deductive closure and different types of negatives and formulated evaluation methods for knowledge base completion. We demonstrate that our embedding methods improve over the baseline ontology embedding in the task of knowledge base or ontology completion.

1. Introduction

Several methods have been developed to embed Description Logic theories or ontologies in vector spaces (Chen et al., 2020a, 2021; Jackermeier et al., 2024; Kulmanov et al., 2019; Mondal et al., 2021; Özcep et al., 2023; Peng et al., 2022; Xiong et al., 2022). These embedding methods preserve some aspects of the semantics in the vector space, and may enable the computation of semantic similarity, inferring axioms that are entailed, and predicting axioms that are not entailed but may be added to the theory. For the lightweight Description Logic ${\mathcal{E}\mathcal{L}}^{++}$ , several geometric embedding methods have been developed (Jackermeier et al., 2024; Kulmanov et al., 2019; Mondal et al., 2021; Özcep et al., 2023; Xiong et al., 2022). They can be proven to “faithfully” approximate a model in the sense that, if a certain optimization objective is reached (usually, a loss function reduced to $0$ ), the embedding method has constructed a model of the ${\mathcal{E}\mathcal{L}}^{++}$ theory. Geometric model construction enables the execution of various tasks. These tasks include knowledge base completion and subsumption prediction via either testing the truth of a statement under consideration in a single (approximate) model or aggregating truth values over multiple models.

Advances on different geometric embedding methods have usually focused on the expressiveness of the embedding methods; originally, hyperballs (Kulmanov et al., 2019) where used to represent the interpretation of concept symbols, yet hyperballs are not closed under intersection. Therefore, axis-aligned boxes were introduced (Jackermeier et al., 2024; Peng et al., 2022; Xiong et al., 2022). Furthermore, ${\mathcal{E}\mathcal{L}}^{++}$ allows for axioms pertaining to roles, and several methods have extended the way in which roles are modeled (Jackermeier et al., 2024; Kulmanov et al., 2019; Xiong et al., 2022). However, there are several aspects of geometric embeddings that have not yet been investigated. In particular, for ${\mathcal{E}\mathcal{L}}^{++}$ , there are sound and complete reasoners with efficient implementations that scale to very large knowledge bases (Kazakov et al., 2013); it may therefore be possible to utilize a deductive reasoner together with the embedding process to improve generation of embeddings that represent geometric models.

We evaluate geometric embedding methods and incorporate deductive inference into the training process. We use the ELEmbeddings (Kulmanov et al., 2019), ELBE (Peng et al., 2022), and ${\mathrm{Box}}^{2}EL$ (Jackermeier et al., 2024) models for our experiments; however, our results also apply to other geometric embedding methods for ${\mathcal{E}\mathcal{L}}^{++}$ .

Our main contributions are as follows:

–

We propose loss functions that incorporate negative samples in all normal forms and account for deductive closure during training.

–

We introduce a fast approximate algorithm for computing the deductive closure of an ${\mathcal{E}\mathcal{L}}^{++}$ theory and use it to improve negative sampling during model training.

–

We formulate evaluation methods for knowledge base completion that account for the deductive closure during evaluation.

This is an extended version of our previous work (Mashkova et al., 2024). Here, we include a more comprehensive treatment of computing the deductive closure and using the deductive closure with ${\mathcal{E}\mathcal{L}}^{++}$ embedding methods (e.g., for model evaluation). We include additional experiments on three benchmark datasets: besides protein function prediction and protein–protein interaction (PPI), we study the subsumption prediction task on Food Ontology (Dooley et al., 2018) and GALEN ontology (Rector et al., 1996). Furthermore, we consider two additional geometric ontology embedding models, ELBE (Peng et al., 2022) and ${\mathrm{Box}}^{2}EL$ (Jackermeier et al., 2024), apart from ELEmbeddings (Kulmanov et al., 2019) for which we also extend our methods. We make our code and data available at https://github.com/bio-ontology-research-group/DELE.

2. Preliminaries

2.1. Description Logic ${\mathcal{E}\mathcal{L}}^{++}$

Let $\mathrm{\Sigma }=(\mathbf{C},\mathbf{R},\mathbf{I})$ be a signature with set $\mathbf{C}$ of concept names, $\mathbf{R}$ of role names, and $\mathbf{I}$ of individual names. Given $r\in \mathbf{R}$ , and $a,b\in \mathbf{I}$ , ${\mathcal{E}\mathcal{L}}^{++}$ concept descriptions are constructed with the grammar $\mathrm{\perp }\mid \mathrm{\top }\mid C\sqcap D\mid \mathrm{\exists }r.C\mid \{a\}$ where $C,D$ are ${\mathcal{E}\mathcal{L}}^{++}$ concept descriptions and $r$ is a role name. ABox axioms are of the form $C(a)$ and $r(a,b)$ , TBox axioms are of the form $C⊑D$ , and RBox axioms are of the form $r_1\circ r_2\circ \cdots \circ r_n⊑r$ . ${\mathcal{E}\mathcal{L}}^{++}$ Generalized concept inclusions (GCIs) and role inclusions (RIs) can be normalized to follow one of the forms listed in Table 1 (Baader et al., 2005). If an ontology contains a non-empty ABox, then, first, concept assertion axioms of the form $C(a)$ are processed: for each complex concept description $C$ , a new concept name $A$ is introduced and axiom $C\equiv A$ is added to the TBox (Peñaloza & Turhan, 2008). Normalization rules for TBox axioms (Baader et al., 2005) can be found in Table 2. They preserve the same models for ontologies (Mendez, 2011) and the normalized ontology is a conservative extension of the original ontology (Suntisrivaraporn, 2009). Following previous works (Jackermeier et al., 2024; Kulmanov et al., 2019; Peng et al., 2022) we develop our methodology specifically for normalized ${\mathcal{E}\mathcal{L}}^{++}$ ontologies. The advantage of normalizing ${\mathcal{E}\mathcal{L}}^{++}$ ontologies is that their deductive closure is finite whereas the deductive closure of non-normalized ${\mathcal{E}\mathcal{L}}^{++}$ ontologies is not. Therefore, in our work, when we refer to the language ${\mathcal{E}\mathcal{L}}^{++}$ , we always intend the normalized ${\mathcal{E}\mathcal{L}}^{++}$ , that is, where all valid axiom types are those listed in Table 1.

Table 1.

Normalized Forms of ${\mathcal{E}\mathcal{L}}^{++}$ Generalized Concept Inclusions (GCIs) and Role Inclusions (RIs).

Acronym	Axiom Type
GCI0	$A⊑B$
GCI1	$A\sqcap B⊑E$
GCI2	$A⊑\mathrm{\exists }r.B$
GCI3	$\mathrm{\exists }r.A⊑B$
GCI0-BOT	$A⊑\mathrm{\perp }$
GCI1-BOT	$A\sqcap B⊑\mathrm{\perp }$
GCI3-BOT	$\mathrm{\exists }r.A⊑\mathrm{\perp }$
RI0	$r⊑s$
RI1	$r_1\circ r_2⊑s$

Here, $r,r_1,r_2,s$ are role names, $A,B,E$ are concept names.

Table 2.

Normalization Rules for TBox for ${\mathcal{E}\mathcal{L}}^{++}$ Based on Baader et al. (2005).

Input	Output
$r_1\circ \cdots \circ r_k⊑s$	$r_1\circ \cdots \circ r_{k-1}⊑u,u\circ r_k⊑s$
$C\sqcap \hat{D}⊑E$	$\hat{D}⊑A,C\sqcap A⊑E$
$\mathrm{\exists }r.\hat{C}⊑D$	$\hat{C}⊑A,\mathrm{\exists }r.A⊑D$
$\mathrm{\perp }⊑D$	$\mathrm{\varnothing }$
$\hat{C}⊑\hat{D}$	$\hat{C}⊑A,A⊑\hat{D}$
$B⊑\mathrm{\exists }r.\hat{C}$	$B⊑\mathrm{\exists }r.A,A⊑\hat{C}$
$B⊑C\sqcap D$	$B⊑C,B⊑D$

Here, $r,r_1,\dots ,r_k,s$ are role names, $C,D,B$ are arbitrary concept descriptions, $\hat{C},\hat{D}$ are complex concept descriptions (i.e., not concept names), $A$ is a fresh concept name and $u$ is a fresh role name.

To define the semantics of an ${\mathcal{E}\mathcal{L}}^{++}$ theory, we use (Baader et al., 2005) an interpretation domain ${\mathrm{\Delta }}^{\mathcal{I}}$ and an interpretation function ${\cdot }^{\mathcal{I}}$ . For every concept $A\in \mathbf{C}$ $A^{\mathcal{I}}\subseteq {\mathrm{\Delta }}^{\mathcal{I}}$ ; individual $a\in \mathbf{I}$ , $a^{\mathcal{I}}\in {\mathrm{\Delta }}^{\mathcal{I}}$ ; role $r\in \mathbf{R}$ , $r^{\mathcal{I}}\subseteq {\mathrm{\Delta }}^{\mathcal{I}}\times {\mathrm{\Delta }}^{\mathcal{I}}$ . Furthermore, the semantics for other ${\mathcal{E}\mathcal{L}}^{++}$ constructs are the following (omitting concrete domains and role inclusions):

$$\begin{array}{rl}{\mathrm{\perp }}^{\mathcal{I}}& =\mathrm{\varnothing }\\ {\mathrm{\top }}^{\mathcal{I}}& ={\mathrm{\Delta }}^{\mathcal{I}},\\ (C\sqcap D{)}^{\mathcal{I}}& =C^{\mathcal{I}}\cap D^{\mathcal{I}},\\ (\mathrm{\exists }r.C{)}^{\mathcal{I}}& =\{x\in {\mathrm{\Delta }}^{\mathcal{I}}\mid \mathrm{\exists }y\in {\mathrm{\Delta }}^{\mathcal{I}}:((x,y)\in r^{\mathcal{I}}\wedge y\in C^{\mathcal{I}})\},\\ {\{a\}}^{\mathcal{I}}& =\{a^{\mathcal{I}}\}\end{array}$$

An interpretation $\mathcal{I}$ is a model for an axiom $C⊑D$ if and only if $C^{\mathcal{I}}\subseteq D^{\mathcal{I}}$ , for an axiom $C(a)$ if and only if $a^{\mathcal{I}}\in C^{\mathcal{I}}$ ; and for an axiom $r(a,b)$ if and only if $(a^{\mathcal{I}},b^{\mathcal{I}})\in r^{\mathcal{I}}$ (Baader, 2003). The relation of semantic entailment, $\models$ , is defined as a relation between a theory $\mathcal{T}$ and axiom $\varphi$ : $\mathcal{T}\models \varphi$ iff every model of $\mathcal{T}$ is also a model of $\varphi$ ( $Mod(\mathcal{T})\subseteq Mod(\{\varphi \})$ ) (Tarski, 1936).

2.2. Knowledge Base Completion

The task of knowledge base completion is the addition (or prediction) of axioms which hold yet are not represented in the knowledge base. We call the task “ontology completion” when exclusively TBox axioms are predicted. The task of knowledge base completion may encompass both deductive (Jiang et al., 2012; Sato et al., 2018) and inductive (Bouraoui et al., 2017; d’Amato et al., 2012) inference processes and give rise to two subtly different tasks: adding only “novel” axioms to a knowledge base that are not in the deductive closure of the knowledge base, and adding axioms that are in the deductive closure as well as some “novel” axioms that are not deductively inferred; both tasks are related but differ in how they are evaluated.

Inductive inference, analogously to knowledge graph completion (Chen et al., 2020b), predicts axioms based on patterns and regularities within the knowledge base. Knowledge base completion, or ontology completion, can be further distinguished based on the information that is used to predict “novel” axioms. We distinguish between two approaches to knowledge base completion: (1) knowledge base completion which relies solely on (formalized) information within the knowledge base to predict new axioms, and (2) knowledge base completion which incorporates side information, such as text, to enhance the prediction of new axioms. Here, we mainly consider the first case.

2.3. Deductive Closure

The deductive closure of a theory $T$ refers to the smallest set containing all statements which can be inferred by deductive reasoning over $T$ ; for a given deductive relation $⊢$ , we call $T^{⊢}=\{\varphi |T⊢\varphi \}$ the deductive closure of $T$ . In knowledge bases, the deductive closure is usually not identical to the asserted axioms in the knowledge base; it is also usually infinite. Representing the deductive closure is challenging since it is infinite, but, in ${\mathcal{E}\mathcal{L}}^{++}$ , any knowledge base can be normalized to one of the seven normal forms; therefore, we can compute the deductive closure with respect to these normal forms, and this set will be finite (as long as the sets of concept and role names are finite). For example, all entailed axioms of type $A⊑B$ will be a subset of the set of all possible axioms of GCI0 type having cardinality $|\mathbf{C}{|}^2$ where $|\mathbf{C}|$ is the cardinality of the set of all concept names. Similarly, the cardinality of GCI0-BOT deductive closure will be limited by $|\mathbf{C}|$ , GCI1 deductive closure cardinality will be limited by $|\mathbf{C}{|}^3$ , and GCI1-BOT deductive closure cardinality by $|\mathbf{C}{|}^2$ since one of the concepts is fixed. GCI2 and GCI3 deductive closures cardinality will depend on the total number of roles $|\mathbf{R}|$ and will be limited by $|\mathbf{C}{|}^2\cdot |\mathbf{R}|$ , and, finally, the number of entailed axioms of GCI3-BOT type will not exceed $|\mathbf{C}|\cdot |\mathbf{R}|$ .

3. Related Work

3.1. Graph-Based Ontology Embeddings

Graph-based ontology embeddings rely on a construction (projection) of graphs from ontology axioms mapping ontology classes, individuals and roles to nodes and labeled edges (Zhapa-Camacho & Hoehndorf, 2023). Embeddings for nodes and edge labels are optimized following two strategies: by generating random walks and using a sequence learning method such as Word2Vec (Mikolov et al., 2013) or by using Knowledge Graph Embedding (KGE) methods (Wang et al., 2017). These type of methods have been shown effective on knowledge base and ontology completion (Chen et al., 2021) and have been applied to domain-specific tasks such as PPI prediction (Chen et al., 2021) or gene–disease association prediction (Althagafi et al., 2024; Chen et al., 2020a). Graph-based methods rely on adjacency information of the ontology structure but cannot easily handle logical operators and do not approximate ontology models. Therefore, graph-based methods are not “faithful,” that is, do not approximate models, do not allow determining whether statements are “true” in these models, and therefore cannot be used to perform semantic entailment.

3.2. Geometric-Based Ontology Embeddings

Multiple methods have been developed for the geometric construction of models for the ${\mathcal{E}\mathcal{L}}^{++}$ language. ELEmbeddings (Kulmanov et al., 2019) constructs an interpretation of concept names as sets of points lying within an open $n$ -dimensional ball and generates an interpretation of role names as the set of pairs of points that are separated by a vector in ${\mathbb{R}}^n$ , that is, by the embedding of the role name. EmEL++ (Mondal et al., 2021) extends ELEmbeddings with more expressive constructs such as role chains and role inclusions. Another extension of ELEmbeddings, EmELvar (Mohapatra et al., 2021), introduces role embeddings which enables handling many-to-many roles and provides a perspective to extending the method to more expressive Description Logics. ELBE (Peng et al., 2022) and BoxEL (Xiong et al., 2022) use $n$ -dimensional axis-aligned boxes to represent concepts, which has an advantage over balls because the intersection of two axis-aligned boxes is a box whereas the intersection of two $n$ -balls is not an $n$ -ball. BoxEL additionally preserves ABox facilitating a more accurate representation of knowledge base’s logical structure by ensuring, for example, that an entity has the minimal volume. Box ${}^2$ EL (Jackermeier et al., 2024) represents ontology roles more expressively with two boxes encoding the semantics of the domain and codomain of roles. Box ${}^2$ EL enables the expression of one-to-many roles as opposed to other methods. The box-based method TransBox (Yang et al., 2025) aims to effectively capture all ${\mathcal{E}\mathcal{L}}^{++}$ logical operations allowing for precise representation of any arbitrarily complex concept description. The fact that any normalized ${\mathcal{E}\mathcal{L}}^{++}$ theory has a finite deductive closure allows the definition of a canonical model where all and only entailed axioms are true. Lacerda et al. (2024a, 2024b) developed FaithEL, a strongly TBox-faithful method based on convex sets in $n$ -dimensional real-valued space which interprets concepts and roles as subsets of unitary hypercubes. FaithEL constructs a “canonical” model of an ontology and is able to predict new assertions consistent with a TBox. Axis-aligned cone-shaped geometric model introduced in Özcep et al. (2023) deals with $\mathcal{A}\mathcal{L}\mathcal{C}$ ontologies and allows for full negation of concepts and existential quantification by construction of convex sets in ${\mathbb{R}}^n$ . This work has not yet been implemented or evaluated in an application.

3.3. Knowledge Base Completion Task

Several recent advancements in the knowledge base completion rely on side information as included in large language models (LLMs). Ji et al. (2023) explores how pretrained language models can be utilized for incorporating one ontology into another, with the main focus on inconsistency handling and ontology coherence. HalTon (Cao et al., 2023) addresses the task of event ontology completion via simultaneous event clustering, hierarchy expansion and type naming utilizing BERT (Devlin et al., 2019) for instance encoding. Li et al. (2024) formulates knowledge base completion task as a Natural Language Inference (NLI) problem and examines how this approach may be combined with concept embeddings for identifying missing knowledge in ontologies. As for other approaches, Mežnar et al. (2022) proposes a method that converts an ontology into a graph to recommend missing edges using structure-only link analysis methods, Shiraishi and Kaneiwa (2024) constructs matrix-based ontology embeddings which capture the global and local information for subsumption prediction. All these methods use side information from LLMs and would not be applicable, for example, in the case where a knowledge base is private or consists of only identifiers; we do not consider methods based on pre-trained LLMs here as baselines.

4. Negative Sampling and Objective Functions

Currently available geometric ontology embedding models which construct a model of an ontology by optimizing some objective function usually sample negative examples during training phase (Jackermeier et al., 2024; Kulmanov et al., 2019; Mohapatra et al., 2021; Mondal et al., 2021; Peng et al., 2022; Yang et al., 2025). This operation prevents overgeneralization of learned embeddings and trivial satisfiability in case a model collapses (Kulmanov et al., 2019; Yang et al., 2025) by incorporating additional constraints within a model. Ontology embedding methods select negatives by replacing one of the concepts with a randomly chosen one (either from the set of all concept names, or a subset thereof). ELEmbeddings, ELBE and ${\mathrm{Box}}^{2}EL$ use a single loss for “negatives,” that is, axioms that are not included in the knowledge base; the loss is used only for axioms of the form $A⊑\mathrm{\exists }r.B$ (GCI2) which are randomly sampled; negatives are not sampled for other normal forms. Correspondingly, the embedding methods were primarily evaluated on predicting GCI2 axioms ( ${\mathrm{Box}}^{2}EL$ was also evaluated on subsumption prediction); this evaluation procedure might have introduced biases towards axioms of type GCI2, and influenced the ability of geometric models to predict axioms of other types. Specifically, the lack of negative examples of other axiom types may lead to geometric models in which many axioms are true even if they are not entailed, leading to a decreased ability to find axioms that can be added to a theory in the task of knowledge base completion. Consequently, we also sample negatives for other normal forms and add “negative” losses (i.e., losses for the sampled “negatives”) for all other normal forms.

4.1. ELEmbeddings Negative Losses

For negative loss construction in ELEmbeddings, we employ notations from the ELEmbeddings method where $r_{\eta }(a),r_{\eta }(b),r_{\eta }(e)$ and $f_{\eta }(a),f_{\eta }(b),f_{\eta }(e)$ denote the radius and the ball center associated with classes $a,b,e$ , respectively; and $f_{\eta }(r_0)$ denotes the embedding vector associated with role $r$ . $\gamma$ stands for a margin parameter, and $\epsilon$ is a small positive number. There is a geometric part as well as a regularization part for each new negative loss forcing class centers to lie on a unit ${\ell }_2-$ sphere.

For ELEmbeddings, as reflected in Eq. (1), we use the original GCI1-BOT loss for disjoint classes; although non-containment of ball corresponding to $A$ within the ball corresponding to B is not equivalent to their disjointness, the loss aims to minimize the classes’ overlap for better optimization:

$$\begin{array}{rl}{\mathrm{loss}}_{A⋢B}(a,b)=& max(0,r_{\eta }(a)+r_{\eta }(b)-\Vert f_{\eta }(a)-f_{\eta }(b))\Vert +\gamma )+\\ & +|\Vert f_{\eta }(a)\Vert -1|+|\Vert f_{\eta }(b)\Vert -1|\end{array}$$

(1)

The same logic applies for the negative loss in Eq. (2) where we minimize overlap between the translated ball corresponding to class $A$ and the ball representing $B$ :

$$\begin{array}{rl}{\mathrm{loss}}_{\mathrm{\exists }r.A⋢B}(r_0,a,b)=& max(0,r_{\eta }(a)+r_{\eta }(b)-\Vert f_{\eta }(a)-f_{\eta }(r_0)-f_{\eta }(b))\Vert +\gamma )+\\ & +|\Vert f_{\eta }(a)\Vert -1|+|\Vert f_{\eta }(b)\Vert -1|\end{array}$$

(2)

Negative loss (3) is constructed similarly to the $A\sqcap B⊑E$ loss: the first part penalizes non-overlap of the classes $A$ and $B$ (we do not consider the disjointness case since, for every class $X$ , we have $\mathrm{\perp }⊑X$ ); furthermore, for negative sampling of axioms of this type, we vary only the $E$ part of GCI1 axioms from the ontology, so the intersection of $A$ and $B$ is non-empty by assumption:

$$\begin{array}{rl}{\mathrm{loss}}_{A\sqcap B⋢E}(a,b,e)=& max(0,-r_{\eta }(a)-r_{\eta }(b)+\Vert f_{\eta }(a)-f_{\eta }(b))\Vert -\gamma )+\\ & +max(0,r_{\eta }(a)-\Vert f_{\eta }(a)-f_{\eta }(e))\Vert +\gamma )+\\ & +max(0,r_{\eta }(b)-\Vert f_{\eta }(b)-f_{\eta }(e))\Vert +\gamma )+\\ & +|\Vert f_{\eta }(a)\Vert -1|+|\Vert f_{\eta }(b)\Vert -1|+|\Vert f_{\eta }(e)\Vert -1|\end{array}$$

(3)

The second and the third part force the center corresponding to $E$ not to lie in the intersection of balls associated with $A$ and $B$ . Here we do not consider constraints on the radius of the ball for the $E$ class and focus only on the relative positions of the $A,B$ and $E$ class centers and the overlapping of $n$ -balls representing $A$ and $B$ . Since the first part of the loss encourages classes to have a non-empty intersection, we use it as a negative loss for GCI1-BOT axioms:

$$\begin{array}{rl}{\mathrm{loss}}_{A\sqcap B⋢\mathrm{\perp }}(a,b)=& max(0,-r_{\eta }(a)-r_{\eta }(b)+\Vert f_{\eta }(a)-f_{\eta }(b))\Vert -\gamma )+\\ & +|\Vert f_{\eta }(a)\Vert -1|+|\Vert f_{\eta }(b)\Vert -1|\end{array}$$

(4)

In the original method losses for axioms of type GCI0-BOT and GCI3-BOT force radii of unsatisfiable classes to become $0$ . For the correspondent negative losses (see Eqs. (5) and (6)) we use the interpretation for satisfiable classes as balls with nonzero radius, that is, with a radius which equals to or greater than some small positive number $\epsilon$ :

$$\begin{array}{rl}{\mathrm{loss}}_{A⋢\mathrm{\perp }}(a)& =max(0,\epsilon -r_{\eta }(a))\end{array}$$

(5)

$$\begin{array}{rl}{\mathrm{loss}}_{\mathrm{\exists }r.A⋢\mathrm{\perp }}(r_0,a)& =max(0,\epsilon -r_{\eta }(a))\end{array}$$

(6)

4.2. ELBE Negative Losses

ELBE is a model that relies on boxes instead of balls. Here, similarly, $\epsilon$ is a small positive number, $e_c(a),e_c(b)$ and $e_o(a),e_o(b)$ denote the box center and the box offset associated with classes $a,b$ , respectively, $e_c(r_0)$ denotes the embedding vector associated with role $r$ , and $e_c(\mathrm{new}),e_o(\mathrm{new})$ correspond to the center and the offset of the box which is the result of intersection of boxes associated with concepts $a$ and $b$ , $\mathrm{margin}$ stands for a margin parameter.

Following the same method of negative loss construction for ELEmbeddings, we use GCI1-BOT loss as a negative loss for $A⊑B$ axioms (see Eq. (7)):

$$\begin{array}{r}{\mathrm{loss}}_{A⋢B}(a,b)=\Vert max(\mathrm{zeros},-|e_c(a)-e_c(b)|+e_o(a)+e_o(b)+\mathrm{margin})\Vert \end{array}$$

(7)

Since axis-aligned hyperrectangles are closed under intersection, we also use GCI1-BOT for the intersection of boxes representing $A$ and $B$ concepts and the $E$ box (see Eq. (8)):

$$\begin{array}{r}{\mathrm{loss}}_{A\sqcap B⋢E}(a,b,e)=\Vert max(\mathrm{zeros},-|e_c(\mathrm{new})-e_c(e)|+e_o(\mathrm{new})+e_o(e)++\mathrm{margin})\Vert \end{array}$$

(8)

This property also allows us to interpret each negative sample for $A\sqcap B⊑\mathrm{\perp }$ axioms as a box intersection with nonzero offset (see Eq. (9)):

$$\begin{array}{r}{\mathrm{loss}}_{A\sqcap B⋢\mathrm{\perp }}(a,b)=max(0,\epsilon -\Vert e_o(\mathrm{new})\Vert )\end{array}$$

(9)

Other negative losses have the form similar to the ones constructed for ELEmbeddings:

$$\begin{array}{rl}{\mathrm{loss}}_{\mathrm{\exists }r.A⋢B}(r_0,a,b)& =\Vert max(\mathrm{zeros},-|e_c(a)-e_c(r_0)-e_c(b))|+e_o(a)+e_o(b)+\mathrm{margin})\Vert \end{array}$$

(10)

$$\begin{array}{rl}{\mathrm{loss}}_{A⋢\mathrm{\perp }}(a)& =max(0,\epsilon -\Vert e_o(a)\Vert )\end{array}$$

(11)

$$\begin{array}{rl}{\mathrm{loss}}_{\mathrm{\exists }r.A⋢\mathrm{\perp }}(r_0,a)& =max(0,\epsilon -\Vert e_o(a)\Vert )\end{array}$$

(12)

4.3. ${\mathrm{Box}}^{2}EL$ Negative Losses

${\mathrm{Box}}^{2}EL$ is also based on boxes but uses a different role modeling compared to ELBE. Additionally making use of the notations from ${\mathrm{Box}}^{2}EL$ (Jackermeier et al., 2024), $\epsilon$ is a small positive number, $\mathrm{Box}(A),\mathrm{Box}(B)$ , $\mathrm{Box}(E)$ are boxes associated with classes $a,b,e$ , respectively, $\gamma$ denotes a margin parameter, $\delta$ is a parameter from the GCI2 negative loss, $Head(r)$ represents the head box of role $r$ interpretation, and $Bump(A)$ corresponds to a bump vector associated with concept $A$ .

Equations (13) and (14) are constructed in a similar fashion as for ELBE based on the GCI1-BOT loss which penalizes the element-wise distance $\mathit{d}$ between axis-aligned boxes:

$$\begin{array}{rl}{\mathrm{loss}}_{A⋢B}(a,b)& =\Vert max(\mathbf{0},-(\mathit{d}(\mathrm{Box}(A),\mathrm{Box}(B))+\gamma ))\Vert \end{array}$$

(13)

$$\begin{array}{rl}{\mathrm{loss}}_{A\sqcap B⋢E}(a,b,e)& =\Vert max(\mathbf{0},-(\mathit{d}(\mathrm{Box}(A)\cap \mathrm{Box}(B),\mathrm{Box}(E))+\gamma ))\Vert \end{array}$$

(14)

Negative losses 15–17 encourage boxes to be non-empty:

$$\begin{array}{rl}{\mathrm{loss}}_{A⋢\mathrm{\perp }}(a)& =max(0,\epsilon -\Vert o(A)\Vert )\end{array}$$

(15)

$$\begin{array}{rl}{\mathrm{loss}}_{A\sqcap B⋢\mathrm{\perp }}(a,b)& =max(0,\epsilon -\Vert o(\mathrm{Box}(A)\cap \mathrm{Box}(B))\Vert )\end{array}$$

(16)

$$\begin{array}{rl}{\mathrm{loss}}_{\mathrm{\exists }r.A⋢\mathrm{\perp }}(r_0,a)& =max(0,\epsilon -\Vert o(A)\Vert )\end{array}$$

(17)

The GCI3 negative loss reflects the structure of the original GCI3 loss:

$$\begin{array}{r}{\mathrm{loss}}_{\mathrm{\exists }r.A⋢B}(r_0,a,b)=(\delta -\mu (\mathrm{Head}(r)-\mathrm{Bump}(A),\mathrm{Box}(B)){)}^2\end{array}$$

(18)

5. Negative Sampling and Entailments

In the case of knowledge base completion where the deductive closure contains potentially many non-trivial entailed axioms, the random sampling approach for negatives may lead to suboptimal learning since some of the axioms treated as negatives may be entailed (and should therefore be true in any model, in particular the one constructed by the geometric embedding method). As an example, let us consider the simple ontology consisting of two axioms: $A\sqcap B⊑E$ and $F⊑B$ . For the $A\sqcap B⊑E$ axiom, random negative sampling will sample $A\sqcap B⊑E^{\prime }$ where $E^{\prime }$ is one of $A,B,E,F$ . Since the knowledge base makes the axioms $A\sqcap B⊑A$ , $A\sqcap B⊑B$ , and $A\sqcap B⊑E$ true, in 75% of cases we will sample a negative for this axiom that is actually true in each model.

We suggest to filter selected negatives during training based on the deductive closure of the knowledge base: for each randomly generated axiom to be used as negative, we check whether it is present in the deductive closure and, if it is, we delete it. ${\mathcal{E}\mathcal{L}}^{++}$ reasoners such as ELK (Kazakov et al., 2013) compute subsumption hierarchies, that is, all axioms of the form $A⊑B$ in the deductive closure, but not entailed axioms for the other normal forms. We use the inferences computed by ELK (of the form $A⊑B$ where $A$ and $B$ are concept names) to design an algorithm that computes (a part of) the deductive closure with respect to the ${\mathcal{E}\mathcal{L}}^{++}$ normal forms. The algorithm implements a sound but incomplete set of inference rules which can quickly generate a partial deductive closure with respect to all normal forms. Algorithm 1 contains inference rules for deriving entailed axioms of type GCI1, GCI2, GCI3, GCI1-BOT and GCI3-BOT from axioms explicitly represented within a knowledge base; GCI0 and GCI0-BOT axioms are precomputed by ELK. Algorithm 2 provides a set of additional rules depending on arbitrary classes and roles represented within a knowledge base after inferred axioms from Algorithm 1 are computed. The purpose of Algorithm 2 is to enrich the approximate deductive closure with axioms involving arbitrary roles and concepts or with axioms of new GCI type which may be missed by applying rules from Algorithm 1 since Algorithm 1 computes entailed axioms based on ontology axioms, concept hierarchy and role inclusions. For example, consider an ontology consisting of two axioms: $A⊑E$ and $B⊑E$ . Since no GCI1 axioms are present in the ontology, no GCI1 axioms will be entailed by Algorithm 1. Algorithm 2 enables inference of a GCI1 axiom $A\sqcap B⊑E$ . Another example is an ontology comprised of axioms $A⊑B$ and $B\sqcap E⊑F$ . The axiom $A\sqcap \mathrm{\perp }⊑B$ , which is entailed, cannot be inferred by applying rules from Algorithm 1: the concept $A$ is not a subclass of neither $B$ or $E$ . Although we can use ELK or similar reasoners to query for arbitrary entailed axioms, the algorithms we propose have an advantage over this method since it does not require the addition of a new concept to an ontology and recomputing the concept hierarchy. We show a detailed example of the algorithm in Appendix E based on the simple ontology example introduced in Section 6.4.

In the task of knowledge base completion with many non-trivial entailed axioms, the deductive closure can also be used to modify the evaluation metrics, or define novel evaluation metrics that distinguish between entailed and non-entailed axioms. So far, ontology embedding methods that have been applied to the task of knowledge base completion have used evaluation measures that are taken from the task of knowledge graph completion; in particular, they only evaluate knowledge base completion using axioms that are “novel” and not entailed. However, any entailed axiom will be true in all models of the knowledge base, and therefore also in the geometric model that is constructed by the embedding method.

We suggest to filter entailed axioms from training or test sets when the aim is to predict “novel” (i.e., non-entailed) knowledge. The geometric embedding methods generate models making all entailed axioms true in all models. It is expected that methods explicitly constructing models preferentially make entailed axioms true and rank them higher than non-entailed axioms. If the evaluation is based solely on non-entailed axioms, it will consider all similar inferred axioms false, and to avoid this, we may filter such axioms from the ranking list. The more axioms are filtered, the more entailed axioms are predicted by a model.

6. Experiments

6.1. Datasets

6.1.1. Gene Ontology & STRING Data

Following previous works (Jackermeier et al., 2024; Kulmanov et al., 2019; Peng et al., 2022) we use common benchmarks for knowledge-base completion, in particular a task that predicts PPIs based on the functions of proteins. We also use the same data for the task of protein function prediction. For these tasks we use two datasets, each of them consists of the Gene Ontology (GO) (Consortium, 2015) with all its axioms, PPIs and protein function axioms extracted from the STRING database (Mering, 2003); each dataset focuses on only yeast proteins. GO is formalized using OWL 2 EL (Golbreich & Horrocks, 2007).

For the PPI yeast network we use the built-in dataset PPIYeastDataset available in the mOWL (Zhapa-Camacho et al., 2022) Python library (release 0.2.1) where axioms of interest are split randomly into train, validation and test datasets in ratio 90:5:5 keeping pairs of symmetric PPI axioms within the same dataset, and other axioms are placed into the training part; validation and test sets are made up of TBox axioms of type $\{P_1\}⊑\mathrm{\exists }interacts\mathrm{\_}with.\{P_2\}$ where $P_1,P_2$ are protein names. The GO version released on 2021-10-20 and the STRING database version 11.5 were used. Alongside with the yeast $interacts\mathrm{\_}with$ dataset we collected the yeast $has\mathrm{\_}function$ dataset organized in the same manner with validation and test parts containing TBox axioms of type $\{P\}⊑\mathrm{\exists }has\mathrm{\_}function.\{GO\}$ . Based on the information in the STRING database, in PPI yeast, the interacts_with role is symmetric and the dataset is closed against symmetric interactions. We normalize each train ontology using the updated implementation of the jcel (Mendez, 2012) reasoner ¹ where we take into consideration newly generated concept and role names. Although role inclusion axioms may be utilized within the ${\mathrm{Box}}^{2}EL$ framework we ignore them since neither ELEmbeddings nor ELBE incorporate these types of axioms. Table in the appendix A shows the number of GCIs of each type in the datasets and the number of concepts and roles after normalization. For more precise evaluation of novel knowledge prediction we remove entailed axioms from the test set for function prediction task based on the precomputed deductive closure of the train ontology (see Section 5).

6.1.2. Food Ontology & GALEN Ontology

Food Ontology (Dooley et al., 2018) contains structured information about foods formalized in $\mathcal{S}\mathcal{R}\mathcal{I}\mathcal{Q}$ DL expressivity (Chen et al., 2021) involving terms from UBERON (Mungall et al., 2012), NCBITaxon (Federhen, 2012), Plant Ontology (Jaiswal et al., 2005), and others. The GALEN ontology (Rector et al., 1996) represents biomedical concepts related to anatomy, diseases, and others (Rector & Rogers, 2004). For the Food Ontology, the data for subsumption prediction was extracted from the case studies used to evaluate OWL2Vec* (Chen et al., 2021) ² ; the train part of the ontology was restricted to the $\mathcal{E}\mathcal{L}$ fragment and normalized using the jcel (Mendez, 2012) reasoner. In case of GALEN ontology, subsumption axioms were randomly split in ratio 90:5:5 among train, validation and test sets. Since the normalization procedure splits each complex axiom into a set of shorter axioms including subsumptions between atomic concepts from the signature, it may result in adding axioms represented in the validation or test part of the ontology to the train part. To avoid this, we filtered out such axioms from the original validation and test datasets after the train ontology for subsumption prediction was normalized. Additionally, as described in Section 6.1.1, we remove entailed axioms from the test dataset. Statistics about the number of axioms of each GCI type, roles and classes can be found in Appendix B for the Food Ontology and in Appendix C for the GALEN ontology.

6.2. Evaluation Scores and Metrics

For GO & STRING data, we predict GCI2 axioms of type $\{P_1\}⊑\mathrm{\exists }interacts\mathrm{\_}with.\{P_2\}$ or $\{P\}⊑\mathrm{\exists }has\mathrm{\_}function.\{GO\}$ depending on the dataset. On Food Ontology and GALEN ontology, we predict GCI0 axioms of type $A⊑B$ , $A$ and $B$ are arbitrary classes from the signature. For each axiom type, we use the corresponding loss expressions to score axioms. This is justified by the fact that objective functions are measures of truth for each axiom within constructed models.

The predictive performance is measured by the Hits@n metrics for $n=10,100$ , macro and micro mean rank, and the area under the ROC curve (AUC ROC). For rank-based metrics, we calculate the score of $A⊑\mathrm{\exists }r.B$ or $A⊑B$ for every class $A$ from the test set and for every $B$ from the set $\mathbf{C}$ of all classes (or subclasses of a certain type, such as proteins or functions for domain-specific cases) and determine the rank of a test axiom $A⊑\mathrm{\exists }r.B$ . For macro mean rank and AUC ROC, we consider all axioms from the test set; for micro metrics, we compute corresponding class-specific metrics averaging them over all classes in the signature:

$$\begin{array}{rl}micro\mathrm{\_}M{R}_{A⊑\mathrm{\exists }r.B}& =\mathrm{Mean}(M{R}_A(\{A⊑\mathrm{\exists }r.B,B\in \mathbf{C}\}))\end{array}$$

(19)

$$\begin{array}{rl}micro\mathrm{\_}M{R}_{A⊑B}& =\mathrm{Mean}(M{R}_A(\{A⊑B,B\in \mathbf{C}\}))\end{array}$$

(20)

$$\begin{array}{rl}micro\mathrm{\_}AUC\mathrm{\_}RO{C}_{A⊑\mathrm{\exists }r.B}& =\mathrm{Mean}(AUC\mathrm{\_}RO{C}_A(\{A⊑\mathrm{\exists }r.B,B\in \mathbf{C}\}))\end{array}$$

(21)

$$\begin{array}{rl}micro\mathrm{\_}AUC\mathrm{\_}RO{C}_{A⊑B}& =\mathrm{Mean}(AUC\mathrm{\_}RO{C}_A(\{A⊑B,B\in \mathbf{C}\}))\end{array}$$

(22)

Additionally, we remove axioms represented in the train set or deductive closures (see Section 5) to obtain corresponding filtered metrics (FHits@n, FMR, FAUC). In related work focusing on knowledge graph completion or knowledge base completion tasks (Bordes et al., 2013; Kulmanov et al., 2019; Peng et al., 2022; Wang et al., 2014), filtered metrics are computed by removing axioms presented within the train set from the list of all ranked axioms. This filtration is applied to eliminate statements learnt by a model during training phase which are therefore likely to have lower rank and to evaluate the predictive performance of a model in a more fair setting.

6.3. Training Procedure

All models are optimized with respect to the sum of individual GCI losses (here we define the loss in most general case using all positive and all negative losses):

$$\begin{array}{rl}\mathcal{L}=& l_{A⊑B}+l_{A\sqcap B⊑E}+l_{A⊑\mathrm{\exists }r.B}+l_{\mathrm{\exists }r.A⊑B}+l_{A⊑\mathrm{\perp }}+l_{A\sqcap B⊑\mathrm{\perp }}+l_{\mathrm{\exists }r.A⊑\mathrm{\perp }}+\\ & +l_{A⋢B}+l_{A\sqcap B⋢E}+l_{A⋢\mathrm{\exists }r.B}+l_{\mathrm{\exists }r.A⋢B}+l_{A⋢\mathrm{\perp }}+l_{A\sqcap B⋢\mathrm{\perp }}+l_{\mathrm{\exists }r.A⋢\mathrm{\perp }}\end{array}$$

(23)

All model architectures are built using the mOWL (Zhapa-Camacho et al., 2022) library on top of mOWL’s base models. All models were trained using the same fixed random seed.

All models are trained for 2,000 epochs for STRING & GO datasets and 800 epochs for the Food Ontology and GALEN datasets with batch size of 32,768. Training and optimization is performed using Pytorch with Adam optimizer (Kingma & Ba, 2015) and ReduceLROnPlateau scheduler with patience parameter $10$ . We apply early stopping if validation loss does not improve for $20$ epochs. For ELEmbeddings, hyperparameters are tuned using grid search over the following set: margin $\gamma \in \{-0.1,-0.01,0,0.01,0.1\}$ , embedding dimension $\{50,100,200,400\}$ , learning rate $\{0.01,0.001,0.0001\}$ ; since none of our datasets contains unsatisfiable classes, we do not tune the parameter $\epsilon$ appearing in GCI0-BOT and GCI3-BOT negative losses. For ELBE, grid search is performed over 60 randomly chosen subsets of the following hyperparameters: embedding dimension $\{25,50,100,200\}$ , margin $\{-0.1,-0.01,0,0.01,0.1\}$ , $\epsilon \in \{0.1,0.01,0.001\}$ (for experiments with all negative losses involved), learning rate $\{0.01,0.001,0.0001\}$ . The same strategy is applied to ${\mathrm{Box}}^{2}EL$ models for embedding dimension $\{25,50,100,200\}$ , margin $\gamma \in \{-0.1,-0.01,0,0.01,0.1\}$ , $\delta \in \{1,2,4\}$ , $\epsilon \in \{0.1,0.01,0.001\}$ (similarly, for experiments with all negative losses involved), regularization factor $\lambda \in \{0,0.05,0.1,0.2\}$ , and learning rate $\{0.01,0.001,0.0001\}$ . For experiments with negatives filtration during training we use the same set of hyperparameters for random and filtered mode of negative sampling. See Appendix D for details on optimal hyperparameters used.

6.4. Results

We evaluate whether adding negative losses for all normal forms will allow for the construction of a better model and improve the performance in the task of knowledge base completion. We test the effect of the expanded negative sampling and negative losses first on a small ontology that can be embedded and visualized in 2D space, and then on a larger application. We formulate and add negative losses for all normal forms given by equations (1)–(17).

First, we investigate a simple example corresponding to the task of protein function prediction using the ELEmbeddings model. Let us consider an ontology consisting of two axioms stating that there are two disjoint functions $\{G{O}_1\}$ and $\{G{O}_2\}$ , and proteins having these functions are also disjoint: $\{G{O}_1\}\sqcap \{G{O}_2\}⊑\mathrm{\perp }$ , $\mathrm{\exists }has\mathrm{\_}function.\{G{O}_1\}\sqcap \mathrm{\exists }has\mathrm{\_}function.\{G{O}_2\}⊑\mathrm{\perp }$ . After normalization, the last axiom is substituted by the following three axioms: $A\sqcap B⊑\mathrm{\perp }$ , $\mathrm{\exists }has\mathrm{\_}function.\{G{O}_1\}⊑B$ , $\mathrm{\exists }has\mathrm{\_}function.\{G{O}_2\}⊑A$ where $A,B$ are new concept names. To visualize the results, we embed these axioms in 2D space. Figure 1(a) shows the embedding generated with the original ELEmbeddings model. Since there are no axioms of type GCI2 represented within the knowledge base, the model learns without any negative examples and demonstrates poor performance compared to the model with incorporated negative losses for all normal forms as demonstrated in Figure 1(b).

Figure 1.

ELEmbeddings example. Dashed circles represent translated classes by role vector corresponding to $has\mathrm{\_}function$ role. The normalized theory $\{\{G{O}_1\}\sqcap \{G{O}_2\}⊑\mathrm{\perp },$ $A\sqcap B⊑\mathrm{\perp },\mathrm{\exists }has\mathrm{\_}function.\{G{O}_1\}⊑B,\mathrm{\exists }has\mathrm{\_}function.\{G{O}_2\}⊑A\}$ is better preserved when negative losses are incorporated to all normal forms (Figure b) rather than only to GCI2 normal form (Figure a). (a) GCI2 negative loss; (b) All negative losses.

Since we are interested in predicting not only axioms of type $A⊑\mathrm{\exists }r.B$ for which negative sampling is used in the original ELEmbeddings, ELBE and ${\mathrm{Box}}^{2}EL$ , we also examine the effect of all negative losses utilization during training on Food Ontology and GALEN ontology for subsumption prediction (see Tables 3 and 4, respectively). We find that the ELEmbeddings model does not improve on the Food Ontology subsumption prediction task, but ELBE with additional losses improves over the original model; ${\mathrm{Box}}^{2}EL$ with additional losses surpasses its version with just GCI2 negative loss in Hits@n metrics. As for the results on GALEN ontology, we find that in case of all three models Hits@n metrics are improved when all negative losses are applied (except Hits@100 metric for ELEmbeddings model) indicating that in this particular case negative losses encourage models to predict more new axioms. Mean rank results are similarly better for ELEmbeddings and ${\mathrm{Box}}^{2}EL$ models.

Table 3.

Subsumption Prediction Experiments on Food Ontology.

Model	H@10		H@100		macro_MR			micro_MR			macro_AUC		micro_AUC
	NF	F	NF	F	NF	F	NF-F	NF	F	NF-F	NF	F	NF	F
ELEm	0.12	0.12	0.21	0.21	4659	4656	3	4662	4659	3	0.84	0.84	0.84	0.84
ELEm+l	0.10	0.11	0.19	0.19	5015	5013	2	5020	5017	3	0.83	0.83	0.83	0.83
ELEm+l+n	0.10	0.11	0.19	0.19	5022	5019	3	5027	5024	3	0.83	0.83	0.83	0.83
ELBE	0.01	0.01	0.09	0.09	6695	6692	3	6688	6686	2	0.77	0.77	0.77	0.77
ELBE+l	0.04	0.04	0.14	0.14	5428	5426	2	5412	5409	3	0.81	0.81	0.82	0.82
ELBE+l+n	0.04	0.04	0.14	0.14	5427	5424	3	5410	5408	2	0.81	0.81	0.82	0.82
${\mathrm{Box}}^{2}EL$	0.01	0.01	0.10	0.10	3900	3898	2	3877	3874	3	0.87	0.87	0.87	0.87
${\mathrm{Box}}^{2}EL$ +l	0.04	0.04	0.13	0.13	7550	7547	3	7555	7553	2	0.74	0.74	0.74	0.74
${\mathrm{Box}}^{2}EL$ +l+n	0.05	0.05	0.14	0.14	6865	6862	3	6869	6866	3	0.76	0.76	0.77	0.77

‘‘l’’ corresponds to all negative losses, ‘‘l+n’’ means a model was trained using all negative losses and negatives filtering. For each model we report non-filtered metrics (NF) and filtered metrics with respect to the deductive closure of the train and the test set combined together (F). For macro_MR and micro_MR we additionally report the difference between filtered and non-filtered metrics (NF-F) to check how much of entailed knowledge is predicted on average. Values in bold indicate best metrics.

Table 4.

Subsumption Prediction Experiments on GALEN Ontology.

	H@10		H@100		macro_MR			micro_MR			macro_AUC		micro_AUC
Model	NF	F	NF	F	NF	F	NF-F	NF	F	NF-F	NF	F	NF	F
ELEm	0.15	0.16	0.35	0.35	9106	9105	1	9106	9105	1	0.82	0.82	0.82	0.82
ELEm+l	0.21	0.22	0.34	0.34	8977	8976	1	8977	8976	1	0.82	0.82	0.82	0.82
ELEm+l+n	0.21	0.22	0.33	0.34	9005	9003	2	9005	9003	2	0.82	0.82	0.82	0.82
ELBE	0.08	0.08	0.22	0.22	11236	11234	2	11236	11234	2	0.77	0.78	0.77	0.77
ELBE+l	0.13	0.13	0.34	0.34	11884	11882	2	11884	11882	2	0.76	0.76	0.76	0.76
ELBE+l+n	0.12	0.12	0.34	0.34	11720	11717	3	11720	11717	3	0.77	0.77	0.76	0.76
${\mathrm{Box}}^{2}EL$	0.14	0.14	0.31	0.31	11724	11721	3	11724	11721	3	0.77	0.77	0.76	0.76
${\mathrm{Box}}^{2}EL$ +l	0.16	0.16	0.37	0.37	11371	11369	2	11371	11369	2	0.77	0.77	0.77	0.77
${\mathrm{Box}}^{2}EL$ +l+n	0.16	0.16	0.37	0.37	11378	11376	2	11378	11376	2	0.77	0.77	0.77	0.77

‘‘l’’ corresponds to all negative losses, ‘‘l+n’’ means a model was trained using all negative losses and negatives filtering. For each model we report non-filtered metrics (NF) and filtered metrics with respect to the deductive closure of the train and the test set combined together (F). For macro_MR and micro_MR we additionally report the difference between filtered and non-filtered metrics (NF-F) to check how much of entailed knowledge is predicted on average. Values in bold indicate best metrics.

Additionally, we evaluate the performance on a standard benchmark set for PPI prediction (see Table 5). For this task, the test axioms are of the type GCI2. We observe that ELEmbeddings and ELBE with negative losses for all normal forms integrated demonstrate superior performance compared to their initial configurations in terms of Hits@n metrics; it also allows ${\mathrm{Box}}^{2}EL$ to lower ranks of test axioms. Generally, for the task of PPI prediction, additional negative sampling improves performance.

Table 5.

Protein–Protein Interaction Prediction Experiments on Yeast Proteins.

Model	H@10	H@100	macro_MR	micro_MR	macro_AUC	micro_AUC
ELEm	0.05	0.31	599.21	701.57	0.90	0.90
ELEm+l	0.06	0.35	532.93	681.02	0.91	0.90
ELEm+l+n	0.06	0.37	519.62	671.19	0.91	0.91
ELBE	0.07	0.37	829.86	1123.47	0.91	0.89
ELBE+l	0.08	0.40	984.92	1259.54	0.84	0.82
ELBE+l+n	0.08	0.40	984.18	1281.20	0.84	0.82
${\mathrm{Box}}^{2}EL$	0.05	0.57	215.07	287.16	0.96	0.96
${\mathrm{Box}}^{2}EL$ +l	0.05	0.57	200.85	250.17	0.97	0.96
${\mathrm{Box}}^{2}EL$ +l+n	0.05	0.58	197.73	250.47	0.97	0.96

‘‘l’’ corresponds to all negative losses, ‘‘l+n’’ means a model was trained using all negative losses and negatives filtering. Non-filtered metrics are reported. Values in bold indicate best metrics.

To summarize the above mentioned observations, we note that in some cases additional negative losses may decrease the ability of models to predict new axioms and encourage models to predict entailed knowledge first (as, e.g., in protein function prediction case) thus leading to construction of a more accurate model of a theory. Since there is a tradeoff between prediction of novel and entailed knowledge, additional negative losses may demonstrate worse performance on novel knowledge prediction.

Using the example introduced above and the ELEmbeddings embedding model, we demonstrate that negatives filtration may be beneficial for constructing a model of a theory. Apart from axioms mentioned earlier, that is, $\{G{O}_1\}\sqcap \{G{O}_2\}⊑\mathrm{\perp }$ , $A\sqcap B⊑\mathrm{\perp }$ , $\mathrm{\exists }has\mathrm{\_}function.\{G{O}_1\}⊑B$ and $\mathrm{\exists }has\mathrm{\_}function.\{G{O}_2\}⊑A$ , we add 10 more axioms about 5 proteins $\{P_1\},\dots ,\{P_5\}$ having function $\{G{O}_1\}$ (i.e., $\{P_i\}⊑\mathrm{\exists }has\mathrm{\_}function.\{G{O}_1\},i=1,\dots ,5$ ), and 5 proteins $\{Q_1\},\dots ,\{Q_5\}$ having function $\{G{O}_2\}$ (i.e., $\{Q_i\}⊑\mathrm{\exists }has\mathrm{\_}function.\{G{O}_2\},i=1,\dots ,5$ ). Figure 2 shows the constructed models with and without negatives filtering. We observe that the model with filtered negatives provides faithful representation of GCI3 axiom $\mathrm{\exists }has\mathrm{\_}function.\{G{O}_2\}⊑A$ and axioms introducing proteins having function $\{G{O}_2\}$ as opposed to its counterpart with random negatives: according to geometric interpretation, for GCI3 axioms $\mathrm{\exists }r.A⊑B$ to be faithfully represented, the $n$ -ball interpreting the concept $A$ translated by $-r_0$ role $r$ vector should lie inside the $n$ -ball interpreting the concept $B$ . We see that this hold true on Figure 2(b) yet not on Figure 2(a).

Figure 2.

ELEmbeddings example. Dashed circles represent translated classes by role vector corresponding to $has\mathrm{\_}function$ role. ‘Red’ classes represent proteins $\{Q_1\},\dots ,\{Q_5\}$ , ‘green’ classes represent proteins $\{P_1\},\dots ,\{P_5\}$ . Axioms $\{Q_i\}⊑\mathrm{\exists }has\mathrm{\_}function.\{G{O}_2\},$ $i=1,\dots ,5$ are better preserved when negatives are filtered based on precomputed deductive closure (Figure b) rather than when random negatives are sampled (Figure a). The same applies for the axiom $\mathrm{\exists }has\mathrm{\_}function.\{G{O}_2\}⊑A$ . (a) With random negatives; (b) With filtered negatives.

Tables 3–5 show results in the tasks of PPI and subsumption prediction. We find that excluding axioms in the deductive closure for negative selection slightly improves or yields similar results. One possible reason is that a randomly chosen axiom is very unlikely to be entailed since very few axioms are entailed compared to all possible axioms to choose from.

Because the chance of selecting an entailed axiom as a negative depends on the knowledge base on which the embedding method is applied, we perform additional experiments on Food Ontology with ELEmbeddings model where we bias the selection of negatives; we chose between 100% negatives to 0% negatives from the entailed axioms. We find that reducing the number of entailed axioms from the negatives has an effect to improve performance and the effect increases the more axioms would be chosen from the entailed ones (see Figure 3).

Figure 3.

Metrics reported for biased fraction of random negatives combined with entailed axioms from the precomputed deductive closure. (a) H@1, H@10, H@100 and ROC AUC; (b) macro_MR.

We compute filtered metrics for the protein function and subsumption prediction tasks. Both of them account for entailed axioms prediction since if, for example, $A⊑B$ is being predicted then first models may predict axioms of type $A⊑B^{\prime }$ where $B^{\prime }$ is any superclass of $B$ ; the same is true for function prediction axioms $\{P\}⊑\mathrm{\exists }has\mathrm{\_}function.\{GO\}$ and all superclasses $\{G{O}^{\prime }\}$ of $\{GO\}$ class. Note that the PPI prediction task is not tailored for evaluation using deductive closures of the train or test set: for each protein $\{P\}$ its subclasses include only $\mathrm{\perp }$ and superclasses include only $\mathrm{\top }$ . As a result, the only inferred axioms will be of type $\mathrm{\perp }⊑\mathrm{\exists }interacts\mathrm{\_}with.\{P\}$ , $\{P_1\}⊑\mathrm{\exists }interacts\mathrm{\_}with.\{P_2\}$ or $\{P\}⊑\mathrm{\exists }interacts\mathrm{\_}with.\mathrm{\top }$ , and filtered metrics may be computed only with respect to the train part of the ontology. For this reason we do not report filtered metrics for PPI prediction task (Table 5).

For function prediction and subsumption prediction, we employ filtration of metrics based on the deductive closure of the train set and of the test set. Tables 3, 4 and 6 contain results for subsumption prediction on Food Ontology, subsumption prediction on GALEN ontology and function prediction on GO, respectively.

Table 6.

Protein Function Prediction Experiments on Yeast proteins.

	H@10		H@100		macro_MR			micro_MR			macro_AUC		micro_AUC
Model	NF	F	NF	F	NF	F	NF-F	NF	F	NF-F	NF	F	NF	F
ELEm	0.01	0.01	0.03	0.03	21198	21150	48	21165	21118	47	0.62	0.62	0.63	0.63
ELEm+l	0.00	0.00	0.03	0.03	9603	9575	28	9449	9423	26	0.83	0.83	0.84	0.84
ELEm+l+n	0.00	0.00	0.03	0.03	9488	9460	28	9334	9307	27	0.83	0.83	0.84	0.84
ELBE	0.03	0.03	0.24	0.24	4229	4209	20	4156	4137	19	0.92	0.92	0.93	0.93
ELBE+l	0.00	0.00	0.01	0.01	12920	12865	55	12797	12745	52	0.77	0.77	0.78	0.78
ELBE+l+n	0.00	0.00	0.01	0.01	12900	12845	55	12772	12719	53	0.77	0.77	0.78	0.78
${\mathrm{Box}}^{2}EL$	0.28	0.31	0.55	0.55	1988	1979	9	1988	1980	8	0.96	0.96	0.97	0.97
${\mathrm{Box}}^{2}EL$ +l	0.24	0.27	0.54	0.55	2129	2120	9	2099	2091	8	0.96	0.96	0.97	0.97
${\mathrm{Box}}^{2}EL$ +l+n	0.24	0.27	0.54	0.55	2161	2152	9	2147	2139	8	0.96	0.96	0.96	0.96

‘‘l’’ corresponds to all negative losses, ‘‘l+n’’ means a model was trained using all negative losses and negatives filtering. For each model we report non-filtered metrics (NF) and filtered metrics with respect to the deductive closure of the train and the test set combined together (F). For macro_MR and micro_MR we additionally report the difference between filtered and non-filtered metrics (NF-F) to check how much of entailed knowledge is predicted on average. Values in bold indicate best metrics.

Our findings suggest that the baseline ELEmbeddings predicts primarily entailed axioms of GCI2 type, yet for GCI0 on Food Ontology the model predicts “novel” knowledge first whereas the model modifications with additional negative losses and negatives filtration derive entailed knowledge in the first place. For the GALEN ontology, however, the situation is similar to the protein function prediction case, that is, novel knowledge is predicted in the first place for modifications with additional negative losses and negatives filtration. This may indicate model construction where many classes overlap or “collapse” for all negative losses and negatives filtering case since the GALEN ontology does not contain disjointness axioms and consequently no classes will be separated by the model. The same holds for ELBE and ${\mathrm{Box}}^{2}EL$ models. Losses for all normal forms and negatives filtering during training aid ELBE and ${\mathrm{Box}}^{2}EL$ to construct model-generated embeddings which first predict logically inferred knowledge and then non-entailed axioms of type GCI2 or GCI0 (on Food Ontology), respectively. The results indicate that models with all types of valid negatives in most cases explicitly construct models.

7. Discussion

We evaluated properties of ELEmbeddings, ELBE and ${\mathrm{Box}}^{2}EL$ , ontology embedding methods that aims to generate a model of an ${\mathcal{E}\mathcal{L}}^{++}$ theory; the properties we evaluate hold similarly for other ontology embedding methods that construct models of ${\mathcal{E}\mathcal{L}}^{++}$ theories. While we demonstrate several improvements over the original model, we can also draw some general conclusions about ontology embedding methods and their evaluation. Knowledge base completion is the task of predicting axioms that should be added to a knowledge base; this task is adapted from knowledge graph completion where triples are added to a knowledge graph. The way both tasks are evaluated is by removing some statements (axioms or triples) from the knowledge base, and evaluating whether these axioms or triples can be recovered by the embedding method. This evaluation approach is adequate for knowledge graphs which do not give rise to many entailments. However, knowledge bases give rise to potentially many non-trivial entailments that need to be considered in the evaluation. In particular, embedding methods that aim to generate a model of a knowledge base will first generate entailed axioms (because entailed axioms are true in all models); these methods perform knowledge base completion as a generalization of generating the model where either other statements may be true, or they may be approximately true in the generated structure. This has two consequences: the evaluation procedure needs to account for this; and the model needs to be sufficiently rich to allow useful predictions.

We have introduced a method to compute the deductive closure of ${\mathcal{E}\mathcal{L}}^{++}$ knowledge bases; this method relies on an automated reasoner and is sound. We use all the axioms in the deductive closure as positive axioms to be predicted when evaluating knowledge base completion, to account for methods that treat knowledge base completion as a generalization of constructing a model and testing for truth in this model. We find that some models (e.g., modified box-based models using valid negatives of all types) can predict entailed axioms well, some (e.g., the original ${\mathrm{Box}}^{2}EL$ model) preferentially predict “novel”, non-entailed axioms; these methods solve subtly different problems (either generalizing construction of a model, or specifically predicting novel non-entailed axioms). We also modify the evaluation procedure to account for the inclusion of entailed axioms as positives; however, the evaluation measures are still based on ranking individual axioms and do not account for semantic similarity. For example, if during testing, the correct axiom to predict is $A⊑\mathrm{\exists }r.B$ but the predicted axiom is $A⊑\mathrm{\exists }r.E$ , the prediction may be considered to be “more correct” if $B⊑E$ was in the knowledge base than if $B\sqcap E⊑\mathrm{\perp }$ was in the knowledge base. Novel evaluation metrics need to be designed to account for this phenomenon, similarly to ontology-based evaluation measures used in life sciences (Radivojac & Clark, 2013). It is also important to expand the set of benchmark sets for knowledge base completion.

Use of the deductive closure is useful not only in evaluation but also when selecting negatives. In formal knowledge bases, there are at least two ways in which negatives for axioms can be chosen: they are either non-entailed axioms, or they are axioms whose negation is entailed. However, in no case should entailed axioms be considered as negatives; we demonstrate that filtering entailed axioms from selected negatives during training improves the performance of the embedding method consistently in knowledge base completion (and, obviously, more so when entailed axioms are considered as positives during evaluation).

While we only report our experiments with ELEmbeddings, ELBE, and ${\mathrm{Box}}^{2}EL$ , our findings, in particular about the evaluation and use of deductive closure, are applicable to other geometric ontology embedding methods. As ontology embedding methods are increasingly applied in knowledge-enhanced learning and other tasks that utilize some form of approximate computation of entailments, our results can also serve to improve the applications of ontology embeddings.