Reason-able embeddings: Learning concept embeddings with a transferable neural reasoner

Abstract

We present a novel approach for learning embeddings of $\mathcal{ALC}$ knowledge base concepts. The embeddings reflect the semantics of the concepts in such a way that it is possible to compute an embedding of a complex concept from the embeddings of its parts by using appropriate neural constructors. Embeddings for different knowledge bases are vectors in a shared vector space, shaped in such a way that approximate subsumption checking for arbitrarily complex concepts can be done by the same neural network, called a reasoner head, for all the knowledge bases. To underline this unique property of enabling reasoning directly on embeddings, we call them reason-able embeddings. We report the results of experimental evaluation showing that the difference in reasoning performance between training a separate reasoner head for each ontology and using a shared reasoner head, is negligible.

1. Introduction

In this paper we design, implement and evaluate a novel method of learning concept embeddings in knowledge bases for the $\mathcal{ALC}$ description logic, using a transferable neural reasoner. Our method of learning embeddings ensures that the resulting embeddings are entirely data-driven, which unlike manually-designed concept vectorization schemes, captures as many useful properties of the given training data as possible. We call the resulting embeddings reason-able embeddings, as they are explicitly constructed in such a way as to maintain the formal semantics of the represented concepts and thus enable approximate reasoning with them.

The neural reasoner consists of two modules – a reasoner head, that is a neural network classifier, trained to classify whether subsumption axioms hold for a given knowledge base, and capable of constructing an embedding for arbitrarily complex $\mathcal{ALC}$ concepts, and an embedding layer responsible for mapping concept names to the vector space. The reasoner head is transferable, because the embedding layers learn how to embed concepts in a space that minimizes the classifier loss, and is shared between all knowledge bases. It must be pointed out that the reasoner head does not explicitly use terminological axioms for entailment classification. Instead, the terminological axioms indirectly shape the learned concept embeddings, so even though it may look like the reasoner head is simply memorizing answers, some form of deductive reasoning is actually performed.

In our work we hypothesize, and experimentally show support for the idea that concepts from different knowledge bases can be represented in a shared embedding space, with a topology that lends itself for approximate reasoning by entailment classifiers based on neural networks. This research is a curiosity-driven intellectual endeavor: to the best of our knowledge, this is the first attempt to force a kind of shared semantics on seemingly separate sets of embeddings, yet without defining the semantics explicitly. While we imagine that there may be practical applications of the proposed idea, such as hot-swapping ontologies in a neural-symbolic system, we do not make any claims about its immediate usefulness.

We purposefully limit ourselves to the taxonomical part of $\mathcal{ALC}$ knowledge bases, and assume the assertional part to be empty. We believe that accounting for a fundamental difference between an individual, and a set of individuals, is a separate difficulty in and of itself. At the same time, solving this difficulty is not necessary for the method to be useful, as many ontologies employ a modeling style such that there are no individuals or they are used very sparingly [27].

The main contributions of the work are as follows:

• A novel architecture consisting of two parts: a reasoner head, employing a recursive neural network, and a collection of embedding layers. Together, they enable the construction of reason-able embeddings.

• A training procedure employing multiple knowledge bases at once to shape the topology of the embedding space.

• Extensive experimental evaluation aimed at establishing the properties of the proposed approach. In the experiments, we show that the proposed approach indeed forces embeddings to represent the formal semantics in a shared way, that the resulting embeddings are of high quality, and that using this kind of shared, latent semantics does not incur substantial penalty on them.

On top of that, we present three open-source technical contributions implemented in Python that may be useful for the community:

• An implementation of the presented method

• A random axiom generator for the $\mathcal{ALC}$ description logic

• An extended interface for the FaCT++ reasoner

Our work is structured as follows. In Section 2, we briefly introduce deep learning (Section 2.1) and description logics (Section 2.2), along with the related notation that we use throughout this text. In Section 3, we give an overview of the current state of the art, in particular of knowledge base embeddings (Section 3.1), and of deep deductive reasoners (Section 3.2). Section 4 presents the details of the architecture for learning reason-able embeddings: in Section 4.1 we present the details of the reasoner head, and in Section 4.2 of the embedding layer. We then introduce a relaxed variant of the architecture in Section 4.3, and describe the training procedure in Section 4.4. The axiom generator for $\mathcal{ALC}$ is described in Section 5. We report the results of the experimental evaluation in Section 6, where we begin by formulating the research questions guiding the experiments in Section 6.1. We describe the setup in Section 6.2, the employed metrics in Section 6.3, and the used data sets in Section 6.4. Subsequently, we summarize the results of four experiments in Sections 6.5–6.8. Lastly, in Section 7 we summarize the results of our work and point out potential improvements, extensions, and applications of our neural reasoner and concept embeddings.

2. Preliminaries

2.1. Deep learning

While machine learning is ubiquitous in research these days, we give a very brief introduction to ensure there is no confusion in terms. In this, we follow [31], and the reader is welcome to the book for a more extended introduction.

Classification is a task where a learning algorithm is supposed to produce a function, called a classifier, mapping an object to a label from a discrete, finite set of labels. A special case, used in this paper, is binary classification, where only two mutually-exclusive labels are considered: 0 (negative) and 1 (positive). Objects are usually represented by vectors of numerical features, albeit different representations are possible. By convention, features are usually denoted by x, and predicted labels are denoted by $\hat{\mathit{y}}$ .

Binary classification is frequently solved by constructing a function predicting the probability of an object to belong to the positive class, and then thresholding on the probability to obtain the label. Throughout this work, we employ this approach, writing $\mathit{P}\mathrm{(}\mathit{x}\mathrm{)}$ to denote the probability that the object represented by x should be labeled with the positive label.

A classifier usually contains a number of parameters, which are automatically adjusted during the training, a process aiming to optimize the value of some metric on a training set. The training set consists of examples, each example being an object assigned a true label that the classifier is supposed to predict. By convention, the true labels are usually denoted as y. The details of the optimization process can vary wildly, but in the context of this work, as the metric we use binary cross-entropy and employ a variant of gradient descent optimization algorithm called AdamW [53] to minimize it.

For gradient descent algorithms, it is established to use a mini-batch training, where the training set is randomly split into small, disjoint subsets called mini-batches, and each mini-batch is used to update the parameters of the model separately. A complete pass over all the mini-batches from a training set is called an epoch. In some special cases, only a subset of parameters is optimized at a given time, while the remaining are called frozen and kept constant.

Due to the stochastic nature of training, too aggressively optimizing the metric can lead to overfitting, a phenomenon in which the classifier labels the training examples very well, but fares poorly on new, unseen data. To evaluate the performance of a classifier on unseen data, a separate set, called test set, is maintained. It is assumed that both sets are independent and identically distributed (the i.i.d. assumption), i.e., the examples in them are drawn independently of each other from the same probability distribution. Usually, for the evaluation, different metrics are used than during training. The details of the metrics used in this paper are given in Section 6.3.

It is frequently the case that a learning algorithm has some configuration options that cannot be automatically adjusted using the training set, e.g., because that would immediately lead to overfitting. These options are called hyperparameters, and are adjusted by observing the value of a metric of choice on yet another set, the validation set. It follows the same i.i.d. assumption as the two other sets.

A feed-forward neural network is a sequence of linear transformations and activation functions, where each transformation is followed by an activation function. By convention, each pair: a linear transformation and an activation function is called a layer. The linear transformation is of form $\mathit{A}\mathit{x}+\mathit{b}$ , where A is a matrix of the type $\mathit{n}\times \mathit{m}$ , b is a vector of the type n, and x is a vector of the type m. A and b are parameters, called weights, while x is an input. By convention, m is called the number of neurons of the layer. Throughout the paper, we denote a single layer with ${\mathit{L}}_{\mathit{m}}^{\mathit{f}}$ , where m is the number of neurons and f is the activation function: ${\mathit{L}}_{\mathit{m}}^{\mathit{f}}\mathrm{(}\mathit{x}\mathrm{)}=\mathit{f}\mathrm{(}\mathit{A}\mathit{x}+\mathit{b}\mathrm{)}$ . In a special case where there is no activation function in a layer, we replace f with ·. In this convention, a feed-forward neural network of two layers, the first consisting of 16 neurons and employing the activation function σ and the second of 1 neuron and employing no activation function, can be represented as: ${\mathit{L}}_1^{·}\mathrm{(}{\mathit{L}}_{16}^{\mathit{\sigma }}\mathrm{(}\mathit{x}\mathrm{)}\mathrm{)}$ . Traditionally the last layer is called the output layer, and the remaining layers – hidden layers.

Over the years, many activation functions were considered. Throughout the paper we use two of them: the sigmoid σ, given in Equation (1), and $\mathrm{ELU}$ in Equation (2) [15]. $$\begin{array}{}\text{(1)}& \mathit{\sigma }\mathrm{(}\mathit{x}\mathrm{)}={\displaystyle \frac{1}{1+{\mathit{e}}^{-\mathit{x}}}}\text{(2)}& \mathrm{ELU}\mathrm{(}\mathit{x}\mathrm{)}=\left\{\begin{array}{ll}{\mathit{e}}^{\mathit{x}}-1& \mathit{x}\mathrm{<}0\\ \mathit{x}& \mathit{x}⩾0\end{array}\right.\end{array}$$

Finally, by an embedding we understand a real-valued vector representing some concept from a non-numeric set, such that the size of the vector is much smaller than the cardinality of the set, and that the dimensions are not directly interpretable, i.e., the vector comes from a latent space.

2.2. Description logics

Description logics (DLs) are a family of logics, that are widely used as a formal way to represent knowledge in the form of knowledge bases (KBs). An advantageous property of DLs is that reasoning in most of them is decidable, since they are fragments of first-order predicate logic [1]. The formalism of DLs has been used as a basis for the Web Ontology Language (OWL) standard, which as part of the Semantic Web is responsible for describing the semantics, or meaning, of data on the Internet [59]. DLs and the foundational technologies of the Semantic Web are considered mature, and are used both in research and the industry [36]. An illustrative example of the potential applications of DLs is SNOMED CT – a knowledge base that is considered to be the most comprehensive multilingual clinical healthcare terminology in the world [5].

As mentioned, DLs are a family of logics, which means that there are many DLs of varying expressiveness. Since in our work we focus on the $\mathcal{ALC}$ DL (abbreviated from attributive language with complements), we also introduce DLs from the perspective of $\mathcal{ALC}$ . There exist DLs that are simpler than $\mathcal{ALC}$ , for example $\mathcal{EL}$ , and there are also vastly more expressive DLs like $\mathcal{SHOI}{\mathcal{N}}^{\mathrm{(}\mathcal{D}\mathrm{)}}$ and $\mathcal{SROI}{\mathcal{Q}}^{\mathrm{(}\mathcal{D}\mathrm{)}}$ , which are the basis of OWL DL and OWL 2 DL respectively. We do not deal with $\mathcal{EL}$ or $\mathcal{SROI}{\mathcal{Q}}^{\mathrm{(}\mathcal{D}\mathrm{)}}$ , but we do discuss a procedure for transforming $\mathcal{SHOI}{\mathcal{N}}^{\mathrm{(}\mathcal{D}\mathrm{)}}$ knowledge bases to $\mathcal{ALC}$ knowledge bases in Section 6.4.2, by removing parts that $\mathcal{ALC}$ does not support.

For readers that are unfamiliar with DLs we begin with an intuitive overview. In later sections we describe the notation we use, and define the syntax and semantics of the $\mathcal{ALC}$ DL. The introduction to DLs is based on [78].

2.2.1. Overview

At a high level, a KB divides knowledge into terminology (also called TBox) and assertions (also called ABox). One may think that the terminology describes general knowledge (e.g. “Dogs are mammals”), and assertions state facts (e.g. “Fido is a dog” or “Fido likes Alice”). Those elements of KB terminology and assertions are called axioms. Axioms are formulated using a vocabulary, which consists of individuals (e.g. “Fido” or “Alice”), concept names (e.g. “dog” or “mammal”), and role names (e.g. “likes”). A concept denotes a set of individuals, and besides concept names in the vocabulary, there are complex concepts that can be constructed from other concepts using concept constructors. The available constructors are different depending on the chosen DL. In particular, in the $\mathcal{ALC}$ DL there are concept complement, intersection, union, universal restriction, and existential restriction constructors.

The TBox contains subsumption axioms, that allow one to state that one concept is subsumed by another concept. There are two kinds of assertion axioms in ABox: a concept assertion axiom allows one to state that an individual is an instance of a concept, and a role assertion axiom allows one to relate two individuals by a binary role.

Semantics of DLs are defined in a model-theoretic way, by providing an interpretation that represents the KB vocabulary in terms of a set called the domain. In particular, an interpretation maps individuals to elements of the domain, concepts to sets of individuals, and roles to binary relations between individuals. There are also special concepts, called the bottom concept and top concept, which correspond to the empty set and the domain, respectively.

The semantics of DLs provide an entailment relation, so that given a set of axioms, logical consequences may be inferred. In other words, KB entails a given axiom if that axiom follows from KB’s axioms.

2.2.2. Notation

Formally, a KB in the $\mathcal{ALC}$ DL is a pair $\mathcal{K}=\mathrm{(}\mathcal{T}\mathrm{,}\mathcal{A}\mathrm{)}$ , where $\mathcal{T}$ is the set of terminological axioms and $\mathcal{A}$ is the set of assertion axioms. The vocabulary of a KB is defined as a triple $\mathrm{(}{\mathit{N}}_{\mathit{C}}\mathrm{,}{\mathit{N}}_{\mathit{R}}\mathrm{,}{\mathit{N}}_{\mathit{I}}\mathrm{)}$ , where ${\mathit{N}}_{\mathit{C}}$ is the set of concept names, ${\mathit{N}}_{\mathit{R}}$ is the set of role names, and ${\mathit{N}}_{\mathit{I}}$ is the set of individual names. Given an arbitrary ordering of the set of concept names ${\mathit{N}}_{\mathit{C}}$ and an arbitrary ordering of the set of role names ${\mathit{N}}_{\mathit{R}}$ , we define ${\mathit{A}}_{\mathit{i}}$ as the i-th concept name, and ${\mathit{R}}_{\mathit{i}}$ as the i-th role name, so that ${\mathit{A}}_{\mathit{i}}\in {\mathit{N}}_{\mathit{C}}$ and ${\mathit{R}}_{\mathit{i}}\in {\mathit{N}}_{\mathit{R}}$ . In text, we write individuals and concept names in PascalCase, and role names in camelCase. In this work we do not deal with assertion axioms and henceforth assume that $\mathcal{A}=\varnothing$ .

2.2.3. Semantics

An interpretation is defined as a pair $\mathcal{I}=\mathrm{(}{\mathrm{\Delta }}^{\mathcal{I}}\mathrm{,}{·}^{\mathcal{I}}\mathrm{)}$ , where ${\mathrm{\Delta }}^{\mathcal{I}}$ is a set called the domain, and ${·}^{\mathcal{I}}$ is an interpretation function. The interpretation function ${·}^{\mathcal{I}}$ maps each concept C to a subset of ${\mathrm{\Delta }}^{\mathcal{I}}$ , each role R to a subset of ${\mathrm{\Delta }}^{\mathcal{I}}\times {\mathrm{\Delta }}^{\mathcal{I}}$ , and each individual to an element of ${\mathrm{\Delta }}^{\mathcal{I}}$ . The syntax and semantics in $\mathcal{ALC}$ is summarized in Table 1.

The set of terminological axioms $\mathcal{T}$ contains subsumption axioms $\mathit{C}⊑\mathit{D}$ , that are read C is subsumed by D, where C and D are concepts. If, for an interpretation $\mathcal{I}$ , ${\mathit{C}}^{\mathcal{I}}\subseteq {\mathit{D}}^{\mathcal{I}}$ then we say that the axiom $\mathit{C}⊑\mathit{D}$ holds in the interpretation, denoted $\mathcal{I}\vDash \mathit{C}⊑\mathit{D}$ , and call the interpretation a model of the axiom. Sometimes $\mathcal{T}$ also contains equivalence axioms $\mathit{C}\equiv \mathit{D}$ , although such axioms can be expressed in terms of two subsumption axioms $\mathit{C}⊑\mathit{D}$ and $\mathit{D}⊑\mathit{C}$ . If an interpretation is a model for all the axioms in $\mathcal{K}$ , we write $\mathcal{I}\vDash \mathcal{K}$ .

The notion can be extended to arbitrary axioms α. We say that α is a logical consequence of $\mathcal{K}$ , denoted $\mathcal{K}\vDash \mathit{\alpha }$ , if for every model $\mathcal{I}$ of $\mathcal{K}$ , the entailment $\mathcal{I}\vDash \mathit{\alpha }$ also holds.

Table 1

Syntax and semantics of $\mathcal{ALC}$

Description	Syntax	Semantics
top	⊤	${\mathrm{\Delta }}^{\mathcal{I}}$
bottom	⊥	∅
intersection	$\mathit{C}\sqcap \mathit{D}$	${\mathit{C}}^{\mathcal{I}}\cap {\mathit{D}}^{\mathcal{I}}$
union	$\mathit{C}\bigsqcup \mathit{D}$	${\mathit{C}}^{\mathcal{I}}\cup {\mathit{D}}^{\mathcal{I}}$
complement	$\neg \mathit{C}$	${\mathrm{\Delta }}^{\mathcal{I}}\setminus {\mathit{C}}^{\mathcal{I}}$
existential restriction	$\exists \mathit{R}.\mathit{C}$	$\{\mathit{x}\in {\mathrm{\Delta }}^{\mathcal{I}}|{\exists }_{\mathit{y}\in {\mathrm{\Delta }}^{\mathcal{I}}}\mathrm{(}\mathrm{(}\mathit{x}\mathrm{,}\mathit{y}\mathrm{)}\in {\mathit{R}}^{\mathcal{I}}\wedge \mathit{y}\in {\mathit{C}}^{\mathcal{I}}\mathrm{)}\}$
universal restriction	$\forall \mathit{R}.\mathit{C}$	$\{\mathit{x}\in {\mathrm{\Delta }}^{\mathcal{I}}|{\forall }_{\mathit{y}\in {\mathrm{\Delta }}^{\mathcal{I}}}\mathrm{(}\mathrm{(}\mathit{x}\mathrm{,}\mathit{y}\mathrm{)}\in {\mathit{R}}^{\mathcal{I}}\to \mathit{y}\in {\mathit{C}}^{\mathcal{I}}\mathrm{)}\}$

2.2.4. Ontologies in the web ontology language

While the Web Ontology Language (OWL) is underpinned by a DL, it uses a different nomenclature. In particular, knowledge bases are called ontologies in OWL. We also use the term ontology, but only when referring to a KB described in OWL. Concept names and role names are called class names and object property names, respectively. In general, concepts are referred to as classes or class expressions.

Terminological axioms are called class axioms, and a subsumption $\mathit{C}⊑\mathit{D}$ is read “C is a subclass of D”. Although we did not discuss role axioms as $\mathcal{ALC}$ does not support them, OWL ontologies do support object property axioms, which are the same thing. In addition to class, object property, and individual axioms, OWL ontologies can also contain annotation properties and annotation axioms, that allow one to provide additional information about classes. For example, one very common annotation property is rdfs:label, which is used to provide human-readable labels.

2.2.5. Semantic reasoners

The main benefit of using a knowledge representation paradigm with well-founded semantics is the ability to perform automated reasoning. In this work, we use the entailment checking reasoning task, i.e., deciding whether $\mathcal{K}\vDash \mathit{\alpha }$ for a given axiom α.

Many semantic reasoners for DLs are available, providing reasoning services for KBs, among others, HermiT [61], Pellet [83], and FaCT++ [89] capable of reasoning in very expressive DLs, or more specialized ones such as ELK [44].

In this work, FaCT++ is the reasoner of choice, as it is capable enough to deal with $\mathcal{ALC}$ knowledge bases, yet it is written in C++, which means that it is easy and efficient to interface with it from Python via a Cython extension. That was not the case for the more popular reasoners implemented directly on the Java Virtual Machine, where the cost of interaction was prohibitive.

3. Related work

The neural and symbolic paradigms of artificial intelligence are vastly different. On one hand, the currently dominant neural paradigm shows how well neural networks perform on large-scale data sets, ranging from simple classification and regression, to language models so powerful, that their output is almost indistinguishable from human-written text [9], and generative image models that can synthesize photorealistic images from text prompts [74]. However, neural models often produce nonsensical results, for example paragraphs of text that contradict themselves. Because neural models are not easily interpretable, it is difficult to find the cause of such problems. On the other hand, the symbolic paradigm offers methods for explicitly describing knowledge, and reliably performing reasoning over that knowledge. Symbolic methods can provide step-by-step explanations of their inferences, because they use deductive reasoning. Unfortunately, symbolic methods are not well suited for learning from data, as real-life knowledge is often seemingly contradictory. Symbolic methods also have trouble with processing large-scale data sets, because of high time complexities of used algorithms. The research field of neuro-symbolic integration aims to combine the large-scale learning ability of neural models, and the ability of symbolic methods to express knowledge and perform reasoning, all while keeping interpretability, thus combining the benefits and avoiding the pitfalls of both paradigms [23]. There seems to be two major avenues where the Semantic Web can benefit from the neural paradigm: ontology and knowledge graphs embeddings and deep deductive reasoners [35].

3.1. Knowledge base embeddings

One way to bridge the neural and symbolic paradigms is to represent symbolic knowledge in terms of vectors in a high-dimensional real vector space. Such vectors can then be used as additional inputs to machine learning models based on neural networks, to improve their performance by allowing them to use an approximation of expert knowledge [87]. A good method of learning embeddings from symbolic knowledge should ideally leverage the structural information present in relations between abstract concepts, and should not try to learn embeddings for abstract concepts with the aid of word embeddings, due to the ambiguity of language, its limited abstraction, and other problems [11].

One of the first embeddings methods tailored to KBs to get traction were NTN proposed by Socher et al. [84], TransE proposed by Bordes et al. [7,8], later unified into a single framework by Yang et al. [97], as both leverage linear and bilinear mappings. The notion of modeling relations as operations in some vector space was further extended by Wang et al. with TransH [94], Lin et al. with TransR [51] and PTransE [50], Sun et al. with RotatE [85], Zhang et al. with HakE [98], Wang et al. with InterHT [90], or Zhou et al. with Path-RotatE [99].

A separate category of embeddings was established by ConvE, employing convolutional neural networks [20]. This line of research was continued by Nguyen et al. with ConvKB [62], Jiang et al. with ConvR [42], Demir et al. with ConEx [19]. Over the time even more complex neural architectures were employed, e.g., capsule networks in CapsE by Nguyen et al. [63], recurrent skipping networks by Guo et al. [33], graph convolutional networks in GCN-Align by Wang et al. [93] or R-GCN by Schlichtkrull et al. [82], recurrent transformers by Werner et al. [95]

Yet another approach was taken in RESCAL by Nickel et al. where tensor-based techniques were used [65]. Similar approaches were taken in TOAST by Jachnik et al. [39], TATEC by García-Durán et al. [28], DistMult by Yang et al. [97], HolE by Nickel et al. [64], ComplEx by Trouillon et al. [88], or ANALOGY by Liu et al. [52].

In the context of the Semantic Web technologies, notable examples of embeddings are RDF2Vec by Ristoski et al. [75], OWL2Vec^* by Chen et al. [12], and TransOWL by d’Amato et al. [17].

Embeddings are used in various downstream tasks, such as disambiguation [80], ontology learning [71], handling concept drift in streams [13], question answering [54] or fact classification [43]. Over the years multiple survey papers were published on the topic of KB embeddings [10,14,16,91,92].

To the best of our knowledge, none of the proposed methods aims to capture the semantics as a topology of the embedding space, nor enables computing the embeddings of complex expressions from the embeddings of their parts.

3.2. Deep deductive reasoners

Compared to the work on embeddings, much less attention has been devoted to the notion of approximating or emulating the results of deductive reasoning with machine learning, and in particular deep learning [23].

Earlier works, employing traditional machine learning approaches, were mostly devoted to approximate a deductive reasoner in order to obtain the results faster than from a deductive reasoner. For example, Rizzo et al. [76,77] proposed terminological trees and forests to tackle this problem, while Paulheim and Stuckenschmidt concentrated on the problem of approximate reasoning with ABox [70].

One of the first works leveraging deep learning for deductive reasoning over ontologies were those of Makni and Hendler on using recurrent neural networks for RDFS reasoning [56], and of Hohenecker and Lukasiewicz on using recursive reasoning networks [37]. Eberhart et al. employed long short-term memory networks (LSTMs) for reasoning in the ${\mathcal{EL}}^{+}$ description logic [22,55]. Badreddine et al. introduced Logic Tensor Networks [2], which were then employed by Bianchi and Hitzler to perform deductive reasoning [6]. Ebrahimi et al. investigated the applicability of pointer networks to the task [24] and of memory networks [25]. Zhu et al. cast the problem of reasoning as a graph-to-graph transformation problem and applied it in the context of RDF [100].

Ebrahimi et al. in [24] introduced a few dimensions along which a reasoner can be classified: logic – which logical formalism is tackled; transfer – the capability of achieving good performance on a previously unknown KB; generative – the capability of generating new inferences, not only answering yes/no queries; scale – the size of considered KBs; performance – high if the system was able to cross the 70% threshold on the F1-score, low otherwise. Following these dimensions, the reasoner presented in this work tackles the $\mathcal{ALC}$ logic, is transferable, is not generative, is of low to moderate scale, and has high performance. To the best of our knowledge, this is the first transferable reasoner capable of tackling such a complex logic, albeit we must underline that the embeddings for a new KB must be trained under the supervision of the reasoner.

4. Neural reasoner

Fig. 1.

A high-level overview of the neural reasoner architecture. An embedding layer for knowledge base $\mathcal{K}$ and a reasoner head are combined to create a neural reasoner for $\mathcal{K}$ . The reasoner predicts whether $\mathcal{K}$ entails subsumption axioms, written $\mathcal{K}\vDash \mathit{C}⊑\mathit{D}$ , where C and D are $\mathcal{ALC}$ concepts. We train the reasoner by contrasting the predicted probability of entailment with the answer provided by a semantic reasoner. The reasoner head generalizes across all $\mathcal{ALC}$ knowledge bases, but for each $\mathcal{K}$ we have to train a separate embedding layer. The reasoner head may be frozen while training an embedding layer for a new knowledge base $\mathcal{K}$ , to make learning faster.

Our neural reasoner (hereafter, reasoner) is a classifier, which given a subsumption axiom, outputs the probability that the axiom is entailed by a given knowledge base. The reasoner consists of the generic reasoner head, and an interchangeable embedding layer specific to a given knowledge base. The generic reasoner head can classify entailment, and construct embeddings for concept complements and intersection of concepts. While an embedding layer contains embeddings for a given knowledge base – that is, it stores embedding vectors for concept names and can construct embeddings for existential restrictions for a given role name. A diagram of the architecture of our neural reasoner is shown in Fig. 1.

The main purpose of the classifier is to facilitate the computation of the gradient of the loss function with respect to the weights of the embedding layer. In other words, when the reasoner is given an axiom, it builds its representation bottom-up from the KB-specific embeddings in the embedding layer. That representation is then used by a neural network in the reasoner head to classify whether the axiom is entailed by that KB. The classification output is contrasted with the target output computed by a semantic reasoner, and the value of the loss function is used to adjust weights of the embedding layer with backpropagation through structure [30].

4.1. Reasoner head

In our work we chose the reasoner to be an entailment classifier for subsumption axioms in $\mathcal{ALC}$ – that is, given an axiom $\mathit{C}⊑\mathit{D}$ , the reasoner head outputs the probability $\mathit{P}\mathrm{(}\mathcal{K}\vDash \mathit{C}⊑\mathit{D}\mathrm{)}$ that the axiom is entailed by a given knowledge base $\mathcal{K}$ . Assume that ${\mathit{h}}_{\mathcal{K}}$ is a function mapping $\mathcal{ALC}$ concepts in knowledge base $\mathcal{K}$ to embedding vectors in real vector space ${\mathbb{R}}^{{\mathit{N}}_{\mathit{e}}}$ , where ${\mathit{N}}_{\mathit{e}}$ is the embedding dimension. The details of how this function is constructed are given in Section 4.2.

An interaction map for a pair of embeddings u, v, denoted $\mathrm{IM}\mathrm{(}\mathit{u}\mathrm{,}\mathit{v}\mathrm{)}$ , as defined by Equation (3), is a concatenation of u, v, and their outer product1

¹

Each element of the outer product of two vectors ${\mathrm{(}\mathbf{u}\otimes \mathbf{v}\mathrm{)}}_{\mathit{i}\mathit{j}}$ is defined as ${\mathbf{u}}_{\mathit{i}}·{\mathbf{v}}_{\mathit{j}}$ .

$\mathit{u}\otimes \mathit{v}$ . Before concatenation, the outer product is reshaped to a row vector. Our preliminary experimentation indicated that including the outer product in the interaction map is of utmost importance, as this helps the classifier to learn high-order correlations [3,34]. By extension, an interaction map for a pair of concepts C, D in knowledge base $\mathcal{K}$ is the interaction map for their embeddings: ${\mathrm{IM}}_{\mathcal{K}}\mathrm{(}\mathit{C}\mathrm{,}\mathit{D}\mathrm{)}=\mathrm{IM}\mathrm{(}{\mathit{h}}_{\mathcal{K}}\mathrm{(}\mathit{C}\mathrm{)}\mathrm{,}{\mathit{h}}_{\mathcal{K}}\mathrm{(}\mathit{D}\mathrm{)}\mathrm{)}$ . $$\begin{array}{}\text{(3)}& \mathrm{IM}\mathrm{(}\mathit{u}\mathrm{,}\mathit{v}\mathrm{)}=[\mathit{u};\mathit{v};\text{flatten}\mathrm{(}\mathit{u}\otimes \mathit{v}\mathrm{)}]\end{array}$$

The classification output for an entailment query $\mathit{P}\mathrm{(}\mathcal{K}\vDash \mathit{C}⊑\mathit{D}\mathrm{)}\in \mathrm{(}0\mathrm{,}1\mathrm{)}$ (see Equation (4)) is defined as the output of the feedforward neural network ${\mathrm{NN}}_{⊑}$ , which accepts the interaction map of C and D, and has a single output neuron with the sigmoid activation function σ and a single hidden layer with 16 neurons, which is followed by the ELU activation function [15]. $$\begin{array}{}\text{(4)}& \mathit{P}\mathrm{(}\mathcal{K}\vDash \mathit{C}⊑\mathit{D}\mathrm{)}={\mathrm{NN}}_{⊑}\mathrm{(}{\mathrm{IM}}_{\mathcal{K}}\mathrm{(}\mathit{C}\mathrm{,}\mathit{D}\mathrm{)}\mathrm{)}={\mathit{L}}_1^{\mathit{\sigma }}\mathrm{\left(}{\mathit{L}}_{16}^{\mathrm{ELU}}\mathrm{\right(}{\mathrm{IM}}_{\mathcal{K}}\mathrm{(}\mathit{C}\mathrm{,}\mathit{D}\mathrm{)}\mathrm{\left)}\mathrm{\right)}\end{array}$$

4.2. Embedding layer

Recursive neural networks have been successfully used for encoding expression trees as fixed-size vectors, that could be used as inputs to machine learning models [30]. The recursively defined DL concepts also have a tree structure [49], so it is appropriate to use a recursive neural network as the embedding layer in our reasoner architecture. The key hypothesis that defines our reasoner is that for each KB, we can train an embedding layer that embeds concepts from that KB in an embedding space with a topology that makes it easy for the reasoner head to classify entailment. An embedding topology that is beneficial to entailment classification is formed by jointly training the reasoner head and embedding layers for multiple KBs, which forces the embedding head to generalize, and the embedding layers to output embeddings in the shared embedding space.

We note in passing that we distinguish between a recursive and a recurrent neural network: in a recursive neural network, different parts of the network are applied to the input multiple times, depending on a structured input, as with recursive grammars, where the same production rule can be applied again and again; in a recurrent neural network (e.g., GRU, LSTM) the output of the network is connected back to the network, as part of its input.

In general, concept embeddings are represented by vectors in real vector space ${\mathbb{R}}^{{\mathit{N}}_{\mathit{e}}}$ , where ${\mathit{N}}_{\mathit{e}}$ is the embedding dimension. In Equations (5)–(11) we define the recursive function ${\mathit{h}}_{\mathcal{K}}:\mathcal{ALC}\to {\mathbb{R}}^{{\mathit{N}}_{\mathit{e}}}$ that maps $\mathcal{ALC}$ concepts for knowledge base $\mathcal{K}$ to embedding vectors in real vector space ${\mathbb{R}}^{{\mathit{N}}_{\mathit{e}}}$ . We begin with the embedding for each concept name ${\mathit{A}}_{\mathit{i}}$ : a vector ${\mathbf{W}}_{\mathcal{K}\mathrm{,}{\mathit{A}}_{\mathit{i}}}$ of dimension ${\mathit{N}}_{\mathit{e}}$ , as given by Equation (5). The vector is optimized during training and it is KB-specific, since different KBs have a different number named concepts with different meanings. $$\begin{array}{}\text{(5)}& {\mathit{h}}_{\mathcal{K}}\mathrm{(}{\mathit{A}}_{\mathit{i}}\mathrm{)}={\mathbf{W}}_{\mathcal{K}\mathrm{,}{\mathit{A}}_{\mathit{i}}}\end{array}$$

The embeddings of the bottom and top concepts are stored by the vectors ${\mathbf{W}}_{\perp }$ and ${\mathbf{W}}_{\top }$ , respectively, as specified by Equation (6). Since the semantics of the top and bottom concepts are independent of any specific KB, the vectors are shared between all KBs. $$\begin{array}{}\text{(6)}& {\mathit{h}}_{\mathcal{K}}\mathrm{(}\top \mathrm{)}={\mathbf{W}}_{\top }{\mathit{h}}_{\mathcal{K}}\mathrm{(}\perp \mathrm{)}={\mathbf{W}}_{\perp }\end{array}$$

To obtain the embedding for the complement of a given concept, one first recursively obtains the embedding of that concept, and then passes it as an input to the complement constructor network ${\mathrm{NN}}_{\neg }$ , consisting of a single layer of ${\mathit{N}}_{\mathit{e}}$ neurons without the activation function, as defined in Equation (7). $$\begin{array}{}\text{(7)}& {\mathit{h}}_{\mathcal{K}}\mathrm{(}\neg \mathit{C}\mathrm{)}={\mathrm{NN}}_{\neg }\mathrm{(}{\mathit{h}}_{\mathcal{K}}\mathrm{(}\mathit{C}\mathrm{)}\mathrm{)}\text{where }{\mathrm{NN}}_{\neg }={\mathit{L}}_{{\mathit{N}}_{\mathit{e}}}^{·}\end{array}$$

We also define in Equation (8) the embedding of a doubly negated concept as the embedding of that concept, by the double negation elimination rule, which speeds the construction of embeddings. $$\begin{array}{}\text{(8)}& {\mathit{h}}_{\mathcal{K}}\mathrm{(}\neg \neg \mathit{C}\mathrm{)}={\mathit{h}}_{\mathcal{K}}\mathrm{(}\mathit{C}\mathrm{)}\end{array}$$

To obtain the embedding for an intersection of concepts, one first recursively obtains the embeddings of the two concepts, computes their interaction map, and passes it as the input to the intersection constructor network ${\mathrm{NN}}_{\sqcap }$ . It is a single layer of ${\mathit{N}}_{\mathit{e}}$ neurons without the activation function, as specified by Equation (9), that maps the interaction map of two concept embeddings, to the embedding of their intersection. Its input is a vector of the size of the interaction map $2{\mathit{N}}_{\mathit{e}}+{\mathit{N}}_{\mathit{e}}^2$ , and the output is a vector of size ${\mathit{N}}_{\mathit{e}}$ . $$\begin{array}{}\text{(9)}& {\mathit{h}}_{\mathcal{K}}\mathrm{(}\mathit{C}\sqcap \mathit{D}\mathrm{)}={\mathrm{NN}}_{\sqcap }\mathrm{(}{\mathrm{IM}}_{\mathcal{K}}\mathrm{(}\mathit{C}\mathrm{,}\mathit{D}\mathrm{)}\mathrm{)}\text{where }{\mathrm{NN}}_{\sqcap }={\mathit{L}}_{{\mathit{N}}_{\mathit{e}}}^{·}\end{array}$$

Fig. 2.

An example interpretation of a knowledge base with role assertions is shown on the left side. The individuals Fido, Salem, and Rocky are elements of the set ${\mathtt{\text{Animal}}}^{\mathcal{I}}$ . Individuals Alice, Bob, and Charlie are elements of set ${\mathrm{(}\exists \mathtt{\text{owns}}.\mathtt{\text{Animal}}\mathrm{)}}^{\mathcal{I}}$ . An illustration of transformations from concept embeddings to existential restriction embeddings is shown on the right side. The embedding of $\exists \mathtt{\text{owns}}.\mathit{C}$ may be obtained by applying ${\mathrm{NN}}_{\mathcal{K}\mathrm{,}\mathtt{\text{owns}}}$ to the embedding of concept C.

To obtain the embedding for an existential restriction, one first obtains the embedding of the given concept, and passes it to the existential restriction constructor network, defined as ${\mathrm{NN}}_{\mathcal{K}\mathrm{,}{\mathit{R}}_{\mathit{i}}}={\mathit{L}}_{{\mathit{N}}_{\mathit{e}}}^{·}$ , i.e., consisting of a single layer of ${\mathit{N}}_{\mathit{e}}$ neurons without the activation function. To understand the intuition behind this choice, consider the following example. On the left side of Fig. 2 we show an interpretation of a KB with a set of human individuals, and a set of individuals that are instances of concept Animal. Some individuals own an animal, which is described with role assertions $\mathtt{\text{owns}}\mathrm{(}\mathtt{\text{Alice}}\mathrm{,}\mathtt{\text{Fido}}\mathrm{)}$ , $\mathtt{\text{owns}}\mathrm{(}\mathtt{\text{Bob}}\mathrm{,}\mathtt{\text{Fido}}\mathrm{)}$ , $\mathtt{\text{owns}}\mathrm{(}\mathtt{\text{Charlie}}\mathrm{,}\mathtt{\text{Rocky}}\mathrm{)}$ (shown as arrows). Alice, Bob and Charlie may be described as instances of $\exists \mathtt{\text{owns}}.\mathtt{\text{Animal}}$ . Notice that if we reversed all owns arrows, then an owns arrow would always start in set Animal, and end in set $\exists \mathtt{\text{owns}}.\mathtt{\text{Animal}}$ , as by definition ${\mathrm{(}\exists \mathit{R}.\mathit{C}\mathrm{)}}^{\mathcal{I}}=\{\mathit{x}\in {\mathrm{\Delta }}^{\mathcal{I}}|{\exists }_{\mathit{y}\in {\mathrm{\Delta }}^{\mathcal{I}}}\mathrm{(}\mathrm{(}\mathit{x}\mathrm{,}\mathit{y}\mathrm{)}\in {\mathit{R}}^{\mathcal{I}}\wedge \mathit{y}\in {\mathit{C}}^{\mathcal{I}}\mathrm{)}\}$ , all elements of ${\mathrm{(}\exists \mathit{R}.\mathit{C}\mathrm{)}}^{\mathcal{I}}$ are related to some element of ${\mathit{C}}^{\mathcal{I}}$ . So in general, if we visualize role assertions as arrows, for any interpretation $\mathcal{I}$ , if we reverse role ${\mathit{R}}_{\mathit{i}}$ arrows, they always begin in set ${\mathit{C}}^{\mathcal{I}}$ and end in set ${\mathrm{(}\exists {\mathit{R}}_{\mathit{i}}.\mathit{C}\mathrm{)}}^{\mathcal{I}}$ . Now consider the right side of the figure. Our embedding layer represents concepts as points in the embedding space ${\mathbb{R}}^{{\mathit{N}}_{\mathit{e}}}$ and does not support individuals at all, so we “squish” the sets $\exists \mathtt{\text{owns}}.\mathtt{\text{Animal}}$ and Animal into points in ${\mathbb{R}}^{{\mathit{N}}_{\mathit{e}}}$ , but we keep the (reversed) arrows, since their start and end points always occur inside the sets. Thus, we think that it is appropriate to model the existential restriction constructor as a function mapping concept embeddings ${\mathit{h}}_{\mathcal{K}}\mathrm{(}\mathit{C}\mathrm{)}$ to embeddings of the existential restriction ${\mathit{h}}_{\mathcal{K}}\mathrm{(}\exists {\mathit{R}}_{\mathit{i}}.\mathit{C}\mathrm{)}$ . There is a separate function ${\mathrm{NN}}_{\mathcal{K}\mathrm{,}{\mathit{R}}_{\mathit{i}}}$ for each role, and functions ${\mathrm{NN}}_{\mathcal{K}\mathrm{,}{\mathit{R}}_{\mathit{i}}}$ need to be learned per knowledge base $\mathcal{K}$ , because different KBs have different numbers of roles, and roles may have different meanings, so there is no obvious way of sharing existential restriction constructors between KBs. The formalization is given in Equation (10). $$\begin{array}{}\text{(10)}& {\mathit{h}}_{\mathcal{K}}\mathrm{(}\exists {\mathit{R}}_{\mathit{i}}.\mathit{C}\mathrm{)}={\mathrm{NN}}_{\mathcal{K}\mathrm{,}{\mathit{R}}_{\mathit{i}}}\mathrm{(}{\mathit{h}}_{\mathcal{K}}\mathrm{(}\mathit{C}\mathrm{)}\mathrm{)}\end{array}$$

In deep learning, activation functions must be used after each hidden layer so that one does not compose multiple linear transformations, which are equivalent to a single linear transformation. Note that even though our constructor networks do not have an activation function, the result of the ${\mathit{h}}_{\mathcal{K}}$ function is not simply a composition of multiple linear transformations, because at every recursive step the constructor networks (i.e. linear transformations) are given different inputs. If one would like to make the constructor networks deep, i.e., add more hidden layers, then one must add an activation function like $\text{ELU}$ after each layer, except the last one.

To extend ${\mathit{h}}_{\mathcal{K}}$ to concept unions and universal restrictions, we use the de Morgan’s laws [78], arriving at the formulas given in Equation (11). $$\begin{array}{}\text{(11)}& \begin{array}{rl}& {\mathit{h}}_{\mathcal{K}}\mathrm{(}\mathit{C}\bigsqcup \mathit{D}\mathrm{)}={\mathit{h}}_{\mathcal{K}}\mathrm{(}\neg \mathrm{(}\neg \mathit{C}\sqcap \neg \mathit{D}\mathrm{)}\mathrm{)}\\ & {\mathit{h}}_{\mathcal{K}}\mathrm{(}\forall {\mathit{R}}_{\mathit{i}}.\mathit{C}\mathrm{)}={\mathit{h}}_{\mathcal{K}}\mathrm{(}\neg \exists {\mathit{R}}_{\mathit{i}}.\neg \mathit{C}\mathrm{)}\end{array}\end{array}$$

4.3. Relaxed architecture

An important property of the embedding layer, that we introduced in the previous section, is that the weights of the neural networks ${\mathrm{NN}}_{\neg }$ and ${\mathrm{NN}}_{\sqcap }$ do not depend on a specific knowledge base. Since the semantics of concept complement and intersection constructors do not change depending on a KB, we think that their neural equivalents should also not change. We call a reasoner, where ${\mathrm{NN}}_{\neg }$ and ${\mathrm{NN}}_{\sqcap }$ network weights are stored in the reasoner head, the restricted reasoner. Unless stated otherwise, we use restricted reasoners.

While sharing constructors mimics the logic, it may not be an appropriate assumption in a deep learning model and thus a hindrance to learn good embeddings. To verify whether this is the case, we consider a variant of the reasoner, that allows the embedding layer to learn KB-specific concept constructor networks ${\mathrm{NN}}_{\mathcal{K}\mathrm{,}\neg }$ and ${\mathrm{NN}}_{\mathcal{K}\mathrm{,}\sqcap }$ , and a KB-specific embedding of the bottom concept ${\mathbf{W}}_{\mathcal{K}\mathrm{,}\perp }$ and the top concept ${\mathbf{W}}_{\mathcal{K}\mathrm{,}\top }$ . We call this variant the relaxed reasoner. All changes to the function ${\mathit{h}}_{\mathcal{K}}$ in the relaxed reasoner are marked (underlined) below. $$\begin{array}{}\text{(12)}& \begin{array}{rl}& {\mathit{h}}_{\mathcal{K}}\mathrm{(}\neg \mathit{C}\mathrm{)}={\mathrm{NN}}_{\underset{\_}{\mathcal{K}}\mathrm{,}\neg }\mathrm{(}{\mathit{h}}_{\mathcal{K}}\mathrm{(}\mathit{C}\mathrm{)}\mathrm{)}\\ & {\mathit{h}}_{\mathcal{K}}\mathrm{(}\mathit{C}\sqcap \mathit{D}\mathrm{)}={\mathrm{NN}}_{\underset{\_}{\mathcal{K}}\mathrm{,}\sqcap }\mathrm{(}{\mathrm{IM}}_{\mathcal{K}}\mathrm{(}\mathit{C}\mathrm{,}\mathit{D}\mathrm{)}\mathrm{)}\\ & {\mathit{h}}_{\mathcal{K}}\mathrm{(}\top \mathrm{)}={\mathbf{W}}_{\underset{\_}{\mathcal{K}}\mathrm{,}\top }\\ & {\mathit{h}}_{\mathcal{K}}\mathrm{(}\perp \mathrm{)}={\mathbf{W}}_{\underset{\_}{\mathcal{K}}\mathrm{,}\perp }\end{array}\end{array}$$

In the relaxed architecture, the only weights shared between different KBs are the ones learned by the classifier ${\mathrm{NN}}_{⊑}$ , which we expected would allow the embedding layers to learn more fitting embeddings, at the cost of limiting generalization and opportunities for transfer learning.

4.4. Training procedure

The training procedure for our reasoner consists of two steps. In the first step, we train both the reasoner head and embedding layers for as many diverse KBs as possible. This results in a reasoner head that learned to classify whether subsumption axioms hold in any KB, given that an appropriate embedding layer is provided. By appropriate embedding layer we mean a layer that learned to embed KB-specific concepts in a space that minimizes the classifier loss. We simply call this step training the reasoner head, and we consider the data used in this step as the training set. Note that, as a result of training the reasoner, we obtain trained embedding layers for KBs in the training set. If obtaining the trained embedding layers was the goal, then the next step is not necessary.

In the second step, we freeze the reasoner head and train the embedding layers for KBs that were not seen in the first step. This results in embedding layers that can embed concepts in a space in which the reasoner is good at classification. We think that if our reasoner can accurately classify whether subsumption axioms are entailed by a KB, then the embeddings used as inputs to the classifier ${\mathrm{NN}}_{⊑}$ faithfully capture the semantics of the KB, even though we did not train the reasoner head in the second step. We call this step training the embedding layers or testing the reasoner head, and we consider the data set used in this step as the test set.

5. Axiom generator

Our reasoner learns embeddings by learning to classify entailment queries $\mathcal{K}\vDash \mathit{C}⊑\mathit{D}$ . A set of query axioms could be created manually, but for learning to be successful a large number of subsumption axioms is needed. One way of obtaining a large number of axioms would be to generate all possible axioms for a given KB, up to some maximum expression tree depth. This approach would suffice for shallow expression trees and KBs with very small vocabularies, but is impractical otherwise. A more practical method of obtaining a large set of axioms is to pseudo-randomly generate them, an approach which is fast, reproducible by setting the initial state of the pseudo-random number generator to a fixed value, yet offering a virtually unlimited number of axioms, at the small cost of possibly generating duplicates.

One algorithm for pseudo-randomly generating $\mathcal{ALC}$ expressions and axioms was introduced by Eberhart et al. [21]. Unfortunately, their implementation requires the Java Virtual Machine, which did not suit our purposes. We thus decided to implement in Python our own custom generator inspired by their algorithm. It has four parameters: the set of concept names ${\mathit{N}}_{\mathit{C}}$ , the set of role names ${\mathit{N}}_{\mathit{R}}$ , the probability ${\mathit{p}}_{\mathit{A}}$ of generating a concept name instead of a top or bottom concept, and the maximum depth of recursion ${\mathit{d}}_{max}$ . Expressions are generated recursively and the current recursion depth d is tracked. At each step, the generator uses a grammar rule by replacing the symbol from the left side of the rule with one of the symbols from the right side.

Axioms are generated by starting with the grammar rule A (Equation (13)), which returns a subsumption axiom or a disjointness axiom with equal probability. The second alternative is intentionally redundant, to ensure that disjointness axioms are generated frequently enough for the reasoner to benefit from them. Regardless of which alternative was returned, two concepts are generated according to grammar rule C (Equation (14)). First, we select at random the maximum recursion depth for the axiom ${\mathit{d}}_{max}^{\prime }$ from the discrete uniform distribution $\mathcal{U}\{1\mathrm{,}{\mathit{d}}_{max}\}$ . Then, the maximum depth of recursion for the first concept ${\mathit{d}}_{max\mathrm{,}1}$ is chosen from the uniform discrete distribution $\mathcal{U}\{1\mathrm{,}{\mathit{d}}_{max}^{\prime }\}$ , and the maximum depth of recursion for the second concept ${\mathit{d}}_{max\mathrm{,}2}$ is chosen from $\mathcal{U}\{1\mathrm{,}max\{1\mathrm{,}{\mathit{d}}_{max}^{\prime }-{\mathit{d}}_{max\mathrm{,}1}\}\}$ . This technique of limiting the expression depth results in axioms where one of the concepts is deeper than the other, or axioms where both concepts are relatively shallow, to ensure satisfactory performance of checking entailment of the generated axioms. $$\begin{array}{}\text{(13)}& \mathbf{A}::=\mathbf{C}⊑\mathbf{C}|\mathbf{C}\sqcap \mathbf{C}⊑\perp \end{array}$$

The grammar rule C returns one of the following alternatives with equal probability: a concept according to the grammar rule T (Equation (15)), a concept complement, a concept intersection, a concept union, an existential restriction, or a universal restriction. For existential and universal restrictions, the role name ${\mathit{R}}_{\mathit{i}}$ is chosen uniformly at random from ${\mathit{N}}_{\mathit{R}}$ . If the recursion depth d is equal to or greater than ${\mathit{d}}_{max}$ , then a concept according to the grammar rule T is always returned. $$\begin{array}{}\text{(14)}& \mathbf{C}::=\mathbf{T}|\neg \mathbf{C}|\mathbf{C}\sqcap \mathbf{C}|\mathbf{C}\bigsqcup \mathbf{C}|\exists {\mathit{R}}_{\mathit{i}}.\mathbf{C}|\forall {\mathit{R}}_{\mathit{i}}.\mathbf{C}\end{array}$$

The grammar rule T returns one of the following alternatives: with probability ${\mathit{p}}_{\mathit{A}}$ , a concept name ${\mathit{A}}_{\mathit{i}}$ chosen uniformly at random from ${\mathit{N}}_{\mathit{C}}$ , the top concept with probability $\frac{1-{\mathit{p}}_{\mathit{A}}}{2}$ , or the bottom concept with probability $\frac{1-{\mathit{p}}_{\mathit{A}}}{2}$ . To ensure proper exposure of the reasoner to the top and bottom concepts during training, we use the parameter ${\mathit{p}}_{\mathit{A}}$ to make the probability of returning ⊤ or ⊥ independent of the number of concept names $|{\mathit{N}}_{\mathit{C}}|$ . $$\begin{array}{}\text{(15)}& \mathbf{T}::={\mathit{A}}_{\mathit{i}}|\top |\perp \end{array}$$

6. Experimental evaluation

6.1. Goals

Although the design of the proposed approach mimics the $\mathcal{ALC}$ logic and is well-grounded in research on deep learning architectures, ultimately its usability and properties must be established experimentally. To this end, we formulate four research questions and design appropriate experiments to answer them.

The main hypothesis of our research is that using a transferable reasoner head forces the embeddings to reflect the semantics of the underlying KBs in a shared way. To address it, we pose the following research question RQ1: Can the reasoner induce a useful topology that it can subsequently exploit, or are the embeddings simply overfitting the KBs? We answer RQ1 in Experiment 1, described in Section 6.5, where we compare the classification metrics of a trained reasoner and a random reasoner in previously unseen ontologies, and show that the trained reasoner indeed brings something to the table.

In Section 4.3, we constructed a relaxed variant of the reasoner, hypothesising that the restricted variant may be too restrictive and impede the learning process. From this, we formulate RQ2: Can concept constructors be shared between KBs, or must they be KB-specific? To answer it, we designed Experiment 2, described in Section 6.6. In there, we follow the same procedure as in Experiment 1, but using the relaxed reasoner. We then compare the results obtained by the restricted reasoner in Experiment 1, and by the relaxed reasoner, and show that the difference is negligible.

The overarching goal of this work is to present a method to train embeddings for $\mathcal{ALC}$ KBs including the semantics of the concepts, and thus we pose RQ3: Can the presented approach learn good embeddings for concepts from a real-world ontology? We propose Experiment 3, described in Section 6.7, where we use the pizza ontology as the real-world ontology. We consider three different training schemes, measure the classification metrics, and visually analyze the obtained embeddings, showing that the reasoner head from Experiment 1 fares only slightly worse than a reasoner head overfitted to the ontology.

We extend RQ1 and RQ3 to a larger set of real-world ontologies, formulating RQ4: Is transfer learning with the presented approach viable for a variety of real-world ontologies? In Experiment 4, described in Section 6.8, we follow the methodology established in Experiment 3 with 6 different ontologies of varying sizes and complexity and show that on average the results of using a pre-trained reasoner head are as good as training a reasoner head from scratch on the considered ontologies.

Finally, we tackle the problem of identifying what is hard for the reasoner by posing RQ5 Are there types of queries that are significantly harder for the reasoner than others?. We briefly address this question in Experiment 5, however, without reaching a satisfactory answer.

6.2. Setup

We implemented the proposed approach in Python 3.9.7 and Cython [4] (a superset of Python that compiles to C or C++), and used PyTorch 1.10.1 to implement the neural reasoner [68]. The source code is available at https://github.com/maxadamski/reasonable-embeddings. We ran the experiments on a computer with an Intel Core i5-4670K CPU, 32 GB of DDR3 RAM, and no dedicated GPU, running Void Linux with kernel 5.15.

We use pseudo-randomly generated numbers to create our data sets and in training. For data generation we use the NumPy implementation of the Permuted Congruential Generator [66]. The internal PyTorch pseudo-random number generator is used during training. To ensure reproducibility of our experiments, we set the initial states of the pseudo-random number generators to a known initial value.

During training, our reasoner uses inferences made by a semantic reasoner as the expected class in classification. The Semantic Web community traditionally uses Java as the language of choice, a language which does not interface well with Python. In particular, using either HermiT [61] or Pellet [83], state-of-the-art semantic reasoners, required executing the reasoners as separate Java Virtual Machine (JVM) processes, a process for every inference. This introduced unacceptable overhead, which we alleviated by employing FaCT++ [89], a semantic reasoner implemented in C++, which makes the task of interfacing with Python much easier. We improved a Python interface for FaCT++ by wrobell,2

²

https://bitbucket.org/wrobell/factplusplus/src/factpp/factpp/

extending it with missing concept constructors, and addressing performance issues.

In the experiments dealing with OWL ontologies, initially we used Owlready2 [47] to parse OWL ontologies in the RDF/XML format. Eventually, to improve performance and avoid some problems with the parser, we switched to converting the ontologies to the OWL functional-style syntax [69] using the ROBOT command-line tool [40], and parsing them using a custom, high-performance parser implemented in Cython.

Both our parser and the extended version of the Python interface for FaCT++ are available at https://github.com/maxadamski/reasonable-embeddings/tree/main/src/simplefact, together with compilation instructions in the root of the repository.

6.3. Evaluation metrics

As the quality of the learned embeddings cannot be measured directly, we perform so-called extrinsic evaluation by computing classification metrics for a trained reasoner. We stipulate that if the classification metrics indicate good performance, then the embeddings learned by the embedding layer must capture the semantics of a given KB well. If that was not the case and the embeddings did not encode useful information for inference in the $\mathcal{ALC}$ DL, then the classifier should not be able to reliably classify entailment of axioms.

For simplicity, in threshold-sensitive metrics we choose a threshold of 0.5, i.e, $\hat{\mathit{y}}=1$ if $\mathit{P}\mathrm{(}\mathcal{K}\vDash \mathit{\alpha }\mathrm{)}⩾0.5$ , and $\hat{\mathit{y}}=0$ otherwise. Given that y is the expected class, TP is the number of true positives, i.e., the number of queries where $\hat{\mathit{y}}=\mathit{y}=1$ ; TN is the number of true negatives, i.e., the number of queries where $\hat{\mathit{y}}=\mathit{y}=0$ ; FP is the number of false positives, i.e., the number of queries where $\hat{\mathit{y}}=1$ and $\mathit{y}=0$ ; and FN is the number of false negatives, i.e., the number of queries where $\hat{\mathit{y}}=0$ and $\mathit{y}=1$ .

Our classifier learns binary classification, so we chose appropriate metrics [86]. Firstly, we include accuracy (Equation (16)) in the set of evaluation metrics. Because the data set that we generated for the experiments is slightly imbalanced we also compute precision (Equation (17)), recall (Equation (18)), and the F1-score (Equation (19)). $$\begin{array}{}\text{(16)}& \text{Accuracy}={\displaystyle \frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+\text{FN}}}\text{(17)}& \text{Precision}={\displaystyle \frac{\text{TP}}{\text{TP}+\text{FP}}}\text{(18)}& \text{Recall}=\text{TPR}={\displaystyle \frac{\text{TP}}{\text{TP}+\text{FN}}}\text{(19)}& \text{F1}={\displaystyle \frac{2·\text{Precision}·\text{Recall}}{\text{Precision}+\text{Recall}}}\end{array}$$

We also compute the values of the area under the ROC curve (AUC-ROC) and the area under the PR curve (AUC-PR), which are threshold-invariant metrics with a minimum value of 0 and a maximum value of 1. The ROC curve shows the performance of a classification model at all classification thresholds, by plotting the true positive rate (TPR; also called recall) against the false positive rate (FPR) (Equation (20)). Similarly, the PR curve shows the performance of a model at all thresholds by plotting precision against recall [79]. For both ROC and PR curves, a larger area under the curve suggests a better classifier [26,79]. $$\begin{array}{}\text{(20)}& \text{FPR}={\displaystyle \frac{\text{FP}}{\text{FP}+\text{TN}}}\end{array}$$

In experiments leveraging multiple KBs, we compute all metrics separately for each query set, and then compute the average and standard deviation across KBs. We do this because the KBs in the test set may be unequally difficult to classify, so it is beneficial to measure the variance of the reasoner performance across different KBs.

6.4. Data sets

6.4.1. Synthetic data set

Finding a large number of $\mathcal{ALC}$ KBs with a small number of concept and role names proved difficult, so for some experiments we generated a synthetic data set, and the code to reproduce it is available in the repository. It consists of 60 randomly generated KBs, such that the number of concept names $|{\mathit{N}}_{\mathit{C}}|$ for each KB is chosen randomly from the discrete uniform distribution $\mathcal{U}\{80\mathrm{,}120\}$ , and the number of role names $|{\mathit{N}}_{\mathit{R}}|$ is chosen randomly from the discrete uniform distribution $\mathcal{U}\{1\mathrm{,}5\}$ . The number of role names is very small, because we observed that the number of role names in real-world KBs is typically much smaller than the number of concept names. The actual concept and role names are simply ordinal numbers and are not considered in any way during learning.

For each KB, we also randomly choose the number of terminological axioms $|\mathcal{T}|$ to generate, from the normal distribution $\lfloor \mathcal{N}\mathrm{(}200\mathrm{,}10\mathrm{)}\rfloor$ . The axioms were generated according to the procedure that we described in Section 5, and we set the maximum depth of axioms ${\mathit{d}}_{max}$ to 3. The parameter ${\mathit{p}}_{\mathit{A}}$ is chosen randomly from the uniform distribution $\mathcal{U}[0.9\mathrm{,}1]$ for each KB. We discard and generate anew the axioms that would make the KB inconsistent, or make more than 10% of the named concepts unsatisfiable.

For each knowledge base $\mathcal{K}$ , we generate 2000 unique queries $\mathcal{K}\vDash \mathit{\alpha }$ , where α is a random subsumption axiom. The same parameters of the random axiom generator are used, as during generating terminological axioms for that KB. For each query, we label it with $\mathit{y}=1$ if the entailment holds according to FaCT++, and with $\mathit{y}=0$ otherwise.

We assign the first 40 KBs as the training set and the remaining 20 KBs as the test set. We then create a validation set from 20% of the queries from the training set. In total, there are 64,000 queries in the training set, 16,000 queries in the validation set, and 40,000 queries in the test set. In every data set, approximately 21.5% of queries have class $\mathit{y}=1$ and the remaining queries have class $\mathit{y}=0$ . This class imbalance does not significantly affect our experiments, so we do not use any methods for dealing with imbalanced data sets. We report detailed statistics of the data set in Appendix B.

6.4.2. Data set of real-world ontologies

Preprocessing We do the following pre-processing steps to make OWL ontologies compatible with our reasoner. Pre-processing is done automatically after parsing the ontology file, so one does not need to edit the ontology manually.

• We ignore object property axioms, since role axioms are not expressible in $\mathcal{ALC}$ (including role hierarchies, inverse roles, transitive roles, and (inverse) functional roles).

• We ignore axioms with number or value restrictions, since they are not expressible in $\mathcal{ALC}$ with KBs where ABox is empty.

• We ignore individuals and ABox axioms, since our reasoner does not support ABox reasoning.

• We remove roles that do not appear in any TBox axiom kept after pre-processing. This is done because of a limitation in FaCT++, which makes it raise an exception, when constructing a concept with an unused role name.

The pizza ontology We use the pizza ontology3

³

The pizza ontology in the RDF/XML format is available at http://owl.cs.manchester.ac.uk/publications/talks-and-tutorials/protg-owl-tutorial/.

from the Manchester University OWL Tutorial [38] We chose the pizza ontology, because it has a similar number of axioms, concept and role names to the randomly generated KBs from the synthetic data set.

The ontology contains 99 classes and 8 object properties, which are equivalent to concept names and role names, respectively. There are 15 equivalence axioms and 15 class disjointness axioms. Two of the classes are unsatisfiable.

The preprocessing is not without some influence on the ontology: InterestingPizza is equivalent to the top concept, because the axiom defining it as a pizza with at least 3 toppings contained a number restriction, so it was removed. The equivalence axiom that defines RealItalianPizza as a pizza with Italy as its country of origin was removed because it contained a value restriction. Since RealItalianPizza was also defined as a subclass of $\exists \mathtt{\text{hasBase}}.\mathtt{\text{ThinAndCrispyBase}}$ , any pizza with a thin and crispy base is inferred to be a subclass of RealItalianPizza. Value restrictions for American, AmericanHot, Napoletana, Veneziana, and MozzarellaTopping were removed, but that did not have a significant effect on reasoning.

CQ2SPARQLOWL To further test the capabilities of our architecture, we use the set of ontologies accompanying the CQ2SPARQLOWL data set4 ⁴

https://github.com/CQ2SPARQLOWL/Dataset

[73,96]. It consists of five ontologies:

• Software Ontology5 ⁵

  Path in the repository: Ontologies/swo_merged.owl.

  (SWO) with 4068 classes, 52 object properties and 7683 axioms [57]

• Stuff ontology6 ⁶

  Path in the repository: Ontologies/stuff.owl.

  (Stuff) with 193 classes, 57 object properties and 717 axioms [45]

• African Wildlife Ontology7 ⁷

  Path in the repository: Ontologies/AfricanWildlifeOntology1.owl.

  (AWO) with 31 classes, 5 object properties and 56 axioms [46]

• Dementia Ambient Care ontology8 ⁸

  Path in the repository: Ontologies/exchangemodel.owl.

  (Dem@Care) with 261 classes, 72 object properties and 695 axioms [18]

• Ontology of Datatypes9 ⁹

  Path in the repository: Ontologies/OntoDT.owl.

  (OntoDT) with 406 classes, 16 object properties and 947 axioms [67]

The ontologies were preprocessed according to the procedure described earlier.

6.5. Experiment 1 – topology induction on the synthetic data set

In this experiment, we consider RQ1: Can the reasoner induce a useful topology that it can subsequently exploit, or are the embeddings simply overfitting the KBs? To answer it, we designed a simple test to check whether a trained reasoner head actually learns to reason in the $\mathcal{ALC}$ description logic, and high classification metric values are not just the effect of embedding layers overfitting to KBs.

We test our architecture by first training the reasoner head and embedding layers on the training set (as usual), which results in a skillful reasoner. Then we create a no-skill reasoner head, that is not trained, but only randomly initialized. For each of the two reasoner heads, we train embedding layers on the test set, but keep the reasoner head weights frozen. After training the embedding layers on the test set for a fixed number of epochs, if the classification metrics for the reasoner with the trained head are significantly greater than for the reasoner with the random head, then the trained reasoner head learned useful relations in the $\mathcal{ALC}$ embedding space that generalize to new KBs. In short, the reasoner head is transferable.

Conversely, if the differences between classification metrics of both reasoner heads are not significant, then the trained reasoner actually has little or no skill and the only skill in classification comes from the embedding layer, which would mean that the reasoner head is not transferable.

6.5.1. Training details

The embedding dimension is set to ${\mathit{N}}_{\mathit{e}}=10$ , which is a much smaller number than $|{\mathit{N}}_{\mathit{C}}|$ . This forces the embeddings for different concept names to share embedding dimensions. As the loss for our classifier we use binary cross entropy. We create one AdamW optimizer [53] per reasoner head with learning rate set to ${\mathit{\eta }}_{\mathit{r}}=0.0001$ , and one AdamW optimizer per embedding layer with learning rate set to ${\mathit{\eta }}_{\mathit{e}}=0.0002$ . In total, for each reasoner variant, we create 1 optimizer for the reasoner head, and 60 optimizers for the embedding layers.

We train both reasoner variants with mini-batch gradient descent for 15 epochs, which was enough for the validation loss to stop decreasing. During testing, we train the embedding layers (while the reasoner head weights are frozen) for 10 epochs, as the test loss stabilized after that. We set the batch size to 32, as small batch sizes have been shown to improve generalization [58]. Unless stated otherwise, all weights are randomly initialized using the Xavier initialization [29].

6.5.2. Evaluation of reasoning ability

We evaluate the reasoning ability of the restricted variant of our reasoner by monitoring the training and validation loss and AUC-ROC values during training. The only goal of this assessment is to verify that the reasoner can effectively learn to classify entailment in the training set, and does not suffer from underfitting. Training and validation losses steadily decreasing, and validation AUC-ROC increasing with each training epoch, suggests that the reasoner architecture is sufficient for learning to classify entailment in a given data set.

Fig. 3.

Training and test progress of the restricted reasoner. The reported training loss for each epoch is the average mini-batch loss in that epoch. We also show the standard deviation of the mini-batch loss for each epoch. In test progress, the trained reasoner is shown in red, while the random reasoner is shown in blue. The reasoner was not trained during epoch 0 of training and testing – during epoch 0 we only compute the initial loss and metric values. Dashed curves show the AUC-ROC for each KB in the test set, while the thicker blue and red curves are the averaged AUC-ROC across KBs in the test set. Note that the random reasoner loss actually decreases, but at a very slow rate.

As shown in Fig. 3, the training loss decreases and validation AUC-ROC increases for the entirety of the training, with smaller gains after epoch 10. Furthermore, the validation loss does not start increasing in later epochs, which suggests that the restricted reasoner variant is not prone to overfitting.

6.5.3. Evaluation of knowledge transfer

After training the restricted reasoner, we froze the reasoner head, and trained embedding layers on the test set. We also trained separate embedding layers in conjunction with a randomly initialized, frozen reasoner head. The test loss decreased quickly for the reasoner with the trained head, while the loss for the random head decreased so slowly, that the test loss curve seems stationary on the plot. For the reasoner with the trained head, the test AUC-ROC quickly increased to almost 0.8 after epoch 2, and approached 1 after the last epoch.

Overall, training embedding layers for the reasoner with the trained head was much faster, than for the reasoner with the randomly initialized head, as the reasoner with the trained head achieved average AUC-ROC greater than 0.8 after epoch 2, while it took the reasoner with the random head 10 epochs to do the same. Furthermore, the trained head allowed the reasoner to achieve an average AUC-ROC close to 1 on the test set after 10 epochs, while the reasoner with the random head only achieved an average AUC-ROC of around 0.8.

The metrics after the last training epoch on the test data are reported in Table 2. The extremely high recall of the reasoner with the randomly initialized head is an artifact of it classifying most queries as class 1. Given the very low precision of the reasoner with the random head, its high recall should be ignored. The restricted reasoner with the trained head has lower variance of AUC-ROC and accuracy, than the reasoner with the random head. However, the restricted reasoner with the random head has lower variance for the F1-score, precision and recall. The trained reasoner outperforms the random reasoner on all measures except recall.

Table 2

Test set metrics for the restricted reasoner. Metric values were averaged across different KBs in the test set. In addition to averages, we standard deviation values are shown

Model	Accuracy	Precision	Recall	F1	AUC-ROC	AUC-PR
Trained head	$0.9610\pm 0.0148$	$0.9690\pm 0.0249$	$0.8469\pm 0.0446$	$0.9036\pm 0.0346$	$0.9838\pm 0.0086$	$0.9642\pm 0.0184$
Random head	$0.2873\pm 0.0275$	$0.2318\pm 0.0305$	$0.9870\pm 0.0120$	$0.3746\pm 0.0399$	$0.7656\pm 0.0242$	$0.5588\pm 0.0522$

6.5.4. Measuring the effect of embedding size and reasoner head width on performance

Table 3

Test set metrics for the restricted reasoners with varying embedding size ( ${\mathit{N}}_{\mathit{e}}$ ), and the number of neurons in the reasoner head (k). Metric values were averaged across different KBs in the test set

k	${\mathit{N}}_{\mathit{e}}$	Accuracy	Precision	Recall	F1	AUC-ROC	AUC-PR
1	1	$0.8168\pm 0.0220$	$0.9145\pm 0.0680$	$0.1661\pm 0.0666$	$0.2762\pm 0.1008$	$0.7372\pm 0.0298$	$0.5863\pm 0.0662$
16	1	$0.8731\pm 0.0255$	$0.8000\pm 0.0922$	$0.5514\pm 0.0848$	$0.6496\pm 0.0809$	$0.8751\pm 0.0303$	$0.7541\pm 0.0802$
1	10	$0.9559\pm 0.0133$	$0.9694\pm 0.0242$	$0.8212\pm 0.0447$	$0.8888\pm 0.0341$	$0.9788\pm 0.0095$	$0.9567\pm 0.0193$
16	10	$0.9611\pm 0.0149$	$0.9691\pm 0.0249$	$0.8470\pm 0.0447$	$0.9036\pm 0.0347$	$0.9838\pm 0.0086$	$0.9643\pm 0.0184$

To determine the effect of the embedding size and the number of neurons in the reasoner head we set the embedding size to 1 and/or the number of neurons to 1, and repeated the training procedure we described earlier. A reasoner with 1 neuron is a linear classifier and thus must rely on the embeddings to convey almost all the necessary information. Conversely, an embedding of size 1 is sufficient if all the necessary information is stored in the reasoner.

We report the results in Table 3. Following [41], we used repeated measures ANOVA on the AUC-ROC and obtained a p-value below 0.001, i.e., for at least one pair $\mathrm{(}\mathit{k}\mathrm{,}{\mathit{N}}_{\mathit{e}}\mathrm{)}$ there is a significant difference in the mean at the significance level $\mathit{\alpha }=0.01$ . We then performed pairwise comparisons using the paired t-test and obtained all p-values below 0.001, indicating the differences in AUC-ROC between every pair of rows in Table 3 are statistically significant at the family-wise error rate 0.01 with the Bonferroni correction.

The variant with $\mathit{k}=1$ , ${\mathit{N}}_{\mathit{e}}=1$ barely works, with accuracy on par with always predicting the positive class (recall the datasets are not balanced, with on average $21.5\mathrm{\%}$ of examples in the positive class). The second-worst variant with $\mathit{k}=16$ , ${\mathit{N}}_{\mathit{e}}=1$ is slightly better but is still characterized by low recall value. Both variants with ${\mathit{N}}_{\mathit{e}}=10$ perform much better than any of the variants for ${\mathit{N}}_{\mathit{e}}=1$ . Albeit there is still a difference in performance between them, it is, on average, rather small.

Answering RQ1, the presented results indicate that the reasoner indeed shapes the embedding space and generalizes well to previously unseen KBs, not storing any substantial amount of knowledge.

6.6. Experiment 2 – comparing restricted and relaxed reasoners

In this experiment we answer RQ2: Can concept constructors be shared between KBs, or must they be KB-specific? We followed the same protocol as in Experiment 1, but used the relaxed reasoner instead of the restricted reasoner.

6.6.1. Evaluation of reasoning ability

First, we trained the relaxed reasoner on the training set. The training progress for this variant is shown in Fig. 4. The training loss decreases and the validation AUC-ROC increases for the entirety of the training, with smaller gains after epoch 10. The validation loss decreases until epoch 10, and then starts increasing, which indicates overfitting.

When comparing Fig. 4 with Fig. 3, we observe that the AUC-ROC metric for the validation set increases slower for the relaxed reasoner than for the restricted reasoner. We attribute the faster convergence to the shared concept constructor networks ${\mathrm{NN}}_{\neg }$ , ${\mathrm{NN}}_{\sqcap }$ , which effectively have n times more training samples, where n is the number of training KBs, because they are trained on every sample from every KB, compared to the relaxed architecture, in which KB-specific concept constructor networks are trained on samples from one KB. As mentioned, the validation loss starts increasing in later epochs, suggesting that the relaxed reasoner is more prone to overfitting than the restricted reasoner.

6.6.2. Evaluation of knowledge transfer

Fig. 4.

Training and test progress of the relaxed reasoner. The reported training loss for each epoch is the average mini-batch loss in that epoch. We also show the standard deviation of the mini-batch loss for each epoch. In test progress, the trained reasoner is shown in red, while the random reasoner is shown in blue. The reasoner was not trained during epoch 0 of training and testing – during epoch 0 we only compute the initial loss and metric values. Dashed lines show the AUC-ROC for each KB in the test set, while the thicker blue and red curves are the averaged AUC-ROC across KBs in the test set.

After training the relaxed reasoner, we froze the reasoner head, and trained embedding layers on the test set. We also trained embedding layers in conjunction with a randomly initialized reasoner head. The test progress is shown on the right side of Fig. 4. At the beginning, the test loss for the trained reasoner head was higher than the test loss for the randomly initialized reasoner head. The test loss decreased quickly for the reasoner with the trained head, while the loss for the random head decreased very slowly. For the reasoner with the trained head, the test AUC-ROC quickly increased to about 0.7 after epoch 2, and approached 1 after the last epoch. The average test AUC-ROC for the random head slowly increased from around 0.5 in the beginning to around 0.8 after the last epoch.

Overall, training embedding layers for the relaxed reasoner with the trained head was much faster, than for the reasoner with the randomly initialized head. Moreover, the trained head allowed the reasoner to achieve an average AUC-ROC close to 1 on the test set after 10 epochs, while the reasoner with the random head only achieved an average AUC-ROC of around 0.8.

We report the final metrics of training on the test data in Table 4. For the relaxed architecture, the trained model is strictly better than the randomly initialized one, as all metric values are higher for the former. Moreover, the trained model has lower variance of metrics across KBs in the test set, than the random model. Similarly as for the restricted reasoner, the reasoner with trained head outperforms the reasoner with random head by a fair margin.

6.6.3. Comparative results

Comparing the metrics for the restricted reasoner in Table 2 and for the relaxed reasoner in Table 4, we observe that as expected, for both relaxed and restricted reasoners, the reasoners with trained heads achieved superior performance on the test set, which shows that both variants are indeed transferable.

Between the two random reasoners, the relaxed variant achieves better metric values, except recall, due to the very high recall of the restricted random reasoner. This was expected, since in the relaxed variant, the complement and intersection constructor networks can adjust to the randomly classifier to minimize the classification error, while in the restricted variant this is not possible.

Table 4

Test set metrics for the relaxed reasoner. Metric values were averaged across different KBs in the test set. In addition to averages, the standard deviation values are shown

Model	Accuracy	Precision	Recall	F1	AUC-ROC	AUC-PR
Trained head	$0.9585\pm 0.0138$	$0.9713\pm 0.0196$	$0.8332\pm 0.0488$	$0.8964\pm 0.0340$	$0.9836\pm 0.0086$	$0.9636\pm 0.0176$
Random head	$0.7755\pm 0.0463$	$0.4901\pm 0.1006$	$0.5494\pm 0.1363$	$0.5107\pm 0.0965$	$0.7898\pm 0.0583$	$0.6352\pm 0.1167$

When comparing the trained reasoners, the relaxed variant achieves slightly higher values with lower variance for all metrics (except precision, for which the restricted variant has slightly lower variance). However, the differences are not statistically significant. For each metric, we conducted a paired t-test, and checked whether the metric values for KBs from the test set, do not significantly differ between the relaxed and restricted reasoners. The resulting p-values, shown in Table 5, indicate that the null hypothesis cannot be rejected for any metric, even on a relatively high significance level $\mathit{\alpha }=0.05$ and not including corrections for the family-wise error rate.

Table 5

p-Values of the t statistic in a paired t-test for the test set metrics of a trained restricted reasoner and trained relaxed reasoner

	Accuracy	Precision	Recall	F1	AUC-ROC	AUC-PR
p	0.1798	0.4431	0.0801	0.1529	0.9236	0.8399

Disregarding the performance metrics, the restricted reasoner has an advantage over the restricted reasoner – it has fewer learnable parameters. Since one of the main goals of our reasoner is transfer learning, a lower number of parameters in the embedding layer is preferable, as it speeds up training for new KBs. The average embedding layer training time per epoch was approximately 21.72 seconds for the restricted variant, and 26.84 seconds per epoch for the relaxed variant, which is 23% slower.

Answering RQ2, introducing restrictions does not significantly change the reasoning performance, as measured by the classification metrics, but increases training efficiency.

6.7. Experiment 3 – visualizing concept embeddings from a real-world ontology

The previous experiments showed that our reasoner is capable of learning good concept embeddings for synthetic KBs, so the next step is to answer RQ3: Can the presented approach learn good embeddings for concepts from a real-world ontology? In this experiment we do that by training the neural reasoner to classify entailment, given randomly generated queries about the pizza ontology as the data set. We repeat this experiment three times. In the first run, we let the reasoner learn without any pre-training. In the second run, we initialize the reasoner head with weights of the restricted reasoner, that we trained in Experiment 1, then freeze it, and only allow the embedding layer to learn. In the third run, we randomly initialize the reasoner head, and also freeze it to only allow the embedding layer to learn KB-specific embedding.

6.7.1. Training set

The data set for this experiment consists of 32,000 unique random queries that we generate using the algorithm described in Section 5. We set the maximum axiom depth to ${\mathit{d}}_{max}=4$ , and the probability of concept names to ${\mathit{p}}_{\mathit{A}}=0.95$ . The answer to each query is obtained using FaCT++. We do not split the data set because we want the learned embeddings to fit the data set as well as possible. The data set is slightly imbalanced, with 41.11% of queries belonging to class 1, and the rest to class 0, although the class imbalance did not significantly affect classification performance.

6.7.2. Training procedure

As mentioned, we repeat learning three times, which results in three reasoners:

• Reasoner with unfrozen head – Both the reasoner head and embedding layers were trained.

• Reasoner with frozen pre-trained head – Only embedding layers were trained. The reasoner head was initialized with the weights of the restricted reasoner, that we obtained in Experiment 1.

• Reasoner with frozen random head – Only embedding layers were trained. The reasoner head was initialized randomly and its weights were immediately frozen.

In every run of the experiment we trained the model for 30 epochs, with learning rate set to ${\mathit{\eta }}_{\mathit{r}}={\mathit{\eta }}_{\mathit{e}}=0.001$ both for the optimizer of the reasoner head and the embedding layers. Similarly to the previous experiments, we use the AdamW optimizer and train the reasoners with batch size of 32. Because we initialize the frozen pre-trained reasoner head with weights of the restricted reasoner head, that we trained in Experiment 1, the embedding dimension ${\mathit{N}}_{\mathit{e}}$ and the layers of the neural network ${\mathrm{NN}}_{⊑}$ needed to be exactly the same as in that experiment. Thus, for all reasoners in this experiment we set ${\mathit{N}}_{\mathit{e}}=10$ and defined ${\mathrm{NN}}_{⊑}$ to be a neural network with one hidden layer with 16 neurons and the ELU activation function. The embedding dimension is adequate, since the pizza ontology has a similar number of concepts to the ontologies in the synthetic data set that we used in Experiment 1.

We expected the reasoner with the unfrozen head to achieve the best metric values and learn the best embeddings out of the three reasoners in the embedding analysis, because no weights are frozen, which means that the reasoner with the unfrozen head can fit to the data set more than the reasoners with frozen heads. The only obstacle to learning good embeddings for the reasoner with unfrozen head are the randomly generated queries, that may not contain useful entailments for the pizza ontology, although the other two reasoners learn with the same data set, so the comparison is at least fair.

Based on the results of Experiment 1, we expected the reasoner with transfer to achieve higher classification metrics than the reasoner with randomly initialized frozen head. In Experiment 1, the trained reasoner head was better at classifying queries for the unseen KBs from the test set, than the randomly initialized reasoner head, so we expected the same to be true for the pizza ontology.

6.7.3. Evaluation of reasoning ability

Table 6

Classification metrics for reasoners in Experiment 3 after training for 30 epochs

Reasoner head	Accuracy	Precision	Recall	F1	AUC-ROC	AUC-PR	Training time
Unfrozen	0.9929	0.9897	0.9930	0.9914	0.9996	0.9995	411.16 s
Frozen pre-trained	0.9527	0.9508	0.9331	0.9419	0.9859	0.9823	354.99 s
Frozen random	0.7733	0.6593	0.9283	0.7710	0.8932	0.8390	328.49 s

The classification metrics of the three reasoners are shown in Table 6. As expected, the unfrozen reasoner achieves strictly better classification performance than the reasoners with frozen heads, as the values of all metrics are higher than for other reasoners. The reasoner with the frozen pre-trained head is the second-best reasoner after the unfrozen reasoner, and is strictly better than the reasoner with the randomly initialized head.

The reasoner with the randomly initialized frozen head was the worst of the three reasoners, although it is better than a random guesser, with AUC-ROC of around 0.90. This reasoner has very high recall, but relatively low precision, which is reflected in the significantly lower F1-score and AUC-PR, when compared to the better reasoners. Even though the reasoner with the randomly initialized head was the worst of the three, it still achieved relatively high metric values, which shows that good embeddings can compensate for a bad reasoner head.

6.7.4. Evaluation of knowledge transfer

In the last section, we discussed the differences between the three reasoners that we trained in Experiment 3. Taking into account the differences between the reasoners with frozen heads, we conclude that the transfer of knowledge from the randomly generated KBs in Experiment 1 to the pizza ontology was a success. The reasoner with the pre-trained head achieved results similar to the unfrozen reasoner, which is very promising, given that the pizza ontology is certainly different from the randomly generated KBs used as the training set in Experiment 1.

It should be mentioned that the total training time of the reasoners with frozen heads is shorter than the training time of the unfrozen reasoner, which is the case because there is no time spent on updating the reasoner head weights.

6.7.5. Embedding analysis

In addition to evaluating the reasoning and transfer ability of our reasoner, which showed that it can successfully learn to classify entailment in a real-world KB, we visualize the learned embeddings. A 2D visualization that tries to preserve the distances between concepts in the high-dimensional embedding space enables visual assessment of the quality of the learned embeddings. We think that if semantically similar concepts are placed closer to each other than to dissimilar concepts, then the learned embeddings capture the semantics of a given KB.

We set the concept embedding dimension to ${\mathit{N}}_{\mathit{e}}=10$ , so to visualize the embeddings we first needed to reduce their dimensionality. We used Uniform Manifold Approximation and Projection for Dimension Reduction (UMAP),10

¹⁰

https://github.com/lmcinnes/umap

because it is a nonlinear dimensionality reduction algorithm with high performance and interpretable hyperparameters [60], that we set as follows: n_neighbors to 50 to emphasize the global structure of the data, and set min_dist by starting at a large value and decreasing it until clusters started to separate and the visualization became readable, which occurred when min_dist was set to 0.3.

The UMAP visualizations of the embeddings learned in this experiment, are shown in Fig. 5. The visualization of embeddings learned by the unfrozen reasoner suggests, that the embeddings capture the semantics of the pizza ontology well. As expected, the unsatisfiable CheesyVegetableTopping is close to ⊥, and the general DomainThing and Food concepts are close to ⊤. Spiciness and pizza bases form their own clusters. Pizzas and toppings are separated, with vegetarian pizzas forming one cluster, and non-vegetarian pizzas forming another cluster together with spicy pizzas. Cheesy toppings, vegetable toppings, spicy toppings, and pepper toppings are also close to related concepts.

The embeddings learned by the reasoner with the pre-trained head look a bit worse than those of the unfrozen reasoner. The pizzas and the toppings are separated, but inside of the topping and pizza clusters the embeddings are not as well organized.

The embeddings learned by the reasoner with the randomly initialized head do not look good when visualized. A few concept names form clusters that make sense, but most look like they are randomly scattered. General concepts are close to the top concept, which is good, but the unsatisfiable CheesyVegetableTopping is far away from the bottom concept.

In general, we think that the embeddings for the unfrozen reasoner are the best, the embeddings for the reasoner with the frozen pre-trained head are good, and the embeddings for the reasoner with the frozen randomly initialized head are not good at all. Again, the results of the visual assessment of the learned embeddings are consistent with the classification metrics of the reasoners.

Overall, the presented results indicate that the answer to RQ3 is affirmative, and that the presented approach can learn embeddings of good quality.

Fig. 5.

UMAP visualization of the learned concept embeddings. We replaced “Topping” in concept names with “T” to improve readability. We use different shapes, sizes, and colors of markers as follows: by default concepts are square and gray; the top concept, bottom concept and concept expressions are cyan; toppings diamond-shaped, and pizzas are round; vegetarian pizzas and toppings are light green, except vegetable toppings, which are dark green; seafood toppings are pink; non-vegetarian pizzas and meat toppings are dark red; cheesy pizzas, and cheese toppings are respectively marked with a yellow disk or yellow diamond inside; pepper toppings are marked with an orange diamond inside; spicy things are marked with a red “x” inside; spiciness levels are light red.

6.8. Experiment 4 – learning concept embeddings in different real-world ontologies

In Experiment 3 we found that a reasoner with a frozen pre-trained head achieves similar classification metrics to a reasoner trained from scratch. To see whether this result holds for a more diverse set of ontologies we posed RQ4: Is transfer learning with the presented approach viable for a variety of real-world ontologies?

6.8.1. Training set

In this experiment we use six real-world ontologies: five ontologies from the CQ2SPARQLOWL data set, and the pizza ontology. For each ontology, we generate 32,000 unique random queries according to the algorithm described in Section 5. We set the maximum axiom depth to ${\mathit{d}}_{max}=4$ , and the probability of concept names to ${\mathit{p}}_{\mathit{A}}=0.95$ . The answer to each query is obtained using FaCT++. Similarly to Experiment 3, we do not split the data set because we want the learned embeddings to fit the data set as well as possible. Class imbalance depends on the ontology, ranging from only 4.24% of the Dem@Care ontology queries belonging to class 1, to 41.11% of the pizza ontology queries in class 1. We reused queries for the pizza ontology from Experiment 3.

6.8.2. Training procedure

Similarly to the Experiment 3, we repeat learning three times, which results in three reasoner heads, each with six embedding layers.

• Reasoner with unfrozen head – Both the reasoner head and embedding layers were trained.

• Reasoner with frozen pre-trained head – Only embedding layers were trained. The reasoner head was initialized with the weights of the restricted reasoner, that we obtained in Experiment 1.

• Reasoner with frozen random head – Only embedding layers were trained. The reasoner head was initialized randomly.

We use the exact reasoner architecture and training setup as in Experiment 3, but in this experiment each mini-batch could contain queries for any of the six ontologies, as the embedding layers were trained in parallel.

6.8.3. Results

The metric values for the pizza ontology are slightly different from the ones in Experiment 3, as this time, the reasoner head was trained on six ontologies at once, in contrast to exclusively training on the pizza ontology. We report the classification metrics for each reasoner head, averaged over all ontologies, in Table 7, and we give the values for each ontology separately in Appendix A.

Table 7

Classification metrics for reasoners for the AWO, Dem@Care, Stuff, SWO, OntoDT, and Pizza ontologies. Values were averaged over all ontologies

Reasoner head	Accuracy	Precision	Recall	F1	AUC-ROC	AUC-PR	Train time
Unfrozen	$0.9766\pm 0.0135$	$0.9425\pm 0.0489$	$0.9017\pm 0.0772$	$0.9207\pm 0.0574$	$0.9907\pm 0.0076$	$0.9637\pm 0.0343$	27 min 18 s
Frozen pre-trained	$0.9600\pm 0.0219$	$0.9121\pm 0.0605$	$0.8449\pm 0.1122$	$0.8755\pm 0.0838$	$0.9776\pm 0.0156$	$0.9284\pm 0.0566$	24 min 6 s
Frozen random	$0.7789\pm 0.0468$	$0.3672\pm 0.2526$	$0.4371\pm 0.1679$	$0.3879\pm 0.2198$	$0.7230\pm 0.0819$	$0.3460\pm 0.2497$	24 min 5 s

In Table 8 we report the p-values obtained from a paired t-test between the unfrozen head, and the frozen pre-trained head. As some of the p-values hint on the possibility of statistical significance, we also report the adjusted p-values $\tilde{\mathit{p}}$ , obtained using the Holm-Bonferroni method. After adjustment, on the significance level $\mathit{\alpha }=0.05$ , no difference is statistically significant.

The results indicate that using a pre-trained head is a viable choice for real-world ontologies, as the performance drop is negligible, and thus the answer for RQ4 is positive.

Table 8

p-Values of the t statistic in a paired t-test for the test set metrics of a reasoner with an unfrozen head and a frozen pre-trained head. Adjusted p-values $\tilde{\mathit{p}}$ were obtained using the Holm–Bonferroni method

	Accuracy	Precision	Recall	F1	AUC-ROC	AUC-PR
p	0.0338	0.0432	0.0319	0.0277	0.0208	0.0335
$\tilde{\mathit{p}}$	0.1383	0.1383	0.1383	0.1383	0.1244	0.1383

6.9. Experiment 5 – what is difficult for the reasoner?

One can wonder what types of axioms are particularly hard for the reasoner. To answer this, we started with the reasoner and the test set from Experiment 1. Then, for each query in the test set, we computed the following features:

${\mathit{n}}_{\perp }$ – the number of occurrences of the bottom concept in the query;

${\mathit{n}}_{\mathit{C}}$ – the number of different concept names in the query;

${\mathit{n}}_{\mathit{R}}$ – the number of different role names in the query;

${\mathit{n}}_{\neg }$ – the number of negations ¬ in the query;

${\mathit{n}}_{\sqcap }$ – the number of conjunctions ⊓ in the query;

${\mathit{n}}_{\exists }$ – the number of existential restrictions ∃ in the query;

d – the depth of the syntax tree of the query (i.e., the recursion depth d as defined in Section 5);

Next, we computed the Pearson correlation coefficient between each feature and the classification error of the reasoner (before thresholding). The absolute values of the obtained coefficients (Table 9) were all below 0.05, indicating that none of the features have a significant impact on the performance of the reasoner.

Table 9

The Pearson correlation coefficient between each feature and the classification error of the reasoner

${\mathit{n}}_{\perp }$	${\mathit{n}}_{\neg }$	${\mathit{n}}_{\sqcap }$	${\mathit{n}}_{\exists }$	${\mathit{n}}_{\mathit{C}}$	${\mathit{n}}_{\mathit{R}}$	d
−0.0106	−0.0051	−0.0062	−0.0015	0.0282	0.0004	−0.0065

Currently, we do not have a decisive answer to RQ5. There do not seem to be any obvious indicators of hardness for a query. We stipulate it might be possible to detect more complex patterns by using, e.g., an adversarial attack [32], or frequent pattern mining [48,72]. However, we deem this to be out of the scope of this work.

7. Conclusions

In this work, we introduced a novel method of learning data-driven concept embeddings, called reason-able embeddings, in $\mathcal{ALC}$ knowledge bases. To our best knowledge, our method of learning concept embeddings is the first one using an entailment classifier based on neural networks. Thanks to of our unique approach, after learning embeddings, one can use the resulting classifier to perform fast approximate reasoning.

We also show that using recursive neural networks for constructing embeddings of arbitrarily complex concepts obviates the need for manually designing concept vectorization schemes, and avoids the pitfalls of recurrent neural networks operating on a textual representations of concepts. Instead, concept embeddings can be learned in a data-driven way, by simply asking entailment queries for a given knowledge base.

Finally, we show that a significant part of our reasoner is transferable across knowledge bases in the $\mathcal{ALC}$ description logic, including real-world knowledge bases like the pizza ontology, or the ontologies accompanying the CQ2SPARQLOWL data set. The advantage of a transferable reasoner is that learning concept embeddings takes less time, and thus is less expensive, when using a pre-trained reasoner head, compared to training an entire reasoner from scratch. Promising results in the transferability of neural reasoners suggest that it is indeed feasible to embed concepts from multiple domains in a single shared embedding space.

We hope that our neural reasoner architecture will allow for greater use of knowledge in models based on neural networks, both by providing an effective way of learning concept embeddings, and learning an accurate entailment classifier for knowledge bases in description logics, thus, making a small step towards the integration of the neural and symbolic paradigms in artificial intelligence.

We identified many opportunities to improve, extend and apply our neural reasoner, that were out-of-scope for this work, but look like promising avenues for future research. In our work we used small neural networks, but deeper and wider concept constructor networks, and subsumption entailment classifier networks could be examined. The number of parameters in the reasoner could also be reduced, while preserving the quality of embeddings and accuracy of entailment classification. Currently, the number of parameters scales quadratically with the embedding dimension, because the reasoner uses the outer product of embeddings as an input to neural networks ${\mathrm{NN}}_{⊑}$ and ${\mathrm{NN}}_{\sqcap }$ . Instead of passing the entire interaction map as input, convolutional neural networks could be used, in turn greatly reducing the number of parameters. The number of parameters used by existential restriction constructor networks could possibly be reduced by representing weights as sparse matrices.

Finally, it would be interesting to see if recursive neural networks could be applied in reverse to how we use them – to generate concepts, given learned concept embeddings. That would make it possible to not only learn concept embeddings by classifying entailment, but also to induce new concepts by sampling the embedding space, e.g., to construct a scalable algorithm for explainable artificial intelligence [81].