Sage Journals: Discover world-class research

Abstract

Relation prediction in knowledge graphs (KGs) aims to anticipate the connections between entities. While both transductive and inductive models are incorporated for context comprehension, we need to focus on two primary issues. First, these models only collate relations at each layer of the subgraph, overlooking the potential sequential relationship between different layers. Second, these methods overlook the homogeneity of subgraphs, thus impeding their ability to effectively learn the importance of relationships within the subgraphs. To address this challenge, we propose a hierarchical and homogenous subgraph learning model for KG relation prediction (HiHo). Specifically, we adopt a subgraph-to-sequence mechanism to learn the potential semantic associations between layers in the subgraph of a single entity, and thus model the hierarchy of the subgraph. Then, we implement a common preference inference mechanism that assigns higher weights to co-occurrence relations while learning the importance of each relation in the subgraphs of two entities, and thus models the homogeneity of the subgraph. In our study, we sequentially employ induction on each layer of subgraphs pertaining to the two entities for relation prediction. To assess the efficacy of our method, we perform experiments on five publicly available datasets. The results of our experiments demonstrate that our method surpasses the current state-of-the-art baselines in both transductive and inductive settings.

Keywords

knowledge graph relation prediction inductive method

1. Introduction

Knowledge graphs (KGs) are structured collections of facts represented as triples (head entity, relation, and tail entity), playing a fundamental role in numerous natural language processing tasks, such as multihop query (Shang et al., 2024), question answering (Yang et al., 2023), recommendation systems (Wang et al., 2023a), and commonsense reasoning (Wu et al., 2023). Despite the success, most of the KGs often suffer from incompleteness, especially the missing relations between entities. Therefore, an increasing number of researchers are committed to predicting the missing relations between entities (Shi et al., 2023), and many different methods have been proposed. These methods can be roughly categorized into transductive methods and inductive methods.

Transductive methods refer to the traditional KG embedding methods, which can be divided into translation-based methods (e.g., TransE [Bordes et al., 2013] and TransR [Lin et al., 2015]), semantic matching methods (e.g., ComplEx [Trouillon et al., 2016] and DistMult [Yang et al., 2015]), path-based methods (e.g., CURL [Zhang et al., 2022a] and CPL [Fu et al., 2019]), and graph-based methods (e.g., TransEQ [Liu et al., 2023] and NoGE [Nguyen et al., 2022]). Despite their different forms, transductive methods mostly focus on learning a matching score for the relations of known inherent entities in KG. Many transductive methods have proven to be effective in predicting relations between entities, but they need expensive re-training to infer the relations between unknown new entities, which is impractical for real-world KGs that are constantly adding new entities.

Different from transductive methods, induction-based methods aim at learning the subgraph structure of entities to infer the relations between entities in KGs, where entities may be unseen in the training process. For instance, Grail (Teru et al., 2020) is a framework based on graph neural networks (GNNs) with strong inductive bias, which can learn relationship semantics unrelated to entities. Grail does not learn embeddings of specific entities, but predicts relationships from subgraph structures around candidate relationships. SNRI (Subgraph Neural Relational Inference) (Xu et al., 2022) incorporates complete relational information into the enclosing sparse subgraph in a global way for relation prediction. TACT (Topology-Aware Correlation Transfer) (Chen et al., 2021) predicts the missing link by learning the topology-aware semantic correlations between relations in an entity-independent manner. However, these methods only collect relations at each layer of the subgraph individually, without considering the back-and-forth associations of relations in different layers, so they lose the global nature of the relations. Furthermore, they approximate the commonalities between subgraphs by simply extracting their intersection, and they prune nodes that are isolated or located at a distance greater than a specified threshold (denoted as ”k”) from either of the target entities. But sometimes the intersection occupies only a small portion of the two subgraphs, which leads to many relations in the subgraphs being ignored.

To solve the above problem, we first reviewed the subgraphs of entities. Through our observation, we found that the subgraph of an entity consists of the neighboring entities and relations surrounding the entity. These neighboring entities and neighboring relations exist in the form of layers, and any level of the subgraph does not exist in isolation. For the $i$ th layer of the subgraph, both its forward and backward layers can provide potential information for it. The assumptions we are based on are that the process of human understanding things usually includes understanding the history of things and predicting their future. So for machines, they should cognize the history and future information of the $i$ th layer of the subgraph to model it. Therefore, we assume that combining the back-and-forth association of each layer of the subgraph should be helpful for relation prediction. Furthermore, we found that the intersection of the subgraphs of the head and tail entity, that is, the co-occurrence relations in the two subgraphs, can provide valuable information for predicting the relations that are connected to both the head and tail entity. These co-occurrence relations are more likely to be true relations than other relations, so they should be given higher weight. We operate under the assumption that the higher the frequency of a relation (denoted as $r_{i}$ ) appearing in the intersection of the subgraphs of the head and tail entities, the more likely it is that $r_{i}$ is simultaneously associated with both entities. Consequently, $r_{i}$ is more likely to represent the true relation between the head and tail entities and should thus be given greater preference. However, it is essential to recognize the significance of other relations as well.

To implement the above process, we propose a subgraph-to-sequence (S2S) mechanism that converts the subgraphs of an entity into a sequence composed of layers, specifically we choose bidirectional gated recurrent unit (Bi-GRU) to encode the history and future information for the layers in the sequence to model the hierarchical nature of the subgraphs. Furthermore, we propose a common preference inference (CPI) mechanism that assigns higher weights to co-occurrence relations while learning the importance of each relation to both subgraphs. Also, to prevent over-preference for certain relations, a smoothing factor is added to CPI to balance the preferred and unpreferred relations. In this way, each neighbor relation is assigned a weight to indicate the importance of the relation in the two subgraphs. Finally, we alternately induct each layer of the subgraph of the head entity and the tail entity to predict the relation between them. In summary, our main contributions are as follows.

We introduce an S2S mechanism to capture the hierarchical structure within the subgraph of an individual entity. This mechanism facilitates the understanding of the entity’s internal relationships and context.

We propose a CPI mechanism to address the homogeneity within the subgraphs of the head and tail entities. CPI aims to model and leverage the shared characteristics present in both subgraphs, enhancing the predictive accuracy of relations between entities.

We propose HiHo to predict the relations between entities by modeling the hierarchy and homogeneity of the subgraphs of entities. In this way, HiHo avoids learning to embed any specific entity, so it can predict the relations between emerging or unseen entities in the reasoning phase.

2. Related Work

Recently, researchers have focused on predicting relationships within KGs because it is essential for many tasks. As a result, more scholars are now studying this area. Existing KG relation prediction methods fall into two main categories: transductive and inductive methods.

2.1. Transductive Method

Transductive methods primarily concentrate on learning representations of entities and relations, employing a score function to determine the likelihood of a relation being associated with an entity (Li et al., 2023). These methods encompass various categories, and the most popular ones are translation-based methods, semantic matching methods, path-based methods, GNN-based methods, and graph-based methods.

Translation-based methods, such as TransE (Bordes et al., 2013), TransR (Lin et al., 2015), TransH (Wang et al., 2014), ExpressivE (Pavlovic & Sallinger, 2023), and ConvRot (Le et al., 2023) usually project entities and relations into the low-dimensional feature spaces, and assume that the distance between the embeddings of the tail entity and head entity is approximately equal to the relation embedding. Despite their simplicity and efficiency, they ignore the features in the multilayered neighborhoods of entities, which limits their ability to infer relations between entities.

Semantic matching methods, such as ComplEx (Trouillon et al., 2016), DistMult (Yang et al., 2015), QuatE (Zhang et al., 2019), and DRUM (Sadeghian et al., 2019) employ a scoring function grounded in similarity rules, building upon the framework of translation-based methods. Through this approach, they gauge the likelihood of connections between relations and entities by aligning the semantic associations of entities and relations within the embedded vector space. Nonetheless, their effectiveness heavily relies on the configuration of rules, and certain missing relations may remain uninferrable through any rule (Lin et al., 2021).

Path-based methods, such as P-INT (Xu et al., 2021), AstarNet (Zhu et al., 2022), HiAM (Ma et al., 2021), CURL (Zhang et al., 2022a), and CPL (Fu et al., 2019a), usually define a path as the sequence of relations and select relevant features from a path, then model the interactions between the paths to infer new relations between entities. However, in many KGs, the length and number of paths are not balanced, and some paths that are too much, too little, too long, or too short can affect the performance of relation reasoning.

Graph-based methods usually design a GNN to learn the subgraph structure of entities and obtain the representation of entities. On the basis of GAT (Velickovic et al., 2018), EIGAT (Zhao et al., 2022), and GGAE (Li et al., 2022) assign distinct attention weights to the neighbor entities or the neighbor relations of the central entity to obtain entity representation. And on the basis of GCN (Kipf & Welling, 2017), CompGCN (Vashishth et al., 2020), KE-GCN (Yu et al., 2021), and DA-GCN (Zhang et al., 2022b) models, the node or relationship features in the full graph are processed by convolution operations to obtain the entity representation. Although their implementation details are different, they all aggregate the neighboring features of nodes into the node representation (Wang et al., 2023b), which is the reference for relation prediction.

However, these methods can only model known entities and predict relations between known entities during training. Therefore, they require expensive re-training to model the constantly added unknown entities in KGs, and to infer the relations between unknown entities and the relations between unknown and known entities, which makes them difficult to apply to the ever-changing KGs.

2.2. Inductive Method

Inductive methods usually infer relations between entities from the subgraphs around entities, avoiding learning to embed any specific entity. They can be naturally generalized to unseen entities or emerging entities in KG, as they learn to reason over subgraph structures independent of any specific entity representation. Recently, inductive methods have been receiving more and more attention. They are classified into rule-based and subgraph-based methods.

Rule-based methods involve the explicit learning of logical rules for reasoning, which are independent of entities and thus considered inductive. Yang et al. (2017) introduced NeuralLP, the first end-to-end differentiable model inspired by TensorLog and neural networks, to learn variable rule lengths. This model merges first-order rule reasoning with sparse matrix multiplication and introduces a neural controller system featuring attention mechanisms and memory, enabling simultaneous learning of the parameters and structure of first-order logical rules. However, this model is constrained by the maximum length of rules and may inadvertently extract incorrect rules with high confidence. Sadeghian et al. (2019) improved NeuralLP with the introduction of Drum, where the learning confidence scores for each rule are related to the low-rank tensor approximation. They use BiRNN to share useful information across different relation learning tasks. However, they often overlook the structure surrounding the target triple, leading to limited expressive ability.

The method based on subgraphs can utilize structural information for inductive reasoning. GraIL (Teru et al., 2020) is the first method to learn entity-independent relational semantics to predict the relation between two entities by reasoning about the local subgraph structure around the two entities. TACT (Chen et al., 2021) transforms a KG to a relational correlation graph, and proposes a relational correlation network to learn topology-aware correlations between relations in entities’ neighborhoods in an entity-independent manner, to predict the missing relation. CFAG (Wang et al., 2022) utilizes a coarse-grained module to generalize the unseen entities with multiple relational semantics, then uses a fine-grained module to generate more accurate entity representations with certain query relations. MorsE (Chen et al., 2022) utilizes an entity initializer to generate each entity’s initial embedding through relation-domain embedding settings and relation-range embedding settings, then learns the neighbor structure of the entity to enhance entity embedding by a GNN modulator. Next, MorsE resorts to meta-learning to output an entity’s final embeddings. These methods model the rules between relations to generate relation preferences for entities. PathCon (Wang et al., 2021) proposes a multilayer relational message passing mechanism, which iteratively aggregates the relational context features of entities and the relational path features between entities to predict missing relations. SNRI (Xu et al., 2022) extracts complete neighboring relations for each entity and constructs neighboring relational paths by the mutual information maximization mechanism. By combining the above two approaches, SNRI can effectively integrate the comprehensive relational information into the subgraph of entities to improve the performance of relation prediction. StATIK (Markowitz et al., 2022) aggregates structure information from the neighborhoods of entities through a message passing neural network, then completes the missing relations by combining underlying textual descriptions of entities and relations through a pretrained language model. RMPI (Geng et al., 2023) utilizes novel techniques such as target relation-guided graph pruning strategies, target relation-aware neighborhood attention, handling empty subgraphs, and ontology-based relationship semantic injection to perform relation prediction through relation message passing. These methods are not simply graph-based iterative methods; they all choose to combine other methods to obtain a better relation prediction effect.

These methods infer the relation between two entities by generalizing the subgraphs around those two entities. However, these methods neglect the heterogeneity and hierarchy natures of subgraphs, which limits their performance in relation prediction.

3. Problem Definition

Definition 1 (KG)

$G (E, R) = {(h, r, t) ∣ h, t \in E, r \in R}$ . $E$ represents a set of entities, $R$ represents a set of relations, and $E \times R \times E$ represents a set of fact triplets. A fact triplet is represented as $(h, r, t)$ , where $h$ denotes the head entity, $t$ denotes the tail entity, and $r$ denotes the relationship between the head and tail entities. A fact triplet such as (Heartbleed Vulnerability exists in OpenSSL Program), where “Heartbleed Vulnerability,” “exists in,” and “OpenSSL Program” are, respectively, the head entity, relationship, and tail entity, represents the fact that the Heartbleed Vulnerability exists in the OpenSSL program.

Definition 2 (Inductive KG)

An inductive KG, $G^{'} (E^{'}, R)$ , consists of a set of invisible entities, $E^{'}$ , and a set of visible relations, $R$ , which are shared with the original KG. $G^{'} (E^{'}, R) = {(h, r, t) ∣ h, t \in E^{'}, r \in R}$ . $E^{'} \cap E = \emptyset$ , where $E^{'}$ represents a set of invisible entities.

Definition 3 (KG Relation Prediction)

KG relation prediction aims to train models to predict missing relationships given a triplet $(h, ?, t)$ .

Definition 4 (Transductive Relation Prediction)

Transductive relation prediction is defined as predicting missing triplets: $F^{1} = {(h, r, t) ∣ (h, t \in E, r \in R)}$ .

Definition 5 (Inductive Relation Prediction)

In the inductive setting, relation prediction aims to quantify the score of each relation $r$ in the graph $G$ and predict the relationship between two invisible entities $h$ and $t$ in the test data. Inductive relation prediction is defined as predicting the missing triplet $F^{2} = {(h, r, t) ∣ h, t \in E \cup E^{″}, r \in R \cup R^{″}}$ , where $E^{″}$ represents a set of invisible entities, and $R^{″}$ denotes a set of hidden relations. As shown in Figure 1, in the inductive relation prediction of KG, the triplet (invisible head entity: Chrome browser; visible relationship: runs on; invisible tail entity: Windows operating system) in the emerging KG shares the visible relationship “runs on” with the triplet (visible head entity: OpenSSL program; visible relationship: runs on; visible tail entity: Linux operating system) in the original KG. However, all triplets in the original KG are visible, while entities in the emerging KG are all invisible.

Figure 1.

Illustration of the Transductive and Inductive Settings for Relation Prediction in Knowledge Graphs (KGs). The Transductive Relation Prediction Method Can Only Identify Known Entities, While Inductive Relation Prediction is Learned From the Training KG to Infer in Another KG Without Shared Entities. Green Entities Represent Known Entities During Training, Red Entities Represent Newly Emerged Entities, and the Red Dashed Lines Indicate the Relationships to be Predicted.

4. Proposed Method

For a given entity pair ( $h$ , $t$ ), where $h$ is the head entity and $t$ is the tail entity. When inferring the relation $r$ between $h$ and $t$ , the subgraph around $h$ and the subgraph around $t$ are very important. They can reveal the properties of $h$ and $t$ (Wang et al., 2021), and provide logical evidence to infer the relationship type between $h$ and $t$ (Teru et al., 2020). However, there are a large number of discrete atomic symbols in these subgraphs, so it is almost impossible to directly model them (Xiong et al., 2017).

To address this issue, we propose a new inductive relation prediction method. Specifically, we design S2S to infer the hierarchical information of a single subgraph. The GRU processes the state sequences within the S2S mechanism, allowing the model to capture the contextual relationships between the states and thus obtain the global characteristics of the subgraph. Then, we propose a CPI mechanism to learn the homogenous information between two subgraphs, and reduce the differences between subgraphs by strengthening their common parts, so as to obtain the local characteristics of subgraphs. The CPI assigns different weights to relations. This weighted information from the CPI is then used as part of the input features when the GRU in the S2S processes the state sequences. The GRU, influenced by the CPI’s weighted relations, further refines its capture of the hierarchical information within the subgraph layers. Moreover, this paper proposes an alternate induction method, which collects the relations in the subgraph of a head entity and the subgraph of a tail entity alternately to infer the relation between entities.

4.1. Subgraph Preparation

Unlike traditional methods that simultaneously collect entities and relations from the neighborhoods of entities into subgraphs, we only collect relations from each entity’s neighborhood into the subgraph. However, the relations in the neighborhood of some entities are too dense to be fully used. For example, we observe that there are thousands of relations in the third hop neighborhoods of some entities in the NELL995 dataset, while the first hop neighborhoods of some entities in some real-world KGs may encompass many thousands of relations, so it is impractical to store and model them all.

The solution to this problem is that, for entity $e$ , we first initialize its subgraph to be empty. Then, starting from the first hop neighborhood of entity $e$ , we sample up to $K$ neighbor relations as the first layer of the subgraph, and add them to the subgraph. Next, for the entities connected with these neighbor relations, we sample up to $K$ neighbor relations of each entity as the second layer of the subgraph and add them to the subgraph, and so on, lasting for $L$ times. Eventually, each entity’s subgraph contains no more than $K^{L}$ neighbor relations. Based on this, for entity $e$ , we obtain its subgraph $N_{r} (e) = {N_{r}^{1} (e), N_{r}^{2} (e), \dots, N_{r}^{L} (e)}$ .

4.2. S2S Mechanism

The subgraph of the entity contains a vast number of relations. These relations exist in the form of layers, and any layer of the subgraph does not exist in isolation. Taking Figure 2 as an example, for the entity “Heartbleed Vulnerability,” each layer of its subgraph does not exist independently and has a semantic association with other layers. For instance, the first layer of “Heartbleed Vulnerability” contains the relation “exists in,” and we can infer that the type of the unknown tail entity 1 connected to “Heartbleed Vulnerability” may be “affects.” And the second layer contains the relation “runs on,” and we can infer that the type of the unknown tail entity 2 may be a larger range of location. By learning the semantic association between the first layer and the second layer in the subgraph, it can be inferred that the relation between “Heartbleed Vulnerability” and the unknown tail entity 2 may be “affects.” Thus, modeling the hierarchical nature of subgraphs, that is, learning the potential semantic associations between layers in individual subgraphs, can generate more accurate relation tendencies for entities.

Figure 2.

The Framework Diagram of the Proposed Method, HiHo, Comprises Four Components: (1) Subgraph Preparation Selectively Samples up to K Neighbor Relations From Each Entity's Neighborhood into the Subgraph; (2) S2Smechanism Used to Capture the Semantic Associations Between Layers Within the Subgraph of a Single Entity and Achieve Hierarchy; (3) CPI Mechanism Which Assigns Higher Weights to Co-occurrence Relations and Learns the Importance of Each Relation in the Subgraphs of Two Entities; and (4) Alternating Induction Mechanism That Learns the Subgraphs of Two Entities to Predict Relations Between Them. S2S: Subgraph-to-sequence; CPI: Common Preference Inference.

Similar to Graph2Seq (Xu et al., 2019), we regard the subgraph $N_{r} (e)$ as a sequence consisting of $N_{r}^{i} (e),$ where $i \in [1, L]$ . Here, “ $e$ ” represents the entity for which we are constructing and analyzing the subgraph. In the multihop subgraph sequence related to the entity, the forward layers $N_{r}^{i - k} (e)$ (the layers that come before a certain point in the sequence) provide historical information, while the backward layers $N_{r}^{i + k} (e)$ (the layers that come after that point) supply future information.

Next, we opted to use the Bi-GRU, which is a time series prediction method based on the gated recurrent unit (GRU). It combines the bidirectional model and the gating mechanism. The overall structure and the cell structure are consistent with those of GRU, thus enabling it to effectively capture the temporal relationships in time series data. The overall structure of Bi-GRU consists of two GRU networks in opposite directions. One network processes the time series data from the front to the back, and the other network processes it from the back to the front. This bidirectional structure can simultaneously capture both past and future information, thereby more comprehensively modeling the temporal relationships in the time series data. We use Bi-GRU to encode the features from the previous and later parts of the sequence for each state. This enables us to capture the contextual relationships between states, which are equivalent to the hierarchical information present between each layer of the subgraph. The output of the Bi-GRU can be described by equations (1) and (2):

\begin{aligned} \vec{e_{r}^{i}} & = GRU (\partial [N_{r}^{1} (e)], \partial [N_{r}^{2} (e)], \dots, \partial [N_{r}^{i} (e)]), \end{aligned}

(1)

\begin{aligned} \overset{\leftarrow}{e_{r}^{i}} & = GRU (\partial [N_{r}^{i} (e)], \partial [N_{r}^{i + 1} (e)], \dots, \partial [N_{r}^{L} (e)]), \end{aligned}

(2)

\begin{aligned} e_{r}^{i} & = [\vec{e_{r}^{i}}, \overset{\leftarrow}{e_{r}^{i}}], \end{aligned}

(3)

where sequences

\vec{e_{r}^{i}}

and

\overset{\leftarrow}{e_{r}^{i}}

are the forward and backward hidden layer state sequences, which are the outputs of the Bi-GRU. Thus, the layer state sequence stitching

e_{r}^{i}

is generated,

i \in [1, L]

is the layer index, as shown in equation (3).

\partial (\cdot)

projects

N_{r}^{i} (e)

as the vector by representing each relation in the subgraph as a one-hot identity vector of the relation type that it belongs to, as follows:

\partial [N_{r}^{i} (e)] = {x_{1}, x_{2}, \dots, x_{m}}, x_{j} = one-hot (x_{j}),

(4)

where

x_{j}

is the one-hot identity vector of

r_{j}

j \in [1, m]

, and

m \leq K

is the number of relations in

N_{r}^{i} (e)

By the above way, S2S transforms the subgraph $N_{r} (e) = {N_{r}^{1} (e), N_{r}^{2} (e), \dots, N_{r}^{L} (e)}$ into the state sequence $S_{r} (e) = {e_{r}^{1}, e_{r}^{2}, e_{r}^{3}, \dots, e_{r}^{L}}$ .

4.3. CPI Mechanism

To better capture the homogeneity between the $i$ th layer subgraphs $N_{r}^{i} (h)$ and $N_{r}^{i} (t)$ of entities $h$ and $t$ , based on the assumption that the more frequent the $r_{i}$ appears in the intersection of $N_{r}^{i} (h)$ and $N_{r}^{i} (t)$ , we introduce a CPI mechanism to infer the importance of each relation in $N_{r}^{i} (h)$ and $N_{r}^{i} (t)$ , and assign higher weights to the co-occurrence relations in $N_{r}^{i} (h)$ and $N_{r}^{i} (t)$ . We further introduce a smoothing factor $λ$ to balance the preferred and unpreferred relations and prevent over-preference for certain relations. Take Figure 1 as an example, the first layer in the subgraphs of both entity “OpenSSL Program” and entity “Sudo Program” contain the same relation “runs on,” and both entities are connected to the tail entity with “Linux Operating System,” then relation “runs on” may be the real relation between these two entities, so it should be given a higher weight.

Specifically, for $N_{r}^{i} (h)$ and $N_{r}^{i} (t),$ we learn the homogeneity between them and obtain the latent subgraph representation of them, as shown in equations (5) and (6):

\begin{aligned} h^{i} & = h_{r}^{i} \cdot g (N_{r}^{i} (h), N_{r}^{i} (t)), \end{aligned}

(5)

\begin{aligned} t^{i} & = t_{r}^{i} \cdot g (N_{r}^{i} (h), N_{r}^{i} (t)), \end{aligned}

(6)

where

h_{r}^{i}

and

t_{r}^{i}

denote the layer state sequence of

N_{r}^{i} (h)

and

N_{r}^{i} (t)

, as shown in equation (3). And

g (N_{r}^{i} (h), N_{r}^{i} (t))

is the relation weight matrix, containing the weights of each relation, as shown in equation (7).

\begin{matrix} g (N_{r}^{i} (h), N_{r}^{i} (t)) = λ \frac{N_{r}^{i} (h) \cap N_{r}^{i} (t)}{N_{r}^{i} (h) \cup N_{r}^{i} (t)} + (1 - λ) \frac{N_{r}^{i} (h) \cup N_{r}^{i} (t) - N_{r}^{i} (h) \cap N_{r}^{i} (t)}{N_{r}^{i} (h) \cup N_{r}^{i} (t)}, \end{matrix}

(7)

\begin{matrix} g (N_{r}^{i} (h), N_{r}^{i} (t)) \leftrightarrow {w_{1}, w_{2}, w_{3}, \dots, w_{n}}, \end{matrix}

(8)

where

λ

is a smoothing factor to balance the preferred co-occurrence relations with the other relations. In this formula, a plus sign connects two terms, and the first term emphasizes the common part between the two subgraphs. The second term represents the difference between the two subgraphs. By subtracting the intersection from the union and normalizing it, we get a measure of the unique parts of each subgraph. When

λ

is larger, the model gives more favor to co-occurrence relations; the smaller

λ

is, the model gives more favor to other relations. Intuitively, for different subgraphs, there exist different optimal

λ

. After extensive experimentation and evaluation on the benchmark datasets, we found that a lambda value of 0.5 yielded the best overall performance in most cases.

Finally, the CPI mechanism functions by assigning different weights $w_{i}$ to each relation $r_{i}$ . These weights are calculated based on the significance of $r_{i}$ within the subgraphs $N_{r}^{i} (h)$ and $N_{r}^{i} (t)$ associated with the head entity $h$ and the tail entity $t$ . The more important a relation $r_{i}$ is in these subgraphs, the higher the weight $w_{i}$ it will be assigned. This weighting allows us to focus on the relations that are more relevant in characterizing the entities $h$ and $t$ . By emphasizing these important relations, we are able to capture more similar features between $h$ and $t$ . For example, if a particular relation $r_{i}$ frequently appears in both $N_{r}^{i} (h)$ and $N_{r}^{i} (t)$ , it will likely have a higher weight, indicating its importance in determining the similarity between the two entities. This, in turn, helps us to reason about the possible relation between $h$ and $t$ . As shown in equation (8), this mechanism is formalized in a way that enables the model to effectively utilize these weighted relations for better inference.

4.4. Alternating Induction Mechanism

After learning the hierarchical features and homogeneous features of each layer of subgraphs of the head entity and tail entity, we alternately induct the $i$ th layer subgraph of the head and tail entities to obtain the inductive subgraph representations of the head and tail entities, as shown in equations (9) and (10):

\begin{aligned} a_{h}^{i} & = \sum_{i \in [2, L]} f (\bar{h^{i}}, \bar{h^{i - 1}}), \end{aligned}

(9)

\begin{aligned} a_{t}^{i} & = \sum_{i \in [2, L]} f (\bar{t^{i}}, \bar{t^{i - 1}}), \end{aligned}

(10)

\begin{aligned} {\bar{h}}^{i} & = σ [(a_{h}^{i - 1} + h^{i} + a_{t}^{i - 1}) W_{h}^{i} + b], i \in [2, L], \end{aligned}

(11)

\begin{aligned} {\bar{t}}^{i} & = σ [(a_{h}^{i - 1} + t^{i} + a_{t}^{i - 1}) W_{t}^{i} + b], i \in [2, L], \end{aligned}

(12)

where

a_{h}^{i}

and

a_{t}^{i}

denote the latent semantic features of h and t at the ith alternate induction.

h^{i}

and

t^{i}

denote the latent subgraph representation, as shown in equations (5) and (6).

\bar{h^{i}}

and

\bar{t^{i}}

denote the inductive representation of the ith subgraph of h and t at the ith iteration. And in equations (9) and (10), we multiply the current inductive representation by the previous one layer inductive representation to learn the direct historical information (Ma et al., 2021).

σ (\cdot)

is a sigmoid function,

f (\cdot)

is the element-level interaction operation, as shown in equation (13):

f (e^{i}, e^{i - 1}) = (\begin{matrix} {(e^{i})}^{(1)} \cdot {(e^{i - 1})}^{(1)} & \dots & {(e^{i})}^{(1)} \cdot {(e^{i - 1})}^{(d)} \\ ⋮ & ⋱ & ⋮ \\ {(e^{i})}^{(d)} \cdot {(e^{i - 1})}^{(1)} & \dots & {(e^{i})}^{(d)} \cdot {(e^{i - 1})}^{(d)} \end{matrix}) .

(13)

The final latent semantic feature $a_{h}^{L}$ and $a_{t}^{L}$ represent the inductive subgraph representations. They are further combined and input to the linear layer to obtain the score of $(h, t)$ , which is used to predict the relation type between $h$ and $t$ . The score is a crucial metric that is calculated as follows. It is obtained by taking the dot product of the latent semantic features $a_{h}^{L}$ and $a_{t}^{L}$ with the weight $W$ , adding the bias $b$ , and then applying the sigmoid function $σ$ . This process is formalized in equation (14), where the score is calculated as equation (14):

Score (h, t) = σ [(a_{h}^{L}, a_{t}^{L}) \cdot W + b],

(14)

where

W

is the weight and

b

is bias. The score provides a quantitative measure that helps the model assess the likelihood or strength of a particular relation existing between the two entities, allowing for more accurate predictions and inferences about the relationships in the KG.

We train our method by minimizing the loss between the predicted relation type and the actual ground truth relation type, as shown in equation (15):

Loss = \sum_{(h_{i}, r_{i}, t_{i}) \in KG} C [Score (h_{i}, t_{i}), r_{i}],

(15)

where

C (\cdot)

is the cross entropy loss function.

4.5. Proposed Algorithm

The algorithm of HiHo is shown in Algorithm ??. First, we collect the relations in the neighborhood of $h$ and $t$ and add them to the subgraphs (lines 1–9). Then, we regard the subgraphs as sequences, and use the Bi-GRU to encode previous features and the later features for each state in sequences (lines 10–17). Next, we learn the homogeneity between the subgraphs and obtain the latent subgraph representation of them (lines 19–24). Finally, we alternately induct the subgraph of $h$ and $t$ to obtain the score of ( $h, t$ ) (lines 25–33), which is used to predict the relation type between $h$ and $t$ .

5. Experiments

In this section, we assess the performance of our proposed method across five benchmark datasets. The subsequent subsections outline the statistics of these benchmark datasets and summarize the parameter settings. Then, the experimental metrics chosen for this paper are presented. Next, we validate the effectiveness of our method through performance comparison and analysis in both the transductive relation prediction task and the iterative relational prediction task. Subsequently, the effectiveness of different mechanisms on relation prediction is demonstrated. Finally, the influence of the parameters on the model is verified.

5.1. Datasets and Experimental Settings

The benchmark datasets selected in this paper are FB15K, FB15K-237, NELL995, WN18, and WN18RR, where FB15K and FB15K-237 are extracted from Freebase (Bollacker et al., 2008), NELL995 extracted from Nell (Mitchell et al., 2018), and WN18 and WN18RR are extracted from WordNet (Miller, 1995). And in addition, FB15K-237 and WN18RR are obtained by removing the inverse relations in FB15K and WN18, respectively. These datasets comprise numerous entities and relations, partitioned into training, validation, and test sets. Table 1 presents the statistics for these datasets.

Table 1.
Statistics of Datasets.

Datasets Entities Relations Train Triples Validations Triples Test Triples Total Triples

FB15K 14,951 1,345 483,142 50,000 59,071 592,213

FB15K-237 14,541 237 272,115 17,535 20,466 310,116

NELL995 63,917 198 137,465 5,000 5,000 147,465

WN18 40,943 18 141,442 5,000 5,000 151,442

WN18RR 40,943 11 86,835 3,034 3,134 93,003

Datasets	Entities	Relations	Train Triples	Validations Triples	Test Triples	Total Triples
FB15K	14,951	1,345	483,142	50,000	59,071	592,213
FB15K-237	14,541	237	272,115	17,535	20,466	310,116
NELL995	63,917	198	137,465	5,000	5,000	147,465
WN18	40,943	18	141,442	5,000	5,000	151,442
WN18RR	40,943	11	86,835	3,034	3,134	93,003

We employ Adam (Kingma & Ba, 2015) to train our method, the initial learning rate $δ$ is selected within 0.001, 0.005, and 0.01. At the same time, the embedding dimension of entities and relations in the triples is set to $d$ , the value of which is selected from 16, 32, 64, 128, and 256. Subsequently, the maximum number of layers in the neighborhood is set as $L_{hop} \in [1, 5]$ , the maximum number of relations in each layer is set as $N_{r} \in [1, 10]$ . Through grid search method, the optimal parameters of HiHo are finally selected as follows: $δ = 0.005$ , $d = 64$ , $L_{hop} = 2$ , $N_{r} = 8$ on FB15K; $δ = 0.005$ , $d = 64$ , $L_{hop} = 2$ , $N_{r} = 8$ on FB15K-237; $δ = 0.005$ , $d = 64$ , $L_{hop} = 2$ , $N_{r} = 8$ on NELL995; $δ = 0.005$ , $d = 64$ , $L_{hop} = 3$ , $N_{r} = 8$ on WN18; $δ = 0.005$ , $d = 64$ , $L_{hop} = 4$ , $N_{r} = 8$ on WN18RR. We train at most 1,000 epochs for both datasets.

Similar to the link prediction task, the relation prediction task aims to forecast the target relation between the head entity $h$ and the tail entity $t$ . Specifically, for a given triple $(h, r, t)$ , we formulate a query for the relation prediction task: $(h, ?, t)$ . Subsequently, each relation in the dataset is considered a potential target. We proceed to rank all candidate triples in descending order of score to determine the ranking of the ground-truth triple $(h, r, t)$ .

5.2. Evaluation Metrics

This paper employs mean reciprocal rank (MRR) and HIT ratio with cut-off values of $K$ (HIT@ $K$ ) as our evaluation metrics. Higher MRR, higher HIT@ $K$ implies better performance of the model. The definitions of MRR and HIT@K are shown in equations (16) and (17):

\begin{aligned} MRR & = \frac{1}{| F |} \sum_{i = 1}^{| F |} \frac{1}{{rank}_{i}}, \end{aligned}

(16)

\begin{aligned} HIT @ K & = \frac{1}{| F |} \sum_{i = 1}^{| F |} I [{rank}_{i} \leq K] . \end{aligned}

(17)

5.3. Transductive Relation Prediction

We conduct a comparative analysis of HiHo with other state-of-the-art transductive methods (TransE [Bordes et al., 2013], DRUM [Sadeghian et al., 2019], QuatE [Zhang et al., 2019], P-INT [Xu et al., 2021], KE-GCN [Yu et al., 2021], GGAE [Li et al., 2022], and ConvRot [Le et al., 2023]) in the transductive relation prediction experiment. Table 2 reports the results on five datasets. Specifically, the MRR increases of HiHo against the second-ranking method are 0.052, 0.006, 0.015, 0.015, and 0.039 on the five datasets, respectively; the HIT@1 increases of HiHo against the second-ranking method are 2.5%, 0.9%, 2.7%, 2.3%, and 3.4% on the five datasets, respectively; the HIT@3 increases of HiHo against the second-ranking method are 6.0%, 0.1%, 0.8%, 1.3%, and 0.3% on the five datasets, respectively.

Table 2.
The Performance of Different Models in the Transductive Relation Prediction Experiment.

Dataset NELL995 FB15K FB15K-237 WN18 WN18RR

Metric MRR HIT@1 HIT@3 MRR HIT@1 HIT@3 MRR HIT@1 HIT@3 MRR HIT@1 HIT@3 MRR HIT@1 HIT@3

TransE 0.841 0.781 0.889 0.962 0.94 0.982 0.966 0.946 0.984 0.971 0.955 0.984 0.784 0.669 0.87

DRUM 0.715 0.64 0.74 0.945 0.945 0.978 0.959 0.905 0.958 0.969 0.956 0.98 0.854 0.778 0.912

QuatE 0.752 0.706 0.783 0.983 0.972 0.991 0.974 0.958 0.988 0.981 0.975 0.983 0.823 0.767 0.852

P-INT 0.706 0.644 0.727 0.885 0.839 0.966 0.941 0.885 0.943 0.954 0.931 0.971 0.832 0.763 0.905

KE-GCN 0.824 0.795 0.844 0.938 0.927 0.989 0.937 0.934 0.971 0.951 0.948 0.978 0.923 0.916 0.961

GGAE 0.845 0.815 0.881 0.934 0.928 0.985 0.937 0.931 0.959 0.968 0.943 0.977 0.945 0.942 0.991

ConvRot 0.856 0.826 0.892 0.935 0.962 0.996 0.975 0.957 0.982 0.983 0.974 0.985 0.948 0.931 0.992

HiHo 0.908 0.874 0.946 0.989 0.981 0.997 0.99 0.984 0.996 0.998 0.997 0.998 0.987 0.974 0.995

Dataset	NELL995	FB15K	FB15K-237	WN18	WN18RR
TransE	0.841	0.781	0.889	0.962	0.94	0.982	0.966	0.946	0.984	0.971	0.955	0.984	0.784	0.669	0.87
DRUM	0.715	0.64	0.74	0.945	0.945	0.978	0.959	0.905	0.958	0.969	0.956	0.98	0.854	0.778	0.912
QuatE	0.752	0.706	0.783	0.983	0.972	0.991	0.974	0.958	0.988	0.981	0.975	0.983	0.823	0.767	0.852
P-INT	0.706	0.644	0.727	0.885	0.839	0.966	0.941	0.885	0.943	0.954	0.931	0.971	0.832	0.763	0.905
KE-GCN	0.824	0.795	0.844	0.938	0.927	0.989	0.937	0.934	0.971	0.951	0.948	0.978	0.923	0.916	0.961
GGAE	0.845	0.815	0.881	0.934	0.928	0.985	0.937	0.931	0.959	0.968	0.943	0.977	0.945	0.942	0.991
ConvRot	0.856	0.826	0.892	0.935	0.962	0.996	0.975	0.957	0.982	0.983	0.974	0.985	0.948	0.931	0.992
HiHo	0.908	0.874	0.946	0.989	0.981	0.997	0.99	0.984	0.996	0.998	0.997	0.998	0.987	0.974	0.995

MRR: mean reciprocal rank.

According to the experimental results shown in Table 2, we compared HiHo with the baselines, and the analysis results are as follows: (1)

Our method outperforms all baselines in all datasets, especially in the NELL995 dataset, where our method has significant advantages. The reason is that NELL995 is sparser than other benchmark datasets, which leads to the overfitting of the baselines and limits their learning ability of the baselines. The improvement of our method in NELL995 is quite significant because our method models the hierarchical and homogeneous nature of entities’ subgraphs, which makes our method easy to extract effective features of entities. In addition, our method has fewer parameters and can be applied to some KGs that are characterized by having relatively fewer connections or edges between entities compared to the total number of possible relationships, that is, sparse KGs.

(2)

As a translation-based method, TransE (Bordes et al., 2013) and ConvRot (Le et al., 2023) models only the inherent characteristics of entities and relations; they ignore the multistep neighborhood features of entities, which limits their performance in relation prediction. As the semantic matching methods, DRUM (Sadeghian et al., 2019) and QuatE (Zhang et al., 2019) reason about what kind of relation exists between two entities by mining the underlying semantic association rules between entities and relations. However, they are very sensitive to rule settings, and some missing relations cannot be inferred from any rule (Lin et al., 2021). As a path-based method, P-INT (Xu et al., 2021) can extract rich features in the paths between entities to predict the relations between entities. However, not all entities have paths between them, which leads to unpredictable relationships between them. As the graph-based methods, KE-GCN (Yu et al., 2021) and GGAE (Li et al., 2022) learn the aggregated representation of entities by collecting information in their neighborhoods. However, the final representations of the head and tail entities are independent of each other, and they may have no common ground and lack the head–tail interactions, so the relation between them cannot be well reasoned.

5.4. Inductive Relation Prediction

We compare HiHo with other state-of-the-art inductive methods (GraIL [Teru et al., 2020], TACT [Chen et al., 2021], PathCon [Wang et al., 2021], StATIK [Markowitz et al., 2022], MorsE [Chen et al., 2022], and RMPI [Geng et al., 2023]) in the inductive relation prediction experiment. Similar to the inductive KG completion experiment conducted by using PathCon (Wang et al., 2021), we randomly sample approximately 20% of the entities present in the test set. Subsequently, we exclude these entities along with their connected relations from the training set. The rest of the training set is utilized to train the models. During evaluation, we reintroduce the removed relations back into consideration (Wang et al., 2021). As can be seen from the results in Figures 3 to 7. Specifically, compared with other baselines, the MRR of HiHo are improved by 0.008–0.184, 0.001–0.056, 0.019–0.066, 0.006–0.042, and 0.006–0.189 on the five datasets, respectively; the HIT@1 of HiHo are improved by 0.5%–21.0%, 0.5%–6.2%, 1.2%–4.9%, 0.2%–6.5%, and 1.6%–32.7% on the five datasets, respectively; the HIT@3 of HiHo are improved by 0.6%–25.4%, 0.2%–2.2%, 0.1%–3.1%, 0.1%–1.9%, and 0.2%–13.1% on the five datasets, respectively.

Figure 3.

The Performance of Each Method on the NELL995 Dataset.

Figure 4.

The Performance of Each Method on the FB15K Dataset.

Figure 5.

The Performance of Each Method on the FB15K237 Dataset.

Figure 6.

The Performance of Each Method on the WN18 Dataset.

Figure 7.

The Performance of Each Method on the WN18RR Dataset.

HiHo outperforms all other inductive methods. The reasons are: (1) GraIL, TACT, and RMPI focus on modeling the intersection of two subgraphs, but sometimes the intersection only accounts for a very small proportion of the two subgraphs, which leads to many relation types in the subgraphs being ignored. (2) StATIK and MorsE model the subgraph of the head entity and the subgraph of the tail entity separately without considering the commonality of the two subgraphs. (3) PathCon predicts relations by alternately generalizing the context of two entities and modeling the relational path between two entities, but the relational path used by PathCon suffers from the same problems as the path-based methods. In addition, the above method only learns the features of the $i$ th layer subgraph of an entity separately, without modeling the hierarchical nature of the subgraph. Our method achieves the best performance by generalizing the hierarchical and homogeneous nature of the subgraphs of entities.

5.5. Ablation Study

To demonstrate the individual effects of the proposed S2S and CPI components, we conduct an ablation experiment on the NELL995 dataset. Specifically, we exclude each component from the proposed method separately and evaluate its impact on the experimental results.

As shown in Table 3, compared to our whole model, the MRR, Hit@1, and Hit@3 of the model without S2S dropped by 0.23, 6.2%, and 1.5%, respectively; and the MRR, Hit@1, and Hit@3 of the model without CPI dropped by 0.46, 8.3%, and 3.9%, respectively. Therefore, we can obtain the following conclusions: (1) Without S2S, the performance of our model degrades severely, which proves that the lack of modeling the hierarchy of subgraph can seriously affect the effectiveness of relation prediction. (2) Without CPI, the performance of our model degrades slightly. The reason is that all the benchmark datasets used in this paper, even the NELL995 dataset, are relatively dense. We quantify density as the ratio of existing edges to the total possible edges [calculated as $n (n - 1) / 2$ for a graph with $n$ entities]. The high density leads to dense relations in entities’ subgraphs, different from some real-world KGs such as DBpedia and Wikidata (not used in current experiments due to complexity and heterogeneity). In summary, the ablation study emphasizes the significance of S2S and CPI, especially S2S.

Table 3.
Ablation Study on NELL995 Dataset.

MRR HIT@1 HIT@3

Without S2S 0.862 0.791 0.907

Without CPI 0.885 0.812 0.931

HiHo 0.908 0.874 0.946

	MRR	HIT@1	HIT@3
Without S2S	0.862	0.791	0.907
Without CPI	0.885	0.812	0.931
HiHo	0.908	0.874	0.946

MRR: mean reciprocal rank; S2S: subgraph-to-sequence.

5.6. Parameter Sensitivity

The maximum number of layers in the neighborhood $L_{hop}$ , and the maximum number of relations in each layer $N_{r}$ are the most important parameters of HiHo, and they largely determine the performance of HiHo in terms of KG relation prediction. In general, the larger the values of $L_{hop}$ or $N_{r}$ , the more valuable information HiHo can utilize to predict relations, thereby enabling a more comprehensive understanding of triple characteristics. However, excessively large values of $L_{hop}$ or $N_{r}$ can escalate the complexity of HiHo, diminish its training speed, and lead to the absorption of redundant information, ultimately resulting in overfitting. Conversely, setting $L_{hop}$ or $N_{r}$ too small may limit the features learned by HiHo, reducing its ability to predict relations effectively, resulting in underfitting. In extreme cases, HiHo may regress to a translation-based or semantic matching method. Balancing these parameters is crucial for achieving optimal performance.

Figure 8 shows the performance of the different values of $L_{hop}$ and $N_{r}$ on HiHo. From Figure 8, we can see that: With the increase of $L_{hop}$ and $N_{r}$ , the MRR of HiHo increases continuously. When $L_{hop} = 3$ and $N_{r} = 8$ , HiHo’s MRR reaches the highest point, and HiHo converges to the optimal solutions. But as $L_{hop}$ and $N_{r}$ continue to increase, the MRR of HiHo stops rising. Therefore, HiHo performs best on WN18 when $L_{h o p} = 3$ and $N_{r} = 8$ .

Figure 8.

The Effect of the Maximum Number of Layers in the Neighborhood and the Maximum Number of Relations in Each Layer.

6. Conclusion

In this paper, we propose the HiHo model to predict the relation between entities in the inductive setting. Specifically, we propose an S2S mechanism to model the hierarchy of subgraphs by learning the semantic associations between layers in the subgraph of a single entity. And we propose a CPI mechanism to model the homogeneity of subgraphs by assigning higher weights to co-occurrence relations and learning the importance of each relation in the subgraphs of two entities. Finally, we alternatively generalize the subgraphs of two entities to infer the relations between them. And compared with the baselines, our method has better performance.

Although the proposed method has shown good performance in the relation prediction task, it may encounter challenges when dealing with rare relations or complex subgraph structures. For rare relations, due to the lack of sufficient data, it is difficult to learn reliable patterns, which may lead to inaccurate predictions. In the future, we plan to adopt data augmentation techniques to alleviate this problem. For complex subgraph structures, the high connectivity and intertwined relationships may exceed the capacity of the model, making it difficult to capture meaningful information. In the future, we plan to develop a more advanced GNN architecture with hierarchical or multiscale processing mechanisms to decompose the complexity and focus on relevant substructures, so as to achieve the effective extraction and utilization of information in complex subgraph structures. In addition, this paper has only explored the application of the proposed method in small- and medium-sized real-world KGs. In future work, we will further explore its application in ultra-large-scale KGs.

Footnotes

ORCID iDs

Jiangtao Ma

Yuke Ma

Yaqiong Qiao

Funding

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: This research was partly funded by National Natural Science Foundation of China (62272163), Henan Provincial Department of Science and Technology (252102210054, 252102210101), Research and Practice Project of Research Teaching Reform in Undergraduate Colleges and Universities in Henan Province (202338870), Zhengzhou University of Light Industry Fund (2022BSJJ016, 23XJCYJ070), and Key Research Projects in Higher Education Institutions in Henan Province (25A520054).

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Bollacker

Evans

Paritosh

Sturge

Taylor

(2008). Freebase: A collaboratively created graph database for structuring human knowledge. In SIGMOD’08 (pp. 1247–1250). Association for Computing Machinery.

Bordes

Usunier

Garcia-Duran

Weston

Yakhnenko

(2013). Translating embeddings for modeling multi-relational data. In Burges C., Bottou L., Welling M., Ghahramani Z. & Weinberger K. (Eds.), Advances in neural information processing systems (vol. 26, pp. 2787–2795). Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2013/file/1cecc7a77928ca8133fa24680a88d2f9-Paper.pdf.

Chen

Wang

(2021). Topology-aware correlations between relations for inductive link prediction in knowledge graphs. Proceedings of the AAAI Conference on Artificial Intelligence, 35(7), 6271–6278. https://doi.org/10.1609/aaai.v35i7.16779

Chen

Zhang

Zhu

Zhou

Yuan

Chen

(2022). Meta-knowledge transfer for inductive knowledge graph embedding. In Proceedings of the 45th International ACM SIGIR conference on research and development in information retrieval. SIGIR’22 (pp. 927–937). Association for Computing Machinery.

Chen

Jin

Ren

(2019). Collaborative policy learning for open knowledge graph reasoning. In Inui K., Jiang J., Ng V. & Wan X. (Eds.), Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th International joint conference on natural language processing (EMNLP–IJCNLP) (pp. 2672–2681). Association for Computational Linguistics. https://doi.org/10.18653/v1/D19-1269

Geng

Chen

Zhang

Pan

J. Z.

Chen

Zeng Chen

Jiang

(2023). Relational message passing for fully inductive knowledge graph completion. In 2023 IEEE 39th international conference on data engineering (ICDE) (pp. 1221–1233). IEEE. https://api.semanticscholar.org/CorpusID:252780623

Kingma

D. P.

(2015). Adam: A method for stochastic optimization. In Bengio Y. & LeCun Y. (Eds.), Third international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, conference track proceedings.

Kipf

T. N.

Welling

(2017). Semi-supervised classification with graph convolutional networks. In International conference on learning representations. https://openreview.net/forum?id=SJU4ayYgl

(2023). Knowledge graph embedding by relational rotation and complex convolution for link prediction. Expert Systems with Applications, 214, 119122. https://doi.org/10.1016/j.eswa.2022.119122

10.

Wang

Feng

Niu

Zhang

(2022). Global graph attention embedding network for relation prediction in knowledge graphs. IEEE Transactions on Neural Networks and Learning Systems, 33(11), 6712–6725. https://doi.org/10.1109/TNNLS.2021.3083259

11.

Zhang

Zhu

Zheng

Zhang

(2023). Knowledge graph representation learning with simplifying hierarchical feature propagation. Information Processing & Management, 60(4), 103348. https://doi.org/10.1016/j.ipm.2023.103348

12.

Lin

Liu

Pan

Zhang

(2021). Rule-enhanced iterative complementation for knowledge graph reasoning. Information Sciences, 575, 66–79.

13.

Lin

Liu

Sun

Liu

Zhu

(2015). Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 29, pp. 2181–2187). AAAI Press. https://doi.org/10.1609/aaai.v29i1.9491

14.

Liu

Grau

Horrocks

Kostylev

(2021). INDIGO: GNN-based inductive knowledge graph completion using pair-wise encoding. In Ranzato M., Beygelzimer A., Dauphin Y., Liang P. & Vaughan J. W. (Eds.), Advances in neural information processing systems (vol. 34, pp. 2034–2045). Curran Associates, Inc., https://proceedings.neurips.cc/paper_files/paper/2021/file/0fd600c953cde8121262e322ef09f70e-Paper.pdf

15.

Liu

Yang

Ding

(2023). Two birds, one stone: An equivalent transformation for hyper-relational knowledge graph modeling. https://openreview.net/forum?id=e3U6bGsfcA

16.

Huang

(2021). Hiam: A hierarchical attention based model for knowledge graph multi-hop reasoning. Neural Networks, 143, 261–270. https://doi.org/10.1016/j.neunet.2021.06.008

17.

Markowitz

Balasubramanian

Mirtaheri

Annavaram

Galstyan

Ver Steeg

(2022). StATIK: Structure and text for inductive knowledge graph completion. In Carpuat M., de Marneffe M. C. & Meza Ruiz I. V. (Eds.), Findings of the association for computational linguistics: NAACL 2022 (pp. 604–615). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.findings-naacl.46

18.

Meltzer

Kallioniemi

Trent

(2002). Chromosome alterations in human solid tumors. In Vogelstein B. & Kinzler K. (Eds.), The genetic basis of human cancer (pp. 93–113). McGraw-Hill.

19.

Miller

G. A.

(1995). Wordnet: A lexical database for english. Communications of the ACM, 38(11), 39–41. https://doi.org/10.1145/219717.219748

20.

Mitchell

Cohen

Hruschka

Talukdar

Yang

Betteridge

Carlson

Dalvi

Gardner

Kisiel

Krishnamurthy

Lao

Mazaitis

Mohamed

Nakashole

Platanios

Ritter

Samadi

Settles

, … Welling

(2018). Never-ending learning. Communications of the ACM, 61(5), 103–115. https://doi.org/10.1145/3191513

21.

Murray

Rosenthal

Kobayashi

Pfaller

(2002). Medical microbiology. 4th ed. Mosby.

22.

Nguyen

D. Q.

Tong

Phung

Nguyen

D. Q.

(2022). Node co-occurrence based graph neural networks for knowledge graph link prediction. In WSDM’22 (pp. 1589–1592). Association for Computing Machinery.

23.

Pavlovic

Sallinger

(2023). Expressive: A spatio-functional embedding for knowledge graph completion. In The eleventh international conference on learning representations. https://openreview.net/forum?id=xkev3_np08z

24.

Sadeghian

Armandpour

Ding

Wang

D. Z.

(2019). Drum: End-to-end differentiable rule mining on knowledge graphs. In Advances in neural information processing systems (vol. 32, pp. 9444–9454). Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2019/file/0c72cb7ee1512f800abe27823a792d03-Paper.pdf

25.

Shang

Zhao

Liu

Wang

(2024). Attention-based exploitation and exploration strategy for multi-hop knowledge graph reasoning. Information Sciences, 653, 119787. https://doi.org/10.1016/j.ins.2023.119787

26.

Shi

Wang

Deng

(2023). Relagraph: Improving embedding on small-scale sparse knowledge graphs by neighborhood relations. Information Processing & Management, 60(5), 103447. https://doi.org/10.1016/j.ipm.2023.103447

27.

Teru

Denis

Hamilton

(2020). Inductive relation prediction by subgraph reasoning. In III H. D. & Singh A. (Eds.), Proceedings of the 37th international conference on machine learning research (vol. 119, pp. 9448–9457). PMLR. https://proceedings.mlr.press/v119/teru20a.html

28.

Trouillon

Welbl

Riedel

Gaussier

Bouchard

(2016). Complex embeddings for simple link prediction. In Balcan M. F. & Weinberger K. Q. (Eds.), Proceedings of the 33rd international conference on machine learning research (vol. 48, pp. 2071–2080). PMLR. https://proceedings.mlr.press/v48/trouillon16.html

29.

Vashishth

Sanyal

Nitin

Talukdar

(2020). Composition-based multi-relational graph convolutional networks. In International conference on learning representations. https://openreview.net/forum?id=BylA_C4tPr

30.

Velickovic

Cucurull

Casanova

Romero

Liò

Bengio

(2018). Graph attention networks. In International conference on learning representations. https://openreview.net/forum?id=rJXMpikCZ

31.

Wang

Ren

Leskovec

(2021). Relational message passing for knowledge graph completion. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, KDD’21 (pp. 1697–1707). Association for Computing Machinery.

32.

Wang

Zhang

Feng

Chen

(2014). Knowledge graph embedding by translating on hyperplanes. Proceedings of the AAAI Conference on Artificial Intelligence, 28(1), 1112–1119. https://doi.org/10.1609/aaai.v28i1.8870

33.

Wang

Zhang

Shi

Cai

Wang

Zhang

Zheng

(2023b). Knowledge graph embedding model with attention-based high-low level features interaction convolutional network. Information Processing & Management, 60(4), 103350. https://doi.org/10.1016/j.ipm.2023.103350

34.

Wang

Zheng

Zhang

Yang

Zhu

(2023a). To see further: Knowledge graph-aware deep graph convolutional network for recommender systems. Information Sciences, 647, 119465. https://doi.org/10.1016/j.ins.2023.119465

35.

Wang

Zhou

Pan

Dong

Song

Sha

(2022). Exploring relational semantics for inductive knowledge graph completion. Proceedings of the AAAI Conference on Artificial Intelligence, 36(4), 4184–4192. https://doi.org/10.1609/aaai.v36i4.20337

36.

Shen

Xia

(2023). Commonsense knowledge graph completion via contrastive pretraining and node clustering. In Rogers A., Boyd-Graber J. L. & Okazaki N. (Eds.), Findings of the Association for Computational Linguistics: ACL 2023 (pp. 13977–13989). Association for Computational Linguistics. https://doi.org/10.18653/V1/2023.FINDINGS-ACL.878.

37.

Xiong

Hoang

Wang

W. Y.

(2017). DeepPath: A reinforcement learning method for knowledge graph reasoning. In Palmer M., Hwa R. & Riedel S. (Eds.), Proceedings of the 2017 conference on empirical methods in natural language processing (pp. 564–573). Association for Computational Linguistics. https://doi.org/10.18653/v1/D17-1060

38.

Wang

Feng

Witbrock

Sheinin

(2019). Graph2seq: Graph to sequence learning with attention-based neural networks. https://openreview.net/forum?id=SkeXehR9t7

39.

Zhang

Chao

Yan

(2022). Subgraph neighboring relations infomax for inductive link prediction on knowledge graphs. In International joint conference on artificial intelligence (pp. 2341–2347). International Joint Conferences on Artificial Intelligence Organization. https://api.semanticscholar.org/CorpusID:250629390

40.

Zhang

Dong

Chen

Liu

(2021). P-INT: A path-based interaction model for few-shot knowledge graph completion. In Moens M. F., Huang X., Specia L. & Yih SWt (Eds.), Findings of the association for computational linguistics: EMNLP 2021 (pp. 385–394). Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.findings-emnlp.35

41.

Yang

Cohen

W. W.

(2017). Differentiable learning of logical rules for knowledge base reasoning. In Neural information processing systems (Vol. 30, pp. 2316–2325). Curran Associates, Inc. https://api.semanticscholar.org/CorpusID:4502993

42.

Yang

Luo

Jiang

Zhang

Wang

(2023). Bert and hierarchical cross attention-based question answering over bridge inspection knowledge graph. Expert Systems with Applications, 233, 120896. https://doi.org/10.1016/j.eswa.2023.120896

43.

Yang

Yih

Gao

Deng

(2015). Embedding entities and relations for learning and inference in knowledge bases. In Bengio Y. & LeCun Y. (Eds.), Third international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, conference track proceedings. https://api.semanticscholar.org/CorpusID:2768038

44.

Yang

Zhang

(2021). Knowledge embedding based graph convolutional network. In WWW ’21 (pp. 1619–1628). Association for Computing Machinery.

45.

Zhang

Huang

Gao

Han

Zhou

(2022b). Knowledge graph embedding by logical-default attention graph convolution neural network for link prediction. Information Sciences, 593, 201–215. https://doi.org/10.1016/j.ins.2022.01.076

46.

Zhang

Tay

Yao

Liu

(2019). Quaternion knowledge graph embeddings. In Advances in neural information processing systems (vol. 32, pp. 2731–2741). Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2019/file/d961e9f236177d65d21100592edb0769-Paper.pdf

47.

Zhang

Yuan

Liu

lin

Xiong

(2022a). Learning to walk with dual agents for knowledge graph reasoning. Proceedings of the AAAI Conference on Artificial Intelligence, 36(5), 5932–5941. https://doi.org/10.1609/aaai.v36i5.20538

48.

Zhao

Feng

Zhou

Yang

Chen

Xie

Zhuang

(2022). Eigat: Incorporating global information in local attention for knowledge representation learning. Knowledge-Based Systems, 237, 107909. https://doi.org/10.1016/j.knosys.2021.107909

49.

Zhu

Yuan

Galkin

Xhonneux

Zhang

Gazeau

Tang

(2022). A*net: A scalable path-based reasoning approach for knowledge graphs. https://api.semanticscholar.org/CorpusID:256597814

Dataset	NELL995			FB15K			FB15K-237			WN18			WN18RR
Metric	MRR	HIT@1	HIT@3	MRR	HIT@1	HIT@3	MRR	HIT@1	HIT@3	MRR	HIT@1	HIT@3	MRR	HIT@1	HIT@3
TransE	0.841	0.781	0.889	0.962	0.94	0.982	0.966	0.946	0.984	0.971	0.955	0.984	0.784	0.669	0.87
DRUM	0.715	0.64	0.74	0.945	0.945	0.978	0.959	0.905	0.958	0.969	0.956	0.98	0.854	0.778	0.912
QuatE	0.752	0.706	0.783	0.983	0.972	0.991	0.974	0.958	0.988	0.981	0.975	0.983	0.823	0.767	0.852
P-INT	0.706	0.644	0.727	0.885	0.839	0.966	0.941	0.885	0.943	0.954	0.931	0.971	0.832	0.763	0.905
KE-GCN	0.824	0.795	0.844	0.938	0.927	0.989	0.937	0.934	0.971	0.951	0.948	0.978	0.923	0.916	0.961
GGAE	0.845	0.815	0.881	0.934	0.928	0.985	0.937	0.931	0.959	0.968	0.943	0.977	0.945	0.942	0.991
ConvRot	0.856	0.826	0.892	0.935	0.962	0.996	0.975	0.957	0.982	0.983	0.974	0.985	0.948	0.931	0.992
HiHo	0.908	0.874	0.946	0.989	0.981	0.997	0.99	0.984	0.996	0.998	0.997	0.998	0.987	0.974	0.995

HiHo: A Hierarchical and Homogenous Subgraph Learning Model for Knowledge Graph Relation Prediction

Abstract

Keywords

1. Introduction

2. Related Work

2.1. Transductive Method

2.2. Inductive Method

3. Problem Definition

Definition 1 (KG)

Definition 2 (Inductive KG)

Definition 3 (KG Relation Prediction)

Definition 4 (Transductive Relation Prediction)

Definition 5 (Inductive Relation Prediction)

4.1. Subgraph Preparation

4.2. S2S Mechanism

5. Experiments

5.1. Datasets and Experimental Settings

Table 3. Ablation Study on NELL995 Dataset. MRR HIT@1 HIT@3 Without S2S 0.862 0.791 0.907 Without CPI 0.885 0.812 0.931 HiHo 0.908 0.874 0.946

Footnotes

ORCID iDs

Funding

Declaration of Conflicting Interests

References

Table 3.
Ablation Study on NELL995 Dataset.

MRR HIT@1 HIT@3

Without S2S 0.862 0.791 0.907

Without CPI 0.885 0.812 0.931

HiHo 0.908 0.874 0.946