Source-linked AI summary
Communicative Message Passing for Inductive Relation Reasoning
Sijie Mai, Shuangjia Zheng, Yuedong Yang, Haifeng Hu
TL;DR
Most knowledge-graph embedding methods assume a fixed entity set, limiting their ability to predict relations for unseen entities. CoMPILE reasons over directed subgraphs with communicative message passing; experiments report state-of-the-art performance on most inductive datasets and evaluation metrics.
Problem
Most knowledge-graph embedding methods assume a fixed entity set and cannot infer relations for entities unseen during training.
Source: Introduction · p. 1Method
CoMPILE reasons over directed enclosing subgraphs and uses node-edge communicative message passing to strengthen relation information.
Source: Introduction · p. 2, Conclusions · p. 7Results
CoMPILE achieves state-of-the-art AUC-PR and Hits@10 across most evaluated inductive datasets and outperforms state-of-the-art methods on most evaluation metrics.
Source: Introduction · p. 2, Conclusions · p. 7Takeaways & Limitations
Filtering triplets without enclosing subgraphs yields new inductive datasets for more accurate evaluation of relation reasoning methods.
Source: Conclusions · p. 7
Abstract
from arXiv · showhide
Relation prediction for knowledge graphs aims at predicting missing relationships between entities. Despite the importance of inductive relation prediction, most previous works are limited to a transductive setting and cannot process previously unseen entities. The recent proposed subgraph-based relation reasoning models provided alternatives to predict links from the subgraph structure surrounding a candidate triplet inductively. However, we observe that these methods often neglect the directed nature of the extracted subgraph and weaken the role of relation information in the subgraph modeling. As a result, they fail to effectively handle the asymmetric/anti-symmetric triplets and produce insufficient embeddings for the target triplets. To this end, we introduce a \textbf{C}\textbf{o}mmunicative \textbf{M}essage \textbf{P}assing neural network for \textbf{I}nductive re\textbf{L}ation r\textbf{E}asoning, \textbf{CoMPILE}, that reasons over local directed subgraph structures and has a vigorous inductive bias to process entity-independent semantic relations. In contrast to existing models, CoMPILE strengthens the message interactions between edges and entitles through a communicative kernel and enables a sufficient flow of relation information. Moreover, we demonstrate that CoMPILE can naturally handle asymmetric/anti-symmetric relations without the need for explosively increasing the number of model parameters by extracting the directed enclosing subgraphs. Extensive experiments show substantial performance gains in comparison to state-of-the-art methods on commonly used benchmark datasets with variant inductive settings.
Introduction
Knowledge graphs have missing links, while many embedding methods cannot handle entities absent during training. CoMPILE addresses inductive relation prediction by reasoning over directed enclosing subgraphs and strengthening communication between entities and relations.
Introduction
- Many knowledge-graph embedding methods assume a fixed entity set, limiting their ability to predict relations for newly added entities without costly retraining.The paper notes that new entities lack learned embeddings because they were unknown during training.Source: Introduction · p. 1
- External-resource methods require extra computation, while rule-based approaches face scalability and expressiveness limitations.Rule-based approaches are entity-independent, but the paper says they are less expressive due to their rule-based nature.Source: Introduction · p. 1
- GraIL predicts links inductively from extracted enclosing subgraphs, without requiring domain-related initial embeddings for emerging entities.Its strategy includes subgraph extraction, relative-position annotation, and GNN scoring.Source: Introduction · pp. 1–2
- GraIL’s undirected extraction, symmetric scoring, and node-based message passing weaken its handling of asymmetric relations and relation information.The paper identifies asymmetric and anti-symmetric relations as particular challenges for this design.Source: Introduction · p. 2
- CoMPILE uses directed enclosing subgraphs and communicative message passing to strengthen entity–relation interactions and handle asymmetric and anti-symmetric relations.It updates edge and entity embeddings simultaneously, with edge-aware attention to aggregate neighborhood features.Source: Introduction · p. 2
Related Work
Transductive knowledge-graph embedding methods learn from graph structure but generally operate on a fixed entity set. Inductive approaches include external-attribute, neighbor-aggregation, and rule-based methods, each with stated limitations.
Transductive Relation Prediction
- Typical knowledge-graph embedding models process triplets independently, while graph neural networks capture neighborhood structure; most such approaches remain transductive.The paper names TransE, DistMult, ComplEx, and ConvE as typical embedding models.Source: Transductive Relation Prediction · p. 2
Inductive Relation Prediction
- Attribute-based inductive methods depend on external resources, while neighbor-aggregation methods require emerging entities to connect to known nodes and cannot handle entirely new graphs.Source: Inductive Relation Prediction · p. 2
- Rule-based methods are entity-independent but face scalability and expressiveness limits; differentiable rule learners also omit neighbor structure and may be less expressive when head–tail paths are sparse.Source: Inductive Relation Prediction · pp. 2–3
CoMPILE
CoMPILE represents target triplets with directed enclosing subgraphs and models information flow between nodes and relation-bearing edges. Its updates and scoring are designed to preserve direction while supporting inductive prediction for unseen entities.
Denotations and Task Definition
- Inductive relation reasoning scores a target triplet whose head and tail representations are unavailable at prediction time, using its enclosing subgraph as the representation.The subgraph contains observed nodes and edges, and relation embeddings are shared across train and test graphs.Source: Denotations and Task Definition · p. 3
Directed Subgraph Extraction
- Directed extraction retains paths from the target head toward the tail, distinguishing a triplet from its reversed counterpart without increasing model complexity.The method prunes the undirected enclosing subgraph; with hop number h, the maximum head-to-tail distance is h+1.Source: Directed Subgraph Extraction · p. 3
Node/Edge Embedding Initialization
- Unseen nodes receive entity-independent embeddings from their directed distances to the target head and from the node to the target tail.Each edge is initialized by concatenating its head-node embedding, relation embedding, and tail-node embedding.Source: Node/Edge Embedding Initialization · p. 3
Directed Subgraph Modeling
- CoMPILE iteratively communicates between nodes and edges, using the target head, relation, and tail to attend to relevant edges and strengthen relation information.Unlike the described CMPNN mechanism, its edge updates account for head, relation, and tail; edge-aware attention uses the whole target triplet.Source: Directed Subgraph Modeling · pp. 3–4
- Each edge updates its tail node but not its head, keeping information flow directed from the target head toward the target tail.The tail embedding can aggregate relations along that path together with their relative positions.Source: Directed Subgraph Modeling · p. 4
- Edge representations aggregate head-node, relation, and tail-node information before being updated, with residual learning and edge dropout also used.The edge aggregation follows the model's directed structure.Source: Directed Subgraph Modeling · pp. 4–5
- The scoring function combines the subgraph representation with target head, tail, and relation representations before a two-layer network predicts triplet plausibility.The design adopts a TransE-inspired form to preserve direction.Source: Directed Subgraph Modeling · p. 5
Experiments
Across the original inductive datasets, CoMPILE improves on most AUC-PR and Hits@10 results; it also outperforms GraIL on most metrics in the extracted datasets. The experiments further examine directed subgraphs, model components, and asymmetric relations.
Datasets
- The authors filter out triplets without enclosing subgraphs and construct negative triplets with subgraphs; extracted datasets cover FB15k-237 and NELL-995, not WN18RR, due to insufficient WN18RR negatives for Hits@10.They cite empty-subgraph triplets as a source of inaccurate performance evaluation.Source: Datasets · p. 5
Experimental Details
- The evaluation compares Neural-LP, DRUM, RuleN, and GraIL using AUC-PR and Hits@10 on inductive train/test graphs, where test entities are unseen during training.For extracted datasets, the sampled negative triplets also have enclosing subgraphs.Source: Experimental Details · p. 5, Datasets · p. 5
Results and Discussions
- CoMPILE improves on most original inductive datasets in AUC-PR and Hits@10, with a significant AUC-PR gain over GraIL on WN18RR.The comparison holds the enclosing subgraph, loss, and node initialization settings the same as GraIL.Source: Table 1 · p. 6, Compare with Baselines on Original… · p. 6, Table 2 · p. 6
- On extracted datasets, CoMPILE outperforms GraIL on most metrics with directed subgraphs in both models, using 35,969 versus 29,264 parameters.The authors attribute CoMPILE’s slightly larger parameter count to modeling both edge and node embeddings.Source: Compare with Baseline on Our Inductive… · p. 6, Table 3 · p. 7
- Directed-subgraph CoMPILE significantly outperforms its undirected version on almost all inductive datasets and metrics, and is more time-efficient.The authors describe the directed subgraph as a cropped version of the undirected one.Source: Directed Subgraph vs. Undirected… · p. 6, Table 4 · p. 7
- Enhanced edge attention adds over 2% on both AUC-PR and Hits@10 for FB15k-237-v1; removing edge updates sharply hurts all metrics, while removing relation information weakens performance.The comparison also finds GraIL’s edge attention yields relatively small improvement and can underperform no attention on FB15k-237-v2.Source: Table 5 · p. 7, Ablation Studies · p. 7
Conclusions
CoMPILE reasons over directed subgraphs for inductive relation prediction, combining enhanced edge attention with edge–node interactions. It outperforms state-of-the-art methods on most evaluation metrics; future work will explore relation-only message passing.
Conclusions
- CoMPILE outperforms state-of-the-art methods on the majority of evaluation metrics.Source: Conclusions · p. 7
- CoMPILE uses directed subgraphs and enhanced edge attention with edge–node interactions to learn node and edge embeddings for inductive relation prediction.Source: Conclusions · p. 7
- The authors filter out triples without subgraphs when constructing new inductive datasets to evaluate methods more accurately.Source: Conclusions · p. 7
- Future work will develop relation-only message passing that discards subgraph nodes and investigate relation enclosing subgraphs.Source: Conclusions · p. 7