Source-linked AI summary
PairRE: Knowledge Graph Embeddings via Paired Relation Vectors
Linlin Chao, Jianshan He, Taifeng Wang, Wei Chu
TL;DR
Knowledge graph embeddings must address both complex relation types and diverse relation patterns, a combination prior methods did not solve well. PairRE uses paired vectors for each relation, and experiments report competitive or state-of-the-art benchmark performance, including on two large Open Graph Benchmark datasets.
Problem
Existing distance-based embedding methods do not simultaneously handle complex relations and multiple relation patterns effectively.
Method
PairRE represents each relation with paired vectors that separately project head and tail entities, with constraints enabling additional subrelation encoding.
Results
PairRE achieves state-of-the-art or highly competitive performance across six benchmarks and outperforms baselines on ogbl-wikikg2 and ogbl-biokg.
Takeaways & Limitations
PairRE combines complex-relation handling with symmetry/antisymmetry, inverse, composition, and subrelation encoding in an efficient model.
Takeaways & Limitations
Entity embeddings are constrained to have L2-norm 1 to remove scaling freedoms, while relation embeddings remain unconstrained for semantic and complex-relation representation.
Abstract
from arXiv · showhide
Distance based knowledge graph embedding methods show promising results on link prediction task, on which two topics have been widely studied: one is the ability to handle complex relations, such as N-to-1, 1-to-N and N-to-N, the other is to encode various relation patterns, such as symmetry/antisymmetry. However, the existing methods fail to solve these two problems at the same time, which leads to unsatisfactory results. To mitigate this problem, we propose PairRE, a model with paired vectors for each relation representation. The paired vectors enable an adaptive adjustment of the margin in loss function to fit for complex relations. Besides, PairRE is capable of encoding three important relation patterns, symmetry/antisymmetry, inverse and composition. Given simple constraints on relation representations, PairRE can encode subrelation further. Experiments on link prediction benchmarks demonstrate the proposed key capabilities of PairRE. Moreover, We set a new state-of-the-art on two knowledge graph datasets of the challenging Open Graph Benchmark.
1 Introduction
Knowledge graph completion addresses missing links, while distance-based embeddings struggle to handle complex relations and relation patterns simultaneously. PairRE uses paired relation vectors to address both challenges and performs competitively across benchmarks.
- Motivation: Missing-link prediction, also called knowledge graph completion, is fundamental because most knowledge graphs are incomplete.Embedding methods address this problem by placing entities and relations in a low-dimensional space.
- Challenges: Distance-based methods must handle 1-to-N, N-to-1, and N-to-N relations while learning symmetry, antisymmetry, inverse, and composition patterns.These requirements can demand different entity representations after relation transformation.
- Prior limitations: Previous methods generally addressed complex relations and relation-pattern encoding separately, leaving complex-relation prediction challenging for pattern-focused methods.RotatE encodes several patterns, but complex relations remain difficult.
- PairRE: PairRE represents each relation with two vectors, enabling simultaneous treatment of complex relations and multiple relation patterns.The vectors project head and tail entities into Euclidean space, where projected distance is minimized.
- Contributions: Paired representations adapt the loss margin to different complex relations, capture symmetry/antisymmetry, inverse and composition, and support subrelations under simple constraints.PairRE is also described as highly efficient and achieves state-of-the-art or highly competitive results on six benchmarks.
2 Background and Notation
This section defines how knowledge graphs, embeddings, complex relations, and relation patterns are represented. It formalizes symmetry, inverse, composition, and subrelation using true triples.
- Notation: A knowledge graph is represented as true fact triples (h, r, t), with entities and relations embedded as column vectors.A scoring function f_r(h,t) measures triple plausibility.
- Complex relations: Relation complexity is classified using average tails per head and heads per tail: thresholds of 1.5 distinguish 1-to-1, N-to-N, and 1-to-N relations.The supplied definition explicitly gives the 1-to-1, N-to-N, and 1-to-N cases.
- Relation patterns: A relation is symmetric when triples remain true after swapping entities, and antisymmetric when the swapped triple is not true.These properties are defined over all entity pairs in the true-triple set.
- Relation patterns: Inverse relations reverse one another, composition links the endpoints of two consecutive relations, and subrelation means every r1 triple also satisfies r2.Each pattern is defined through implications or equivalences over true triples.
3 Related Work
Related work spans distance-based, semantic-matching, and neural approaches, each with different expressive or practical trade-offs. PairRE is presented as combining complex-relation handling with four relation-pattern capabilities.
- Distance based models: TransE is efficient but struggles with symmetry and complex relations, motivating extensions such as TransH, TransR, TransD, and TranSparse.These methods primarily improve complex-relation modeling.
- Distance based models: RotatE encodes symmetry/antisymmetry, inverse, and composition patterns, but complex relations remain challenging.GC-OTE improves RotatE’s complex-relation modeling by adding graph context, with additional computational cost.
- Comparison: Table 1 compares distance-based methods using symmetry, antisymmetry, inverse, composition, and subrelation capabilities, including constrained capabilities marked with an asterisk.The table is described as showing PairRE’s ability to handle complex relations and encode four key patterns.
- Semantic matching models: Semantic-matching models include bilinear and neural-network approaches, but the cited models have difficulty encoding composition relations.Some models are fully expressive only when embedding dimensions satisfy requirements that are difficult to meet in practice.
- Neural methods: Neural methods show promising performance but are difficult to analyze because they function as black boxes.This creates an interpretability trade-off relative to more explicitly structured approaches.
- Subrelation encoding: PairRE encodes subrelations through constraints on relation representations while retaining symmetry/antisymmetry, inverse, and composition capabilities.The paper characterizes this approach as relatively low cost compared with rule-based graph augmentation.
4 Methodology
PairRE represents each relation with paired vectors that project head and tail entities into Euclidean space, minimizing their distance for true triples. This design supports adaptive handling of complex relations and encodes several relation patterns through constrained relation representations.
- PairRE scoring: PairRE represents each relation with paired vectors, rH and rT, that project head and tail entities through entry-wise products before measuring their distance.For a true triple, h ◦rH should be close to t ◦rT; the scoring function uses the negative distance.
- PairRE scoring: The scoring function is defined as the negative L1 distance between the projected head and tail vectors.The paper states that it uses the L1-norm to measure distance.
- Complex relations: Paired relation vectors allow entity representations to be distributed across different relations and adapt the loss margin to complex relation structures.For 1-to-N relations, the model adjusts rT; for N-to-1 relations, it can adjust rH, while larger relation-specific sets require smaller absolute scaling values.
- Relation patterns: PairRE omits a relation-specific translational vector, enabling it to encode symmetry/antisymmetry, inverse, and composition patterns.The paper states these capabilities as propositions for the paired-vector representation.
- Relation patterns: With an inequality constraint, PairRE can encode subrelations by forcing the superrelation triple to receive a more plausible score.For a subrelation pair, the model enforces fr2(h, r2, t) ≥ fr1(h, r1, t).
- Optimization: Training uses the self-adversarial negative sampling loss, with a fixed margin γ and weights for negative samples.The model defines the negative-sample weights through p(h′i).
5 Experimental results
PairRE is evaluated across six knowledge graph benchmarks using standard link-prediction metrics and comparisons with established baselines. Results cover large-scale datasets, relation categories, and relation-pattern and subrelation capabilities.
- Experimental setup: The experiments evaluate link prediction on six benchmarks using MR, MRR, and Hit@n after filtering observed triples.The benchmark suite includes large-scale, complex-relation, symmetry, inverse, composition, and subrelation-focused settings.
- Main results: PairRE achieves state-of-the-art performance on the large-scale ogbl-wikikg2 and ogbl-biokg datasets.PairRE performs best on ogbl-wikikg2 under both limited and increased embedding dimensions and outperforms all baselines on ogbl-biokg.
- Main results: PairRE improves MRR over RotatE by 1.4% on FB15k and 1.3% on FB15k-237 across all evaluation metrics.These comparisons support PairRE’s effectiveness for encoding inverse and composition relations.
- Subrelation: Adding subrelation and inverse-rule constraints improves PairRE by 0.7% on MRR and 1.2% on Hit@1 on DB100k.PairRE already outperforms SeeK and ComplEx-based models with large margins on all evaluation metrics.
- Complex relations: PairRE performs well on N-to-N and N-to-1 relations, while 1-to-N performance is weaker on ogbl-wikikg2.Only 2.2% of test triples belong to the 1-to-N category in ogbl-wikikg2.
- Complex relations: RotatE+PairRelation improves RotatE by 20.8%, 27.5%, 14.4%, and 39.1% on 1-to-1, 1-to-N, N-to-1, and N-to-N relations respectively.The experiments use the same embedding dimension of 50 and attribute the improvements to paired relation vectors’ ability to handle complex relations.
6 Conclusion
PairRE represents each relation with paired vectors to handle complex relations and multiple relation patterns, while also supporting subrelations through simple representation constraints. It outperformed state-of-the-art baselines on ogbl-wikikg2 and ogbl-biokg.
- PairRE represents each relation with paired vectors to handle complex relations and multiple relation patterns.
- With simple constraints on relation representations, PairRE can encode subrelations alongside symmetry/antisymmetry, inverse, and composition relations.
- PairRE outperforms all state-of-the-art baselines on the large-scale ogbl-wikikg2 and ogbl-biokg benchmarks.
- Experiments on other well-designed benchmarks demonstrate PairRE’s effectiveness for its focused key abilities.