Source-linked AI summary
RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, Jian Tang
TL;DR
Missing-link prediction in incomplete knowledge graphs requires representations that model relation patterns. RotatE uses complex-space rotations and self-adversarial negative sampling, achieving state-of-the-art results across the reported benchmarks while modeling symmetry/antisymmetry, inversion, and composition.
Problem
Knowledge graph embedding seeks representations for predicting missing links, but existing models do not model and infer all symmetry/antisymmetry, inversion, and composition patterns.
Method
RotatE represents entities and relations in complex space, defines relations as rotations, and uses self-adversarial negative sampling for training.
Results
RotatE significantly outperforms existing state-of-the-art approaches across four large-scale benchmarks and achieves state-of-the-art results on a composition-focused benchmark.
Takeaways & Limitations
RotatE models all three relation patterns, and its relation embeddings implicitly represent symmetry/antisymmetry, inversion, and composition.
Abstract
from arXiv · showhide
We study the problem of learning representations of entities and relations in knowledge graphs for predicting missing links. The success of such a task heavily relies on the ability of modeling and inferring the patterns of (or between) the relations. In this paper, we present a new approach for knowledge graph embedding called RotatE, which is able to model and infer various relation patterns including: symmetry/antisymmetry, inversion, and composition. Specifically, the RotatE model defines each relation as a rotation from the source entity to the target entity in the complex vector space. In addition, we propose a novel self-adversarial negative sampling technique for efficiently and effectively training the RotatE model. Experimental results on multiple benchmark knowledge graphs show that the proposed RotatE model is not only scalable, but also able to infer and model various relation patterns and significantly outperform existing state-of-the-art models for link prediction.
1 INTRODUCTION
Knowledge graph embedding addresses missing-link prediction by learning representations that capture relation patterns. RotatE represents relations as rotations in complex space, models symmetry/antisymmetry, inversion, and composition, and is trained with self-adversarial negative sampling.
- Problem: Knowledge graphs are often incomplete, making missing-link prediction a fundamental task for knowledge graph embedding.
- Motivation: Existing models capture only some relation patterns, leaving a need for one approach that models symmetry/antisymmetry, inversion, and composition.
- RotatE: RotatE maps entities and relations to complex vectors and defines each relation as a rotation from the source entity to the target entity.
- RotatE: For a triplet (h, r, t), RotatE expects t = h ◦ r, with unit-modulus relation embeddings and element-wise multiplication.
- Properties: RotatE can model symmetry/antisymmetry, inversion, and composition, while remaining linear in both time and memory.
- Training and evaluation: Self-adversarial negative sampling generates negative samples from current entity and relation embeddings, and experiments span multiple benchmark knowledge graphs.
2 RELATED WORK
Related work includes score-function-based knowledge graph embedding, compact-group translation models, relational-path approaches, and adversarial negative sampling. RotatE is positioned as an implicitly pattern-learning approach with broader representation capacity and scalable entity and relation embeddings.
- Pattern modeling: Table 2 compares models by their abilities to model and infer relation patterns.
- Knowledge graph embedding: Knowledge graph embedding methods score factual triplets above corrupted triplets to support missing-link prediction.
- Related models: TorusE can be viewed as a special case of RotatE with fixed embedding moduli, whereas RotatE operates on the entire complex space.
- Relational approaches: Relational approaches mainly model relational paths explicitly, while RotatE implicitly learns relation patterns and provides embeddings for entities and relations.
- Negative sampling: Self-adversarial sampling avoids the simultaneous training of an embedding model and discrete negative-sample generator required by a prior generative adversarial framework.
3 ROTATE: RELATIONAL ROTATION IN COMPLEX VECTOR SPACE
RotatE represents relations as element-wise rotations in complex vector space, targeting the modeling and inference of symmetry, inversion, and composition patterns. The section also introduces self-adversarial negative sampling for optimization.
- 3.1 MODELING AND INFERRING RELATION PATTERNS: Knowledge-graph link prediction relies on inferring symmetry, inversion, and composition patterns from observed facts.
- 3.2 MODELING RELATIONS AS ROTATIONS IN COMPLEX VECTOR SPACE: RotatE maps entities and relations to complex embeddings and defines each relation as an element-wise rotation from head to tail.Each relation embedding has unit modulus, so it changes entity phases through complex multiplication.
- 3.2 MODELING RELATIONS AS ROTATIONS IN COMPLEX VECTOR SPACE: RotatE can model and infer symmetry/antisymmetry, inversion, and composition patterns.The paper states these results as Lemmas 1–3 and summarizes them in Table 2.
- 3.2 MODELING RELATIONS AS ROTATIONS IN COMPLEX VECTOR SPACE: RotatE addresses TransE’s limitation on symmetric relations by representing them with arbitrary vectors whose elements satisfy ri = ±1.In TransE, symmetric relations correspond to a 0 translation vector; RotatE uses rotations instead.
- 3.3 OPTIMIZATION: Self-adversarial negative sampling draws negative triples according to the current embedding model and uses their probabilities as loss weights.This is proposed because uniform sampling becomes inefficient when many negatives are obviously false during training.
4 EXPERIMENTS
RotatE is evaluated across benchmark knowledge graphs whose difficulty reflects different relation patterns, with results generally supporting its ability to model those patterns. The experiments also examine scalability, relation categories, and the contribution of modulus information.
- Main results: RotatE outperforms existing state-of-the-art models on FB15k and WN18, while its performance is similar to pRotatE on these datasets.On FB15k-237 and WN18RR, RotatE improves more substantially over pRotatE.
- Main results: RotatE’s larger gains on FB15k-237 and WN18RR indicate that modulus information is important for modeling and inferring composition patterns.These datasets contain many composition patterns.
- Main results: Performance across datasets is consistent with the relation-pattern analysis: ComplEx performs well when symmetry/antisymmetry and inversion dominate, whereas TransE performs well when composition dominates.DistMult’s performance varies with entity-type structure and its inability to model antisymmetry and inversion.
- Countries dataset: On the Countries dataset, RotatE outperforms all previous models under AUC-PR, with the largest advantage on S3, the most difficult task.In a separate comparison, TransE slightly outperforms RotatE on Countries S3 because the tasks do not require symmetric relations.
- Relation categories: RotatE performs well on non-injective relations, especially many-to-many relations, in the FB15k relation-category evaluation.The study also reports that KG2E KL(bern) consistently outperforms its corresponding embedding model, motivating uncertainty modeling as future work for RotatE.
5 CONCLUSION
The paper presents RotatE as a complex-space embedding method with self-adversarial negative sampling, reporting broad benchmark performance and pattern modeling. It also identifies future evaluation and uncertainty modeling as next steps.
- RotatE represents entities as complex vectors and relations as rotations in complex vector space.
- Self-adversarial negative sampling is proposed to train RotatE efficiently and effectively.
- RotatE outperforms existing state-of-the-art models on four large-scale benchmarks and achieves state-of-the-art results on a composition-focused benchmark.
- The paper plans to evaluate RotatE on more datasets and use a probabilistic framework to model entity and relation uncertainties.
- The analysis reports that existing models cannot model all three relation patterns, motivating RotatE's broader pattern coverage.
B PROOF OF LEMMA 1
The proof derives constraints for symmetric and antisymmetric relations from RotatE's rotation equations.
- For a symmetric relation, applying the relation in both directions yields r ◦ r = 1.
- The proof starts from r(x, y) and r(y, x) holding simultaneously to characterize symmetry.
- For an antisymmetric relation, the proof instead assumes r(x, y) holds while r(y, x) does not.
C PROOF OF LEMMA 2
The proof considers inverse relations by combining their rotations across reversed entity pairs.
- The proof assumes r1(x, y) and r2(y, x) hold for the two relations.
- Reversing the entity pair connects the two relation embeddings through RotatE's rotation formulation.
- The lemma establishes the algebraic condition associated with inverse relations in the complex rotation space.
D PROOF OF LEMMA 3
The section connects RotatE's relation operations to inverse acquisition, angular composition, and a TransE-like distance behavior.
- For a composition pattern, the proof assumes r1(x, z), r2(x, y), and r3(y, z), deriving r1 = r2 ◦ r3.
- Complex conjugation provides the inverse of a relation and treats head and tail entities uniformly for efficient 1-N scoring.
- With fixed entity moduli, the RotatE distance depends on the angular difference between head, relation, and tail embeddings.
- The angular distance form is similar to TransE's distance function, and RotatE can degenerate into TransE.
- The modulus contributes a lower bound to the distance function through the difference between entity moduli.
F PROOF OF THEOREM 4
The proof restricts embedding magnitudes and rewrites complex embeddings through phase-based trigonometric forms. It also connects these phase representations to TransE embeddings through scaling, while the surrounding results report performance on YAGO3-10.
- F PROOF OF THEOREM 4: Restricting |h_i| and |t_i| to a constant enables rewriting the embeddings in phase-based complex form.The proof introduces constant-magnitude representations before expanding them into cosine and sine components.
- F PROOF OF THEOREM 4: The resulting distance expression depends on the phase difference θ_h + θ_r − θ_t through cosine and sine terms.This expresses the rotation relation geometrically in complex space.
- F PROOF OF THEOREM 4: Setting θ_h = ch′, θ_r = cr′, θ_t = ct′ and C = 1/c connects the phase formulation to TransE embeddings.The proof uses scaled TransE vectors as the corresponding phase parameters.
- G LINK PREDICTION ON YAGO3-10: RotatE also outperforms state-of-the-art models on YAGO3-10.The supplied result passage reports the comparison without giving metric values.
I ABLATION STUDY
The ablation study separates the effects of self-adversarial sampling and negative sampling loss, and examines result stability across random initializations. Self-adversarial sampling helps both RotatE and TransE, while RotatE's negative sampling loss has an additional model-specific effect.
- Ablation Study: Self-adversarial sampling boosts performance for both RotatE and TransE on FB15k-237.The study also re-implements a 1000-dimension TransE for this comparison.
- Ablation Study: Negative sampling loss is effective on RotatE but not on TransE in the reported ablation.The comparison is conducted on FB15k-237.
- Ablation Study: RotatE's MRR performance is stable across different random initializations on FB15k, WN18, FB15k-237 and WN18RR.Average and variance are calculated from three runs with different random seeds.
K MORE RESULTS OF IMPLICIT BASIC PATTERN INFERENCE
Additional visualizations examine embedding phases and phase additions associated with implicit relation patterns. The supplied captions identify the datasets, relation categories, and histogram quantities but do not state comparative outcomes.
- Implicit pattern inference: The paper provides additional embedding-phase histograms in Figures 3–5.These figures cover WN18 and FB15k-237 visualizations.
- Embedding-phase histograms: Figure 3 shows histograms of embedding phases for two general relations and four symmetric relations on WN18.The figure uses k = 500.
- Embedding-phase histograms: Figure 4 shows histograms of embedding phases for six symmetric relations on FB15k-237.The figure uses k = 1000.
- Implicit pattern inference: Figure 5 shows histograms of element-wise additions of inversed relation embedding phases on WN18.The figure uses k = 500.