Source-linked AI summary
ChronoR: Rotation Based Temporal Knowledge Graph Embedding
Ali Sadeghian, Mohammadreza Armandpour, Anthony Colas, Daisy Zhe Wang
TL;DR
Temporal link prediction must infer time-stamped facts despite the limited focus of prior work on static graphs and the complexity of temporal data. ChronoR uses relation- and time-parameterized k-dimensional rotations and reports stronger benchmark performance than previous state-of-the-art methods, while one cited prior approach is demonstrated only on medical and sensory data.
Problem
Temporal knowledge graph completion is underexplored relative to static graph reasoning, although temporal models must handle evolving facts and complex temporal dependencies.
Method
ChronoR learns entity, relation, and time representations with a k-dimensional rotation and scaling transformation parameterized by relation and time.
Results
ChronoR consistently outperforms competing methods on link prediction MRR and is at least as strong as prior work on Hits@10 across the evaluated datasets.
Takeaways & Limitations
ChronoR provides state-of-the-art temporal knowledge graph performance while modeling temporal and relational patterns economically in parameter count.
Takeaways & Limitations
One related temporal embedding model is demonstrated only on medical and sensory data.
Abstract
from arXiv · showhide
Despite the importance and abundance of temporal knowledge graphs, most of the current research has been focused on reasoning on static graphs. In this paper, we study the challenging problem of inference over temporal knowledge graphs. In particular, the task of temporal link prediction. In general, this is a difficult task due to data non-stationarity, data heterogeneity, and its complex temporal dependencies. We propose Chronological Rotation embedding (ChronoR), a novel model for learning representations for entities, relations, and time. Learning dense representations is frequently used as an efficient and versatile method to perform reasoning on knowledge graphs. The proposed model learns a k-dimensional rotation transformation parametrized by relation and time, such that after each fact's head entity is transformed using the rotation, it falls near its corresponding tail entity. By using high dimensional rotation as its transformation operator, ChronoR captures rich interaction between the temporal and multi-relational characteristics of a Temporal Knowledge Graph. Experimentally, we show that ChronoR is able to outperform many of the state-of-the-art methods on the benchmark datasets for temporal knowledge graph link prediction.
1 Introduction
Knowledge graphs support structured reasoning but remain incomplete, while temporal facts add evolving relationships and time-dependent context. ChronoR addresses temporal link prediction with a rotation-based embedding model and reports stronger benchmark performance than prior methods.
- Background: Knowledge graph completion predicts missing relations between entities, enabling computers to use structured information in applications such as recommendation and question answering.The paper situates temporal completion within broader knowledge graph completion research.
- Motivation: Temporal knowledge graphs extend ordinary knowledge graphs with time, capturing how entities and relations evolve across events.Temporal facts represent dynamic real-world interactions, such as changing relationships among political actors and organizations.
- Problem: Temporal link prediction remains underexplored, and existing models can be parameter-heavy or rely on datasets that are sparse in time.These limitations motivate methods designed specifically for temporally rich knowledge graphs.
- Contribution: ChronoR learns entity representations and a rotation operator parameterized by each fact’s relation and temporal elements.The paper also presents a generalized scoring function and a tensor-nuclear-norm-inspired regularization method.
2 Related Work
Related work spans static knowledge graph embeddings, temporal extensions, sequence models, and time-aware representations. Existing approaches differ in whether they evolve entities, relations, or scoring functions over time.
- Static KG Embeddings: Static knowledge graph embeddings include translational, bilinear, complex-valued, rotational, and quaternion-based models.Examples include TransE, DistMult, ComplEx, RotatE, and QuatE.
- Temporal Embeddings: Temporal link prediction methods may aggregate timestamps into static graphs, weight recent links, or learn time-dependent representations.These approaches extend static reasoning by incorporating temporal information in different ways.
- Temporal Embeddings: Sequence-based temporal models use mechanisms such as Kalman filters and recurrent neural networks to model temporal data.One cited RNN approach dynamically represents relations but not entities.
- Temporal Embeddings: Other temporal methods encode temporal order or time representations, while some applications remain limited to medical and sensory data.The cited limitation narrows the demonstrated application scope of one early approach.
- Temporal Embeddings: Diachronic approaches map entity- or relation–timestamp pairs to hidden representations, whereas other methods keep entity embeddings fixed and modify temporal scoring.This reflects differing assumptions about where temporal dynamics should be represented.
3 Problem Definition
The paper formulates a temporal knowledge graph as time-stamped quadruples and defines completion as inferring missing facts within observed timestamps. The task excludes forecasting facts at future times.
- Notation: The section establishes notation for scalars, vectors, matrices, tensors, element-wise products, concatenation, and norms.These conventions support the mathematical definitions used in later model sections.
- Problem Definition: A temporal knowledge graph consists of quadruples (h, r, t, τ), where entities, relations, and timestamps describe a temporal fact.The timestamp is often discretized and belongs to the set of possible time points.
- Problem Definition: Temporal link prediction infers missing facts from a given subset of temporal knowledge graph facts.The task is framed as temporal knowledge graph completion.
- Problem Definition: This work predicts temporal facts within the observed timestamp set rather than forecasting future facts.That distinction defines the temporal scope of the evaluated completion problem.
4 Temporal KG Representation Learning
ChronoR represents temporal knowledge graph facts with relation- and time-parameterized rotations and scaling. Its scoring function compares transformed heads with tails using angular or inner-product similarity rather than Euclidean distance alone.
- ChronoR: ChronoR parameterizes a k-dimensional linear transformation by both relation and time, combining rotation with scaling.The operator is applied to the head representation so that true facts align with their corresponding tails.
- ChronoR: Higher-dimensional transformations can be constructed from lower-dimensional rotation representations and scalar multiplication.Complex numbers and quaternions represent two- and four-dimensional cases, respectively.
- Scoring Function: The model scores a fact by the angle between the transformed head and tail, unlike RotatE’s Euclidean-distance score.The paper motivates angular similarity partly by the limits of Euclidean proximity in high dimensions.
- Scoring Function: The scoring function uses an inner product, and the paper shows that related complex- and quaternion-valued scoring functions are special cases under suitable representations.The equivalence with ComplEx is established for k = 2.
- Scoring Function: ChronoR expects the cosine of the relative angle between transformed head and tail to be higher for true facts.This makes small relative angles correspond to higher scores.
- ChronoR: Relation and time embeddings are concatenated to parameterize the transformation, with an additional relation-only operator allowed for static and dynamic facts.This parameterization is used in the paper’s experiments.
5 Optimization
ChronoR optimizes temporal link prediction with likelihood-based training, tensor-nuclear-norm-inspired regularization, and temporal smoothness. The proposed Λ4(θ) regularization outperforms other compared regularization methods.
- 5.3 Loss Function: The model trains by minimizing the negative log-likelihood of correct tail prediction for each training quadruple.The likelihood assigns probabilities to candidate tails using the scoring function, with θ denoting all model parameters.
- 5.3 Loss Function: Likelihood training requires summing over every candidate entity but avoids generating negative samples when computationally feasible.The paper reports that this approach performed better at the scale used in its experiments.
- 5.1 Regularization: ChronoR treats the temporal knowledge graph as an order-4 tensor and derives regularization from tensor nuclear norm connections.Entity, relation, and time embeddings form the tensors associated with the decomposition.
- 5.1 Regularization: Λ4(θ) outperforms the other regularization methods in empirical comparisons.The paper states that theoretical results supporting Equation 9 are provided in the Appendix.
- 5.2 Temporal Regularization: Using the same 4-norm for ΛΓ and Λ4 makes the loss components similarly scaled and simplifies tuning λ1 and λ2.The paper relates this choice to practical hyperparameter tuning and scale matching across loss terms.
- 5.3 Loss Function: The training objective combines edge-estimation terms with Λ4(θ) and a temporal smoothness term.The first two terms estimate TKG edges, while the third incorporates temporal smoothness behavior.
6 Experiments
ChronoR is evaluated on three temporal knowledge graph benchmarks under parameter-matched experimental settings. It consistently outperforms competitors on MRR, matches or exceeds them on Hits@10, and benefits from the proposed regularization.
- 6.2 Experimental Setup: Embedding dimensions are selected without tuning so each model has the same parameter count as the comparison models.The appendix lists the resulting dimensions for each dataset.
- 6.1 Datasets: The evaluation covers ICEWS14, ICEWS05-15, and Yago15K, with ICEWS datasets providing timestamps at 24-hour granularity.ICEWS14 spans 2014, while ICEWS05-15 spans 2005–2015; Yago15K is also included as a benchmark.
- 6.2 Experimental Setup: Evaluation ranks all candidate entities for missing heads and tails and reports filtered MRR together with Hits@1, Hits@3, and Hits@10.The protocol fills both (?, r, t, τ) and (h, r, ?, τ) for each test quadruple.
- 6.3 Results: ChronoR consistently outperforms all competitors in link prediction MRR and is greater than or equal to prior work on Hits@10 across the evaluated datasets.Table 1 reports the comparison across ICEWS14, ICEWS05-15, and Yago15K.
- 6.3 Results: Three-dimensional rotations improve ICEWS05-15 but underperform planar rotations on ICEWS14 and Yago15K, while higher dimensions provide no significant gain.The authors relate this pattern to the greater number of relations and timestamps in ICEWS05-15 and leave more sophisticated tests for future work.
- 6.3 Results: The proposed Λ4 regularizer increases MRR by 2 points, and selecting its regularization weight carefully can increase MRR by up to 7 points.Λ4 is linked directly to the 4-tensor factorization, while Figure 2 compares it with Ω3 from TNTComplEx.
7 Conclusion
The conclusion presents ChronoR as a k-dimensional rotation-based embedding model for temporal knowledge graphs. It states that the method captures temporal and relational dynamics, achieves state-of-the-art benchmark performance, and remains economical in parameter count.
- 7 Conclusion: ChronoR models temporal knowledge graphs with k-dimensional rotations and scaling operations applied to entity representations.The model accounts for changes in entities and relations with respect to time.
- 7 Conclusion: ChronoR provides state-of-the-art performance on benchmark temporal knowledge graphs while using relatively few parameters.The conclusion also states that the model can represent temporal and relational patterns.
- 7 Conclusion: The work generalizes rotation-based models from static knowledge graphs to the temporal domain and connects prior scoring and regularization functions.Future work considers combining other geometric transformations and rotations and exploring additional regularization techniques.