Source-linked AI summary
TeRo: A Time-aware Knowledge Graph Embedding via Temporal Rotation
Chengjin Xu, Mojtaba Nayyeri, Fouad Alkhoury, Hamed Shariat Yazdi, Jens Lehmann
TL;DR
Temporal knowledge graphs require reasoning over facts whose validity changes with time, while existing KGE and TKGE models have limitations in handling temporary relations and relation patterns. TeRo models temporal evolution through complex-space rotations and dual embeddings for interval endpoints, and experiments report superior link-prediction performance across four TKG datasets while examining time granularity.
Problem
Temporal KGs require time-aware reasoning, but traditional KGE ignores time and many TKGE extensions are not fully expressive for some relation patterns.
Method
TeRo represents temporal entity evolution as complex-space rotations and uses dual relation embeddings for interval beginnings and endings.
Results
TeRo remarkably outperforms existing state-of-the-art KGE and TKGE models on link prediction across four established TKG datasets.
Takeaways & Limitations
TeRo handles multiple relation patterns and varied time-annotation forms while the study shows that time granularity affects its performance.
Takeaways & Limitations
Temporal extensions of DistMult cannot model asymmetric relations, while temporal extensions of TransE have difficulty modeling multiple reflexive relations.
Abstract
from arXiv · showhide
In the last few years, there has been a surge of interest in learning representations of entitiesand relations in knowledge graph (KG). However, the recent availability of temporal knowledgegraphs (TKGs) that contain time information for each fact created the need for reasoning overtime in such TKGs. In this regard, we present a new approach of TKG embedding, TeRo, which defines the temporal evolution of entity embedding as a rotation from the initial time to the currenttime in the complex vector space. Specially, for facts involving time intervals, each relation isrepresented as a pair of dual complex embeddings to handle the beginning and the end of therelation, respectively. We show our proposed model overcomes the limitations of the existing KG embedding models and TKG embedding models and has the ability of learning and inferringvarious relation patterns over time. Experimental results on four different TKGs show that TeRo significantly outperforms existing state-of-the-art models for link prediction. In addition, we analyze the effect of time granularity on link prediction over TKGs, which as far as we know hasnot been investigated in previous literature.
1 Introduction
Temporal knowledge graphs add time annotations to multi-relational facts, creating a need for models that reason over changing relations. TeRo addresses limitations of existing temporal embeddings through complex-space temporal rotation, interval-aware relation representations, and time-granularity analysis.
- 1 Introduction: TKGs represent facts as quadruples with time annotations, but traditional KGE models disregard time and perform poorly on temporary relations.Examples include visits and live in, whose validity depends on time.
- 1 Introduction: Existing TKGE models encode time and improve link prediction, yet many extend TransE or DistMult and are not fully expressive for some relation patterns.The paper motivates a model that addresses these expressiveness limitations.
- 1 Introduction: TeRo models temporal entity evolution as a rotation from initial to current time in complex vector space.This is the paper’s central temporal representation mechanism.
- 1 Introduction: For interval facts, TeRo uses dual complex relation embeddings to represent relation beginnings and endings, accommodating varied time-annotation forms.The design supports time points, beginning or end times, and time intervals.
- 1 Introduction: The study varies time-division strategies across TKG datasets to investigate how time-step length affects model performance.This addresses a time-granularity choice that prior work typically fixed for each dataset.
- 1 Introduction: Experiments on link prediction and time prediction over four TKGs report significant improvements over state-of-the-art KGE and TKGE baselines.The reported gains are attributed to inferring relation patterns and encoding time information.
2 Related Work
Related work groups KGE models into distance-based and semantic-matching approaches, then extends both families with temporal information. These temporal extensions improve TKG reasoning but inherit limitations from their underlying static models.
- 2 Related Work: KGE models are broadly classified as distance-based models or semantic-matching models.The distinction concerns how fact plausibility is computed from entity and relation embeddings.
- 2 Related Work: Distance-based models score facts using entity distance after relation-based translation or rotation, with TransE as a representative example.TransE’s deficiencies for 1-n relations motivated multiple extensions using different entity-to-relation mappings.
- 2 Related Work: Semantic-matching models compare latent entity and relation semantics; RESCAL and DistMult cannot capture asymmetric relations, unlike several later models.The passage names ComplEx, QuatE, and GeomE as models proven able to capture various relation patterns.
- 2 Related Work: Temporal KGE models extend either TransE or DistMult to incorporate time information into knowledge-graph reasoning.Examples include TTransE, TA-TransE, HyTE, ATiSE, Know-Evolve, TDistMult, and TA-DistMult.
3 A Novel TKGE Approach based on Temporal Rotation
TeRo models temporal entity evolution as rotation in complex space and uses dual relation embeddings for interval endpoints. It is designed to capture temporary, asymmetric, and reflexive relation patterns while keeping complexity near TransE when time steps are fewer than entities.
- Relation patterns: TeRo addresses limitations of static and TransE- or DistMult-based temporal models in representing temporary, asymmetric, and multiple reflexive relations.The cited limitations include time-invariant static scores for temporary relations, symmetric scores in temporal DistMult extensions, and collapsed reflexive relations in temporal TransE extensions.
- Temporal rotation: TeRo represents temporal evolution by rotating time-independent entity embeddings into time-specific embeddings in complex vector space.Each time step induces an element-wise rotation, while relation embeddings translate the time-specific subject toward the conjugate object embedding.
- Interval modeling: Interval facts are split into beginning- and end-time quadruples using dual relation sets, with their scores averaged.Facts missing one endpoint use the score from the known endpoint only.
- Relation patterns: TeRo captures temporary relations through time-dependent mismatch, asymmetric relations through directional mismatch, and multiple reflexive relations through distinct complex embeddings.For reflexive relations, the model uses the condition Im(r) = 2Im(st1) and conjugate object operations.
- Complexity: TeRo and TransE have close space complexity when the number of time steps is smaller than the number of entities.The paper states that tuning time granularity can achieve this condition in practice.
4 Experiments
The experiments evaluate TeRo on four temporal knowledge-graph datasets using time-aware link prediction, varied time granularity, and comparisons with established KGE and TKGE baselines.
- 4.1 Temporal Knowledge Graph Datasets: The study uses ICEWS14, ICEWS05-15, YAGO11k, and Wikidata12k as temporal knowledge-graph datasets.ICEWS14 and ICEWS05-15 are event-based datasets; YAGO11k and Wikidata12k are also included.
- 4.1 Temporal Knowledge Graph Datasets: Time granularity is fixed at one day for ICEWS datasets, while YAGO and Wikidata aggregate dates using frequency-based time steps.The experiments also vary ICEWS time units and YAGO/Wikidata minimum thresholds.
- 4.2 Experimental Settings: Link prediction is evaluated under a time-wise filtered setting that excludes only quadruples already existing in the dataset.Facts absent at the test time remain candidate answers, unlike time-unwise filtering.
- 4.3 Evaluation Metrics: Mean Reciprocal Rank and Hits@k measure ranking quality, with MRR averaging reciprocal ranks and Hits@k measuring top-k test coverage.These metrics evaluate the rank of each test quadruple among filtered candidates.
- 4.4 Baselines and Implementation: TeRo is compared with state-of-the-art KGE and TKGE baselines, and its implementation uses PyTorch with validation-based hyperparameter selection.The listed baselines include TransE, DistMult, ComplEx-N3, RotatE, QuatE, TTransE, TA-TransE, TA-DistMult, DE-SimplE, and ATiSE.
5 Results and Analysis
TeRo outperforms baseline models across nearly all link-prediction metrics and datasets, while its performance depends strongly on time granularity. The model also captures temporal relation behavior with comparable space complexity and efficient interval handling.
- Comparative Study: 1.2, 6.7, 1.7, and 1.9 MRR points were TeRo’s improvements over ATiSE on ICEWS14, ICEWS05-15, YAGO11k, and Wikidata12k, respectively.These improvements are reported in dataset order.
- Ablation Study: Small time granularities improved ICEWS14 performance, while one-year granularity made all ICEWS14 annotations share a temporally unmeaningful embedding.For ICEWS14, 1-day, 2-day, and 3-day settings performed better than larger granularities on MRR and Hits@3.
- Ablation Study: TeRo with u = 1 often predicted ICEWS14 examples correctly, whereas u = 365 produced wrong predictions that could remain valid without the time constraint.The examples involve temporary relations such as Host a visit and Criticize or denounce.
- Efficiency Study: TeRo has the same space complexity as TTransE and HyTE, while ATiSE and DE-SimplE use 1.8 and 2.2 times as much memory as TeRo on ICEWS14.With 500-dimensional embeddings, training takes 4.3, 25.9, 1.9, and 4.1 seconds per epoch on the four datasets, respectively.
- Ablation Study: For interval relations, short-term pairs of beginning and ending embeddings are highly similar, whereas long-term pairs retain semantic similarity in real parts but temporal dissimilarity in imaginary parts.The analysis compares deadIn and isMarriedTo in YAGO11k.
6 Conclusion
The paper introduces TeRo, a temporal knowledge-graph embedding model that represents temporal changes as rotations in complex vector space. It reports strong link-prediction performance across four datasets and examines how time division affects performance.
- Conclusion: TeRo represents entities or relations with single or dual complex embeddings and models temporal changes as rotations in complex vector space.Dual embeddings support varied time annotations, including points, beginning or end times, and intervals.
- Conclusion: TeRo outperforms existing state-of-the-art KGE and TKGE models on link prediction across four established temporal knowledge graphs.The conclusion also reports an investigation of time granularity across different datasets.