Source-linked AI summary
Temporal Knowledge Graph Reasoning Based on Evolutional Representation Learning
Zixuan Li, Xiaolong Jin, Wei Li, Saiping Guan, Jiafeng Guo, Huawei Shen, Yuanzhuo Wang, Xueqi Cheng
TL;DR
Future-fact prediction in temporal knowledge graphs requires modeling both within-timestamp structure and across-timestamp history, yet existing methods provide limited effectiveness and efficiency. RE-GCN recurrently represents the whole KG sequence with relation-aware graph convolution, gated temporal modeling, and static entity properties. Across six benchmarks, it reports consistently better temporal reasoning performance and 17-to-82-times speedup in entity prediction over RE-NET.
Problem
Temporal KG reasoning for future facts remains relatively unexplored, while existing methods incompletely capture concurrent structural dependencies, sequential patterns, and static entity properties.
Method
RE-GCN recurrently models the whole KG sequence using relation-aware GCNs, gated recurrent components, and static graph constraints to learn entity and relation representations.
Results
Across six benchmarks and both temporal reasoning tasks, RE-GCN consistently improves performance, with up to 11.46% MRR improvement and 17-to-82-times speedup in entity prediction over RE-NET.
Takeaways & Limitations
Modeling the KG sequence as a whole enables evolutional representations that support entity and relation prediction while substantially improving efficiency.
Takeaways & Limitations
On GDELT, abstract concepts make some raw-setting temporal predictions almost impossible, so models predict only partial facts and obtain similar results.
Abstract
from arXiv · showhide
Knowledge Graph (KG) reasoning that predicts missing facts for incomplete KGs has been widely explored. However, reasoning over Temporal KG (TKG) that predicts facts in the future is still far from resolved. The key to predict future facts is to thoroughly understand the historical facts. A TKG is actually a sequence of KGs corresponding to different timestamps, where all concurrent facts in each KG exhibit structural dependencies and temporally adjacent facts carry informative sequential patterns. To capture these properties effectively and efficiently, we propose a novel Recurrent Evolution network based on Graph Convolution Network (GCN), called RE-GCN, which learns the evolutional representations of entities and relations at each timestamp by modeling the KG sequence recurrently. Specifically, for the evolution unit, a relation-aware GCN is leveraged to capture the structural dependencies within the KG at each timestamp. In order to capture the sequential patterns of all facts in parallel, the historical KG sequence is modeled auto-regressively by the gate recurrent components. Moreover, the static properties of entities such as entity types, are also incorporated via a static graph constraint component to obtain better entity representations. Fact prediction at future timestamps can then be realized based on the evolutional entity and relation representations. Extensive experiments demonstrate that the RE-GCN model obtains substantial performance and efficiency improvement for the temporal reasoning tasks on six benchmark datasets. Especially, it achieves up to 11.46\% improvement in MRR for entity prediction with up to 82 times speedup comparing to the state-of-the-art baseline.
1 INTRODUCTION
Temporal KG reasoning predicts future facts from historical graph sequences, but existing methods incompletely capture concurrent structure, temporal patterns, and static entity properties. RE-GCN models these signals jointly and efficiently for entity and relation prediction.
- TKGs represent time-stamped facts as a sequence of KGs, making future-fact prediction an important but relatively unexplored reasoning problem.
- Extrapolation predicts facts after the observed period, supporting applications such as disaster relief and financial analysis.
- Concurrent facts create structural dependencies, while temporally adjacent facts reveal sequential patterns that influence entity and relation behavior.
- Earlier methods either cannot model concurrent facts or encode only query-specific histories, reducing coverage and efficiency while often omitting static entity properties.
- RE-GCN treats the TKG as a KG sequence, using relation-aware GCNs and recurrent components to encode historical facts into entity and relation representations.
2 RELATED WORKS
Prior work covers static KG reasoning and temporal interpolation, while this paper focuses on extrapolating future facts from historical TKGs. RE-GCN is presented as a sequence-based alternative to query-specific and continuous-time approaches.
- Static KG Reasoning: Static KG reasoning models missing facts in fixed multi-relational graphs, but static models cannot predict future facts.
- Temporal KG Reasoning: Temporal interpolation methods infer missing historical facts by incorporating timestamps or time-specific geometric structure.
- Temporal KG Reasoning: Extrapolation methods predict new facts at future timestamps from historical facts, which is the setting addressed by this paper.
- Temporal KG Reasoning: Some related approaches use temporal point processes for continuous-time prediction, whereas RE-GCN models a discrete sequence of timestamped KGs.
3 PROBLEM FORMULATION
A TKG is formalized as timestamped directed multi-relational graphs, and future entity or relation prediction uses recent historical KGs under a finite-history assumption.
- A TKG is a sequence of timestamped KGs, with each KG containing entities, relations, and facts occurring at one discrete timestamp.
- Each fact is represented as a quadruple (s, r, o, t), and its inverse quadruple is appended to the dataset.
- The static graph separately represents entity and relation properties through its own entity, relation, and edge sets.
- Entity prediction fills a missing subject or object, while relation prediction fills the missing relation at timestamp t+1.
- Future predictions are assumed to depend on the KGs at the latest m timestamps, whose information is encoded in evolutional embedding matrices.
4 THE RE-GCN MODEL
RE-GCN models a temporal knowledge graph as a recurrent sequence of graphs, combining within-timestamp structure, cross-timestamp patterns, and static entity properties. Its evolution unit produces entity and relation representations for future temporal reasoning with a computationally efficient design.
- Evolution Unit: RE-GCN combines a relation-aware GCN, recurrent components, and a static graph constraint component to encode temporal knowledge graphs.The evolution unit models structural dependencies, historical sequence patterns, and static entity properties.
- Structural Dependencies among Concurrent Facts: A relation-aware GCN captures structural dependencies among concurrent facts through relation-sensitive message passing.Entity embeddings receive information from subject entities while incorporating relation embeddings; self-loops support entities without facts.
- Sequential Patterns across Temporally Adjacent Facts: The model uses recurrent processing to capture sequential patterns across temporally adjacent facts for entities and relations.Entity evolution uses a time-gated component, while relation evolution uses a GRU informed by related entities and prior relation embeddings.
- Static Properties: Static entity properties are incorporated through a one-layer R-GCN and an angle-based constraint on evolutional and static embeddings.The permitted angle increases over time, with a maximum angle of 90°.
- Computational Complexity Analysis: The evolution unit has time complexity O(m(|E|ω + |R|D) + |E_s|).This combines recurrent GCN and pooling costs across m historical steps with static-embedding computation.
5 EXPERIMENTS
Experiments on six benchmark TKG datasets evaluate RE-GCN for entity and relation prediction, with comparisons to static and temporal baselines, ablations, and runtime analysis. RE-GCN generally performs strongly, while performance is constrained on GDELT by abstract entities and noise.
- Entity prediction: RE-GCN consistently outperforms baselines on entity prediction across ICEWS18, ICEWS14, ICEWS05-15, WIKI, and YAGO.The paper attributes gains over static models to sequential-pattern modeling and gains over temporal models to additionally modeling temporal patterns and static entity properties.
- Entity prediction: 8.97/11.46% improvements in MRR, 10.60/12.91% in Hits@3, and 12.61/14.01% in Hits@10 over the best baseline are reported on WIKI and YAGO.The paired values correspond to the two datasets discussed in the results.
- Dataset-specific analysis: GDELT yields similarly poor results across models because many entities are abstract concepts, making some raw-setting predictions nearly impossible and introducing noise.Among the top 50 frequent entities, 28 are abstract concepts, and 43.72% of corresponding facts involve abstract concepts.
- Relation prediction: RE-GCN performs better than all selected baselines on relation prediction, although its performance gap is smaller than for entity prediction.The paper explains that relation prediction has fewer candidates and is therefore easier than entity prediction.
- Efficiency: RE-GCN is 66, 36, 17, 82, 82, and 22 times faster than RE-NET on ICE18, ICE14, ICE05-15, WIKI, YAGO, and GDELT, respectively.The runtime comparison uses entity prediction on the test set with ground-truth history under the same environment.
- Ablation studies: Removing the evolution unit substantially harms results on all datasets except GDELT, indicating that historical-information modeling is important for the evaluated tasks.The ablation uses the ConvTransE score function with randomly initialized learnable embeddings.
- Ablation studies: The static graph constraint improves RE-GCN on ICEWS datasets, while removing the time gate causes performance to decline rapidly as the historical sequence length increases.Static entity type and location information enrich entity representations, and the time gate supports longer historical sequences.
- Case study: Case studies show RE-GCN combining sequential patterns across timestamps with structural dependencies among concurrent facts for top-ranked predictions.The examples include entity prediction and relation prediction cases from ICEWS18.
6 CONCLUSIONS
RE-GCN improves temporal reasoning by modeling structural dependencies among concurrent facts, sequential patterns across adjacent timestamps, and static entity properties. Across six benchmarks, it outperforms on two temporal reasoning tasks and achieves substantial entity-prediction speedups.
- RE-GCN captures concurrent-fact structure, adjacent-timestamp patterns, and static entity properties in evolutional entity and relation representations.
- RE-GCN demonstrates significant performance and superiority across two temporal reasoning tasks on six benchmark datasets.
- 17 to 82 times speedup in entity prediction is achieved compared with RE-NET, the state-of-the-art baseline.