Source-linked AI summary
HiTS-CL: A Continual Learning Framework for Long-Horizon Temporal Knowledge Graph Extrapolation
Yansong Liu, Rui Liu, Yuan Zuo, Hongwei Zhao, Da Fu, Fuwei Zhang, Fuzhen Zhuang, Yong Chen, Zhe Li
TL;DR
Extrapolative temporal knowledge graph reasoning must predict future facts, but fixed-prefix models remain static as the stream evolves. HiTS-CL formulates the task as continual learning and combines fine-tuning, distillation, and selective memory. Across five backbones and four benchmarks, it consistently improves extrapolation accuracy and reduces long-horizon degradation.
Problem
Fixed-prefix methods train once on early snapshots and then remain unchanged, although the stream evolves through new facts, shifting dependencies, and recurring signals.
Source: §1 · pp. 1–2Method
HiTS-CL treats extrapolative TKGR as continual learning, using fine-tuning for current dynamics, adaptive multi-teacher distillation for stable knowledge, and selective memory for recurring evidence.
Source: §1 · p. 2Results
HiTS-CL consistently improves extrapolation accuracy and reduces long-horizon degradation across five backbones and four benchmarks; against +Replay, relative MRR improves 6.0% on ICEWS14.
Source: §5.2 · p. 7, §7 · p. 10Takeaways & Limitations
The results support continual learning over streaming snapshots as a framework for improving extrapolative TKGR under the evaluated settings.
Source: §1 · p. 2At each evaluation time, only facts timestamped earlier than that time are available as historical context; earlier validation and test snapshots enter history as evaluation proceeds.
Source: §5.1.2 · pp. 6–7
Abstract
from arXiv · showhide
Extrapolative temporal knowledge graph reasoning (TKGR) predicts future facts from historical snapshots. Most existing methods train once on an early prefix of the timeline and then use a frozen model for all future timestamps. We argue that this fixed-prefix protocol is misaligned with extrapolation. It learns from a static prefix, whereas the target stream is non-stationary: new entities and facts emerge, temporal dependencies shift across regimes, and recurring historical signals must be refreshed online. As a result, models trained only on early snapshots become outdated and degrade over long horizons. We address this mismatch by formulating extrapolative TKGR as continual learning over streaming snapshots. Under this view, effective extrapolation must jointly handle current dynamics, stable knowledge, and recurring historical evidence. Based on these requirements, we propose History-enhanced Two-Step Continual Learning (HiTS-CL), a backbone-agnostic continual learning framework for extrapolative TKGR. HiTS-CL tracks current dynamics via continual fine-tuning, preserves stable knowledge via multi-teacher adaptive distillation, and retains recurring historical evidence via a selective memory of recent and frequent facts. We integrate HiTS-CL into five representative TKGR backbones and evaluate it on four benchmark datasets. HiTS-CL consistently improves extrapolation accuracy, reduces long-horizon degradation, and outperforms strong continual-learning baselines, including a recent method for temporal knowledge graphs. Source code and data are available at https://github.com/liuyansong98/HiTS-CL.
1 Introduction
Fixed-prefix training leaves a static predictor facing an evolving stream, while extrapolative TKGR requires current dynamics, stable knowledge, and recurring historical evidence. HiTS-CL addresses this mismatch with continual learning and consistently improves extrapolation across evaluated backbones and datasets.
1 Introduction
- Fixed-prefix methods train once on an early timeline prefix, then remain unchanged despite a non-stationary future stream.They show long-horizon performance decay and can underperform continually updated models on the first test snapshot.Source: §1 · p. 1, Figure 1 · p. 2
- Three challenges are emerging facts, shifting temporal dependencies, and recurring historical signals that must be refreshed without storing the full history.Recurring evidence includes periodic or high-frequency interaction patterns.Source: §1 · p. 2
- The paper reframes extrapolative TKGR as continual learning over streaming snapshots, distinguishing it from prior continual-TKG work focused on within-snapshot completion.The target setting is strict next-step forecasting under a no-leakage protocol.Source: §1 · p. 2
- HiTS-CL combines continual fine-tuning, multi-teacher adaptive distillation, and selective memory to track current dynamics, preserve stable knowledge, and retain recurring evidence.The selective memory is inspired by cache replacement strategies.Source: §1 · p. 2
- HiTS-CL consistently improves extrapolation accuracy, reduces long-horizon degradation, and outperforms strong continual-learning baselines across five backbones and four datasets.The comparisons include a recent temporal-knowledge-graph continual-learning method adapted to the no-leakage extrapolation setting.Source: §1 · p. 2
2 Related Work
Prior work studies temporal knowledge graphs, continual learning, and continual graph learning, but strict streaming extrapolation remains distinct from common fixed-prefix and within-snapshot settings. HiTS-CL targets that next-step forecasting setting.
2.1 Temporal Knowledge Graph Reasoning
- Most extrapolative TKGR methods train on early snapshots and remain unchanged afterward, rather than adapting to evolving streams.This motivates treating extrapolative TKGR as continual learning over streaming snapshots.Source: §2.1 · p. 2
2.2 Continual Learning and Continual Graph Learning
- Prior continual temporal-KG work typically studies within-snapshot completion or incremental completion, not strict next-step streaming extrapolation.DGAR [42] is identified as the closest work, but uses within-snapshot interpolation and delayed task-level evaluation.Source: §2.2 · pp. 2–3
- Continual graph learning addresses evolving nodes, edges, and propagated influence, while many benchmarks emphasize node or graph classification rather than temporal link prediction.Extrapolative TKGR involves many incremental stages and is described as a distinct, under-explored setting [9].Source: §2.2 · pp. 2–3
3 Preliminary and Problem Definition
The paper defines extrapolative TKGR as predicting facts beyond observed history, then formulates deployment as continual updates on streaming snapshots followed by next-snapshot evaluation. It also specifies a previous-snapshot validation choice.
3.1 Extrapolative Temporal Knowledge Graph Reasoning
- A temporal knowledge graph is a sequence of snapshots, each a directed multi-relational graph whose facts are subject-relation-object-time quadruples.For an extrapolative query, the model scores candidate entities at a future timestamp.Source: §3.1 · p. 3
3.2 Extrapolative TKGR as Continual Learning
- The proposed continual-learning formulation replaces the fixed-prefix protocol with stage-wise updates on the current snapshot and evaluation on the next snapshot.The previous snapshot is used for validation to preserve the full current snapshot for training; the authors report slightly better results across datasets with this choice.Source: §3.2 · p. 3
4 Method
HiTS-CL adapts a pretrained TKGR backbone to streaming snapshots through continual fine-tuning and multi-teacher distillation, while selective memory supplies recurring historical evidence at inference. Its stages are designed to capture current dynamics, preserve stable knowledge, and incorporate historical patterns without using future snapshots.
4 Method
- HiTS-CL combines continual fine-tuning, multi-teacher distillation, and selective historical memory to adapt a pretrained TKGR backbone to streaming snapshots.Fine-tuning and distillation address current dynamics and stable knowledge; memory contributes recurring historical evidence at prediction time.Source: §4 · pp. 3–4
4.1.1 Step 1: Continual Fine-Tuning.
- At each stage, the fine-tuned teacher updates on the newly arrived snapshot, with the preceding snapshot used for MRR-based early stopping.Using only past data for early stopping follows the no-leakage extrapolation protocol.Source: §4.1.1 · p. 4
4.1.2 Step 2: Multi-Teacher Adaptive Distillation.
- Multi-teacher distillation combines a pretrained teacher for stable knowledge with a fine-tuned teacher for current patterns, routing samples to the teacher with lower loss.Per-sample temperature scaling sharpens supervision for lower-entropy predictions.Source: §4.1.2 · p. 4
- The student is initialized by reliability-weighted interpolation of the teachers, using their MRR on the current snapshot to estimate reliability.This starts the student closer to the teacher that is more reliable on the current stage.Source: §4.1.2 · p. 4
4.2 History Enhancement
- For each query pair, selective memory tracks historical facts by recency and frequency, then re-ranks retrieved candidates using recency and frequency priors.Candidates absent from memory receive zero for both priors; memory capacity can grow adaptively with the stream.Source: §4.2 · pp. 4–5
4.3 Training and Inference
- Each stage updates the fine-tuned teacher, trains a student with prediction supervision and distillation, then applies memory-based re-ranking at inference.The optimization steps use only current or past snapshots, preserving the no-leakage protocol.Source: §4.3 · p. 5
5 Experiments
The experiments evaluate continual extrapolative TKGR on four benchmarks using chronological splits with extended test horizons. They compare five backbone integrations and continual-learning baselines under time-aware filtered link prediction metrics.
5.1 Experimental Setup
- Evaluation uses ICEWS14, ICEWS18, ICEWS05-15, and GDELT with chronological 30%/5%/65% training, validation, and test splits to study long-horizon performance.The test period is processed as a streaming deployment phase, with updates and evaluation snapshot by snapshot in chronological order.Source: §5.1.1 · pp. 5–6
5.1.1 Datasets and Baselines.
- HiTS-CL is instantiated on five representative backbones and compared with DGAR, a continual-learning method adapted from temporal interpolation to extrapolation.The backbones span snapshot-based GNNs, temporal sequence encoders, and contrastive learning frameworks.Source: §5.1.1 · p. 6
5.1.2 Evaluation Metrics.
- The study reports time-aware filtered MRR and Hits@K, ranking candidate entities while filtering other valid facts at the query timestamp.At time t, evaluation uses only facts timestamped earlier than t; earlier validation and test snapshots become history for later predictions.Source: §5.1.2 · pp. 6–7
5.1.3 Implementation Details.
- The streaming pipeline pretrains on the initial split, then processes test snapshots sequentially, using each inferred snapshot for the next update and memory.Early stopping uses validation-snapshot MRR with patience 2; distillation uses α=2 and β=1.Source: §5.1.3 · p. 7
5.2 Main Results
HiTS-CL improves results over static and continual-learning baselines across four datasets and five backbones, while reducing long-horizon degradation. Continual updating also outperforms fixed-prefix training on the immediately following test snapshot.
5.2 Main Results
- HiTS-CL improves relative MRR over +Replay by 6.0%, 6.4%, 2.8%, and 15.7% on ICEWS14, ICEWS18, ICEWS05-15, and GDELT, respectively.Against original static models, average MRR gains are 27.4%, 17.7%, 16.1%, and 29.1% on those datasets, respectively.Source: §5.2 · p. 7
- Continual fine-tuning beats static training in nearly all settings, but remains behind +Replay and HiTS-CL, while +Reg lacks consistent gains.The results indicate that adaptation alone and simple regularization are insufficient under temporal drift.Source: §5.2 · pp. 7–8
- LogCL+HiTS-CL surpasses adapted continual-learning method DGAR by 11.5%, 11.4%, 11.9%, and 18.4% MRR on ICEWS14, ICEWS18, ICEWS05-15, and GDELT, respectively.DGAR improves over static TKGR baselines but remains weaker than standard backbones equipped with HiTS-CL.Source: §5.2 · p. 8
- Static models decay as the horizon grows on ICEWS05-15, whereas HiTS-CL stabilizes the trend and often improves performance over time.Generic continual-learning baselines alleviate some decay but remain behind HiTS-CL.Source: §5.2 · p. 8
5.3 Training Paradigm Comparison
- Continual learning yields a stronger backbone than fixed-prefix training on the same immediately following test snapshot.The comparison evaluates both paradigms on the first snapshot after the training period.Source: §5.3 · p. 8
5.4 Ablation Study
Ablations indicate that continual fine-tuning, multi-teacher distillation, and history enhancement each contribute to HiTS-CL. Validating on the previous snapshot performs slightly better than holding out part of the current snapshot.
5.4 Ablation Study
- Removing continual fine-tuning causes the largest performance drop, weakening adaptation because the model cannot track current dynamics.The ablations cover four datasets and multiple TKGR backbones.Source: §5.4 · p. 8
- Removing multi-teacher distillation consistently degrades performance because recent-snapshot adaptation alone does not combine stable pretrained knowledge with current dynamics.Distillation combines the pretrained and fine-tuned teachers, reducing forgetting.Source: §5.4 · p. 9
- Disabling history enhancement further reduces performance, with the largest drops on GDELT, where recurring historical evidence complements recent snapshots.The drop varies across backbones, and historical cues matter especially when observations are sparse or irregular.Source: §5.4 · p. 9
5.5 Validation Strategy Analysis
- Validating on G_t−1 consistently performs slightly better than reserving 15% of G_t, while preserving the full current-snapshot training signal.The authors relate improvements on G_t−1 after learning from G_t to backward transfer.Source: §5.5 · p. 9
5.6 Efficiency Analysis
HiTS-CL adds limited inference overhead and reduces total training time relative to fixed-prefix models. Its history enhancement also promotes historically frequent and recent candidates in the reported case study.
5.6 Efficiency Analysis
- Inference uses only the distilled student, and History Enhancement adds 0.064s per snapshot on ICEWS14 while cache memory remains below 1GB even on GDELT.The inference cost otherwise matches that of the backbone.Source: §5.6.1 · p. 9
5.6.2 Training Efficiency.
- In Figure 6, HiTS-CL reduces total training time by 15.4% on average compared with the original fixed-prefix models across the reported model comparisons.The reduction comes from replacing one full training run with stage-wise updates on smaller increments, which typically require fewer epochs and converge faster.Source: §5.6.2 · p. 9
- On LogCL, pretraining takes 1,618s, continual fine-tuning 736s, and distillation 1,153s, for a total of 3,507s.Pretraining and distillation account for most of the total cost; continual fine-tuning is relatively lightweight across the three tested backbones.Source: §5.6.2 · p. 10
5.7 Case Study
- In the case study, entities with frequent historical events and recent occurrences, such as China, move to higher ranks after applying History Enhancement.The module uses simple historical statistics such as frequency and recency to guide temporally consistent predictions.Source: §5.7 · p. 10
6 Hyperparameter Sensitivity Analysis
The sensitivity analysis finds broad robustness to α over a range of values, while β behaves differently across datasets. Extremely high or low α values can degrade performance.
6 Hyperparameter Sensitivity Analysis
- α is generally robust across a wide range of values on both datasets, but extremely high or low settings can degrade performance.The analysis uses RE-GCN, LogCL, and DiMNet on ICEWS14 and GDELT.Source: §6 · p. 10
- β shows broad robustness on GDELT but sensitivity for certain base models on ICEWS14.β is the distillation-loss weight.Source: §6 · p. 10
7 Conclusion
The conclusion presents HiTS-CL as a continual-learning framework for extrapolative TKGR, addressing long-horizon degradation by combining continual fine-tuning, adaptive distillation, and selective memory. Experiments across five backbones and four benchmarks support continual learning as a practical and effective paradigm for long-running temporal knowledge graph streams.
7 Conclusion
- Across five TKGR backbones and four benchmarks, HiTS-CL consistently improves extrapolation accuracy, mitigates long-horizon degradation, and outperforms strong continual-learning baselines and a representative continual TKG method.The comparisons use a unified no-leakage extrapolation protocol.Source: §7 · p. 10
- HiTS-CL formulates extrapolative TKGR as continual learning over streaming snapshots and captures current dynamics, stable knowledge, and recurring historical evidence.It uses continual fine-tuning, multi-teacher adaptive distillation, and selective memory, respectively.Source: §7 · p. 10
- The results support continual learning as a practical and effective paradigm for long-running temporal knowledge graph streams.Source: §7 · p. 10
GenAI Usage Disclosure
The authors report using generative AI tools only for language polishing and grammar checking.
GenAI Usage Disclosure
- The authors used generative AI tools only for language polishing and grammar checking; they developed and verified the technical work and conclusions.Source: GenAI Usage Disclosure · p. 10