Source-linked AI summary
Denoising the Future: Context-Aware Spectral Diffusion for Temporal Knowledge Graph Extrapolation
Yanglei Gan, Peng He, Run Lin, Peiyuan Jiang, Yifan Wang, Qiao Liu
TL;DR
TKG extrapolation must predict future objects despite uncertainty and insufficient separation of query-specific evidence from noisy subject histories. FreqDiff uses query-slot denoising with temporal modeling, spectral calibration, and frequency-domain supervision, achieving state-of-the-art performance on four public benchmarks. Its generalizability beyond primarily political and international-relations datasets and its finite basis-filter design remain open questions.
Problem
Existing diffusion-based TKG methods may insufficiently distinguish query-specific evidence from non-salient subject-history facts, while generic losses may inadequately capture frequency-specific cues.
Method
FreqDiff formulates future object prediction as query-slot denoising and uses a dual-stream denoiser with temporal dependency modeling, context-aware spectral calibration, and frequency-domain consistency regularization.
Results
FreqDiff achieves state-of-the-art performance on four public TKG benchmarks, with ablations supporting its spectral calibration and frequency-domain regularization.
Takeaways & Limitations
FreqDiff combines temporal and spectral signals for frequency-aware target reconstruction in TKG extrapolation, with time-domain modeling remaining the core component and frequency modeling providing complementary information.
Takeaways & Limitations
Evaluation focuses on four benchmarks centered mainly on political and international-relations events, and the model relies on a finite set of learnable basis filters.
Abstract
from arXiv · showhide
Temporal Knowledge Graph (TKG) extrapolation seeks to infer future facts from time-varying relational histories. Recent diffusion-based approaches improve uncertainty modeling through generative denoising, but their aggregated conditioning on subject histories may insufficiently distinguish query-specific evidence from non-salient historical facts, thereby diluting target-discriminative signals. To bridge this gap, we propose FreqDiff, a Frequency-aware Diffusion framework for TKG extrapolation. Specifically, FreqDiff formulates future object prediction as query-slot denoising and develops a dual-stream denoiser that integrates temporal dependency modeling with context-aware spectral calibration. The spectral branch synthesizes history-conditioned filters from learnable bases to adaptively re-calibrate denoising representations, while a frequency-domain regularizer is proposed to align the denoised target with the gold object in spectral space. Experiments on four public TKG benchmarks demonstrate that FreqDiff achieves state-of-the-art performance.
1 Introduction
TKG extrapolation predicts future facts from temporal histories, but existing deterministic and diffusion-based methods may fail to isolate query-specific evidence. FreqDiff addresses this with query-slot denoising, dual-stream temporal and spectral modeling, and frequency-domain supervision.
- TKGs represent time-evolving facts as quadruples and support extrapolation, which predicts facts after the latest observed timestamp.
- Deterministic TKG methods encode historical dependencies and rank future candidates, but they struggle to capture uncertainty and diversity in future events.
- Unified processing of subject histories can mix query-discriminative evidence with weakly relevant facts, obscuring critical temporal signals.
- FreqDiff formulates future object prediction as query-slot denoising and combines temporal dependency modeling with context-aware spectral calibration.
- A frequency-domain consistency regularizer aligns denoised targets with gold object embeddings in spectral space, while experiments on four benchmarks report state-of-the-art performance.
2 Related Works
Related work spans discriminative temporal reasoning, explicit selection of useful historical evidence, and generative approaches that model uncertainty in TKG extrapolation.
- Discriminative TKG methods model event occurrence or encode temporal signals using recurrent reasoning, graph propagation, structural evidence, and related mechanisms.
- Some approaches explicitly select or bottleneck useful historical evidence through temporal relational attention, reinforcement learning, contrastive learning, or masking.
- Generative methods introduce uncertainty-aware modeling through conditional Gaussian diffusion, negative-aware diffusion, graph-node diffusion, and periodic contrastive learning.
- Other generative directions use LLMs to generate multi-step event chains or refine rules alongside diffusion-based reasoning.
3 Preliminary
The preliminary section defines temporal knowledge graphs as time-stamped relational facts and formalizes reasoning as completing a missing object from a sliding history window.
- A temporal knowledge graph is a collection of time-stamped quadruples over finite entity, relation, and timestamp sets.
- The graph can be viewed as an ordered sequence of static snapshots, with each snapshot aggregating triples valid at its timestamp.
- The standard bidirectional-relation convention augments each quadruple with an inverse relation encoding reverse semantics.
- TKG reasoning uses a query with a missing object and a sliding history window of preceding snapshots to learn a scoring function.
- Each candidate object receives a score, and the highest-scoring entity completes the query quadruple.
4 Method
FreqDiff predicts missing future objects by denoising a query-slot representation conditioned on a subject’s recent temporal history. Its dual-stream denoiser combines sequential modeling with history-conditioned spectral calibration, followed by reconstruction and frequency-domain consistency objectives.
- Temporal Representation Learning: FreqDiff constructs a subject-centric sequence of recent events and the query slot, embedding objects, relations, and relative times in a shared space.The query asks for a missing object at a future timestamp, while each historical event is represented with its relation and interval from the query time.
- Forward Diffusion Process: Gaussian noise is injected only into the query target representation, while historical object representations remain unchanged to provide deterministic conditioning.The forward process samples a diffusion step using an accumulated signal ratio and a scaled noise schedule; reverse denoising reconstructs the target from this corrupted representation.
- Dual-stream Denoiser: The temporal branch processes conditioned hidden states with a Transformer to model sequential dependencies in the historical event trajectory.The denoising input combines historical and diffused target representations with relation, relative-time, and diffusion-step information.
- Dual-stream Denoiser: The spectral branch summarizes history to synthesize context-aware filters from learnable basis filters, applies them to grouped spectra, and maps calibrated features back through inverse FFT.The resulting spectral representation is fused with the time-domain representation by interpolation before extracting the target representation from the query position.
- Training Objective: Training combines reconstruction loss with a frequency-domain regularizer that aligns the denoised target and gold object in both embedding and spectral spaces.Candidate entities are scored by dot-product matching, while the auxiliary regularizer is weighted by λ; inference can directly predict the clean target from a corrupted representation.
5 Experiments
Experiments across four public TKG benchmarks show that FreqDiff consistently outperforms baselines, while ablations and robustness analyses support the contributions of temporal modeling, spectral calibration, and frequency-domain supervision.
- Overall Performance: 16.14% and 6.73% are FreqDiff’s largest relative gains on GDELT for Hit@1 and MRR, respectively.The results associate these improvements with the dataset’s more complex temporal evolution patterns.
- Ablation Studies: Removing the time branch causes the most severe degradation, while removing frequency modeling or the regularizer also consistently reduces performance.The time branch preserves event order and relation-specific temporal context; spectral modeling supplies complementary signals, and explicit spectral supervision remains beneficial.
- Spectral Learning: FreqDiff’s learned spectral responses adapt across frequency indices and latent channels rather than uniformly amplifying temporal patterns.The no-frequency variant shows a more concentrated response dominated by overall temporal patterns.
- Robustness Analysis: As historical corruption increases from 10% to 70%, both models decline, but FreqDiff consistently outperforms the no-spectral variant on ICEWS18 and GDELT in MRR and Hits@1.Its advantage becomes more evident at higher corruption ratios, consistent with frequency-aware re-calibration attenuating corrupted responses.
- Frequency Regularizer: The frequency regularizer improves DiffuTKG, NADEx, and CENET by 3.73%/5.76%, 3.40%/3.52%, and 9.01%/11.20% in average MRR/H@1.These gains are reported across ICEWS14, ICEWS18, and GDELT without changing the backbone architectures or training settings.
6 Conclusion
FreqDiff formulates future object prediction as query-slot denoising and reconstructs targets with a dual-stream denoiser combining temporal dependency modeling and context-aware spectral calibration. Experiments and ablations support its state-of-the-art performance and the effectiveness of its spectral components.
- FreqDiff formulates future object prediction as query-slot denoising.
- Its dual-stream denoiser combines temporal dependency modeling with context-aware spectral calibration.
- A frequency-domain consistency regularizer provides explicit spectral supervision for target reconstruction.
- Experiments on four public TKG benchmarks demonstrate state-of-the-art performance, while ablations verify the effectiveness of spectral calibration and frequency-domain regularization.
Limitation
FreqDiff's evaluation and spectral design leave two scope boundaries: its benchmark generalizability beyond political and international-relations events is unexamined, and finite learnable basis filters may limit flexibility for irregular dynamics.
- FreqDiff is evaluated on four benchmarks primarily centered on political and international-relations events.The datasets include ICEWS14, ICEWS05–15, ICEWS18, and GDELT.
- Its generalizability to scientific, financial, public-health, and natural-disaster temporal knowledge graphs remains to be examined.
- The context-aware spectral calibration relies on a finite set of learnable basis filters, which may limit flexibility for highly irregular or domain-specific temporal dynamics.
Ethics Statement
The study uses public benchmark data collected and processed by prior research, without new data collection, human-subject interaction, or private personal information. The supplied statement also situates spectral analysis as a broader processing approach across several application areas.
- The study uses publicly available benchmark datasets collected and processed by prior research.
- It involves no new data collection, human-subject interaction, or use of private personal information.
- Spectral analysis decomposes signals into frequency components and has been adopted across vision, language, recommendation, and time-series applications.
A.2 Diffusion Models on Discrete Data
Diffusion models have expanded from continuous generation to discrete symbolic and structured prediction through latent or sequence-level denoising. The section introduces spectral analysis through the Discrete Fourier Transform and its inverse, while dataset and implementation statistics are reported separately.
- Diffusion-LM performs text denoising in continuous word-embedding space, while DiffuSeq uses sequence-level corruption and denoising for non-autoregressive generation.
- DiffusionNER formulates named entity recognition as span-boundary denoising, refining noisy boundaries into entity predictions.
- DiffRec and DreamRec apply diffusion to user–item interaction modeling to capture uncertain preference distributions.
- The Discrete Fourier Transform maps a length-N time-domain sequence into frequency-domain components, while the inverse DFT reconstructs the original sequence.
- Dataset statistics and benchmark implementation details are provided in Tables 8 and 9.
C.1 Dataset Statistics
The study uses chronological data splitting and largely shared implementation settings across datasets, while comparing FreqDiff with established temporal and static baselines. Spectral analysis on ICEWS14 shows FreqDiff preserves a broader range of frequency components than comparison variants.
- Evaluation protocol: The chronological protocol assigns the earliest 80% of facts to training, the next 10% to validation, and the latest 10% to testing.This preserves temporal order and avoids future-timestamp leakage during training.
- Implementation settings: All datasets use a hidden dimension of 200, dropout and embedding dropout of 0.2, maximum history length 128, input length 64, 100 epochs, and 200 diffusion steps.Learning rate and filterbank configuration vary with dataset characteristics, while most hyperparameters remain fixed for fair comparison.
- Baselines: Static and time-aware scoring baselines include DistMult, ConvE, RotatE, TTransE, TA-DistMult, and DE-SimplE.These methods model plausibility through bilinear scoring, convolution, complex rotations, temporal translation, time-aware embeddings, or diachronic embeddings.
- Baselines: The compared temporal baselines include recurrent, graph-convolutional, cyclic, convolutional, contrastive, and language-model-assisted approaches for future link prediction.Examples include RE-NET, Re-GCN, CyGNet, CEN, TiRGN, CENET, LLM-DA, MESH, AnRe, and TV-LLM.
- Spectral analysis: On ICEWS14, FreqDiff learns distributed spectral energy across low, middle, and high frequency bands, unlike variants whose responses concentrate mainly in low frequencies.The pattern is described as preserving both stable long-term trends and fluctuating short-term relational changes.
D.2 Sensitivity Analysis
Sensitivity experiments on ICEWS14 and ICEWS18 identify λ=0.3 as the best regularization setting and α≈0.7 as a strong fusion setting. The results indicate that temporal modeling remains dominant while spectral calibration contributes complementary information.
- Weighting coefficient λ: λ=0.3 yields the best performance on both ICEWS14 and ICEWS18 in the weighting-coefficient sensitivity analysis.MRR and Hit@1 improve noticeably as λ increases from 0.1 to 0.3.
- Spectral visualization: Figure 6 plots frequency indices on the y-axis and latent channels on the x-axis, with red denoting higher spectral energy.The visualization compares learned spectral energy distributions on ICEWS14.
- Sensitivity visualization: Figure 7 presents hyper-parameter sensitivity analyses for ICEWS14 and ICEWS18.The figure is organized with ICEWS14 on the left and ICEWS18 on the right.
- Fusion coefficient α: α = 0.7 produces strong performance on both datasets, indicating that temporal modeling should remain dominant in the fusion.Spectral calibration is characterized as complementary information rather than the dominant stream.
D.3 Case Study
Case studies show FreqDiff ranking query-appropriate entities above plausible but less discriminative alternatives. Its frequency-aware design is associated with emphasizing temporally informative evidence rather than raw historical frequency.
- Case 1: 0.7906 confidence places Muhammadu Buhari first for the Democratic Party (Nigeria), Criticize or denounce query in Case 1.The score is substantially higher than those assigned to other candidates.
- Case 2: 0.7782 confidence places Sudan first for the Ethiopia, Sign formal agreement query in Case 2.FreqDiff w/o. Freq ranks Sudan second, while DiffuTKG gives higher ranks to Portugal, China, and Iran.
- Evidence discrimination: FreqDiff ranks Muhammadu Buhari first despite fewer direct connections, whereas competing models favor the more frequent but generic candidate Citizen (Nigeria).The comparison links FreqDiff’s ranking to temporally informative and query-specific evidence rather than raw historical frequency.
- Case-study setup: Table 10 reports the top-five predicted entities and confidence scores for FreqDiff, FreqDiff w/o. Freq, and DiffuTKG, with red entries marking ground-truth entities.The table covers future entity prediction on ICEWS14 with the target entity masked.