Source-linked AI summary
Differentiable Koopman Operator for Contrastive Learning on Dynamic Graphs
Md Abrar Jahin, Taufikur Rahman Fuad, Md Rizwan Parvez
TL;DR
Dynamic graph encoders often lack an explicit model of how node representations evolve, making predictable changes difficult to distinguish from anomalies under distribution shift. KAIROS combines self-supervised graph contrastive learning with a differentiable Koopman operator and a dual-view encoder. It achieves state-of-the-art anomaly detection on all nine datasets, with gains of up to +23.15 ROC-AUC points over CLDG++.
Problem
Many dynamic-graph encoders do not explicitly model embedding transitions, making predictable evolution and anomalous deviation difficult to distinguish under distribution shift.
Source: §1 · p. 1Method
KAIROS jointly trains a dual-view graph encoder and differentiable Koopman operator within self-supervised contrastive learning, then combines three signals for unsupervised anomaly scoring.
Source: §1 · p. 2Results
KAIROS achieves state-of-the-art anomaly detection on all nine datasets, with gains over CLDG++ ranging from +5.81 to +23.15 ROC-AUC points on the primary benchmarks.
Source: §5.5 · p. 7Takeaways & Limitations
Results across nine temporal graph benchmarks suggest explicit dynamics modeling is an effective inductive bias for temporal graph representation learning, particularly for fully unsupervised anomaly detection.
Source: §6 · p. 9KAIROS depends on discrete-time snapshot windowing, and its effectiveness may decrease on highly irregular or event-driven graphs that violate the linear Koopman assumption.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Real-world interaction networks are inherently dynamic: edges form and dissolve as node behavior shifts over time. Most snapshot-based contrastive methods encode temporal dependencies implicitly in encoder weights, without an explicit model of how node representations evolve, making them brittle under distribution shifts. We propose KAIROS (Koopman-Aligned Invariant Representations for Open Dynamic Systems), a self-supervised framework that embeds a differentiable Koopman operator within a dynamic graph contrastive learning loop to linearize temporal evolution in the learned embedding space. A dual-view encoder pairs raw node features with a graph-diffused structural view and is optimized with multi-granularity contrastive objectives across temporal windows. For anomaly detection, KAIROS uses the Koopman prediction residual together with temporal inconsistency and local neighborhood deviation to separate irregular behavior from predictable graph evolution. Evaluated on nine dynamic graph benchmarks, KAIROS achieves state-of-the-art anomaly detection results on all nine datasets, with gains of up to 23.15 ROC-AUC points over prior work, while remaining competitive for unsupervised node classification. These results show that explicit dynamics modeling provides a scalable and effective inductive bias for temporal graph representation learning.
1 Introduction
Dynamic graph encoders often absorb temporal change into their weights rather than explicitly modeling embedding transitions, leaving predictable evolution difficult to distinguish from anomalies under distribution shift. KAIROS adds a differentiable Koopman operator to self-supervised contrastive learning and uses its prediction residual in an unsupervised anomaly score.
Motivation
- Many dynamic-graph encoders lack a parameterized one-step operator over node embeddings, complicating separation of predictable evolution from anomalous deviation under distribution shift.Snapshot and continuous-time designs encode temporal dependencies through their architectures rather than an explicit transition operator [2] [3] [4].Source: §1 · p. 1
- Temporal-view agreement in dynamic contrastive baselines can penalize legitimate non-smooth evolution, while existing dynamic anomaly detectors do not use learned-transition residuals.CLDG and CLDG++ maximize agreement between temporal views [9]; other detectors use dynamic embeddings, spatiotemporal transformers, or semi-supervised pseudo-labeling [13].Source: §1 · pp. 1–2
Contribution
- KAIROS couples a graph encoder and Koopman operator end-to-end under contrastive learning, using the resulting prediction residual as an unsupervised dynamic-graph anomaly signal.The paper reports that, to its knowledge, prior work had not combined these elements in this way.Source: §1 · p. 2
Approach
- A dual-view GNN processes raw and PPR-diffused features, while a linear operator predicts next-snapshot embeddings and anomaly scoring combines its residual with temporal inconsistency and neighborhood deviation.The framework also supports unsupervised node classification with the frozen encoder and a linear probe, disabling the Koopman regularizer for that task.Source: §1 · p. 2
2 Related Work
Prior dynamic graph methods encode temporal dependencies through message passing, contrastive view agreement, or other temporal mechanisms, without explicitly parameterizing inter-snapshot embedding dynamics. KAIROS combines self-supervised graph contrastive learning with a differentiable Koopman operator and uses its residual for anomaly detection.
Dynamic Graph Representation Learning
- Snapshot and continuous-time DGRL models encode temporal information through their respective mechanisms but do not explicitly model how representations evolve between snapshots.The cited approaches include temporal self-attention, functional time encodings, causal anonymous walks, neighborhood influence aggregation, and frequency-domain features [2] [3] [4].Source: §2 · p. 2
Self-Supervised and Contrastive Learning on Graphs
- Dynamic contrastive methods such as CLDG and CLDG++ align temporally sampled views without modeling their intervening dynamics, making the implicit smoothness assumption brittle under non-smooth evolution.The cited work also discusses broader static graph contrastive-learning design choices and their structural fragilities [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [37] [38] [39].Source: §2 · p. 2
Graph Anomaly Detection
- Existing graph anomaly detectors include static methods that ignore temporal evolution and dynamic methods with limitations in expressiveness, scalability, or reliance on pseudo-labels.The cited examples include NetWalk, TADDY, SAD [13], and methods targeting multivariate time series rather than evolving graph topologies.Source: §2 · p. 3
Koopman Operator Theory in Machine Learning
- Graph Koopman work remains limited, and the closest cited study links GNN message passing to dynamic mode decomposition for supervised dynamics modeling rather than contrastive learning or anomaly detection.KAIROS instead places a differentiable Koopman operator in self-supervised dynamic graph contrastive learning and uses its prediction residual as an unsupervised anomaly signal.Source: §2 · p. 3
3 Background and Preliminaries
The setup represents a dynamic graph as temporal snapshots over a fixed node universe and uses raw-feature and PPR-diffused views. KAIROS targets unsupervised node classification and node-level anomaly detection, with a linear Koopman operator modeling embedding evolution.
Problem Setup
Learning Objectives
- The shared self-supervised encoder supports frozen-linear-probe node classification and scalar node-level anomaly scoring, with a linear Koopman operator whose prediction residual is one anomaly signal.Source: §3 · p. 3
4 KAIROS
KAIROS combines temporal view construction, separate encoders for raw and PPR-diffused inputs, multi-granularity contrastive learning, and a Koopman operator. At inference, its frozen encoder supports classification or a unified anomaly score built from three signals.
4 KAIROS
- KAIROS couples snapshot construction and view sampling, dual GNN encoders, multi-granularity contrastive learning, and a Koopman operator that regularizes training and yields an inference-time residual.The three anomaly signals are z-normalized and summed at inference.Source: §4 · p. 3
4.1 Temporal Graph Construction and View Generation
- Each temporal snapshot is paired with raw features and a PPR-diffused structural view; training samples temporal views and anchor nodes, using neighborhood sampling to bound memory on larger graphs.The passage describes canonical snapshots and multiple temporal sampling schemes.Source: §4.1 · pp. 3–4
4.2 Dual-Encoder Architecture
- Separate GNN branches encode raw and structural views; their disjoint weights prevent trivially correlated outputs from collapsing the cross-view contrastive signal.The original-view embedding feeds the Koopman head, while projection heads are discarded at inference.Source: §4.2 · p. 4
4.3 Multi-Granularity Contrastive Objective
- The contrastive objective aligns local-view embeddings over time, aligns local and structural views within snapshots, and stabilizes the structural branch across temporal perturbations.The local-temporal signal receives weight 1/2 and auxiliary cross-granularity terms receive 1/4 each; equal weighting can collapse the local representation toward the diffused view.Source: §4.3 · pp. 4–5
4.4 Koopman Temporal Linearization
- KAIROS fits a linear operator over consecutive-view embeddings and uses a squared cosine prediction loss, with gradients updating both the local encoder and operator.The operator is initialized to identity, making temporal smoothness a special case; when the Koopman weight is zero, its prediction and orthogonality terms are excluded.Source: §4.4 · p. 5
4.5 Unified Anomaly Scoring and Downstream Evaluation
- For node classification, the frozen encoder averages the two pre-projection branches and feeds an equal-budget logistic-regression probe.Source: §4.5 · p. 5
- The anomaly readout combines temporal inconsistency and Koopman prediction residuals; when the Koopman weight is zero, the residual reduces to a temporal-smoothness penalty.Source: §4.5 · p. 6
- Neighborhood deviation measures within-snapshot distance from the neighborhood mean, and the three signals are independently z-normalized and summed without learned weights.The neighborhood mean includes the node itself; learning combination weights would require anomaly labels during training.Source: §4.5 · p. 6
5 Experiments
Across nine temporal graph benchmarks, KAIROS achieves state-of-the-art anomaly detection, while node-classification results vary by dataset. The experiments evaluate benchmark coverage, competing methods, task protocols, and ablations of the model’s design choices.
5.1 Datasets
- The evaluation covers nine temporal graph benchmarks, with all datasets represented as timestamped edge lists without raw node attributes.The study synthesizes anomaly labels using injected structural cliques and feature replacements, with clique counts set per dataset.Source: §5.1 · pp. 6–7, Table 1 · p. 6
5.2 Baselines
- Node-classification comparisons span fourteen methods, while anomaly detection adds seven dedicated methods alongside CLDG and CLDG++.CLDG++ is the primary node-classification baseline because it shares KAIROS’s input modalities and evaluation protocol.Source: §5.2 · p. 7
5.3 Evaluation Protocol
- For classification, the encoder is trained without labels and evaluated with a frozen-embedding logistic-regression probe; anomaly scores combine three z-normalized signals.The anomaly signals are temporal inconsistency, Koopman prediction residual, and neighborhood deviation, averaged over the specified snapshot comparisons.Source: §5.3 · p. 7
5.4 Node Classification
- On node classification, KAIROS trails CLDG++ by 1.08 points on BITotc, 2.49 on BITalpha, and 1.42 on Reddit, while gains are largest on DBLP and TAX51.Reddit results are 70.31±0.11% for KAIROS versus 71.73% for CLDG++; the PPR view adds limited class-discriminative signal at this scale.Source: §5.4 · p. 7, Table 2 · p. 8
5.5 Dynamic Graph Anomaly Detection
- KAIROS exceeds CLDG++ on all nine anomaly-detection datasets, with margins of +5.81 to +23.15 ROC-AUC points and a +13.27-point average.The largest gains occur on Bitcoin trust graphs; mini-batch neighborhood sampling keeps memory tied to the sampled computation graph rather than |V|.Source: §5.5 · pp. 7–8, Table 3 · p. 8
5.6 Ablation Studies
- Fixing InfoNCE temperature at τ=0.5 improves DBLP and TAX51 classification accuracy over τ≈0.07 and stabilizes Reddit anomaly detection, with performance stable across τ ∈[0.3, 0.7].At low temperatures, Reddit’s temporal signals S1 and S2 collapse.Source: §5.6 · p. 8, Figure 2 · p. 9
- No backbone dominates across datasets: GCN performs best on Bitcoin graphs, SAGE on DBLP, and GAT on TAX51.The cumulative gain from the canonical configuration to final KAIROS is +4.11 on DBLP and +4.71 on Bitcoinotc.Source: §5.6 · p. 8, Table 4 · p. 9
- The combined anomaly score beats its strongest standalone signal by +0.41 to +4.60 ROC-AUC points, while the Koopman regularizer has dataset-dependent effects.The decomposition compares temporal inconsistency, Koopman residual, neighborhood deviation, and their normalized sum; KAIROS uses τ=0.5.Source: §5.6 · pp. 8–9, Table 5 · p. 8
6 Conclusion
KAIROS couples dynamic graph contrastive learning with a differentiable Koopman operator to model temporal evolution in embedding space. Results across nine temporal graph benchmarks suggest explicit dynamics modeling is an effective inductive bias, particularly for fully unsupervised anomaly detection.
6 Conclusion
- Across nine temporal graph benchmarks, explicit dynamics modeling is presented as an effective inductive bias, particularly for fully unsupervised anomaly detection.Source: §6 · p. 9
- KAIROS couples dynamic graph contrastive learning with a differentiable Koopman operator that models nonlinear dynamics through a linear embedding-space transition.Source: §6 · p. 9, §A.3 · p. 16
- KAIROS uses discrete-time snapshots, so it depends on the temporal windowing scheme; highly irregular or event-driven graphs may violate its linear Koopman assumption and reduce regularizer effectiveness.Source: §6 · p. 9
- Anomaly evaluation uses synthetic injection, so performance on naturally occurring anomalies may vary across domains.Source: §6 · p. 9
- The work is foundational dynamic-graph representation-learning research, not tied to a specific deployment setting; the authors foresee no direct societal impacts when used as intended.Source: §6 · p. 9
A.4 Studied Problems
- The study addresses unsupervised dynamic node classification and anomaly detection with a shared encoder, setting the Koopman-loss weight to zero for classification and tuning it per dataset for anomaly detection.Source: §A.4 · p. 17
F Backbone Ablation
The backbone ablation reports five-seed classification accuracy and identifies the backbones used for KAIROS’s main results.
F Backbone Ablation
- The supplied ablation passage identifies Table 8 as the source of the full five-seed backbone results.Source: F Backbone Ablation · p. 20, Table 8 · p. 21
- The backbone ablation reports five-seed accuracy as mean±std, with the best backbone marked for each dataset.Source: Table 8 · p. 21
- KAIROS uses SAGE for DBLP, GCN for Bitcoinotc and BITalpha, SGC for BITotc, and GAT for TAX51 in its main results.Source: Table 8 · p. 21
G Koopman Regularizer Ablation
The Koopman regularizer ablation reports dataset-dependent effects, while the statistical validation caption describes bootstrap comparisons against CLDG++ paper values.
G Koopman Regularizer Ablation
- The Koopman regularizer helps only on BITotc and BITalpha; for four of six datasets, including Reddit, the operator learns equally well without the residual-regularization term.Source: Table 9 · p. 21
- The ablation indicates that λ_koop should be tuned per dataset because its effects are mixed across datasets.Source: Table 9 · p. 21
H Statistical Validation
- Table 10 reports bootstrap probabilities, using 10,000 resamples from five seeds, that KAIROS’s best-of-five result exceeds the CLDG++ paper value; its caption says anomaly dominates and classification mirrors the head-to-head verdict.Source: Table 10 · p. 21
I Anomaly Signal Analysis
Figure 4 presents per-signal anomaly ROC-AUC across nine datasets, while Figure 3 illustrates a temperature-induced collapse of S1 and S2 on Reddit at τ=0.07.
I Anomaly Signal Analysis
- Figure 4 displays a heatmap of per-signal ROC-AUC across all nine datasets.The heatmap compares S1, S2, S3, and the combined score.Source: I Anomaly Signal Analysis · p. 21
- S1 is temporal inconsistency, S2 is the Koopman residual, and S3 is neighborhood deviation.Figure 4 also includes their combined score.Source: I Anomaly Signal Analysis · p. 21
- At τ=0.07, Figure 3 shows a temperature-induced collapse of S1 and S2 on Reddit.Figure 4 complements Table 5 with the per-signal scores.Source: I Anomaly Signal Analysis · p. 21
J Per-Dataset Configuration
The configuration tables distinguish node-classification and anomaly-detection settings while holding several shared parameters fixed. The anomaly-signal analysis finds that the combined score beats individual signals on primary datasets, although S3 collapses on long-trajectory graphs.
J Per-Dataset Configuration
- Both tasks share the same backbone, τ=0.5, and α=0.15, while anomaly configurations differ in snapshot count, view count, and sampling strategy for several datasets.Table 12 gives anomaly-detection settings corresponding to the node-classification configurations in Table 11.Source: J Per-Dataset Configuration · p. 21
- The z-normalized combined anomaly score outperforms every individual signal on all primary datasets.On long-trajectory Reddit, MOOC, and Arxiv graphs, S3 collapses while S1 and S2 carry the anomaly signal load.Source: Figure 4 · p. 22
- Node-classification configurations use τ=0.5 and PPR α=0.15 across datasets, with SAGE/GAT for larger graphs and GCN/SGC for small rating graphs (|V| ≤8 K).Table 11 configurations were selected on held-out validation accuracy.Source: Table 11 · p. 22
K KAIROS vs. CLDG++ Efficiency Analysis
The efficiency analysis places KAIROS on the accuracy–efficiency Pareto frontier, while the regularizer's effect varies by dataset. Implementation details specify task configurations, model and training settings, evaluation, and compute requirements.
L Koopman Regularizer Visualization
- On Reddit, setting λkoop=0 yields +3.03 AUC; the regularizer helps on BITotc and BITalpha, indicating that λkoop must be tuned per dataset.Figure 6 visualizes the per-dataset effect of the Koopman regularizer.Source: L Koopman Regularizer Visualization · p. 22, Figure 6 · p. 24
K KAIROS vs. CLDG++ Efficiency Analysis
- At only ∼1.5× the inference cost of CLDG++ (41 K params), KAIROS (GCN, 60 K params) achieves the highest average accuracy.Figure 5 plots average five-seed classification accuracy against trainable parameter count for CLDG and three KAIROS backbone configurations.Source: K KAIROS vs. CLDG++ Efficiency Analysis · p. 22, Figure 5 · p. 23
M Implementation Details
- On Reddit, the largest dataset at 898 K nodes, one training run takes approximately 3.5 hours per seed on one GPU.A five-seed Reddit configuration therefore requires approximately 17.5 GPU-hours.Source: M Implementation Details · p. 23
- All five backbone variants use two message-passing layers, 128 hidden dimensions, and 64-dimensional output embeddings; GAT adds feature and attention dropout.The GAT configuration uses dropout of 0.1 for both.Source: M Implementation Details · p. 23
- All classification runs set λkoop=0 (overridden in code), whereas anomaly detection uses default λkoop=1.0, with Reddit set to λkoop=0.0.The PPR structural view uses α=0.15, and λreg=0.01 is fixed across all experiments. Anomaly snapshot and view counts differ from classification for BITalpha, TAX51, and Reddit.Source: Table 12 · p. 23, M Implementation Details · p. 24
- Training uses Adam at 4×10^-3 with weight decay 5×10^-4, cosine annealing, gradient clipping at 1.0, and a checkpoint selected by lowest total training loss.Anomaly evaluation uses full-neighbor sampling across all canonical snapshots.Source: M Implementation Details · p. 24
- Node classification evaluates frozen encoder weights with a logistic-regression probe and averages scores across five seeds.Within each class, the split is 10% training, 10% validation, and 80% test.Source: M Implementation Details · p. 24
N Dataset Details
The nine datasets use timestamped edge lists and node labels, with sinusoidal node-index encodings rather than supplied raw attributes. The appendix specifies graph preprocessing, benchmark coverage, anomaly injection, and classification splits.
N Dataset Details
- The nine datasets are stored as timestamped edge lists with node labels in separate files, and none provides raw node attributes.Each node instead receives a 128-dimensional sinusoidal positional encoding keyed to its integer index, following the CLDG evaluation protocol [9].Source: N Dataset Details · pp. 24–25
- Each temporal subgraph is canonicalized by removing duplicate edges, symmetrizing the graph, and adding self-loops.Source: N Dataset Details · p. 25
- The primary six datasets replicate the CLDG/CLDG++ benchmark [9], while MOOC, Arxiv, and Elliptic extend coverage to larger graphs, a 40-class citation setting, and a blockchain transaction network.The extended three are not part of the CLDG or CLDG++ benchmark.Source: N Dataset Details · p. 25
- Anomaly injection creates node-disjoint structural and attribute sets, totaling exactly 2nm anomaly nodes per dataset.Structural anomalies use n cliques of m=15 nodes; attribute anomalies replace features for nm nodes using the most distant of k=50 sampled candidates.Source: N Dataset Details · p. 25, Table 13 · p. 25
- Classification labels are split within each class into 10% training, 10% validation, and 80% test, matching CLDG and CLDG++.Source: N Dataset Details · p. 25