PaperlayerView exact revision on arXiv

Source-linked AI summary

Do Temporal Link Predictors Need Learned Memory? A Smoothed-Count Baseline with a Handful of Parameters

Lisi Qarkaxhija, Ingo Scholtes

arXiv:2609.35364v1cs.LGcs.AI

TL;DR

Temporal link predictors often summarize interaction histories with learned node representations, while the predictive value of count-based patterns is less understood when sources rarely revisit destinations. This paper combines pooled, smoothed temporal counts with recency features in a shared scoring rule, and achieves the highest MRR on 7 of 16 datasets while outperforming EdgeBank and Base3 on all 16.

  • Problem

    Many temporal predictors use learned node representations, while the predictive value of simple count-based patterns when sources rarely revisit destinations remains less understood.

    Source: §1 · p. 1

  • Method

    The model pools transition and co-occurrence counts across sources, smooths sparse estimates, and combines them with popularity, source history, and recency using shared weights without node embeddings.

    Source: §1 · p. 2

  • Results

    The model achieves the highest MRR among compared methods on 7 of 16 datasets, exceeds EdgeBank and Base3 on 16 of 16, and exceeds the heuristic rule on 14 of 16.

    Source: §5 · p. 6

  • Takeaways & Limitations

    With 9–13 learned parameters, the model provides a simple, competitive baseline for evaluating future neural temporal link predictors.

    Source: §7 · p. 9

    The timing analysis keeps the configuration and weights fixed; whether query-specific choices between marginal and continuation backoff improve prediction remains for future work.

    Source: §7 · p. 9

Abstract

from arXiv · show

Many temporal link predictors summarize past interactions through learned node representations. We examine whether simple counts of recurring interaction patterns can provide competitive predictions without learning these representations. We propose a temporal link predictor based on statistical language modelling. It pools transition and co-occurrence counts across sources to predict links that a source has never formed. We smooth sparse estimates using destination frequencies or Kneser-Ney continuation counts. A shared log-linear rule combines these estimates with popularity, source history, and recency, without node embeddings. In our main evaluation, the model achieves the highest MRR among the compared methods on 7 out of 16 datasets from TGB and TGB-Seq. It also outperforms EdgeBank and Base3 on all 16 datasets and the heuristic family on 14. These gains extend to datasets designed to limit repeated edges. With only 9--13 learned parameters, our model provides a simple and competitive baseline for evaluating future neural temporal link predictors.

1 INTRODUCTION

Temporal link predictors often rely on learned representations, while recurring interaction patterns also carry predictive information. This work applies statistical language modelling to pooled temporal counts and evaluates whether they provide a strong baseline, including when sources rarely revisit destinations.

1 INTRODUCTION

  • The introduction frames strong statistical baselines as a way to measure predictive gains from learned representations, given that simple temporal patterns also carry predictive information.The paper's contributions include controlled comparisons of context depth, co-occurrence, backoff distributions, and context coverage.Source: §1 · pp. 1–2
  • Existing count-based methods capture edge recurrence, decayed destination frequency, or fixed combinations of recency, popularity, and co-occurrence, but their value when sources rarely revisit destinations is less understood.TGB-Seq tests this setting.Source: §1 · p. 1
  • Statistical language modelling pools recurring contexts and smooths sparse estimates toward broader distributions, addressing the limited transition evidence in individual source histories.The model considers marginal destination-frequency backoff and Kneser-Ney continuation backoff.Source: §1 · p. 1
  • The proposed predictor pools smoothed transition and co-occurrence counts across sources, combining them with popularity, source history, and recency through shared learned weights without node embeddings.At context depth K = 5, it has 9–13 learned parameters shared across nodes.Source: §1 · pp. 1–2
  • Across 16 TGB and TGB-Seq datasets, the model exceeds EdgeBank and Base3 on all datasets and the heuristic family on 14.The evaluation uses matched event-wise and batch-wise updates.Source: §1 · p. 2

2 RELATED WORK

The related work spans learned temporal graph encoders, count-based predictors, sequential recommenders, evaluation protocols, and statistical language modelling. The paper positions pooled smoothed counts as a shared-scoring alternative and motivates careful evaluation of candidate sets and update timing.

Graph-based models

  • Temporal graph methods learn node states, temporal walks, neighborhood sequences, or higher-order dependencies; simpler encoders and parameter-free message passing are also discussed.Examples include JODIE, DyRep, and TGN (Rossi et al., 2020), and GraphMixer.Source: §2 · pp. 2–3

Count-based methods

  • Count-based methods use edge recurrence, decayed destination frequencies, or fixed priority rules, while Base3 interpolates several such signals.The proposed model adds smoothed conditional estimates and learns their contribution to the score.Source: §2 · p. 3

Sequential recommenders

  • Sequential recommenders predict items from recent interactions using session co-occurrence, learned session graphs, or candidate-to-history cross-attention.SGNN-HN and CRAFT provide the strongest reported TGB-Seq scores used in the comparison.Source: §2 · p. 3

Evaluation

  • Temporal link prediction rankings can depend on negative sampling, metric choice, and history-update timing, motivating controlled comparisons and recomputed count baselines.The paper also uses context coverage to measure prior observations of a transition context.Source: §2 · p. 3

Statistical language modelling

  • Statistical language modelling combines estimates across contexts with different evidence levels, using overall frequencies or distinct preceding contexts as backoff distributions.The paper applies these ideas to source histories and destination transitions, with exponential decay accounting for age.Source: §2 · p. 3

3 PROBLEM SETTING AND EVALUATION

The task ranks an observed destination against benchmark negative candidates at each temporal query. The evaluation uses benchmark-provided candidates and MRR, with candidate construction differing between TGB and TGB-Seq.

3 PROBLEM SETTING AND EVALUATION

  • At each query, the observed destination is ranked above benchmark negatives, and MRR uses its rank with a midrank rule for ties.Only the observed destination's rank enters MRR; equal scores can occur for unobserved candidates.Source: §3 · p. 3
  • TGB-Seq supplies 100 uniformly sampled negative destinations per test event, whereas TGB uses dataset-specific candidate sets.Historical-random negatives include destinations the source visited during training, and the benchmarks are analysed separately because candidate distributions differ.Source: §3 · p. 3
  • The evaluation uses the splits, negative candidates, and MRR supplied by TGB and TGB-Seq, compares reported neural results, and recomputes count baselines.Appendix A documents score sources and evaluation details.Source: §3 · p. 3
  • The vocabulary is fixed per dataset and includes candidate destinations first observed after training; u denotes the query source and z a candidate destination.The K most recent source destinations are ordered from most recent, and hatted probabilities denote smoothed estimates.Source: §3 · p. 4

4 A STATISTICAL MODEL FOR TEMPORAL LINK PREDICTION

The model predicts temporal links from recurring interaction contexts, smoothed count estimates, and recency features, combining them with a shared scoring rule rather than learned node representations.

4 A STATISTICAL MODEL FOR TEMPORAL LINK PREDICTION

  • The model stores count tables and recent-destination lists, then uses a shared log-linear rule to combine count estimates and recency features.Stored counts and lists follow fixed update rules; learning determines each feature’s contribution to the score.Source: §4 · p. 4

4.1 FROM COMPLETE HISTORIES TO RECURRING CONTEXTS

  • Shorter recurring contexts replace complete interaction histories: pooled transitions and windowed co-occurrences estimate destinations using evidence from other sources.Transitions capture consecutive interactions; co-occurrence also covers destinations separated by intervening events, allowing support for links the source has never formed.Source: §4.1 · p. 4

4.2 STREAMING COUNT STATE

  • The streaming state tracks decayed destination, source-destination, transition, and windowed co-occurrence counts, plus each source’s recent destinations.Popularity, source history, and consecutive transitions update in O(1) operations per event; co-occurrence requires up to K updates, and the decay time constant is not tuned.Source: §4.2 · p. 4

4.3 SMOOTHED CONDITIONAL ESTIMATORS

  • Each context conditional uses Dirichlet pseudocounts from a base distribution, and an unobserved context returns that base exactly.Marginal backoff tracks destination frequency, while continuation backoff counts distinct predecessors without decay; windowed co-occurrence sums counts across recent-history rows, counting repeated destinations once per position and returning B when unobserved.Source: §4.3 · p. 5

4.4 RECENCY FEATURES

  • Recency features represent the latest source-specific visit, the latest interaction by any source, and source-specific recency at three fixed scales.The scales are τr/10, τr, and 10τr, allowing learned weights to combine faster and slower recency effects.Source: §4.4 · p. 5

4.5 LEARNED COMBINATION

  • A shared log-linear rule scores candidates from count-estimate log-probabilities and untransformed recency values, using one learned weight per feature and a learned bias.Weights are unconstrained, so a negative weight can make a larger feature value evidence against a candidate; unavailable transition positions are zero for every candidate.Source: §4.5 · pp. 5–6
  • At K = 5, evaluated configurations use 9–13 learned parameters, with no node or item embeddings, message passing, or learned per-node state.The statistical state still grows with observed destinations and pairs, so parameter count alone does not describe memory cost.Source: §4.5 · p. 6

4.6 FITTING

  • Fitting extracts features in one chronological pass, reuses them across epochs, and trains only scoring weights with sampled softmax using one positive and one uniformly sampled negative per query.Test candidates are provided by the benchmark.Source: §4.6 · p. 6

5 RESULTS AND DISCUSSION

Across the benchmark comparisons, the model leads all compared methods on 7 of 16 datasets and beats recomputed EdgeBank and Base3 on all 16. The analyses examine backoff choice, co-occurrence, learned feature weights, and context depth.

5 RESULTS AND DISCUSSION

  • The model achieves the highest MRR among compared methods on 7 of 16 datasets, while remaining below reported references on 4 TGB and 4 TGB-Seq datasets.It exceeds EdgeBank and Base3 on all 16 datasets and the heuristic rule on 14.Source: §5 · p. 6, Table 1 · p. 7, Table 2 · p. 7

5.1 THE EFFECT OF THE BACKOFF DISTRIBUTION

  • Continuation backoff improves MRR by 9.13 points on tgbl-review and 3.83 points on tgbl-comment, while marginal backoff is stronger on Yelp by 5.01 points.The three largest backoff effects occur on datasets with low source-specific transition coverage; differences on the remaining datasets are at most 1.39 points.Source: §5.1 · p. 7

5.2 THE CONTRIBUTION OF CO-OCCURRENCE AND RECENCY

  • Adding co-occurrence gives a larger median gain than global recency or further recency scales under either backoff, and the main model gains 1.44 median MRR points over its reference configuration.The largest gain is 13.01 points on Yelp; pooled windowed observations capture associations beyond consecutive transitions.Source: §5.2 · p. 7, Figure 2 · p. 8

5.3 THE LEARNED SCORING RULE

  • The most recent destination has the largest transition coefficient on all 16 datasets, while source-history and recency coefficients vary across TGB and TGB-Seq.Both source-history and recency coefficients are negative on 5 of 7 TGB-Seq datasets; source-history coefficients are positive on all 9 TGB datasets and recency on 8.Source: §5.3 · p. 8

5.4 THE VALUE OF LONGER CONTEXTS

  • Increasing context depth from K = 1 to K = 20 improves mean test MRR on all 7 TGB-Seq datasets and 5 of 9 TGB datasets.Most TGB-Seq improvement occurs within the first 5 destinations; extending K from 5 to 20 adds at most 0.41 MRR points there.Source: §5.4 · p. 8

6 WHEN IS CONTEXT AVAILABLE?

The analysis distinguishes transition evidence pooled across sources from evidence specific to the queried source. Pooled evidence is widely available, but context availability alone does not determine whether the model beats reported references.

6 WHEN IS CONTEXT AVAILABLE?

  • Pooled coverage measures whether any source has prior observations for the queried transition context, while pair coverage requires prior observations from the queried source.The model uses pooled transitions and backs off exactly for empty rows.Source: §6 · p. 8
  • Pooled transition evidence is available on at least 96.1% of test queries on every dataset, while 5 TGB-Seq datasets have zero pair coverage.ML-20M has complete pooled coverage, so transitions observed for other sources remain available.Source: §6 · p. 8
  • Among datasets with zero pair coverage, the model beats reported references on ML-20M, Flickr, and YouTube but not GoogleLocal or WikiLink.It also remains below references on tgbl-coin and tgbl-flight despite pair coverage of at least 87.1%.Source: §6 · p. 9
  • Pair coverage concerns one transition context, not the source’s entire history, and learned node memory may still encode earlier interactions when pair coverage is zero.Pair coverage also includes observations after training.Source: §6 · p. 9

7 CONCLUSION

The model uses pooled transition and co-occurrence patterns to predict temporal links beyond repeated edges, and provides a competitive alternative to learned node states. It achieves the highest MRR among compared methods on 7 of 16 datasets, with 9–13 learned parameters.

7 CONCLUSION

  • On 7 of 16 TGB and TGB-Seq datasets, the model achieves the highest MRR among compared methods.It also outperforms EdgeBank and Base3 on all 16 datasets and the heuristic family on 14.Source: §7 · p. 9
  • The model uses pooled transition and co-occurrence patterns to predict links beyond repeated edges, including links without the same source-specific transition.These patterns can support new links even when the current source has not encountered the transition before.Source: §7 · p. 9
  • The timing analysis keeps the model configuration and weights fixed.Future work could test choosing marginal or continuation backoff separately for each query.Source: §7 · p. 9
  • With 9–13 learned parameters, the model provides a simple, competitive baseline for evaluating future neural temporal link predictors.Source: §7 · p. 9

A DATASETS AND REFERENCE SCORES

The appendix describes the benchmark references and compared methods, then reports fitted weights for the reference configuration and added features.

A DATASETS AND REFERENCE SCORES

B THE COMPARED METHODS

  • The compared temporal models use learned representations or encoders, while EdgeBank stores observed source-destination pairs and PopTrack maintains decayed destination counts.Other baselines include a fixed-priority heuristic family and Base3’s combination of EdgeBank membership, popularity, and windowed co-occurrence.Source: B THE COMPARED METHODS · pp. 13–14

C THE FITTED WEIGHTS

  • Figure 5 reports mean reference-configuration weights across three runs, distinguishing marginal from continuation backoff and positive from negative coefficients.The right panel groups transition coefficients by history position, benchmark, and backoff distribution.Source: Figure 5 · p. 14, C THE FITTED WEIGHTS · p. 14
  • Figure 6 reports mean weights for co-occurrence and added recency features when each is added separately to the reference configuration.Its multi-scale panels include slow (10τr) and fast (τr/10) recency features.Source: C THE FITTED WEIGHTS · p. 14, Figure 6 · p. 15

D SENSITIVITY TO THE CONTEXT DEPTH

The context-depth sweep varies the number of transition features while holding other settings fixed. Results differ across datasets: deeper context helps on TGB-Seq and some TGB datasets, but hurts on others.

D SENSITIVITY TO THE CONTEXT DEPTH

  • The sweep varies K from one to 20 with marginal backoff and holds smoothing, decay, and recency settings fixed; the main experiments use K = 5.Each depth uses popularity, source history, source recency, and the first k transition features.Source: D SENSITIVITY TO THE CONTEXT DEPTH · p. 15
  • TGB-Seq gains from K = 1 to K = 20 range from 0.57 to 3.19 MRR points.The largest gain is on Flickr, where source-specific transition coverage is zero; pooled transition estimates can still draw on earlier destinations.Source: D SENSITIVITY TO THE CONTEXT DEPTH · p. 15
  • On TGB, mean MRR increases on tgbl-uci, tgbl-subreddit, tgbl-review, tgbl-coin, and tgbl-comment, but decreases on the remaining datasets.The largest decline is 1.94 points on tgbl-lastfm.Source: D SENSITIVITY TO THE CONTEXT DEPTH · p. 15

E A WORKED EXAMPLE OF THE ESTIMATORS

The worked stream illustrates smoothed destination, source-history, and transition estimates, plus continuation backoff and windowed co-occurrence. Recency features capture activity across sources and time since a source last visited a candidate.

E A WORKED EXAMPLE OF THE ESTIMATORS

  • In the six-event example, marginal smoothing gives candidate b a destination estimate of 0.30, source conditional estimate of 0.38, and transition estimate of 0.65.For an unobserved successor of c, smoothing gives a transition estimate of 0.10.Source: E A WORKED EXAMPLE OF THE ESTIMATORS · p. 15
  • Continuation backoff assigns b an estimate of 0.43 because it has followed two distinct predecessors, while an empty transition row returns the continuation base.Source: E A WORKED EXAMPLE OF THE ESTIMATORS · pp. 15–16
  • Windowed co-occurrence counts destinations within a history window even when intervening events separate them, as with the pair (a, c).The model combines these estimates with learned weights, and smoothing gives positive probability to a destination the source has never visited.Source: E A WORKED EXAMPLE OF THE ESTIMATORS · p. 16, Table 3 · p. 16
  • Global recency distinguishes a candidate visited by another source from one not visited by the current source; multi-scale recency applies three fixed decay scales.Figure 7 also shows source recency as zero for a candidate the source has never visited.Source: E A WORKED EXAMPLE OF THE ESTIMATORS · p. 16, Figure 7 · p. 17

F CONTROLLED FEATURE COMPARISONS

Controlled comparisons show that co-occurrence often improves MRR, while adding more features does not consistently help. The section also documents fixed hyperparameters, fitting choices, and sensitivity to the smoothing constant.

F CONTROLLED FEATURE COMPARISONS

  • Co-occurrence gives the largest median gain among individual additions and improves MRR on 14 of 16 datasets with marginal backoff; combining all features gives a larger median gain but helps fewer datasets.Each addition is compared with the reference configuration under the same backoff distribution, using three runs.Source: F CONTROLLED FEATURE COMPARISONS · p. 17, Figure 8 · p. 18

G HYPERPARAMETERS AND THEIR PROVENANCE

  • The reference uses α = 1, K = 5, and τr = 0.01; τ is set to the validation duration and recency uses the training duration, with both constants fixed.Each split duration is its last timestamp minus its first; the appendices examine α, context depth, and additional recency scales.Source: G HYPERPARAMETERS AND THEIR PROVENANCE · p. 18
  • The reference configuration has K + 4 learned parameters; co-occurrence and global recency add one each, multi-scale recency adds two, and all features yield K + 8.Table 4 lists the feature sets used in the main comparisons, fixed across five runs.Source: G HYPERPARAMETERS AND THEIR PROVENANCE · p. 18, Table 4 · p. 18
  • Training uses Adam at learning rate 10^-2 for up to 100 epochs, with early stopping after ten validation-MRR epochs without improvement.Fits stop by the patience criterion before reaching the epoch limit; test evaluation uses the best-validation-MRR weights.Source: G HYPERPARAMETERS AND THEIR PROVENANCE · p. 19

H SENSITIVITY TO SMOOTHING

  • Across smoothing constants, the median range of mean test MRR is 0.75 points, with the largest range, 3.03 points, on tgbl-review.The comparison varies α over {10^-1, 1, 10} in a fixed reference configuration; it does not select α for the main results.Source: Figure 9 · p. 19, H SENSITIVITY TO SMOOTHING · p. 19

I SENSITIVITY TO EVALUATION TIMING

The evaluation varies when test events enter history while keeping model weights and candidates fixed. The model's advantage over matched count baselines persists under batch-wise updates, while strict-timestamp evaluation isolates the effect of excluding earlier events at the same timestamp.

I SENSITIVITY TO EVALUATION TIMING

  • Event-wise updates add each scored event to history immediately; batch-wise updates wait until 200 consecutive test events are scored, while strict-timestamp scoring excludes earlier events at the query timestamp.Strict-timestamp evaluation keeps events with earlier timestamps in history; the event-wise schedule retains file order for tied timestamps.Source: I SENSITIVITY TO EVALUATION TIMING · p. 20
  • Under event-wise and batch-200 updates, the model exceeds EdgeBank and Base3 on all 16 datasets and the heuristic family on 14; with batch-200 updates it has the highest MRR on 4 of 16.The heuristic is stronger on tgbl-uci and tgbl-comment under event-wise updates, and tgbl-enron and tgbl-comment under batch-wise updates.Source: Table 5 · p. 20, I SENSITIVITY TO EVALUATION TIMING · pp. 20–21, Table 6 · p. 21
  • Strict-timestamp evaluation produces the largest reductions on tgbl-enron, WikiLink, Flickr, and YouTube; configurations, weights, and candidates stay fixed, and baselines are not evaluated under this schedule.Table 7 reports test MRR and changes from event-wise evaluation.Source: Table 7 · p. 21, I SENSITIVITY TO EVALUATION TIMING · p. 21

Related papers

Loading 2609.35364v1…