PaperlayerView exact revision on arXiv

Source-linked AI summary

Realism VS Accuracy: Event Sequence Forecasting from a Generative Modeling Perspective

Dmitry Osin, Egor Surkov, Petr Mokrov, Igor Udovichenko, Ivan Karpukhin, Evgeny Burnaev, Egor Shvetsov, Alexander Korotin

arXiv:2610.12586v1cs.LG

TL;DR

The paper addresses long-horizon forecasting of event sequences with heterogeneous numerical and categorical features. It frames forecasting as conditional generation and benchmarks accuracy and realism using adapted models and metrics. The evaluation finds a trade-off between the two, while Wasserstein BaryBooster uses diffusion samples to achieve state-of-the-art prediction accuracy.

  • Problem

    The paper studies long-horizon forecasting of timestamped event sequences with heterogeneous numerical and categorical features.

    Source: §1 · p. 1

  • Method

    The authors benchmark adapted statistical, predictive, and generative models using metrics for prediction accuracy and distributional realism.

    Source: §1 · p. 2

  • Results

    No method excels at both TMS and cDFS, while Wasserstein BaryBooster applied to LDM produces the highest TMS across datasets while retaining non-trivial cDFS.

    Source: §1 · p. 2, §6.2 · p. 9

  • Takeaways & Limitations

    Aggregating samples from a realistic generative model with BaryBooster yields a state-of-the-art point forecast without additional training for the target metric.

    Source: §7 · p. 9

    The empirical study uses relatively few datasets focused narrowly on card transactions, and BaryBooster's inference-time computation may be prohibitive in latency-sensitive or resource-constrained deployments.

    Source: §7 · p. 9

Abstract

from arXiv · show

Event sequences are chains of random events, each described by heterogeneous (numerical and categorical) features. The problem of event sequence forecasting occurs in finance, healthcare, business process intelligence, and other fields. In this work, we study multi-horizon forecasting of event sequences as a conditional generative modeling problem. We evaluate forecasts not only for prediction accuracy, but also for distributional realism. To this end, we present a benchmark comprising metrics for both quality aspects, strong statistical baselines, recent models adapted from adjacent fields, and curated real-world datasets. Careful evaluation reveals that no method excels at both accuracy and realism, highlighting a fundamental trade-off between the two. Finally, we introduce Wasserstein BaryBooster, an algorithm that aggregates samples from a realism-strong diffusion model into a state-of-the-art predictor, demonstrating that distributional realism can be converted into prediction accuracy. Code is available at https://github.com/xaosina/event-sequence-forecasting.

1 Introduction

The paper studies long-horizon forecasting of heterogeneous event sequences as conditional generation, evaluating both prediction accuracy and distributional realism. It benchmarks adapted models and metrics, finds a trade-off between the two quality dimensions, and introduces Wasserstein BaryBooster to improve prediction accuracy from diffusion samples.

1 Introduction

  • Event sequences contain timestamped events with heterogeneous numerical and categorical features, and the paper studies forecasting them over long horizons from recorded histories.Source: §1 · p. 1
  • The benchmark evaluates forecasts for both closeness to the actual continuation and resemblance to genuine future trajectories, dimensions prior evaluations largely overlooked.Source: §1 · pp. 1–2
  • The authors adapt statistical baselines, generative diffusion models, and predictive models, and provide datasets and metrics for both forecast-quality dimensions.Source: §1 · p. 2
  • No method excels at both TMS and cDFS, exposing a fundamental trade-off between prediction accuracy and distributional realism.Source: §1 · p. 2
  • Wasserstein BaryBooster aggregates samples from a diffusion model to produce state-of-the-art prediction accuracy.Source: §1 · p. 2

2 Problem Statement and Related Work

The paper formalizes history-conditioned forecasting of heterogeneous event sequences and positions its benchmark among existing generative models, benchmarks, and metrics. It adapts three model classes and introduces metrics designed to assess both accuracy and distributional realism.

2 Problem Statement and Related Work

  • Given a history S1:H, the task is to forecast the next L events, each represented by a timestamp and numerical and categorical feature vectors.Source: §2 · p. 2

2.1 Generative Models for Event Data

  • The paper adapts representatives of latent diffusion, data-space diffusion, and non-diffusion methods to general event-sequence forecasting, a setting prior work had not studied to the authors’ knowledge.Source: §2.1 · p. 3

2.2 Benchmarks for Event Sequences

  • The benchmark extends heterogeneous-event work to forecasting and complements TPP benchmarks by supporting heterogeneous features and evaluating both accuracy and distributional realism.Source: §2.2 · p. 3

2.3 Metrics for Generative Sequence Models

  • TMS uses data-driven normalization and one-to-one horizon matching for heterogeneous features, while cDFS adapts a discriminator-based two-sample test to conditional, heterogeneous continuations.Source: §2.3 · p. 4
  • Existing metrics were not fully suitable for assessing either accuracy or realism on heterogeneous event sequences, motivating the two new measures.Source: §2.3 · p. 4

3 Models for Evaluation

The evaluation combines statistical baselines, GPT-style predictors, and adapted generative models spanning latent diffusion, data-space diffusion, and non-diffusion approaches. The adaptations accommodate heterogeneous event features and fixed-length forecasting.

3 Models for Evaluation

  • The evaluation compares statistical baselines, GPT-style methods, and representative latent-diffusion, data-diffusion, and non-diffusion models adapted for event sequences.Source: §3 · p. 4

3.1 Baselines

  • Repeat copies the last L historical events, HistSampler samples historical events with replacement, and Mode uses historical categorical modes and numerical medians.Source: §3.1 · p. 4

3.2 GPT-based

  • The GPT-2-style AR model predicts events successively, whereas MT predicts L future events in parallel from the history.Source: §3.2 · p. 4

3.3 Latent Diffusion (LDM)

  • LDM encodes events with a β-VAE, diffuses over sequences of latent event representations, and conditions generation on a history embedding plus recent-event latent codes.Source: §3.3 · pp. 4–5

3.4 Data-space Diffusion Model (DDM)

  • DDM adapts CDiff by applying Gaussian diffusion to inter-arrival times and numerical features and categorical diffusion to each categorical feature.Source: §3.4 · p. 5

3.5 DEF

  • DEF is adapted to predict full heterogeneous feature vectors at candidate events and selects the top L candidates by presence score for the fixed-length horizon.Source: §3.5 · p. 5

4 Benchmark Description

The benchmark evaluates forecasting of heterogeneous event sequences with datasets and metrics designed to assess both prediction accuracy and distributional realism. Its public-data scope is constrained, while an additional evaluation probes whether the observed accuracy–realism pattern extends to TPP corpora.

4.1 Datasets

  • Only three qualifying public corpora—Gender, Age, and Alphabattle—were found, all card-transaction datasets; no non-financial corpus met the size, length, and structure criteria.Table 2 reports sequence counts, average lengths, feature counts, and time spans for the datasets.Source: Table 2 · p. 5, §4.1 · p. 6
  • Despite shorter, smaller TPP corpora with only timestamps and categorical marks, the accuracy–realism trade-off and leading models on each axis carry over.Source: §4.1 · p. 6

4.2 Metrics

  • The benchmark pairs TMS for prediction accuracy with cDFS for distributional realism, addressing the lack of metrics for arbitrary numbers of heterogeneous features.The paper also reports Shape and Trend as auxiliary distributional diagnostics.Source: §4.2 · p. 6
  • cDFS tests whether a discriminator can distinguish generated from real continuations, optionally including recent history; 0 indicates perfect separation and 1 chance-level performance.Because the discriminator sees history, it can detect continuations that seem plausible alone but conflict with preceding events.Source: §4.2 · p. 6
  • TMS scores per-feature predictions and jointly matches horizon positions with one shared permutation, using a Hungarian assignment that includes time and penalizes worsened temporal alignment.Numerical and time scores are rescaled against a horizon-naive baseline; categorical scores use per-pair macro-F1, placing heterogeneous features on a common scale.Source: §4.2 · pp. 6–7

5 Wasserstein BaryBooster

Wasserstein BaryBooster aggregates stochastic continuations from a generator into one forecast. It selects a continuation optimal under TMS, using a discrete analogue of a Wasserstein barycenter, without retraining the generator.

5 Wasserstein BaryBooster

  • Wasserstein BaryBooster turns N stochastic samples from any base generator into a single continuation optimal under TMS.The procedure is post hoc and does not require retraining the underlying generator.Source: §5 · p. 7
  • Given a history, the procedure draws N continuations from the base model and aggregates them using a discrete TMS analogue of a Wasserstein barycenter.The reported experiments use N = 30, and another metric can replace TMS in the aggregation objective.Source: §5 · p. 7

6 Experiments

Experiments compare forecast accuracy and distributional realism across fixed horizons. Results expose a trade-off, while Wasserstein BaryBooster improves diffusion-model prediction and retains some realism.

6.1 Evaluation Protocol

  • The protocol evaluates fixed horizons L ∈ {4, 16, 32, 64}, reports both cDFS and TMS, and compares direct forecasts with semi-autoregressive cross-horizon rollouts.Main results use L = 32; the authors report that key conclusions hold at L ∈ {4, 16, 64}.Source: §6.1 · p. 7

6.2 Results

  • Methods with high prediction accuracy show low distributional realism and vice versa; Repeat lies on the baseline Pareto front across all datasets.Figure 2 plots TMS against cDFS at horizon L = 32, with the solid line marking the baseline Pareto front.Source: §6.2 · p. 8, Figure 2 · p. 8
  • BaryBoosting LDM achieves the highest TMS across datasets while retaining non-trivial cDFS; DDM gains are modest and boosting gains are absent for DEF.Table 3 reports TMS and cDFS at horizon L = 32.Source: Table 3 · p. 8, §6.2 · p. 9
  • Aggregating multiple LDM samples around their TMS barycenter converts part of the model’s realism into the highest TMS, while leaving the original LDM available when realism matters more.The effect is strongest at L = 4, where boosting shifts LDM from near-bottom TMS to best-in-class prediction.Source: §6.2 · p. 9
  • BaryBoosting LDM improves many prediction metrics beyond the optimized TMS, and DDM also improves on independent metrics with a few exceptions.Table 4 presents an excerpt of prediction metrics beyond the optimized TMS at L = 32.Source: §6.2 · p. 9, Table 4 · p. 9

7 Discussion

The discussion presents sample aggregation as a way to derive point forecasts from realistic generative models, while noting limits in dataset scope and inference compute.

Outlook

  • Aggregating samples from a realistic generative model yields a state-of-the-art point forecast without additional training for the target metric.The authors propose that one generative model trained once could support multiple downstream event-sequence tasks.Source: §7 · p. 9

Limitations

  • The study uses relatively few datasets focused narrowly on card transactions, and BaryBooster’s inference overhead may be prohibitive in latency-sensitive or resource-constrained deployments.At N = 5, it retains approximately 75% of the TMS gain at one sixth of the sampling cost; the samplings parallelize across GPUs.Source: §7 · p. 9

Broader impacts

  • The forecasts may support planning and risk analysis by letting analysts explore multiple plausible client futures rather than a single average forecast.The authors note possible misuse of realistic synthetic transactions to evade fraud detection; their models use public, anonymized data.Source: §7 · p. 9

A Generative Models for Event Data: Extended Discussion

The extended discussion groups related event-data generative models into tabular diffusion, temporal point processes, and methods for heterogeneous event sequences, noting differences in their data and task scope.

Tabular data

Temporal point processes

  • Temporal point processes model event times, with spatio-temporal variants adding numerical coordinates and marked variants adding one categorical mark per event.Examples include Add-and-Thin [Lüdke et al., 2023] for point processes and DEF [Karpukhin and Savchenko, 2026], which predicts and scores candidate future events.Source: A Generative Models for Event Data:… · p. 13

General event sequences

  • Some marked-TPP methods extend to heterogeneous events, whereas TabDiT [Garuti et al., 2025] targets attribute-conditional generation and UniTTab uses representations for discriminative tasks.The passage identifies these as differing from history-conditional generative forecasting.Source: A Generative Models for Event Data:… · p. 13

B Benchmarks for Event Sequences: Extended Discussion

The extended discussion distinguishes earlier event-sequence benchmarks from heterogeneous, realism-aware forecasting and details how TMS evaluates full continuations through feature scores and horizon alignment.

B Benchmarks for Event Sequences: Extended Discussion

  • EasyTPP and HoTPP forecast only within the single-mark MTPP setting and report accuracy-style metrics, while EBES handles heterogeneous events but targets sequence classification, not forecasting.The benchmark discussion identifies these as the closest prior efforts but notes that none covers the paper’s full forecasting setting.Source: B Benchmarks for Event Sequences:… · pp. 13–14

C.2 Paired R1 for numerical features and time

  • R1 rescales numerical-feature and time errors against history-based naive forecasts: 1 means zero error, 0 baseline-level error, and values below 0 mean worse-than-baseline performance.Baseline rescaling makes features on incomparable units commensurable for averaging.Source: §C.2 · p. 14

C.3 Paired F1 for categorical features

  • Paired F1 compares categorical labels at matching horizon positions and averages macro-F1 over classes present in either continuation, avoiding inflation when many labels are absent.The score is averaged over present classes per pair, a useful choice for high-cardinality features.Source: §C.3 · p. 15

C.4 Permutation-dependent scores

  • Permutation-dependent scores re-align predicted and true horizon positions, so an otherwise correct continuation is not penalized solely for having its events in a different order.The alignment is one-to-one across horizon slots.Source: §C.4 · p. 15

C.5 Matched per-feature scores

  • Matched per-feature scores independently choose each feature’s best horizon alignment, forming optimistic upper envelopes of the paired scores.Each matched score is at least its paired counterpart.Source: §C.5 · pp. 15–16

C.6 Per-pair decomposition and Hungarian optimization

  • A per-pair decomposition yields a precomputable cost matrix, whose maximum alignment is solved exactly with the Hungarian algorithm.The optimization applies to each sequence-prediction pair.Source: §C.6 · p. 16

C.7 TMS

  • TMS scores time and all numerical and categorical features under one shared horizon reordering, rather than independently matching each feature.Because the alignment is shared, TMS generally lies below the average of the independently matched per-feature scores.Source: §C.7 · p. 16

D.1 Technical Details of cDFS

  • The cDFS evaluation combines real and generated continuations into a binary detection dataset, splits it into train, validation, and test sets, and reports held-out discriminator performance.Sequences are truncated to the last 2L events, and the discriminator is a two-layer GRU with hidden size 512.Source: §D.1 · p. 16

D.2 Robustness of cDFS to the discriminator choice

The appendix finds that cDFS rankings are robust to discriminator choice, while Shape and Trend provide useful but incomplete distributional diagnostics.

D.2 Robustness of cDFS to the discriminator choice

  • Across 24 dataset–configuration pairs, ground truth scores approximately 1 and LDM is the most realistic; cDFS rankings correlate 0.91–1.00 with the GRU-512-2L ranking.Tables report cDFS under eight discriminator configurations for Age, Gender, and Alphabattle; the selected GRU-512-2L catches repetition across all three datasets and is cheaper and more reliable than the largest configurations.Source: §D.2 · pp. 16–17, Table 5 · p. 17, Table 6 · p. 17

D.3 Shape and Trend scores

  • Shape and Trend separate models even when cDFS is near zero, but ignore event order and history, so they serve as auxiliary feature-distribution checks rather than primary realism measures.The table reports both scores alongside cDFS at horizon L = 32.Source: §D.3 · pp. 17–18, Table 8 · p. 18

E.1 Replacing cDFS with simpler diagnostics.

  • High Shape or Trend scores do not imply high cDFS: many baselines score well on these diagnostics while collapsing on cDFS, which captures sequential and conditional structure.The pooled figure compares cDFS with Shape and Trend across all three datasets at horizon L = 32.Source: §E.1 · pp. 18–19, Figure 3 · p. 19

E.2 Wasserstein BaryBooster Independent Evaluation

Wasserstein BaryBooster improves prediction accuracy, while the evaluation also tests how well TMS tracks per-feature metrics and whether results hold across horizons and qualitative examples. The appendix details statistical variation and the optimization procedure.

E.2 Wasserstein BaryBooster Independent Evaluation

  • BaryBooster improves all LDM prediction-accuracy metrics consistently across datasets; DDM also improves, with a few exceptions.The reported results support the claim of improved prediction accuracy beyond TMS.Source: §E.2 · p. 19, Table 9 · p. 19

E.3 TMS and matched submetrics

  • TMS grows consistently with matched amount, time, and MCC submetrics, which use separate best permutations rather than TMS’s shared permutation.The correlation indicates that TMS aligns with interpretable per-feature metrics and that gains reflect improvements across forecasting dimensions.Source: §E.3 · p. 19, Figure 4 · p. 20

E.4 Other Horizons

  • At L = 64, LDM is the most realistic model and LDM-WB has the highest TMS on all datasets; Repeat’s high cDFS on Gender and Age reflects detector failure to catch history copying.Repeat’s cDFS is 0.73 on Gender and 0.83 on Age, so those scores should be read with caution.Source: §E.4 · pp. 20–21, Figure 6 · p. 20, Figure 7 · p. 21

E.5 Statistical Significance

  • Metric gaps are much larger than across-seed standard deviations, while LDM-WB’s cDFS of 0.224 is within one standard deviation of the full detector estimate.The three-seed mean is 0.202 ± 0.005, and detector cross-validation contributes an internal standard deviation of about 0.02 (0.024 for LDM-WB).Source: §E.5 · p. 21, Table 10 · p. 21

E.6 Qualitative Samples

  • For one Age client at L = 32, LDM reproduces irregular timing and amount variability, while MT and AR collapse numerical features, Mode repeats one event, Repeat copies history, and HistSampler places events at implausible times.Figure 8 shows continuations conditioned on the same observed history; the original is the ground-truth continuation.Source: §E.6 · p. 21, Figure 8 · p. 22

E.7 Comparison of LDM with ADiff4TPP

  • The LDM implementation performs on par with or slightly better than ADiff4TPP, so the authors omit it as a separate comparison model.Table 11 compares ADiff4TPP and LDM at L = 32 on Age, with and without BaryBooster.Source: §E.7 · p. 22, Table 11 · p. 22

F Wasserstein BaryBooster: optimization details

  • Alternating minimization updates alignments exactly with Hungarian matching, then sets numerical features and times to aligned medians and categorical features to aligned modes.The procedure initializes with the sample minimizing the sum of pairwise TMS costs; categorical matching uses a 0/1 surrogate, so it finds a local solution and terminates in finitely many steps.Source: F Wasserstein BaryBooster:… · p. 22

F.1 Effect of the sample size N

The sample-size study finds that relatively few LDM samples recover most of BaryBooster’s TMS gain. The appendix also reports how boosting behaves on other models and documents data preparation and the LDM–ADiff4TPP comparison.

F.1 Effect of the sample size N

  • N = 5 recovers about 75% of the full TMS gain at one sixth of the sampling cost, N = 10 recovers about 90%, and gains saturate after N ≈10–15.N = 1 is the unboosted LDM; the study is at horizon L = 32.Source: §F.1 · p. 23, Table 12 · p. 23

G Wasserstein BaryBooster on remaining models

H Preprocessing Details

  • Preprocessing applies log1p to transaction amounts and uses out-of-sample test splits, dataset-specific user splits and filters, and fixed evaluation cutoffs per sequence.Training cutoffs are sampled uniformly over admissible times and resampled each epoch; Gender uses three fixed test cutoffs per sequence, while Age and Alphabattle use one.Source: H Preprocessing Details · pp. 23–24

I Implementation Details

  • The implementation appendix records training and validation protocols, shared backbones, diffusion schedules, and non-learned baselines.For a like-for-like ADiff4TPP comparison, the authors train conditional variants of both models and apply classifier-free guidance to LDM at inference.Source: I Implementation Details · p. 24, §I.1 · pp. 24–25, Table 13 · p. 24

I.1 Comparison of LDM and ADiff4TPP [Mukherjee et al., 2026]

  • The LDM implementation performs on par with or slightly better than ADiff4TPP, so it is not included as a separate model in the comparison.The comparison uses conditional variants of both models.Source: §I.1 · pp. 24–25, Table 13 · p. 24

I.2 Schematic Illustration of Methods

The appendix illustrates the adapted methods and describes shared training and calibration protocols. Runtime analysis shows that base-model sampling accounts for most BaryBooster wall time, with matching taking a larger share at longer horizons.

I.2 Schematic Illustration of Methods

I.3 Shared protocol and infrastructure

  • Diffusion models use classifier-free guidance, which modulates conditioning strength at inference to balance sample diversity and generation fidelity; validation calibration selects among guidance scales and other inference settings.The appendix also specifies common feature representations and reconstruction losses, optimizer settings, batching, and data handling.Source: §I.3 · p. 26
  • AR, MT, LDM, and DDM share a six-layer, eight-head transformer trunk; AR predicts autoregressively, while MT predicts L events in parallel from a pooled hidden state.DEF instead uses a one-layer GRU history encoder and retains the top-L candidates; DDM and LDM use GRU history encoders, with LDM also conditioning on the last L history-event latents.Source: §I.3 · pp. 26–27

J Runtime and memory consumption

  • Base-model sampling accounts for about 85% of BaryBooster time at L = 32 and 62% at L = 64; matching takes a larger share at longer horizons, while independent samples can be distributed across GPUs.The decomposition uses N = 30 and 10 test batches per dataset; parallel sampling can reduce that stage’s time almost proportionally.Source: J Runtime and memory consumption · p. 27

K Public datasets considered

This section describes the public datasets used and the criteria for their inclusion, including transfer experiments. The authors report relying only on secondary public datasets, with no crowdsourcing or primary human-subjects research.

K Public datasets considered

  • The survey compares public corpora from prior benchmarks and adjacent generative-modeling work using two protocol-based quantitative criteria and five task-based qualitative criteria.Corpora sharing an underlying source are consolidated into canonical rows; Alphabattle uses the official Alfa-Bank test split.Source: K Public datasets considered · pp. 28–29
  • Screening excludes corpora with repetitive identical events, recommender-scale vocabularies, shuffled order, time-series structure, or many tied timestamps.These properties can trivialize Repeat and collapse evaluation axes, mismatch recommender metrics and assumptions, break temporal ordering, or weaken ordering signal; examples include Taobao, Megamarket, Zvuk, MBD, PhysioNet 2012, and MIMIC-IV.Source: K Public datasets considered · pp. 28–29
  • Only Sberbank Sirius (Age) and Alfa-Bank Alphabattle meet both quantitative thresholds and have heterogeneous fields and unshuffled temporal order.The thresholds are at least 5,000 sequences and mean length at least 64 events; the benchmark’s three datasets—Gender, Age, and Alphabattle—each exceed 5,000 sequences.Source: K Public datasets considered · pp. 28–29

L Transfer to non-transactional TPP corpora

NeurIPS Paper Checklist

  • The paper reports that methods, metrics, datasets, splits, and training/evaluation protocols are specified for reproducibility, and that statistical significance is addressed in Appendix E.5.The checklist identifies the work as empirical, without numbered theoretical results requiring proofs.Source: NeurIPS Paper Checklist · pp. 33–34

K Public datasets considered

Related papers

Loading 2610.12586v1…