Source-linked AI summary
Wasserstein Learning of Deep Generative Point Process Models
Shuai Xiao, Mehrdad Farajtabar, Xiaojing Ye, Junchi Yan, Le Song, Hongyuan Zha
TL;DR
The paper addresses restrictive intensity parameterizations and likelihood-based learning for temporal point processes. It introduces an intensity-free neural generative model trained with Wasserstein distance, which performs robustly across synthetic and real-world processes without prior knowledge of the true process. The Wasserstein-distance computation itself includes a computationally intractable formulation for which a dual form is used.
Problem
Conventional point-process models impose parametric assumptions on latent event dynamics, while intensity-based sampling can be computationally expensive and likelihood-based learning can miss modes.
Method
The paper transforms a noise point process into target sequences with a neural generator and trains it adversarially using Wasserstein distance rather than intensity modeling or likelihood estimation.
Results
WGANTPP estimates empirical intensity robustly across different synthetic point-process types and recovers real-world intensity better than other models when the true generative process is unknown.
Takeaways & Limitations
The approach can estimate point-process behavior across a wide scope of theoretical and real-world processes without prior knowledge of the underlying true process.
Takeaways & Limitations
The Wasserstein-distance formulation is computationally highly intractable, so the method uses its dual form.
Abstract
from arXiv · showhide
Point processes are becoming very popular in modeling asynchronous sequential data due to their sound mathematical foundation and strength in modeling a variety of real-world phenomena. Currently, they are often characterized via intensity function which limits model's expressiveness due to unrealistic assumptions on its parametric form used in practice. Furthermore, they are learned via maximum likelihood approach which is prone to failure in multi-modal distributions of sequences. In this paper, we propose an intensity-free approach for point processes modeling that transforms nuisance processes to a target one. Furthermore, we train the model using a likelihood-free leveraging Wasserstein distance between point processes. Experiments on various synthetic and real-world data substantiate the superiority of the proposed point process model over conventional ones.
1 Introduction
Temporal point processes model event timing directly, but conventional intensity-based and likelihood-based approaches impose restrictive assumptions and can struggle with complex sequence distributions. The paper proposes an intensity-free, Wasserstein-trained generative model for point processes.
- Motivation: Temporal point processes represent events as discrete times and model intervals between events without requiring time-window aggregation.They have been applied to social networks, health records, e-commerce, and finance.
- Limitations of existing models: Conventional models use conditional intensity functions whose parametric assumptions can miss the latent dynamics generating observed event patterns.The paper links model misspecification to degraded performance.
- Limitations of existing models: Intensity-based neural models avoid fixed parametric forms but still explicitly model intensity, which is unnecessary for some generation and prediction tasks.Sampling from intensity-based models commonly uses computationally expensive thinning algorithms.
- Proposed approach: The paper proposes a neural point-process model that bypasses intensity modeling and likelihood-based estimation through generative adversarial learning.Its generator produces synthetic sequences and its discriminator distinguishes real from synthetic data.
- Proposed approach: Wasserstein training is used because its Earth Mover distance correlates with generated-sample quality, reduces mode dropping, and incorporates event-sequence geometry.The paper extends WGANs to point processes with recurrent neural networks and evaluates them using statistical and quantitative measures.
2 Proposed Framework
The framework represents point processes as measurable event-count objects, defines a sequence distance for temporal realizations, and trains an intensity-free generator with Wasserstein adversarial learning. Its distance compares event locations through optimal transport and, for temporal sequences, equals the volume difference between counting measures.
- Point-process representation: A point process maps outcomes of a probability space to counting measures, with realizations represented as collections of Dirac masses or event times.Counting processes record event counts on measurable subsets, while temporal point processes restrict events to [0, T).
- Point-process representation: Temporal point processes are commonly characterized by conditional intensity functions, including Poisson, self-exciting, self-correcting, and recurrent-neural-network forms.These forms specify event rates using fixed functions, previous events, or recurrent hidden states.
- Wasserstein distance: The Wasserstein distance compares two point-process distributions through couplings whose marginals are the real and generated process distributions.Its underlying sequence distance is defined using optimal transport between discrete event-time measures.
- Wasserstein distance: For temporal sequences, the proposed distance matches ordered events and penalizes unmatched events using an anchor at T.The resulting metric is computationally efficient and interpretable as the volume by which the two counting measures differ.
- Wasserstein learning: The Wasserstein dual is computationally intractable in full generality, so the method approximates its Lipschitz function class with a parameterized discriminator.The discriminator family is optimized over parameters while enforcing the Lipschitz constraint.
- Wasserstein learning: WGANTPP uses a generator that maps noise to point-process samples and a discriminator that distinguishes generated from real sequences, with Lipschitz regularization added to the RNN loss.The generator and discriminator are trained using empirical expectations over real and generated sequences.
3 Experiments
The experiments evaluate WGANTPP on synthetic and real-world point-process data using QQ-plot and empirical-intensity deviations. WGANTPP is tested against parametric maximum-likelihood models and RNN-based alternatives under known, misspecified, and mixed generating processes.
- Datasets: The study simulates 20,000 sequences over [0,15) from inhomogeneous, self-exciting, self-correcting, and recurrent neural point processes.Four additional datasets mix triplets of these processes to test mode dropping.
- Datasets: The real-data evaluation uses healthcare, public-media, stock-exchange, and publication-citation sequences, with all time scales normalized to [0,15].The datasets include MIMIC-III, MemeTracker, NYSE, and publication citations.
- Evaluation: Performance is measured by QQ-plot slope deviation and empirical-intensity deviation, using empirical intensity for every model.The QQ procedure assesses whether generated sequences match the theoretical quantiles associated with the learned or ground-truth process.
- Synthetic results: On synthetic data, WGANTPP performs robustly across process types and outperforms parametric and RNN approaches when the generating form is misspecified.A correctly specified parametric estimator performs better when the true parametric form is known.
- Synthetic results: For mixtures of point processes, WGANTPP produces better empirical intensities than alternatives that fail to capture data heterogeneity.The mixture setting is used to examine mode dropping, while QQ plots are not feasible for mixture models.
- Real-world results: Across real-world datasets with no prior knowledge of the true process, WGANTPP recovers intensity better than all other models.The analysis also reports that WGANTPP outperforms the MLE-NN approach despite using the same RNN architecture.
4 Conclusion and Future Work
The paper presents Wasserstein learning as an intensity-free approach that estimates point processes without prior knowledge of the underlying true process across theoretical and real-world settings. Future work targets optimal transport, alternative distances, causal transforms, marked processes, and structured spaces.
- Conclusion: The proposed approach estimates point processes accurately across a wide scope of theoretical and real-world processes without prior knowledge of the true process.The authors identify this as the central conclusion of the work.
- Future work: Future work will investigate optimal transport, alternative distances, causal transforms, marked point processes, and processes over structured spaces.These directions extend both the learning objective and the types of point processes represented.
A Data flow of Wassterstein learning for point process
The data flow maps a Poisson prior sequence through a generator into a synthetic event sequence, then compares generated and real sequence distributions using a Wasserstein discriminator.
- Generator: The generator maps an input sequence ζ sampled from a Poisson prior into an output event sequence ρ.The prior parameter λ_z is estimated from real data.
- Discriminator: The discriminator computes the Wasserstein distance between the generated sequence distribution ρ and the real sequence distribution ξ.This distance supplies the comparison used by the Wasserstein learning scheme.
B Proof that ∥· ∥⋆is a norm
The paper establishes that ∥· ∥⋆ satisfies the triangle inequality, completing its characterization as a norm alongside nonnegativity and symmetry.
- If ∥ξ − ρ∥⋆ = 0, the sequences have equal length and an assignment matches every x_i to an identical y_σ(i).
- The distance is nonnegative and symmetric, providing the remaining stated norm properties.
- The triangle inequality follows by composing the optimal matching with a fixed permutation and minimizing over all permutations.The proof assumes sequence sizes n ≤ k ≤ m and concludes ∥ξ − ζ∥⋆ = ∥ξ − ρ∥⋆ + ∥ρ − ζ∥⋆.
C Proposed ∥· ∥⋆Distance on the Real Line
For temporal point-process sequences on [0,T), the proposed distance compares event times through an optimal permutation after padding the shorter sequence.
- The construction considers ordered sequences ξ = {t1 < ... < tn} and ρ = {τ1 < ... < τm} on [0,T).
- The shorter sequence is padded with T, allowing both sequences to have equal length before matching.
- The identity permutation minimizes the matching cost, so event points are paired in increasing order.A non-identity first mismatch can be exchanged for a lower-cost pairing, contradicting optimality.
D Equivalence of the ∥· ∥⋆Distance and Difference in Count Measures
The proposed sequence distance equals the L1 area between the corresponding count measures when the shorter sequence is anchored at T.
- The construction uses the temporal point process count measure N(B) over B ⊂ [0,T).
- For ordered event times, the count-measure difference consists of blocks with sizes |t_i − τ_i| and, for extra events, T − τ_{n+i}.
- The total area between the count measures is S = Σ_i s_i.
- The sequence distance also equals Σ_i s_i, so it exactly matches the area between the two count measures.Choosing s = T supplies the anchor point for padding the shorter sequence.
E WGANTPP algorithm
WGANTPP alternates critic updates with generator updates, using specified regularization, optimization, batching, and Poisson prior settings.
- WGANTPP initializes critic and generator parameters, then iterates until generator convergence.
- Each generator iteration performs ncritic critic updates using Adam with regularization coefficient ν and hyperparameters α, β1, and β2.
- The default configuration uses ν = 0.3, α = 1e−4, β1 = 0.5, batch size m = 256, and ncritic = 5.
- The algorithm samples real point-process realizations for critic training and draws generated realizations from a Poisson process prior.