Source-linked AI summary
Modeling Events with Cascades of Poisson Processes
Aleksandr Simma, Michael I. Jordan
TL;DR
The paper addresses how to model dependent events in continuous time, where standard Poisson processes do not capture their dependencies. It models each event as triggering a Poisson process and uses distributed EM; its results show gains from modeling event timing and, especially, label dependencies.
Problem
Standard Poisson processes assume independent counts across nonoverlapping intervals, unlike dependent event collections.
Source: §2 · p. 2Method
The model represents each event as triggering a Poisson process of successor events and fits latent causes with EM that can be distributed.
Source: §1 · p. 1, §2.1 · p. 2Results
Modeling periodicity and less-periodic timing variability yields roughly equivalent gains, while a labeling distribution conditioned on previous events provides the largest boost.
Source: §3.3 · p. 6Takeaways & Limitations
The framework is applied to financial Twitter messages and Wikipedia revision histories, demonstrating scalability on a massive dataset.
Source: §1 · p. 1, §5 · p. 10Latent root causes, Bayesian parameter integration, and time-varying parameters remain proposed extensions, with inference complexity and scalability concerns.
Source: §5 · p. 10
Abstract
from arXiv · showhide
We present a probabilistic model of events in continuous time in which each event triggers a Poisson process of successor events. The ensemble of observed events is thereby modeled as a superposition of Poisson processes. Efficient inference is feasible under this model with an EM algorithm. Moreover, the EM algorithm can be implemented as a distributed algorithm, permitting the model to be applied to very large datasets. We apply these techniques to the modeling of Twitter messages and the revision history of Wikipedia.
1 Introduction
The paper models dependent continuous-time events through cascades of triggered Poisson processes, with EM inference that can be distributed for large datasets. Its applications include financial Twitter messages and Wikipedia revisions, and its model differs from prior approaches in supporting structured labels and non-adjacent-time interactions.
1 Introduction
- Each event induces a Poisson process of triggered events, and EM inference can be distributed across clusters for massive datasets.The case studies model financial Twitter messages and Wikipedia revisions, including billions of potential interactions.Source: §1 · p. 1
- Unlike cited continuous-time graphical models, this approach supports structured label spaces rather than only a small set of event labels [2005; 2002, 2005].The passage identifies Rajaram et al. [2005] and Nodelman et al. [2002, 2005] as examples of those approaches.Source: §1 · p. 1
- Unlike Wingate et al. [2009], this model is continuous-time and does not restrict interactions to adjacent time steps.The paper describes its work as an extension and generalization of the continuous-time noisy-or of Simma et al. [2008].Source: §1 · p. 1
- The model's fundamental stochastic process is a mutually self-exciting point process, related to earthquake-prediction models [Ogata, 1988].The cited Epidemic Type Aftershock-Sequences framework involves a similar model applied to earthquake prediction.Source: §1 · p. 1
2 Modeling Events with Poisson and Cox Processes
The section develops a dependent-event model in which events trigger later Poisson processes, then describes EM fitting, feature-based fertility models, and computational strategies for large datasets.
2 Modeling Events with Poisson and Cox Processes
- Standard Poisson models use fixed event rates, time densities, and feature distributions, while Cox processes allow random mean measures to represent dependencies.The standard Poisson process assumes counts in nonoverlapping intervals are independent; the paper instead models triggered events.Source: §2 · p. 2, §2.1 · p. 2
2.1 Events Causing Other Events
- Each event triggers a Poisson process of later events; augmenting each event with its cause yields a random forest whose roots arise from the background process.The triggering kernel must be zero for nonpositive delays, so events can cause only later events.Source: §2.1 · p. 2
2.2 Model Fitting
- EM estimates latent event causes as soft assignments, enabling expected sufficient statistics for parameter estimation from weighted samples.The model also constrains total mean-measure intensity to equal the observed event count, applying the constraint during or after the M step.Source: §2.2 · p. 3
2.3 The Fertility Model
- Fertility α(x), the expected number of events, must be modeled parametrically when feature spaces are too large for direct estimation.The section develops linear and multiplicative fertility forms; its presentation assumes binary feature vectors, and linear fertility restricts coefficients to be nonnegative.Source: §2.3 · p. 3
2.4 Computational Efficiency
- For small label sets and exponential delays, memorylessness permits collapsed statistics that avoid explicitly enumerating each event's possible causes.With sparse transitions, the method has complexity O(|D|z̄) and is more efficient than the direct method for many closely spaced events, but applies only to restricted models.Source: §2.4 · p. 4
- Wikipedia fitting uses Hadoop MapReduce to run page-neighborhood EM iterations and aggregate shared hyperparameter statistics.Each mapped object must fit in memory; splitting oversized neighborhoods requires many more chained MapReduce jobs.Source: §2.4 · p. 4
3 Twitter Messages
The Twitter study models messages as triggered events with timing and feature-based transition distributions. Across increasingly sophisticated models, the largest improvement comes from allowing successor features to depend on prior events; the results also distinguish timing and transition effects.
3 Twitter Messages
- The dataset contains 54,717 ticker-related messages over 39 days, represented by user, timestamp, and binary features.Features include stock tickers, links, and keywords such as “buy” or “option.”Source: §3 · pp. 4–5
- The periodic baseline divides the day into intervals, assumes a uniform rate within each interval, and repeats the pattern daily.Its gain over the time-homogeneous baseline is smaller than the gains from other model components.Source: §3 · p. 5
3.1 Intensity and Delay Distributions
- The timing model uses a gamma delay distribution with shape below 1 as the best-performing candidate, while delay choice matters less than transition choice.The candidate delay families include uniform, piecewise uniform, exponential, and gamma distributions.Source: §3.1 · p. 5, Figure 2 · p. 5
3.3 Results and Discussion
- The model cascade improves log-likelihood, with the largest gain from a labeling distribution that lets successor features depend on previous events.The preceding gains come from modeling periodicity and less-periodic timing variability.Source: Table 1 · p. 6, §3.3 · p. 6
3.2 Transition Distribution
- Models 4 and 5 outperform the first three, and model 5 performs significantly better than model 4 when each transition component has its own fertility and delay functions.Figure 3 tracks component proportions and delay means across EM iterations; component means differ even as the mixture-wide mean remains almost constant.Source: §3.2 · p. 6, Figure 3 · p. 6
3.3 Results and Discussion
- The overall-intensity proxy has the highest mean delay because average activity across labels changes more slowly than activity for a particular stock or topic.For distribution shape, lower k and higher-variance gamma distributions work best.Source: §3.3 · p. 7
- The feature-dependent fertility model allows some events to generate more successors than others, but contributes less to log-likelihood than the other model components.Source: §3.3 · p. 7
4 Wikipedia
The Wikipedia case study models revisions using page and neighbor relationships, while limiting possible causes to each page’s link neighborhood for computational reasons. Across 414,540 pages and 71,073,739 revisions, the learned transition patterns assign substantial activity to baseline, neighbor, and same-page processes.
4.1 Structure in Wikipedia’s History
- Edits are categorized as minor fixes, major inserts, major deletes, major changes, reverts, or other edits.Reverts return a page to an earlier state and are typically responses to vandalism, though good-faith edits can also be reverted.Source: §4.1 · p. 7
4.2 Delay Distributions
- A three-exponential delay mixture captures short-, medium-, and longer-term effects; its components have means of 3.6 minutes for same-page delays and 13.8 minutes for adjacent-page delays.The fast component can capture edits made and saved in sequence or immediate responses; the other components capture periodicity, changing interest, and reactions to specific edits.Source: §4.2 · p. 7
4.3 Transition Distribution
- The model predicts revision features rather than exact edit content, splitting features into revision-wide attributes and page identity, with transition probabilities regularized toward shared values.Individual neighbor intensities are difficult to estimate accurately for most pages because data are insufficient.Source: §4.3 · p. 8
- Regularized transition matrices perform significantly better on unseen data but worse on training data, indicating strong regularization.The likelihood comparisons cover successive model iterations; the baseline-only result is not shown in Figure 5.Source: Figure 5 · p. 8, §4.3 · p. 8
4.4 Learned Transition Matrices
- 23.87% of observed events come from the baseline, 19.11% from edits on neighboring pages, and 57.02% from edits on the same page.The dataset contains 414,540 pages and 71,073,739 revisions, with billions of considered interactions.Source: §4 · p. 7, §4.4 · p. 9
- Same-type edits on neighboring pages make the same edit type more likely, while large changes tend to trigger other large changes regardless of their specific type or editor.The diagonal is predominantly positive, and transitions among massive inserts, deletions, and changes have similar magnitudes.Source: §4.4 · p. 9
4.5 Regularizing Intensity Estimates
- Estimating separate neighbor intensities improves held-out likelihood on pages with more than 500 revisions, but the gain is very small.The authors report that estimating individual intensities is impractical without many revisions and requires strong regularization.Source: §4.5 · pp. 9–10
5 Conclusions
The paper presents and applies a scalable framework for modeling events as cascades of Poisson processes. Its distributions support varied domains, while latent root causes, Bayesian parameter integration, and time-dependent parameters remain directions for further investigation with computational trade-offs.
5 Conclusions
- The paper presents a framework for event models based on cascades of Poisson processes, demonstrates applications, and demonstrates scalability on a massive dataset.Source: §5 · p. 10
- The techniques support a wide range of delay, transition, and fertility distributions, allowing applications to many different domains.Source: §5 · p. 10
- Further work could add latent root causes, Bayesian parameter integration, and time-dependent parameters, but these extensions raise inference and scalability concerns.Bayesian integration may help fit complex fertility or transition distributions with insufficient observations; time-dependent parameters are described as useful, though computational issues remain a concern.Source: §5 · p. 10