PaperlayerView exact revision on arXiv

Source-linked AI summary

CAUSE: Learning Granger Causality from Event Sequences using Attribution Methods

Wei Zhang, Thomas Kobber Panum, Somesh Jha, Prasad Chalasani, David Page

arXiv:2002.07906v1cs.LGstat.ML

TL;DR

Existing point-process approaches trade model flexibility for explainability, complicating Granger-causality discovery across diverse event interactions. CAUSE fits a neural point process and uses axiomatic attribution to derive a causality statistic; experiments report stronger performance than state-of-the-art methods across five datasets.

  • Problem

    Existing point-process approaches trade flexibility for explainability, making Granger-causality inference challenging for event sequences with diverse interactions.

    Source: §1 · p. 1, §2.3 · p. 3

  • Method

    CAUSE fits a neural point process to capture event interdependency, then uses an axiomatic attribution method to derive a Granger causality statistic.

    Source: §1 · p. 2

  • Results

    Across five datasets, CAUSE outperforms state-of-the-art methods in fitting and Granger-causality discovery and computes its statistic three orders-of-magnitude faster.

    Source: §4 · p. 7

  • Takeaways & Limitations

    Experiments show CAUSE can identify inter-type Granger causality on synthetic and real-world datasets with diverse event interactions.

    Source: §5 · p. 9

    CAUSE's attribution setup uses methods satisfying linearity, completeness, null-player, fidelity-to-control, and batchability properties.

    Source: §3 · p. 4

Abstract

from arXiv · show

We study the problem of learning Granger causality between event types from asynchronous, interdependent, multi-type event sequences. Existing work suffers from either limited model flexibility or poor model explainability and thus fails to uncover Granger causality across a wide variety of event sequences with diverse event interdependency. To address these weaknesses, we propose CAUSE (Causality from AttribUtions on Sequence of Events), a novel framework for the studied task. The key idea of CAUSE is to first implicitly capture the underlying event interdependency by fitting a neural point process, and then extract from the process a Granger causality statistic using an axiomatic attribution method. Across multiple datasets riddled with diverse event interdependency, we demonstrate that CAUSE achieves superior performance on correctly inferring the inter-type Granger causality over a range of state-of-the-art methods.

1. Introduction

Multi-type event sequences arise in asynchronous processes, but existing point-process models can lack either the flexibility to represent diverse interactions or the explainability needed to infer Granger causality. CAUSE fits a neural point process and applies axiomatic attribution to estimate causality, with experiments reporting stronger performance than state-of-the-art methods.

1. Introduction

  • Hawkes-based models assume past events independently and additively excite future events, limiting their fit to sequences with inhibitive effects or event interactions.Examples include maintenance reducing breakdown risk and multiple medicines contributing to unexpected adverse events.Source: §1 · p. 1
  • CAUSE fits a flexible neural point process to capture event interdependency, then applies an axiomatic attribution method to derive a Granger causality statistic.The framework aims to address the tension between model flexibility and explainability.Source: §1 · p. 2
  • CAUSE outperforms other state-of-the-art methods and uses two-level batching to compute Granger causality at a scale of millions of events.The paper evaluates on synthetic and real-world datasets with diverse event interdependency.Source: §1 · p. 2

2. Background

The background establishes notation and introduces multivariate point processes, Granger causality for event sequences, and attribution methods.

2.1. Notations

  • The notation describes each subject's event sequence using timestamps, event-type labels, one-hot type vectors, and a sequence length.The text omits the subject index when discussing a single sequence without ambiguity.Source: §2.1 · p. 2

2.2. Multivariate Point Process

  • A multivariate point process models multiple event types in continuous time; Hawkes processes represent excitation through nonnegative pairwise kernels.Neural point processes parameterize conditional intensity functions with deep neural networks; when relevant integrals lack closed forms, Monte Carlo approximations can be inefficient, high-variance, and slow to converge.Source: §2.2 · p. 2

2.3. Granger Causality for Multi-Type Event Sequences

  • Type k is Granger non-causal for type k′ when its history adds no information about type k′'s future conditional intensity given other types' histories.The background notes that complex interactions and non-excitatory effects make inference challenging and motivate restrictions to parametric process classes such as Hawkes models.Source: §2.3 · p. 3

2.4. Attribution Methods

  • An attribution method assigns per-feature contributions to a prediction relative to a baseline, with axioms and properties used to characterize suitable methods.The paper also identifies fidelity-to-control and batchability as useful properties.Source: §2.4 · p. 3

3. Proposed: CAUSE

CAUSE fits a neural point process (NPP) to capture event interdependencies, then applies an axiomatic attribution method to the trained model to compute a Granger causality statistic. Its design combines a flexible yet computationally feasible decoder with a two-level batching scheme for computing that statistic.

3. Proposed: CAUSE

  • CAUSE trains an NPP on event sequences and inspects its predictions with an axiomatic attribution method to compute Granger causality.The framework assumes the attribution method satisfies linearity, completeness, null player, fidelity-to-control, and batchability properties.Source: §3 · p. 4

3.1. A Semi-Parametric Neural Point Process

  • The semi-parametric decoder is designed to model a wide variety of conditional-intensity patterns while keeping their computation feasible.It combines neural-network-computed weights with pre-specified basis functions, which allow the relevant integral to be evaluated analytically.Source: §3.1 · p. 4

3.2. From Event Contributions to a Granger Causality Statistic

  • CAUSE attributes cumulative intensity over each inter-event interval to past events, then aggregates and normalizes those contributions to score Granger causality between event types.The normalization accounts for differences in event-type frequency, and negative scores can represent inhibitive influence.Source: §3.2 · p. 5

3.3. Computing the Granger Causality Statistic

  • Two-level batching reduces attribution-method invocations from O(SNK) to O(SK/B), with up to three orders-of-magnitude speedup when sequence length and batch size are relatively large.Intra-sequence batching computes the attribution sum with one call; inter-sequence batching processes sequences in a mini-batch together.Source: §3.3 · pp. 5–6

3.4. Choice of Attribution Methods

  • Integrated Gradients is used in the experiments, while other attribution methods satisfying CAUSE's stated properties can also be used.The text gives DeepLIFT as an example; methods that violate batchability, such as Shapley values, remain applicable if that efficiency property is not required or is handled alternatively.Source: §3.4 · p. 6

4. Experiments

Across synthetic and real-world event sequences, CAUSE is evaluated for fit, Granger-causality discovery, and scalability. It outperforms baselines overall, while its speedup depends on sequence length and batch size.

4.1. Experimental Setup

  • The setup tests excitation, inhibition, and synergy synthetics alongside IPTV and MemeTracker, with five-fold cross-validation and NLL, Kendall’s τ, and AUC evaluation.IPTV lacks ground-truth causality; MemeTracker uses URL links between sites as an approximate weighted ground-truth matrix.Source: §4.1 · p. 7

4.2.1. GOODNESS-OF-FIT

  • CAUSE achieves smaller hold-out NLLs than every baseline on all datasets, nearly matching ground-truth-model NLLs on the three synthetic datasets.The experiments use hold-out NLL for goodness-of-fit evaluation.Source: §4.2.1 · p. 7, §4.1 · p. 7, Figure 1 · p. 8

4.2.2. CAUSALITY DISCOVERY

  • CAUSE significantly outperforms all baselines on Inhibition, Synergy, and MemeTracker, and is a close second on Excitation despite the Hawkes-based methods’ expected advantage.The reported comparison covers the four datasets with ground-truth causality.Source: §4.2.2 · p. 7, Table 1 · p. 8
  • For inhibition and synergy, which violate Hawkes-process assumptions through non-excitatory or complex interactions, Hawkes-based methods tend to perform poorly.RPPN also performs significantly worse than CAUSE on all datasets; the authors suspect its positive-only attention weights and possible attribution errors contribute.Source: §4.2.2 · p. 8
  • CAUSE’s IPTV matrix shows large positive diagonal entries and positive cross-category associations, as well as negative associations such as ads to drama and education to entertainment.The reported positive links include military, laws, finance, and education to news, and kids and music to drama; the negative links are unavailable to excitation-only models such as HSG.Source: §4.2.2 · p. 8, Figure 2 · p. 8

4.2.3. SCALABILITY

  • With batch size at least 16 and average sequence length at least 100, Algorithm 1 achieves over three orders-of-magnitude speedup over the naive implementation.Speedup scales almost linearly when sequence length is at most 150 and batch size at most 16, then becomes sublinear as the tested GPU reaches maximum utilization.Source: §4.2.3 · pp. 8–9, Figure 3 · p. 9

5. Conclusion

CAUSE combines a flexible neural point process with an axiomatic attribution method to estimate Granger causality, and uses two-level batching to compute the statistic efficiently. The paper evaluates it on synthetic and real-world datasets with diverse event interactions.

5. Conclusion

  • CAUSE trains a flexible neural point process to capture event interdependency, then applies axiomatic attribution to the trained model to compute a Granger-causality statistic.A two-level batching algorithm is derived for efficient computation.Source: §5 · p. 9
  • The experiments on synthetic and real-world data support CAUSE’s effectiveness for identifying Granger causality across diverse event interactions.Source: §5 · p. 9

A. Additional Related Work

  • Hawkes processes model mutual excitation but assume past events influence future events independently and additively, restricting their flexibility for complex real-world interactions.The paper describes more flexible alternatives, including PCIM variants and neural point processes, as generally more predictive but often uninterpretable for causality statistics.Source: A. Additional Related Work · p. 12
  • Methods for synchronous multivariate time series are not directly applicable to asynchronous event sequences without discretizing the continuous observation window.Source: A. Additional Related Work · p. 12
  • Prior event-sequence Granger-causality methods are described as largely limited to Hawkes processes, whose causality structure is encoded in excitation kernels.Source: A. Additional Related Work · p. 12
  • Prediction attribution assigns input features scores for their contribution to a black-box model’s prediction; perturbation methods can be sensitive to perturbation choices and costly for high-dimensional inputs.Source: A. Additional Related Work · p. 12
  • Integrated Gradients uses a path integral of gradients to address gradient saturation, while attention weights have been questioned as reliable explanations of neural predictions.Jain & Wallace (2019) report that learned attention weights can be uncorrelated with gradient-based feature importance and can be permuted without changing predictions.Source: A. Additional Related Work · pp. 12–13

B.1. Proof of Proposition 1

The proof establishes batchability for Integrated Gradients and DeepLIFT, using a separability argument for Integrated Gradients and a related construction for DeepLIFT.

B.1. Proof of Proposition 1

  • The proof treats batchability (P2) as the remaining property to establish for Integrated Gradients and DeepLIFT after citing their satisfaction of axioms A1–A4.It states that P1 follows directly from the definition of either method.Source: §B.1 · p. 13
  • For Integrated Gradients, the derivation swaps summation and gradients, then uses separability of gradients across terms to establish the batch formula.Source: §B.1 · p. 13
  • For DeepLIFT, the proof uses its representation as an input difference times modified gradients, noting that a final linear sum preserves each input’s modified gradient.Source: §B.1 · p. 13

B.2. Proof of Proposition 2

The section verifies Shapley-value attribution properties and describes dyadic Gaussian basis functions designed to reflect time-varying conditional intensities.

B.2. Proof of Proposition 2

  • The Shapley-based attribution assigns each input a score using a value function that evaluates spliced inputs combining selected features with a baseline.Source: §B.2 · p. 13
  • The proof establishes linearity, implementation variance, completeness, null-player behavior, and fidelity-to-control for the Shapley attribution method.Source: §B.2 · p. 14

B.3. Dyadic Gaussian Basis

  • The dyadic Gaussian basis encodes a smoothness bias for conditional intensities over time, with early bases capturing short-term effects and later bases capturing mid- or long-term effects.Source: §B.3 · p. 14

B.4. Proof of Proposition 3

The proof rewrites the statistic and applies attribution properties to establish its formula. The accompanying basis illustration distinguishes short-term from mid/long-term effects.

B.4. Proof of Proposition 3

  • The proof rewrites the statistic before deriving the stated formula.The proof then says that this establishes the formula.Source: §B.4 · pp. 14–15
  • The null-player property makes attributions zero for events before the i-th event.The proof applies this to the integral term that does not depend on events before the i-th event.Source: §B.4 · p. 14
  • The first few dyadic Gaussian bases capture short-term effects, while later bases characterize mid/long-term effects.The example uses L = 100 and R = 5.Source: Figure 4 · p. 15
  • The derivation uses linearity and replaces f with F in an intermediate step.The proof identifies linearity as property A1.Source: §B.4 · pp. 14–15

C.1. The settings for synthetic and real-world datasets.

The appendix describes the synthetic and real-world datasets, including their generation or preprocessing, and points to a table of dataset statistics. The synthetic data are designed around excitation, inhibition, and synergistic event interactions.

C.1. The settings for synthetic and real-world datasets.

  • Table 2 summarizes statistics for the five datasets considered in the paper.The appendix directs readers to it for the dataset statistics.Source: §C.1 · p. 15, Table 2 · p. 16
  • Excitation data come from a multivariate Hawkes process, while inhibition data come from a multivariate self-correcting point process.Both use 1,000 sequences and 10 event types; their sparse off-diagonal weights are sampled from positive values for excitation and negative values for inhibition.Source: §C.1 · p. 15
  • The Synergy dataset uses paired A-and-B occurrence to create a large joint effect on E, alongside C's isolated excitation and D as a complicating type.The five-type structure is replicated once to create ten event types and 1,000 sequences.Source: §C.1 · pp. 15–16
  • The IPTV data are normalized to days, and long sequences are split so each contains at most 1,000 events..Source: §C.1 · p. 16
  • The MT dataset filters MemeTracker phrases to two months and the top 100 domains, then keeps cascades with lengths from 3 to 500.Timestamps are normalized into hours.Source: §C.1 · p. 16

C.2. Implementation Details and Hyperparameter Configurations for Various Methods

CAUSE uses a GRU and a fully connected attribution network, with dataset-dependent hidden sizes and integration steps. The appendix also reports implementation choices for baselines and the runtime on the largest dataset.

C.2. Implementation Details and Hyperparameter Configurations for Various Methods

  • CAUSE uses a single-layer GRU and a two-layer fully connected attribution network with skip connections.Event embeddings and GRU hidden units are 64-dimensional for synthetic data; real-world GRU hidden units are 128. The line integral uses 20 steps on MT and 50 on other datasets.Source: §C.2 · p. 16
  • Hawkes-process baselines use the tick package, with relevant hyperparameters tuned by cross-validation.RPPN was implemented by the authors because public code was unavailable, using settings similar to CAUSE.Source: §C.2 · p. 16

C.3. Platform and Runtime

  • On MT, CAUSE's training and Granger-causality computation together took less than 3 hours.Experiments ran on a server with a 16-core CPU, 512G memory, and two Quadro P5000 GPUs.Source: §C.3 · p. 17

C.4. Qualitative Analysis on MT

The MT analysis visualizes estimated Granger-causality links between domains in two communities. The primer defines multivariate point processes and relates Granger non-causality to whether an event type's history affects another type's conditional intensity.

C.4. Qualitative Analysis on MT

  • The two MT communities show directed estimated Granger-causality links, with news.google.com and blog.myspace.com at their respective centers.The analysis also identifies links among investment-related sites and craigslist.org subdomains.Source: §C.4 · p. 17

D. A Primer on Measure and Probability Theory

  • A multivariate point process couples multiple counting processes; one realization is a sequence of timestamped events labeled by event type.The event types correspond to dimensions of the coupled process.Source: D. A Primer on Measure and Probability… · p. 18
  • A conditional intensity function gives an event type's expected instantaneous occurrence rate conditioned on the process history.The primer defines the history using the natural filtration.Source: D. A Primer on Measure and Probability… · p. 18
  • Type k is Granger non-causal for type k′ when, given other types' history, type k′'s conditional intensity does not depend on type k's past.Otherwise, type k is Granger causal for type k′.Source: D. A Primer on Measure and Probability… · p. 18
Loading 2002.07906v1…