PaperlayerView exact revision on arXiv

Source-linked AI summary

TREND: TempoRal Event and Node Dynamics for Graph Representation Learning

Zhihao Wen, Yuan Fang

arXiv:2203.14303v1cs.SI

TL;DR

Temporal graph learning must represent evolving links, but snapshot-based, non-inductive, or non-exciting approaches leave important aspects of that process unmodeled. TREND uses a Hawkes process-based GNN with event and node dynamics, and experiments on four real-world datasets demonstrate its effectiveness. The paper limits its setting to graph growth, leaving node and edge deletions for future work.

  • Problem

    Temporal graph methods may use discrete snapshots, fail to handle new nodes inductively, or omit event excitation and the individual and collective characteristics of link events.

    Source: §1 · pp. 1–2

  • Method

    TREND combines a Hawkes process-based GNN with event dynamics for individual characteristics and node dynamics for collective characteristics.

    Source: §1 · p. 2

  • Results

    Experiments on four real-world datasets demonstrate TREND's advantages and superior performance.

    Source: §1 · p. 3, §6 · p. 8

  • Takeaways & Limitations

    TREND models temporal link formation by accounting for both individual event characteristics and collective node dynamics.

    Source: §1 · p. 2, §6 · p. 8

    The study considers graph growth only and leaves node and edge deletions for future work.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

Temporal graph representation learning has drawn significant attention for the prevalence of temporal graphs in the real world. However, most existing works resort to taking discrete snapshots of the temporal graph, or are not inductive to deal with new nodes, or do not model the exciting effects which is the ability of events to influence the occurrence of another event. In this work, We propose TREND, a novel framework for temporal graph representation learning, driven by TempoRal Event and Node Dynamics and built upon a Hawkes process-based graph neural network (GNN). TREND presents a few major advantages: (1) it is inductive due to its GNN architecture; (2) it captures the exciting effects between events by the adoption of the Hawkes process; (3) as our main novelty, it captures the individual and collective characteristics of events by integrating both event and node dynamics, driving a more precise modeling of the temporal process. Extensive experiments on four real-world datasets demonstrate the effectiveness of our proposed model.

1 INTRODUCTION

Temporal graphs evolve through time as links form, so future-link prediction depends on historical link-formation dynamics. TREND combines a Hawkes process-based GNN with event and node dynamics to model individual and collective event characteristics.

1 INTRODUCTION

  • Links form chronologically as events, and predicting future links depends on the dynamics embodied in historical link formation.The introduction frames temporal graph representation learning as learning representations that evolve over time.Source: §1 · p. 1
  • Temporal graph methods face gaps in modeling continuous-time excitation, inductive node representations, and both individual and collective event characteristics.The cited prior work describes snapshot methods, continuous-time methods that miss explicit event excitation, and Hawkes-based embeddings that are transductive.Source: §1 · pp. 1–2
  • TREND models event uniqueness with an adaptable event prior, addressing the distinct contexts and times associated with link formation.The paper motivates this prior as a way to specialize general link-formation knowledge to individual events.Source: §1 · p. 2
  • TREND is inductive through its GNN architecture and models graph-wide link formation with a Hawkes process.The framework treats link formation as an event whose occurrence may be influenced by historical events.Source: §1 · p. 2
  • TREND integrates event and node dynamics in a Hawkes process-based GNN to model individual and collective event characteristics.The event prior captures event uniqueness, while node dynamics regularizes events at the node level.Source: §1 · p. 2
  • Experiments on four real-world datasets demonstrate TREND's advantages.The introduction reports extensive experiments without specifying particular metrics or comparison values in these passages.Source: §1 · p. 3

2 RELATED WORK

Prior temporal graph methods either discretize evolving graphs into snapshots or model continuous-time events. The related work identifies gaps in holistic event excitation, inductive learning, and joint event- and node-level dynamics.

2 RELATED WORK

  • Snapshot methods lose temporal-dynamics information, while continuous-time methods still do not explicitly model graph-wide excitation between sequential events.The cited work contrasts snapshot-based methods with continuous-time approaches including random walks and GNNs.Source: §2 · p. 3
  • Hawkes-based network embeddings model graph-wide formation but are transductive, while DyREP is inductive and addresses association and communication events instead.The paper focuses on link formation as the event type.Source: §2 · p. 3
  • Existing approaches do not integrate event and node dynamics to capture individual and collective event characteristics, respectively.The related-work discussion identifies this as an unaddressed combination among prior methods.Source: §2 · p. 3

3 PRELIMINARIES

The preliminaries define temporal graphs through nodes, edges, time, and node features, then formalize inductive representation learning and introduce Hawkes processes as a model of event intensity and excitation.

3.1 Temporal Graph Representation Learning

  • A temporal graph comprises nodes, edges, a time domain, and an input feature matrix; events denote edge formation at a particular time.The same node pair may form a link more than once, and the paper considers graph growth while leaving node and edge deletions for future work.Source: §3.1 · p. 3
  • Inductive temporal graph learning seeks a parametric model that maps nodes in the feature space, including future new nodes, to representations at any time.The definition requires the model to handle nodes outside the observed node set.Source: §3.1 · p. 3

3.2 Hawkes Process

  • A Hawkes process models event intensity conditional on past events, whose influence can excite later events through a time-decaying kernel.The passage describes this property as suitable for modeling graph-wide link formation.Source: §3.2 · p. 3

4 PROPOSED APPROACH

TREND combines a temporal GNN with Hawkes-process event modeling and node dynamics to represent link formation inductively. Its components model both individual event characteristics and the collective tendencies of events sharing a node.

4.1 Overview of TREND

  • The temporal GNN aggregates each node’s self-information and historical neighbors’ information to form representations used in modeling event and node dynamics.Source: §4.1 · p. 4, §4.2 · pp. 4–5

4.2 Hawkes process-based GNN

  • Temporal representations materialize Hawkes conditional intensity, with self-information representing base intensity and historical-neighbor information capturing excitement from past events.Source: §4.2 · p. 4

4.3 Modeling Event Dynamics

  • An event-conditioned learnable transformation adapts an event prior into event-specific transfer-function parameters to capture individual event characteristics.Source: §4.3 · p. 5

4.4 Modeling Node Dynamics

  • Node dynamics estimates each node’s number of new events over time, and smooth L1 loss fits those estimates while limiting skew from high-activity nodes.Source: §4.4 · p. 6

4.5 Overall Model: TREND

  • TREND integrates event and node losses to jointly optimize event-level and node-level dynamics within a Hawkes process-based GNN.Source: §4.1 · p. 3, §4.5 · p. 6
  • Training jointly optimizes the temporal GNN, event prior, transformation model, and node-dynamics estimator; the stated complexity is linear in the number of events when sampling parameters are constant.Source: §4.5 · p. 6

5 EXPERIMENTS

Experiments evaluate TREND on four public temporal networks, chiefly for temporal link prediction and also for temporal node-dynamics prediction. TREND performs best among the link-prediction methods and achieves the smallest node-dynamics prediction MAE on all four datasets.

5.1 Experimental Setup

  • Four public temporal networks are evaluated, with temporal link prediction as the main task and temporal node-dynamics prediction as a secondary task.The datasets are CollegeMsg, cit-HepTh, Wikipedia, and Taobao.Source: §5.1 · pp. 6–7

5.2 Temporal Link Prediction

  • TREND performs best among all methods on temporal link prediction.Table 2 reports performance in percent with 95% confidence intervals.Source: §5.2 · p. 7, Table 2 · p. 7

5.3 Ablation Study

  • The full TREND model achieves the best ablation performance when event and node dynamics are integrated.Performance generally increases as components are added to TGNN; event dynamics typically outperform node dynamics when added separately.Source: §5.3 · p. 8, Figure 3 · p. 8

5.4 Hyperparameter Study

  • On all datasets, one negative sample per positive event is a robust, efficient choice.More negative samples generally do not improve TREND's performance.Source: §5.4 · p. 8, Figure 4 · p. 8
  • Performance is stable across η2 values, with smaller values from 0.0001 to 0.01 tending to perform better.Larger η2 values correspond to very little scaling and shifting, similar to removing the event-conditioned transformation.Source: §5.4 · p. 8, Figure 4 · p. 8
  • TREND is generally robust when the node-loss coefficient η1 is around 0.01, while values that are too small or large are suboptimal.The passage associates a balanced node and event loss with improved stability and performance.Source: §5.4 · p. 8, Figure 4 · p. 8

5.5 Temporal Node Dynamics Prediction

  • TREND achieves the smallest mean absolute error for temporal node-dynamics prediction on all four datasets.This result extends the reported evaluation beyond temporal link prediction.Source: §5.5 · p. 8, Table 3 · p. 8

6 CONCLUSION

TREND is a Hawkes process-based GNN framework for temporal graph representation learning that integrates event and node dynamics. Experiments on four real-world datasets demonstrate its superior performance.

6 CONCLUSION

  • TREND uses a Hawkes process-based GNN to model temporal graphs inductively and capture a holistic view of link formation.It integrates event dynamics for individual event characteristics and node dynamics for collective characteristics.Source: §6 · p. 8

A CONNECTION BETWEEN TRANSFER FUNCTION AND CONDITIONAL INTENSITY

TREND’s transfer function can be chosen to approximate the Hawkes conditional intensity using temporal representations from stacked temporal GNN layers. Its training loop samples temporal events, computes representations and intensity, and updates model parameters from the overall loss.

A CONNECTION BETWEEN TRANSFER FUNCTION AND CONDITIONAL INTENSITY

  • A well-fit transfer function, such as a neural network, can approximate the Hawkes conditional intensity from temporal GNN representations.The section constructs base intensity from self-information and excitement from historical-neighbor information, then composes these into the conditional intensity.Source: A CONNECTION BETWEEN TRANSFER FUNCTION… · p. 11

B PSEUDOCODE

  • TREND training initializes temporal GNN, event-prior, transformation, and node-dynamics parameters, then samples batches of temporal events.For each event it calculates node representations, adapts the event prior, calculates intensity and overall loss, and backpropagates the loss to update parameters.Source: B PSEUDOCODE · p. 11

C ADDITIONAL DESCRIPTION OF DATASETS

The four datasets represent messaging, citation, Wikipedia editing, and e-commerce purchase events, with node features derived from IDs or textual information.

C ADDITIONAL DESCRIPTION OF DATASETS

  • CollegeMsg records private messages as temporal edges and uses one-hot node IDs because nodes have no features.cit-HepTh represents papers citing papers from 1993 to 2003 and uses word2vec embeddings of abstracts as node features.Source: C ADDITIONAL DESCRIPTION OF DATASETS · p. 11
  • Wikipedia temporal edges represent user edits, whose textual features are converted to 172-dimensional LIWC vectors and aggregated by user.The user-level feature is the sum of edit vectors, normalized.Source: C ADDITIONAL DESCRIPTION OF DATASETS · p. 11
  • Taobao temporal edges represent user purchases of items, and node features are preprocessed textual-feature embeddings.Source: C ADDITIONAL DESCRIPTION OF DATASETS · p. 11

D DETAILS OF TASK SETUP

The temporal link-prediction task trains on events before a cutoff and tests on events at or after it, using a logistic-regression classifier at test time.

D DETAILS OF TASK SETUP

  • Node representations are learned from events before the training cutoff, and events on or after the cutoff are prediction targets.Source: D DETAILS OF TASK SETUP · p. 11
  • Experiments train on all events before the final time step and test on events at that final step.For cit-HepTh, training excludes the 78th time step and prediction targets links formed then.Source: D DETAILS OF TASK SETUP · p. 11
  • At test time, logistic regression predicts temporal links using positives from the last time step and an equal number of randomly sampled non-link pairs.Source: D DETAILS OF TASK SETUP · p. 11

E ADDITIONAL DESCRIPTION OF BASELINES

The baselines span static graph methods, temporal methods, and Hawkes process-based methods, with shared settings used for selected model dimensions and sampling choices.

E ADDITIONAL DESCRIPTION OF BASELINES

  • Static baselines train on the graph formed before testing without temporal information, including DeepWalk, Node2vec, VGAE, and GAE.GraphSAGE is also included as a static-graph GNN that supports inductive representation learning.Source: E ADDITIONAL DESCRIPTION OF BASELINES · pp. 11–12
  • Temporal baselines include CTDNE, EvolveGCN, GraphSAGE+T, and TGAT, using time-respecting walks, evolving GCN parameters, time decay, or temporal attention and time encoding.Source: E ADDITIONAL DESCRIPTION OF BASELINES · p. 12
  • Hawkes process-based baselines train node embeddings from the temporal graph, including HTNE and MMDNE.HTNE models historical-neighbor influence; MMDNE distinguishes link-formation micro-dynamics from network-scale macro-dynamics.Source: E ADDITIONAL DESCRIPTION OF BASELINES · p. 12
  • Network-embedding baselines use 128-dimensional embeddings, while GNN baselines match TREND’s layer count, embedding dimensions, and learning rates.HTNE, MMDNE, and EvolveGCN use a five-neighbor historical window; neighborhood sample sizes vary by dataset for GraphSAGE, GraphSAGE+T, and TGAT.Source: E ADDITIONAL DESCRIPTION OF BASELINES · p. 12

F SCALABILITY STUDY

The scalability study finds that TREND’s per-epoch training time grows linearly with the number of training events on Taobao, supporting the model’s scalability.

F SCALABILITY STUDY

  • Training time per epoch grows linearly with training-event count on Taobao, consistent with the complexity analysis and indicating scalability.The study uses five subgraphs formed from 20%–100% of Taobao’s more than 4 million events.Source: F SCALABILITY STUDY · p. 12

Related papers

Loading 2203.14303v1…