Representation Learning for Dynamic Graphs: A Survey

• Graphs augment feature-based learning with relations that support inference among entities in social networks, recommender systems, ontologies, biology, and finance.These domains include evolving interactions such as communications, transactions, ratings, and supply-chain relations.Source: §1 · p. 1
• Dynamic graphs challenge static-graph methods because nodes, edges, and interactions change over time, motivating recent research focused on evolving applications.Examples include streaming messages, new users and products, daily ratings, transactions, and continuously changing supply-chain relations.Source: §1 · p. 2
• The survey organizes recent dynamic-graph representation learning methods as encoders that create time-dependent embeddings and decoders that support node classification, event prediction or interpolation, and link prediction.Its scope includes techniques for discrete-time and continuous-time events, with encoders capturing graph aspects and decoders turning embeddings into predictions.Source: §1 · p. 2
• The survey covers temporal encoders, dynamic-graph decoders, methods outside the encoder-decoder framework, applications, datasets, and open research problems.The planned coverage includes temporal aggregation, time regularization, decompositions, random walks, recurrent processes, time-predicting and time-conditioned decoders, statistical relational learning, dynamic knowledge graphs, and spatiotemporal graphs.Source: §1 · p. 2

2.1 Static Graphs

• Static graphs use vertices and edges, with adjacency, degree, and Laplacian matrices representing connectivity and edge structure; knowledge graphs add labeled relations.Knowledge graphs are multi-digraphs whose edges are source-relation-target triples, while heterogeneous information networks explicitly map nodes and edges to types.Source: §2.1 · p. 3

2.2 Dynamic Graphs

• Dynamic graphs are modeled either as continuous-time event streams that update an initial graph or as regularly sampled snapshot sequences, with snapshot models potentially losing temporal information.Continuous-time models can represent diverse events, whereas discrete-time models are generally easier; continuous-time models can be used for discrete-time data, but not necessarily vice versa.Source: §2.2 · pp. 4–5

2.3 Prediction Problems

• The survey focuses on node classification, edge prediction, and graph classification, while distinguishing interpolation of missing historical information from extrapolation of future events.Extrapolation is usually more challenging than interpolation, and streaming settings require real-time updates after rapidly arriving observations, often producing extrapolation problems.Source: §2.3 · pp. 5–6

2.4 The Encoder-Decoder Framework

• The encoder-decoder framework represents an encoder as producing node or relation embeddings and a decoder as using them for predictions, allowing compatible components from different models to be recombined.Models may train encoder and decoder parameters end-to-end; link prediction commonly converts positive-only observations into classification using generated negative samples.Source: §2.4 · pp. 6–7, §2.4.1 · p. 7

2.5 Expressivity

• Expressivity measures whether representations preserve distinctions needed for prediction: identical embeddings force deterministic decoders to assign identical outputs, causing underfitting.Full expressivity is defined separately for node classification, link prediction, and graph classification, with symmetry constraints for nodes and non-isomorphic graph sets.Source: §2.5 · pp. 8–9

2.6 Sequence Models

• Sequence-model foundations address variable-length dynamic observations through autoregressive windows or hidden-state models, including recurrent neural networks that capture complex long-range dynamics.The surveyed predecessors include hidden Markov models, Kalman filters, dynamic Bayesian networks, and dynamic conditional random fields.Source: §2.6 · p. 9
• RNNs update hidden representations from the current input and prior history, while LSTMs and GRUs provide successful gated architectures for sequential modeling.In dynamic graphs, LSTMs and GRUs can generate node representations from a node’s history.Source: §2.6 · pp. 9–10
• Fully attentive models replace recurrence with self-attention, allowing each sequence element to aggregate information from other elements using positional encodings, queries, keys, and values.Causal masks restrict attention to earlier points, while multi-head attention combines blocks with different learned projections; in dynamic graphs, sequences may represent node states across time.Source: §2.6 · p. 10

2.7 Temporal Point Processes

• Temporal point processes model asynchronous discrete events in continuous time through a conditional intensity function, which determines event density jointly with a survival function.The next-event time can be predicted from the resulting density.Source: §2.7 · pp. 10–11
• Dynamic-graph temporal point processes can learn or specify an intensity parameterized by node representations to predict when events occur for individual nodes or node pairs.Traditional processes include Hawkes, Poisson, self-correcting, and autoregressive conditional duration models, while recent work learns intensity functions from data.Source: §2.7 · p. 11

3. Representation Learning for Static Graphs

• Static-graph representation learning is introduced to support the subsequent discussion of dynamic graphs, with models organized around encoder construction and decoder use.The section directs readers to existing surveys for broader coverage of static graphs and knowledge graphs.Source: §3 · p. 11, §3.1 · p. 11

3.1.1 High-Order Proximity Matrices

• High-order proximity encoders represent nodes with rows or columns of similarity matrices, typically without learned parameters, using measures such as common neighbors, Jaccard, Adamic–Adar, Katz, and preferential attachment.These matrices encode multi-step or weighted neighborhood relationships beyond the adjacency matrix.Source: §3.1.1 · pp. 11–12

3.1.2 Shallow Encoders

• Shallow encoders directly learn embedding-vector and relation-matrix parameters as lookup functions, while decomposition methods learn unsupervised node vectors that place connected nodes nearby.Decomposition approaches use graph-Laplacian or adjacency spectral decompositions, and directed graphs can use separate source and target representations.Source: §3.1.2 · p. 12, §3.1.3 · pp. 12–13

3.1.4 Random Walk Approaches

• Random-walk encoders treat node sequences as sentences and apply language-model techniques, with walk strategies controlling whether embeddings capture local, global, or longer-term dependencies.Knowledge-graph variants constrain walks with hand-crafted, learned, or graph-structured meta-path alternatives.Source: §3.1.4 · pp. 13–14

3.1.5 Autoencoder Approaches

• Autoencoder approaches compress each node’s neighborhood into a deep representation and reconstruct it, optionally incorporating node attributes or high-order proximity matrices.SDNE reconstructs adjacency rows, while related methods add attributes or derive inputs from random-walk co-occurrences.Source: §3.1.5 · p. 15

3.1.6 Graph Convolutional Network Approaches

• GCNs repeatedly aggregate permutation-invariant neighbor information with node representations, allowing each node to receive information from at most L hops after L layers.Architectures include normalized aggregation, adaptive multi-head attention, neighborhood sampling, and relation-specific transformations for knowledge graphs.Source: §3.1.6 · pp. 15–17
• GCN expressivity results relate graph distinguishability to the Weisfeiler–Lehman test and show that certain G-invariant networks are universal approximators of continuous permutation-invariant functions.Xu et al. (2019b) and Morris et al. (2019) report the Weisfeiler–Lehman comparison, while Maron et al. (2019) and Keriven and Peyré (2019) establish universality; Chen et al. (2019c) connect the views.Source: §3.1.6 · p. 17

3.2.1 Decoders for Static Graphs

• Static-graph decoders classify nodes from single embeddings and predict links using dot products or symmetric vector combinations, while directed graphs require source–target-sensitive classifiers or projections.Common symmetric combinations include averaging, Hadamard multiplication, absolute difference, and squared difference; hyperbolic distance is another option.Source: §3.2 · p. 17, §3.2.1 · pp. 17–18
• Graph classification decoders compress node representations by averaging, DeepSet aggregation, virtual nodes, or deterministic or learned hierarchical clustering.Ying et al. (2018b) jointly learns the hierarchical structure with the classifier end-to-end, unlike deterministic clustering approaches.Source: §3.2.1 · p. 18

3.2.2 Decoders for Link Prediction in Static KGs

• Translational decoders vary restrictions on relation matrices: TransE (Bordes et al. (2013)) uses identity matrices, whereas TransR (Lin et al. (2015)) ties them and STransE (Nguyen et al. (2016)) imposes none.Kazemi and Poole (2018c) show that these and other translational variants are not fully expressive and have severe restrictions on modeled relation types.Source: §3.2.2 · pp. 18–19
• Bilinear decoders range from fully expressive but parameter-heavy RESCAL (Nickel et al. (2011)) to parameter-reduced DistMult (Yang et al. (2015)), which models only symmetric relations.ComplEx (Trouillon et al. (2016)), CP (Hitchcock (1927)), and SimplE (Kazemi and Poole (2018c)) reduce RESCAL’s parameters without sacrificing full expressivity; ComplEx distinguishes source and target nodes through complex conjugation.Source: §3.2.2 · pp. 19–20
• Deep-learning decoders score knowledge-graph edges with feed-forward or convolutional networks applied to concatenated or structured node and relation embeddings.Examples include feed-forward scoring by Dong et al. (2014) and Santoro et al. (2017), convolutional scoring by Dettmers et al. (2018) and HypER (Balazevic et al. (2018)), and neural tensor networks (Socher et al. (2013)).Source: §3.2.2 · p. 20

• Extrapolation requires node and relation embeddings based only on observations in the past, while interpolation uses observations before, at, and after the target time.Some encoders may support both problems, but the two settings typically require different encoder types.Source: §4 · p. 20

4.1 Aggregating Temporal Observations

• Aggregating adjacency matrices across snapshots collapses a dynamic graph into a static graph for embedding with static graph encoders or decoders.Liben-Nowell and Kleinberg (2007) sum or union adjacency-matrix entries, while Hisano (2018) unions recent formation and dissolution matrices.Source: §4.1 · p. 21
• Aggregation methods offer simplicity, scalability, and reuse of static-graph learning methods, but information loss can hinder accurate predictions when timing is important.Recent-weighted aggregation may suit extrapolation, whereas interpolation can use weights based on distance from the prediction timestamp.Source: §4.1 · pp. 21–22
• Recency-weighted aggregation assigns greater importance to newer snapshots, with θ controlling the emphasis on recent observations.Larger θ values increase the weight of recent adjacency matrices; the resulting aggregate can support interpolation or extrapolation with suitable weighting.Source: §4.1 · pp. 21–22
• Uniform aggregation can conflate distinct interaction histories, such as edge formation and dissolution, while weighted aggregation retains more but not all temporal information.The weighted aggregate cannot reveal that an edge disappeared only recently, despite preserving more temporal information than the unweighted aggregate.Source: §4.1 · pp. 21–22

Core approach

• Static encoders independently produce node features for every snapshot, which are then aggregated into one feature vector for a decoder.The method avoids first collapsing the dynamic graph into a static graph and instead aggregates representations after per-snapshot encoding.Source: §4.2 · p. 22

Temporal aggregation

• Explicit aggregators assign greater weights to recent snapshots, including exponentially decaying older features or prefixed weighted sums.Yao et al. (2016) and Zhu et al. (2012) exemplify this strategy.Source: §4.2 · p. 22
• Time-series alternatives predict next-snapshot features from prior representations using ARIMA, moving averages, or reinforcement-learning-based approaches.Huang and Lin (2009) and G¨une¸s et al. (2016) use ARIMA; da Silva Soares and Prudˆencio (2012) use ARIMA and moving averages; Moradabadi and Meybodi (2017) use basic reinforcement learning.Source: §4.2 · p. 22

Example

• Any static encoder can generate the independently computed snapshot embeddings before temporal aggregation, as illustrated with the common-neighbor encoder.The example aggregates embeddings from adjacency snapshots using θ = 0.5, producing one embedding per node.Source: §4.2 · pp. 22–23

Scalability

• Computing features independently across many snapshots can be computationally expensive, although later methods reuse previous computations for some static encoder families.The cited examples include decomposition and random-walk approaches that save computations in later snapshots.Source: §4.2 · p. 23

4.3 Time as a Regularizer

• Time-as-regularizer methods combine a static encoder for each snapshot with a smoothness constraint that keeps persistent nodes’ consecutive embeddings close.The constraint applies to nodes present in both snapshots and commonly uses Euclidean distance.Source: §4.3 · p. 23
• Distance variants align embeddings with a rotation before Euclidean comparison or measure angular distance between unit-norm vectors.Other distance spaces may also be used.Source: §4.3 · pp. 23–24
• Liu et al. (2019) formulate regularized representation learning as an efficiently approximated constrained optimization, enabling rapid updates for streaming data and newly added nodes.Pei et al. (2016) similarly encode temporal consistency for node classification by exponentially reducing the influence of older labels; these methods support interpolation and extrapolation.Source: §4.3 · p. 24
• Penalizing consecutive-embedding distances prevents sharp changes, which can be undesirable when nodes undergo major events such as acquisition.Initializing time t from time t−1 and then optimizing with a static encoder implicitly enforces smoothness while permitting necessary abrupt changes, as in Goyal et al. (2017).Source: §4.3 · p. 24
• Du et al. (2018) extend LINE to dynamic graphs and recompute embeddings only for nodes strongly influenced by the preceding snapshot.Source: §4.3 · p. 24

Tensor decomposition

• Tensor decomposition stacks adjacency matrices across timestamps to learn temporal factors and node or edge embeddings for dynamic graphs with fixed nodes.CP, Tucker, and HOSVD are applicable decomposition choices.Source: §4.4 · p. 25

Temporal prediction

• Holt-Winters extrapolates temporal factors to predict adjacency tensors for future snapshots, while the learned factors also support interpolation.Dunlavy et al. (2011) use Holt-Winters to predict the next L temporal steps.Source: §4.4 · p. 25

Time-dependent factorization

• Yu et al. (2017a) decompose dynamic graphs into static node features U and time-dependent features V_t, predicting future adjacency matrices as U(V_T+l)′.The time-dependent matrix is modeled as a polynomial in time, with temporal regularization and a timestamp window.Source: §4.4 · p. 25

Temporal knowledge graphs

• For temporal knowledge graphs, order-4 tensor decomposition approaches are studied in connection with human cognitive memory functions.Tresp et al. (2017) study this connection.Source: §4.4 · p. 26

The streaming scenario

• Incremental eigenvalue and SVD updates support streaming graphs and node or edge changes without recomputing each decomposition from scratch, but approximation error accumulates and requires periodic recalculation.Zhang et al. (2018d) propose notifying when a tolerance threshold is exceeded so recalculation can depend on graph dynamics rather than a fixed heuristic.Source: §4.4 · p. 26

4.5 Random Walk Encoders

• Random-walk updates reuse valid previous-snapshot walks and generate new walks only for affected nodes, reducing computation while updating node embeddings incrementally.Mahdavi et al. (2018) initialize each snapshot’s model from the previous one; Bian et al. (2019) apply a similar affected-node strategy to knowledge graphs using metapath2vec.Source: §4.5 · p. 27
• Naively reusing valid walks biases transition frequencies because earlier walks could not traverse edges introduced later, and additional walks do not remove this bias.In the example, the v2-to-v4 transition occurs once versus three times for v2-to-v1 or v2-to-v3, although from-scratch sampling would equalize the expected counts.Source: §4.5 · p. 27
• Sajjad et al. (2019) generate unbiased new-snapshot walks while reusing valid previous walks, whereas NetWalk learns embeddings by minimizing pairwise distances among nodes in each walk.These approaches exploit temporal structure mainly to reduce computation and can support feature aggregation at each snapshot, but may not capture node evolution or temporal patterns.Source: §4.5 · p. 28
• For continuous-time dynamic graphs, Nguyen et al. (2018b,a) constrain walks to move forward through time, with transition distributions that can depend on event timing.A next edge may be sampled with probability proportional to how long after the previous edge it was added; only temporally valid transitions are permitted.Source: §4.5 · p. 28
• De Winter et al. (2018) apply time-respecting walks to discrete-time graphs, while Bastas et al. (2019) combine static walks on older observations with temporal walks after a cutoff.De Winter et al. (2018) report that discretizing a continuous-time graph can sometimes improve performance; Bastas et al. (2019) use both walk types to learn node embeddings.Source: §4.5 · pp. 28–29

4.5.1 Analysis of Random Walk Encoders

• Random-walk encoders provide standalone embedding functions for unsupervised tasks such as clustering and community detection, but are not typically trained end-to-end for supervised prediction.They also require new walks and several gradient-update rounds when observations arrive, making them unsuitable for many streaming settings; extending them to attributed graphs remains open.Source: §4.5.1 · p. 29

4.6 Sequence-Model Encoders

• Sequence models process observations sequentially, making extrapolation their primary use, while bidirectional models can provide interpolation from information before and after a target time.RNN-based approaches for CTDGs update embeddings without further gradients after training, although retraining can periodically incorporate accumulated test-time data.Source: §4.6 · p. 29, §4.6.4 · p. 33

4.6.1 RNN-based Encoders for DTDGs

• For DTDGs, RNN encoders combine per-snapshot graph representations with recurrent hidden states, separating structural information from node-history dynamics.GCN-RNN variants learn the graph encoder and recurrent model jointly across snapshots; embedded architectures instead incorporate graph convolutions into recurrent gates or generate GCN weights recurrently.Source: §4.6.1 · pp. 30–31

4.6.2 Other Sequence-Model Encoders for DTDGs

• DySAT applies spatial attention, positional embeddings, and multi-head temporal self-attention to produce final node representations for DTDGs (Sankar et al. (2018)).Kalman filters provide another alternative by tracking node embeddings through successive adjacency-matrix observations (Sarkar et al. (2007)); continuous-time encoding via Time2Vec is proposed as a future extension.Source: §4.6.2 · p. 31

4.6.3 RNN-based Encoders for CTDGs

• For CTDGs, custom RNNs update source, target, and sometimes relation embeddings whenever a new edge arrives, incorporating elapsed time, edge features, or temporal neighborhoods.JODIE maintains static and recurrent node components, Know-Evolve separately projects time differences, and Trivedi et al. (2019) use temporally attended neighbor aggregation.Source: §4.6.3 · pp. 32–33

4.6.4 Discussion and Analysis of RNN-based Encoders

• RNN-based CTDG models can propagate a new edge’s information to neighboring nodes, incorporate static or changing attributes, and naturally support streaming updates with frozen recurrent weights.Models for changing attributes exist for DTDGs, but attributed CTDGs with time-varying attributes remain an open research direction.Source: §4.6.4 · p. 33

4.7 Autoencoder-based Encoders

• Goyal et al. (2017) encode each node’s adjacency row and reconstruct it, constraining adjacent nodes’ representations to remain close.For later snapshots, the autoencoder is initialized from the previous snapshot and trained on the current adjacency matrix.Source: §4.7 · pp. 33–34
• Goyal et al. (2017) adapt autoencoder size heuristically to graph changes, using Net2WiderNet and Net2DeeperNet to preserve previous-function initialization.This initialization accelerates learning and implicitly regularizes embeddings toward temporal smoothness, although it may not capture node evolution.Source: §4.7 · p. 34
• Bonner et al. (2018) reconstruct future neighborhoods from current snapshots with a two-layer GCN autoencoder and propose a variational version sampling embeddings from learned Gaussian distributions.The variational model learns the mean and variance through separate two-layer GCNs with tied first-layer parameters.Source: §4.7 · p. 34
• Goyal et al. (2018) use one autoencoder that maps multiple preceding adjacency rows to the current embedding and reconstructs the current adjacency row.The encoder can use concatenated inputs with a feed-forward network or sequential inputs with an LSTM, with analogous reconstructor architectures.Source: §4.7 · p. 34
• Rahman and Al Hasan (2016) represent node pairs using graphlet-transition features from prior snapshots to predict edge additions or deletions in the next snapshot.The concatenated node-pair features are fed to an autoencoder that learns a vector representation for each pair.Source: §4.7 · p. 34

4.8 Diachronic Encoders

• Diachronic encoders map each node–timestamp or relation–timestamp pair to a hidden representation, unlike static encoders that represent nodes or relations alone.Source: §4.8 · p. 34
• They support interpolation by producing node and relation embeddings at arbitrary points in time.Source: §4.8 · p. 35
• Goel et al. (2020) use explicit time-dependent node parameters, with only a specified fraction of features varying over time; the approach can also encode relations and potentially extrapolate.The embedding at a desired time is obtained by substituting that timestamp into the time-dependent function.Source: §4.8 · p. 35
• Xu et al. (2019a) model embeddings as Gaussian distributions combining a shallow embedding with linear trends and sinusoidal seasonality.Source: §4.8 · p. 35
• Dasgupta et al. (2018) project entity and relation embeddings using learnable timestamp vectors, whereas García-Durán et al. (2018) feed timestamp-character embeddings into an LSTM that handles missing date values.The timestamp-vector approach works only for predefined timestamps, while the character-level method can also be applied to node embeddings and incorporates time modifiers when present.Source: §4.8 · p. 35

Staleness problem

• Embeddings become stale when a node’s last update at time t_v is followed by prediction at a later time t without a new observation involving that node.Staleness depends on the elapsed interval t − t_v.Source: §4.9 · p. 36

Staleness handling

• Kumar et al. (2018b) construct a learnable time-difference representation z_Δt from Δt = t − t_v and combine it with the node’s stored embedding to obtain its representation at time t.The time-difference vector uses learnable parameters, following the approach of Beutel et al. (2018).Source: §4.9 · p. 36

Extrapolation

• For prediction at time t, Kumar et al. (2018b) use the updated z_v^t instead of the potentially stale z_v; diachronic encoders can likewise update embeddings without new observations when used for extrapolation.This benefit is noted for several diachronic encoders, including those associated with Equations (47), (48), and (50).Source: §4.9 · p. 36

5. Decoders for Dynamic Graphs

• Dynamic-graph decoders comprise time-predicting and time-conditioned categories, covering predictions of when events occur and what happens at specified timestamps.Both categories support extrapolation and interpolation, although they differ in whether time is predicted or provided as input.Source: §5 · p. 36, §5.1 · p. 36, §5.2 · p. 38

5.1 Time-Predicting Decoders

• Time-predicting decoders use path-based density models, temporal point processes, or Hawkes processes to estimate event times from graph structure and interaction history.Sun et al. (2012) count paths matching predefined templates and fit distribution-based densities, while later methods use relation embeddings and conditional intensities.Source: §5.1 · pp. 36–38
• Time-predicting approaches face concurrency and scalability limitations: the Trivedi et al. (2017) intensity can become zero for concurrent events, while timestamp enumeration may not scale.For KG interpolation, some methods score every observed timestamp and select the highest-scoring one.Source: §5.1 · pp. 37–38
• Trivedi et al. (2019) model communications and associations with separate temporal point processes because different relation types evolve at different rates.Training still requires approximating survival-function integrals computed across node and relation combinations, using sampling.Source: §5.1 · pp. 37–38

5.2 Time-Conditioned Decoders

• Time-conditioned decoders extend static models such as TransE, DistMult, ComplEx, and RESCAL by incorporating timestamp-dependent embeddings or scores.These models support interpolation and can be used for extrapolation when their temporal representations generalize beyond observed timestamps.Source: §5.2 · pp. 38–39
• Shallow timestamp embeddings cannot represent unseen dates, limiting extrapolation and risking overfitting; character-level timestamp embeddings and time-specific node features address these issues.Goel et al. (2020) show that sine activation with SimplE is fully expressive for temporal knowledge-graph link prediction.Source: §5.2 · pp. 38–39

6.1 Statistical Relational Models

• Statistical relational models combine logic and probability through weighted soft rules, representing node features as random variables rather than encoder-decoder vectors and matrices.They support probabilistic inference over relational facts, with rules and weights that may be learned from data.Source: §6.1 · p. 40
• Statistical relational models capture uncertainty and enable interpretable joint inference, but often require tractable structures or approximate inference because of computational cost.Encoder-decoder inference usually scales better and can adjust operator weights through end-to-end learning, despite potentially questionable operators.Source: §6.1 · p. 40

6.2 Spatiotemporal Graphs

• Spatiotemporal graph methods connect nodes across snapshots and learn on a static graph spanning space and time, supporting applications such as video recognition and traffic prediction.Pandhre et al. (2018) report superior results to several baselines on temporal prediction tasks using two random-walk encoders for spatiotemporal graphs.Source: §6.2 · p. 41

6.3 Constructing (Dynamic) KGs from Text

• Dynamic knowledge graph construction from text uses information extractors to add probabilistic facts that exceed confidence thresholds, sometimes incorporating graph-based predictive priors.Relevant work includes Carlson et al. (2010), Dong et al. (2014), Zhang (2015), and Das et al. (2018b), while detailed discussion is outside the paper’s scope.Source: §6.3 · p. 42

7.1 Applications

• Dynamic-graph applications include link, entity/relation, time, node, and graph prediction, recommender systems, network clustering, and temporal question answering, with task-specific evaluation metrics.Applications cover predicting links in the past or future, missing entities or relations, event timestamps, evolving node labels, whole-graph classes, communities, recommendations, and temporal KG questions.Source: §7.1 · pp. 42–44

7.2 Datasets

• The survey presents a representative sample of widely used graph datasets and directs readers to major network-data repositories for additional collections.Table 2 summarizes dataset types, node and edge counts, and temporal granularity.Source: §7.2 · p. 45, Table 2 · p. 46

7.2.1 Temporal Datasets

• The cited datasets exhibit temporal evolution under different conditions: HEP-TH grows from 1,424 to 7,980 nodes and 2,556 to 21,036 edges, while Autonomous Systems has 733 daily snapshots.GDELT has 15-minute temporal granularity, 14,018 nodes, 20 relation types, and 31.29M edges; MovieLens-10M contains 10 million ratings and 100,000 tag applications.Source: §7.2.1 · pp. 47–48

7.2.2 Static Datasets

• Static benchmarks include Blog, CiteSeer, and Flickr datasets for node or graph learning, with reported differences in graph size, edge count, and class structure.CiteSeer is a citation network with 3,312 nodes, 4,732 edges, and 6 node classes; Blog has 5,196 nodes, 171,743 edges, and 6 classes.Source: §7.2.2 · p. 49

7.3 Open-Source Software

• Open-source libraries and repositories provide implementations for surveyed graph and tensor-based models across PyTorch and MXNet, with Table 3 linking available software.Fey and Lenssen (2019) cover many GCN-based papers, while Wang and Ga (2018) provide graph and transformer implementations.Source: Table 3 · p. 49, §7.3 · p. 50

Continuous-time dynamic graphs

• CTDG models should support edge deletion, node and node-structure changes, since existing methods mainly handle edge additions and often ignore graph evolution for new-node embeddings.Ma et al. (2018a) handles node addition but embeds new nodes from only the current graph state.Source: §8 · p. 50

Model generality

• Dynamic-graph encoders should extend across graph types, including attributed graphs and knowledge graphs, because current random-walk and autoencoder methods are specialized.The survey notes that adapting random-walk approaches to attributed graphs and autoencoder-based encoders to KGs is not straightforward.Source: §8 · p. 50

Continuous-time dynamic graphs

• Streaming CTDG research should move beyond RNNs toward self-attention and neural ordinary differential equations, while handling simultaneous observations without arbitrary ordering.Transformer architectures replace recurrence with self-attention (Vaswani et al. (2017)); neural ODEs are another proposed direction (Chen et al. (2018b)).Source: §8 · p. 51

Tasks and expressiveness

• Dynamic-graph classification remains underdeveloped, with current methods often staticizing graphs; future work should design native classifiers and study model expressiveness and its empirical impact.Existing expressiveness studies include Trouillon et al. (2017), Kazemi and Poole (2018c), Xu et al. (2019b), and Morris et al. (2019).Source: §8 · p. 51

Higher-order relations

• Dynamic knowledge-graph models should represent hyperedges directly, because converting multi-node relations into binary edges can leave no test-time embeddings for reified entities.A purchase relation may connect buyer, seller, and item; relevant static-graph work includes Wen et al. (2016), Feng et al. (2018b), Yadati et al. (2018), Bai et al. (2019), and Fatemi et al. (2019b).Source: §8 · p. 51

Evaluation

• Standard train, validation, and test splits are needed because different datasets currently make comparisons among dynamic-graph models difficult.The survey frames standardized benchmarks as a way for future models to use the same datasets and splits.Source: §8 · pp. 51–52