PaperlayerView exact revision on arXiv

Source-linked AI summary

How to Build a Graph-Based Deep Learning Architecture in Traffic Domain: A Survey

Jiexia Ye, Juanjuan Zhao, Kejiang Ye, Chengzhong Xu

arXiv:2005.11691v6eess.SPcs.LG

TL;DR

Traffic data contain complex dependencies, and many traffic networks are graph-structured, limiting grid-based spatial modeling. This survey synthesizes graph-based deep learning architectures, their formulations, techniques, challenges, datasets, codes, and future directions. It aims to provide comprehensive guidance for understanding and applying these methods in traffic tasks.

  • Problem

    Many traffic networks are graph-structured, while existing traffic reviews lack a systematic and elaborated survey of graph-based deep learning techniques.

  • Method

    The survey formulates traffic problems as graphs, guides graph construction from traffic datasets, decomposes architectures into shared techniques, and organizes solutions by common challenges.

  • Results

    The survey provides a comprehensive review of graph-based deep learning architectures across traffic applications, together with benchmark datasets, open-source codes, and future research directions.

  • Takeaways & Limitations

    The survey offers practical guidance intended to help researchers select models and apply emerging graph-based approaches in real-world traffic tasks.

Abstract

from arXiv · show

In recent years, various deep learning architectures have been proposed to solve complex challenges (e.g. spatial dependency, temporal dependency) in traffic domain, which have achieved satisfactory performance. These architectures are composed of multiple deep learning techniques in order to tackle various challenges in traffic tasks. Traditionally, convolution neural networks (CNNs) are utilized to model spatial dependency by decomposing the traffic network as grids. However, many traffic networks are graph-structured in nature. In order to utilize such spatial information fully, it's more appropriate to formulate traffic networks as graphs mathematically. Recently, various novel deep learning techniques have been developed to process graph data, called graph neural networks (GNNs). More and more works combine GNNs with other deep learning techniques to construct an architecture dealing with various challenges in a complex traffic task, where GNNs are responsible for extracting spatial correlations in traffic network. These graph-based architectures have achieved state-of-the-art performance. To provide a comprehensive and clear picture of such emerging trend, this survey carefully examines various graph-based deep learning architectures in many traffic applications. We first give guidelines to formulate a traffic problem based on graph and construct graphs from various kinds of traffic datasets. Then we decompose these graph-based architectures to discuss their shared deep learning techniques, clarifying the utilization of each technique in traffic tasks. What's more, we summarize some common traffic challenges and the corresponding graph-based deep learning solutions to each challenge. Finally, we provide benchmark datasets, open source codes and future research directions in this rapidly growing field.

I. INTRODUCTION

Traffic systems generate heterogeneous, structurally complex data, while many traffic networks are graph-structured rather than grid-based. This survey addresses the resulting need for a systematic account of graph-based deep learning methods, formulations, challenges, datasets, and research directions.

  • Motivation: Traffic infrastructures produce large heterogeneous datasets containing numeric, image/video, and textual data with complex spatiotemporal patterns.The paper identifies an urgent need for more intelligent approaches to process these data.
  • Motivation: CNN-based architectures model spatial correlations on grids, but their learned features are not optimal for graph-structured road and subway networks.GNNs are introduced to exploit graph structure information and capture spatial dependency in traffic architectures.
  • Contributions: The survey systematically outlines traffic problems, research directions, shared challenges, and deep learning techniques to help researchers locate or expand their work.It also compares multiple solutions for common challenges to support model selection in traffic tasks.
  • Contributions: It provides general formulations for traffic problems and specific guidance for constructing graphs from several typical raw traffic datasets.The paper presents this guidance as practical for accelerating graph-based applications in traffic.
  • Contributions: The survey reviews GNNs, RNNs, temporal convolution networks, Seq2Seq models, and GANs, including their theoretical aspects, advantages, limitations, and variants.The review focuses on how these techniques are used in graph-based traffic tasks.
  • Contributions: It collects benchmark datasets and open-source codes, proposes future research directions, and synthesizes prior traffic and GNN surveys.The paper identifies a remaining lack of a systematic, elaborated survey focused on graph-based deep learning in traffic.

A. Traffic Problems

The survey organizes traffic applications around goals including congestion relief, travel-demand service, transportation safety, surveillance, and autonomous driving. Existing graph-based work concentrates on forecasting tasks, while traffic incident and vehicle detection remain unexplored from a graph perspective.

  • Transportation objectives include relieving congestion, satisfying travel demand, improving traffic management, ensuring safety, and enabling automatic driving.
  • Graph-based traffic studies focus mainly on traffic-state, travel-demand, and trajectory prediction; incident detection and vehicle detection have not yet been explored from a graph view.
  • Traffic-state prediction covers flow, speed, travel time, and density, while travel-demand prediction estimates future demand for traffic services.
  • Traffic signal control reduces vehicle staying time and can optimize flow while reducing congestion and vehicle emissions.
  • Transportation-safety research includes incident detection, accident prediction, injury-severity prediction, and accident-prevention prediction.

C. Challenges and Techniques Overview

Traffic tasks share spatial, temporal, and external-factor challenges, motivating architectures that combine graph-based spatial modeling with temporal and other deep learning techniques. The survey formalizes these problems using graph nodes, features, edges, and adjacency matrices.

  • Challenges: Most traffic problems share challenges involving spatial dependency, temporal dependency, and external factors.
  • Challenges: Spatiotemporal prediction depends on prior conditions, surrounding-region conditions, surrounding-vehicle behavior, and an agent’s historical trajectory.
  • Techniques: Graph neural networks model spatial dependency, while recurrent neural networks and temporal convolution networks generally model temporal dependency.
  • Survey scope: The survey provides guidance on graph formulation, graph construction, and the relationships between traffic challenges and deep learning techniques.
  • Problem formulation: A traffic network is represented as G = (V, E, A), with nodes representing traffic objects and edges representing connectivity.
  • Problem formulation: The adjacency matrix A encodes topology and may represent binary connectivity or weighted relationships between nodes.
  • Problem formulation: Given P historical time slices, the forecasting problem predicts traffic indicators over the next Q time slices.

C. Graph Construction from Traffic Datasets

Graph construction maps traffic infrastructures and datasets to nodes, features, and relationships, while adjacency matrices encode alternative spatial assumptions. The survey distinguishes fixed and adaptive matrices for representing traffic dependencies.

  • Graph construction: Traffic graphs consist of nodes, node features, edges, and edge weights, with adjacency matrices representing edges and weights.
  • Dataset types: Traffic datasets include road sensors, GPS trajectories, ride-hailing orders, and public-transit transaction records.
  • Nodes and node features: Sensor graphs use sensors as nodes, whereas road-segment graphs use segments as nodes with aggregated measurements from corresponding sensors or GPS points.
  • Nodes and node features: Road-intersection graphs use intersections as nodes and aggregate traffic measurements passing through each intersection.
  • Nodes and node features: Ride-hailing datasets represent equal-size city-grid regions as nodes, with each node feature recording orders during a time interval.
  • Adjacency matrices: Adjacency matrices capture heterogeneous pairwise relationships and may be fixed, dynamic, or evolving according to traffic scenarios.
  • Adjacency matrices: Fixed matrices encode predefined connectivity, distance, functional similarity, or transportation connectivity relationships.
  • Adjacency matrices: Adaptive matrices are learned from data and can more precisely capture hidden spatial dependency in some traffic tasks than predefined matrices.

V. DEEP LEARNING TECHNIQUES PERSPECTIVE

Graph-based traffic architectures typically combine GNNs with complementary modules, while graph convolutions extract spatial representations from traffic networks. Spectral graph convolution progresses from costly global spectral filtering toward localized, efficient layers that aggregate neighborhood information.

  • V. DEEP LEARNING TECHNIQUES PERSPECTIVE: Most graph-based traffic architectures combine GNNs with RNNs, TCNs, Seq2Seq models, or GANs to address traffic-task challenges.The survey attributes state-of-the-art performance in many traffic scenarios to cooperation between GNNs and other deep learning techniques.
  • A. GNNs: Traffic studies mainly use convolutional GNNs, divided into spectral-based and spatial-based approaches, with most traffic works focusing on convolutional graph neural networks.Other branches include graph auto-encoders and graph attention networks, but the survey reports relatively few traffic applications of them.
  • 1) Spectral Graph Convolution: Spectral graph convolution transforms graph signals using the Laplacian eigendecomposition, applies a diagonal spectral kernel, and returns the filtered signal to the spatial domain.This formulation is theoretically guaranteed but requires multiplication with the eigenvector matrix and eigendecomposition of the graph Laplacian.
  • 1) Spectral Graph Convolution: Polynomial spectral filters localize convolution: K determines the maximum neighborhood radius, while Chebyshev approximation reduces complexity to O(K|E|).The localized operation recursively computes Chebyshev polynomials, where |E| is the number of graph edges.
  • 1) Spectral Graph Convolution: The simplified spectral graph convolution sets K = 2, restricts the filter to first-order terms, and uses a renormalized adjacency-based propagation.The derivation introduces self-loops through ˜A = A + I_N and uses the corresponding degree matrix ˜D.
  • 1) Spectral Graph Convolution: A graph convolution layer aggregates 1-hop neighbor information, applies a shared trainable transformation, and can expand its receptive field by stacking layers.The input has FI features per node and the output has FO features per node, followed by an activation function.

2) Diffusion Graph Convolution:

Diffusion graph convolution models spatial dependency through transition-matrix diffusion and supports directed or undirected traffic graphs. Traffic variants modify graph structure, attention, temporal coverage, or feature domains to capture richer spatial-temporal correlations.

  • 2) Diffusion Graph Convolution: Unlike spectral graph convolution, diffusion graph convolution has no graph-direction constraint and can operate on directed or undirected graphs.Spectral convolution requires a symmetric Laplacian, whereas diffusion convolution uses graph diffusion represented by powers of a transition matrix.
  • 2) Diffusion Graph Convolution: Diffusion convolution approximates multi-step movement probabilities with a finite K-step process and trainable weights for each diffusion step.The transition process includes a restart probability, and bidirectional diffusion captures spatial correlations on directed graphs.
  • 2) Diffusion Graph Convolution: Graph convolution is applied directly to naturally graph-structured road and subway networks to exploit spatial information more fully than grid-based modeling.The surveyed literature primarily uses spectral graph convolution, with diffusion graph convolution and other graph techniques appearing less often.
  • 2) Diffusion Graph Convolution: Spectral and diffusion convolutions encode different spatial assumptions: adjacency emphasizes nearby nodes, while transition matrices represent stochastic and potentially dynamic dependency.The distinction concerns whether spatial correlation is treated as fixed by adjacency or governed by diffusion and restart behavior.
  • 2) Diffusion Graph Convolution: Traffic graph convolution generalizes from 2-D node-feature signals to 3-D spatial-temporal signals by applying the same convolution kernel across time steps.This parallel operation preserves the shared spatial convolution while processing temporal observations.
  • 2) Diffusion Graph Convolution: Variants incorporate spatial attention, temporal neighborhoods, physical road properties, geospatial positions, or edge-based representations.Examples include adaptive spatial attention, K-order graph and Kt-order temporal neighborhoods, weighted adjacency matrices, and convolutions defined on edges.

B. RNNs

RNNs process traffic time series by recurrently carrying hidden information across timestamps. Their recurrent hidden state enables historical information retention and produces an output sequence aligned with the input sequence.

  • B. RNNs: RNNs are commonly used in traffic tasks because traffic data are time series with temporal dependencies.The survey introduces RNN, LSTM, and GRU as classical recurrent models for selecting temporal architectures.
  • B. RNNs: An RNN hidden layer receives both the current input X_t and previous hidden state H_{t−1}, allowing it to memorize past information.The hidden state is recurrent because information is transmitted back into the hidden layer itself.
  • B. RNNs: The recurrent hidden state is computed with a nonlinear transformation of [H_{t−1}, X_t], while outputs apply an activation to the hidden representation.The parameters W_h, W_y, b_h, and b_y are trainable, and t ranges from 1 to the input sequence length P.
  • B. RNNs: RNNs map an input sequence [X_1, · · · , X_P] to an output sequence [Y_1, · · · , Y_P] of the same length.The hidden layer transforms each time-step input, and the output layer maps the resulting hidden state to the output.

2) LSTM:

LSTM and GRU address recurrent modeling difficulties in traffic sequences, while architectural augmentations add attention, external features, residuals, dilation, or graph convolutions. These designs target longer-term or joint spatial-temporal dependencies with different complexity trade-offs.

  • 2) LSTM: Vanishing or exploding gradients hinder standard RNNs from learning long-term sequential dependencies.Repeated matrix multiplication across long sequences causes gradients to decrease or increase dramatically over time.
  • 2) LSTM: LSTM adds a memory cell and input, forget, and output gates to control information flow and preserve features from previous time steps.The gates range in [0,1], and the memory cell combines retained and newly processed information.
  • 2) LSTM: GRU simplifies LSTM with two gates, reducing parameters and training time while remaining equally effective as LSTM according to the survey passage.The cited passage identifies reset and update gates for the GRU hidden layer.
  • 2) LSTM: RNNs remain widely used for traffic temporal correlation, but the survey reports that more than half of reviewed works adopt GRU and some use LSTM.The passage attributes this pattern to the ability of LSTM and GRU to handle severe gradient disappearance or explosion more successfully.
  • 2) LSTM: Traffic recurrent models augment temporal dynamics with attention, gating, residual shortcuts, external attributes, and dilated skip connections.These mechanisms model timestamp correlations, sudden historical changes, and multiple temporal scales.
  • 2) LSTM: Replacing recurrent matrix multiplication with SGC, DGC, or GAT enables GRU and LSTM variants to capture spatial-temporal correlations jointly.Most surveyed replacements occur in GRU, with fewer examples in LSTM; the cited works use DGC, SGC, or GAT.

C. TCN

TCN models temporal dependencies with non-recursive dilated causal convolutions, offering efficient long-sequence processing for traffic data. Its graph-based extension applies shared kernels across nodes and can combine stacked layers, gating, and residual blocks.

  • Sequence Modeling: TCN performs causal sequence modeling, producing each output from current and historical inputs without using future observations.The input and output sequences can have the same length, while causality restricts yt to x1 through xt.
  • 1-D TCN: Dilated causal convolution uses a dilation rate to control skipped inputs, while zero padding preserves the output length.Without padding, the output length is shortened by (K − 1)d.
  • 1-D TCN: Stacking layers with d = 2^l expands the receptive field exponentially without requiring many layers or larger filters.This design supports longer sequences with fewer layers and reduced computation resources.
  • 1-D TCN: Compared with RNNs, TCN captures long sequences non-recursively, enabling parallel computation, faster training, and more stable gradients.The survey also notes that 1D-CNNs have simpler structures and no dependence on previous-step iteration but generally lack long-sequence memory.
  • 3-D TCN: For graph-based traffic signals, 3-D TCN applies the same convolution kernel to every node, producing FO new features per node.The input represents historical observations across P time slices, N nodes, and FI features; the output has FO features.
  • Traffic Applications: Traffic models enhance TCNs with stacked layers, gating mechanisms, and residual blocks to represent multiscale or complex temporal dependencies.Gated TCNs control information flow, while residual blocks add the block input to the final TCN output.

D. Seq2Seq

Seq2Seq converts an input sequence into an output sequence through an encoder and decoder, supporting different input and output lengths. Attention and scheduled sampling address limitations arising from fixed context and training–testing discrepancies.

  • Seq2Seq: Seq2Seq consists of an encoder that maps input sequence X to a fixed latent vector C and a decoder that generates output sequence Y.The input and output sequences may have different lengths.
  • Seq2Seq: The decoder recursively updates its hidden state from C, the previous output, and the previous decoder state before producing each output.The formulation identifies Xi as the encoder input at step i and Yj as the decoder output at step j.
  • Seq2Seq: The encoder and decoder can use RNN, LSTM, GRU, or other sequence-processing models, and they need not be identical.This flexibility makes the component design a central part of graph-based traffic architectures.
  • Attention: A fixed latent vector C can inadequately represent input elements with different relevance to each output step.Attention lets the decoder focus on task-relevant input portions using correlations between inputs and outputs.
  • Scheduled Sampling: Scheduled sampling mitigates decoder error accumulation by addressing the difference between true-label inputs during training and self-generated predictions during testing.The technique is introduced because testing predictions can accumulate errors across decoding steps.

2) Seq2Seq in Traffic Domain:

In traffic applications, Seq2Seq encodes historical observations and decodes future conditions, often using spatial-temporal encoders and decoders for multi-step prediction. GANs can incorporate Seq2Seq components to generate realistic future traffic observations.

  • Seq2Seq in Traffic Domain: Seq2Seq is used for multi-step traffic prediction because it accepts an input sequence and produces an output sequence with different length.The encoder maps historical traffic data into a latent vector, and the decoder generates future traffic conditions.
  • Seq2Seq in Traffic Domain: Attention is incorporated to model the varying influence of historical traffic observations at different time slots on future predictions.This extends the basic encoder–decoder framework by weighting previous observations differently.
  • Seq2Seq in Traffic Domain: Traffic Seq2Seq encoders and decoders commonly capture spatial-temporal dependencies, with DCGRU providing one example that models both jointly.The encoder and decoder are not necessarily the same, and their design is often the novel part of each work.
  • Seq2Seq in Traffic Domain: RNN-based Seq2Seq decoders can accumulate inference errors because each previous prediction becomes input for the next step.Reported responses include scheduled sampling and decoders that use only the last-step prediction through short-term and long-term components.
  • GAN: GANs combine a generator that produces samples with a discriminator that distinguishes generated samples from real training data.The two networks are trained adversarially, with G producing xf = G(z) from random vector z and D comparing fake and real samples.
  • GAN: In traffic prediction, GAN generators typically produce future observations from historical data, while discriminators compare them with future real data.After training, the generator can learn the distribution of real traffic-flow data and predict future traffic states.

A. Spatial Dependency

Traffic networks are naturally graph-structured, making graph-based methods better suited than grid assumptions for representing spatial relationships. Surveyed solutions address locality, multirelationships, global connectivity, temporal patterns, and joint spatial-temporal interactions.

  • Motivation: Grid-based CNN modeling can misrepresent traffic topology by treating opposite-side regions as similarly related even when only same-side roads influence one another.Graph modeling connects genuinely related roads and disconnects weakly related ones.
  • Spatial Locality: Spatial locality methods aggregate information from nearby graph nodes, commonly using K-localized spectral graph convolution over 0 to K − 1 hops.This reflects the relevance of connected stations or adjacent regions.
  • Spatial Locality: Adaptive graph methods learn changing spatial correlations or location-dependent behavior instead of relying only on a fixed, location-independent adjacency and kernel.Attention can adapt adjacency-based correlations, while location-dependent kernels account for regional differences in local statistics.
  • Multirelationship: Multirelationship methods capture correlations beyond geographic proximity, including functional similarity, transportation connectivity, and semantic neighbors.These relationships connect distant regions through non-Euclidean associations.
  • Global Connectivity: Global connectivity methods model network-wide traffic changes with diffusion processes, multigranularity pooling, or self-adaptive adjacency matrices.Diffusion graph convolution uses transition-matrix power series, while pooling extracts features at multiple spatial scopes.
  • Temporal Dependency: TCN-based methods offer simple structures, parallel computation, and stable gradients as alternatives to recurrent approaches for temporal dependency.Other approaches decompose temporal dependency into recent, daily, and weekly patterns or assign adaptive attention to historical time slices.
  • Spatial-Temporal Dependency: Separately modeling spatial and temporal dependencies can ignore their interactions, motivating graph-convolutional recurrent models that capture them jointly.The survey links this need to location- and time-varying impacts on future traffic conditions.

D. External Factors

External factors such as time, weather, events, points of interest, incidents, and historical statistics can influence traffic prediction, but they are difficult to collect and represent. Surveyed works use concatenation or dedicated components to process them, while the survey identifies sparse coverage and simple processing as open challenges.

  • External factors include holidays, temporal attributes, weather, special events, POIs, traffic incidents, and historical statistical speed information.
  • Discrete external factors are commonly one-hot encoded, whereas continuous factors are normalized using Min-Max normalization or Z-score normalization.
  • Surveyed models either concatenate external factors with other features or process them through a separate external component.
  • Dedicated external components commonly use two fully connected layers, while some approaches use multi-LSTM layers to extract external-factor representations.
  • The survey provides public datasets, open-source codes, and future directions to support further graph-based traffic research.
  • External-factor data sparsity and heterogeneous formats limit current approaches, which often use naive processing such as a simple fully connected layer.
Loading 2005.11691v6…