PaperlayerView exact revision on arXiv

Source-linked AI summary

Forecasting Transportation Network Speed Using Deep Capsule Networks with Nested LSTM Models

Xiaolei Ma, Yi Li, Zhiyong Cui, Yinhai Wang

arXiv:1811.04745v1cs.LGstat.ML

TL;DR

Network-level traffic forecasting must address spatial dependencies and temporal patterns in complicated traffic networks. The paper presents a CapsNet–NLSTM approach and reports that it outperforms other baseline models, while identifying dynamic routing as an area for future improvement.

  • Problem

    CNN pooling can discard information, and CNN-based spatial extraction has difficulty capturing interdependencies in networks with viaducts, intersections, and side roads.

    Source: §1 · p. 2

  • Method

    The paper proposes CapsNet+NLSTM for network-level traffic prediction, using CapsNet rather than conventional CNNs to extract road-segment speed features.

    Source: §1 · p. 3, §5 · p. 14

  • Results

    The CapsNet+NLSTM model outperforms other baseline models and achieves the best prediction performance for complicated spatiotemporal traffic patterns.

    Source: §5 · p. 14

  • Takeaways & Limitations

    The study presents CapsNet+NLSTM as an approach for network-level prediction that addresses drawbacks of conventional CNNs.

    Source: §5 · p. 14

    The authors identify improving prediction accuracy and model efficiency as future needs because dynamic routing is the CapsNet’s core component.

    Source: §5 · p. 15

Abstract

from arXiv · show

Accurate and reliable traffic forecasting for complicated transportation networks is of vital importance to modern transportation management. The complicated spatial dependencies of roadway links and the dynamic temporal patterns of traffic states make it particularly challenging. To address these challenges, we propose a new capsule network (CapsNet) to extract the spatial features of traffic networks and utilize a nested LSTM (NLSTM) structure to capture the hierarchical temporal dependencies in traffic sequence data. A framework for network-level traffic forecasting is also proposed by sequentially connecting CapsNet and NLSTM. On the basis of literature review, our study is the first to adopt CapsNet and NLSTM in the field of traffic forecasting. An experiment on a Beijing transportation network with 278 links shows that the proposed framework with the capability of capturing complicated spatiotemporal traffic patterns outperforms multiple state-of-the-art traffic forecasting baseline models. The superiority and feasibility of CapsNet and NLSTM are also demonstrated, respectively, by visualizing and quantitatively evaluating the experimental results.

1. Introduction

The introduction identifies challenges in capturing spatial dependencies across complicated roadway networks and hierarchical temporal patterns in traffic data. It proposes sequentially combining CapsNet and NLSTM, and reports that the framework outperforms multiple state-of-the-art traffic forecasting baselines.

  • CNN pooling discards information, and conventional CNNs miss interlink dependencies in complicated structures such as viaducts, intersections, and side roads.Source: §1 · p. 2
  • Traffic depends on short-term history and periodic long-term patterns, but severe-weather or disaster histories may add little short-term information and require selective capture.Source: §1 · p. 3
  • CapsNet uses vector capsules and dynamic routing to account for slightly active features, while NLSTM selectively accesses inner memories to capture hierarchical temporal dependencies.Capsule direction and length encode high-level feature state and feature detection probability, respectively.Source: §1 · p. 3
  • The proposed CapsNet–NLSTM framework outperforms multiple state-of-the-art traffic forecasting baseline models.Source: §1 · p. 3
  • The paper proposes a network-level forecasting framework combining CapsNet and NLSTM and reports that this is the first traffic-forecasting use of both methods.Source: §1 · p. 3
  • The authors demonstrate CapsNet and NLSTM’s superiority and feasibility through visualization and quantitative evaluation of experimental results.Source: §1 · p. 3

2. Literature review

The literature review covers statistical and artificial-intelligence approaches to traffic prediction, including deep learning models and CNN-based spatial feature extraction. It identifies remaining difficulties with conventional CNNs on complex roadway structures and motivates CapsNet alongside nested LSTM.

  • Traffic prediction approaches are grouped into statistical methods and artificial intelligence, with statistical models applied to traffic-flow parameters.Source: §2 · pp. 3–4
  • Deep learning models including DBNs, autoencoders, CNNs, and RNNs have been adopted for traffic forecasting, with CNNs and hybrid CNN approaches used to extract spatial dependencies.Source: §2 · p. 4
  • Conventional CNN approaches still struggle with overlapping roadways, spatially interlaced viaduct links, and physical traffic-network features.Source: §2 · p. 4
  • The review presents CapsNet for network-level spatial feature extraction and nested LSTM for time-series learning, connected sequentially for traffic prediction.Source: §2 · p. 4

3. Methodology

The methodology describes a network-level traffic forecasting framework that represents road-link speeds as grid-image sequences and uses CapsNet with dynamic routing to extract spatial features. It also explains how NLSTM captures long-term temporal dependencies and how the two components are connected.

3.1. Network representation

  • Traffic states are encoded as 2-minute grid-image sequences, with empty grids set to zero, single-link grids assigned that link’s speed, and overlapping-link grids assigned their average speed.Each grid is a pixel in a one-channel image, and the images preserve the road network’s spatial structure and relative link topology.Source: §3.1 · p. 5

3.2. Spatial features captured by CapsNet

  • CNNs can lose distant-link information through successive convolutions or pooling and struggle with nonconnected links and low-resolution overlaps, motivating CapsNet for network-level spatial features.CapsNet replaces pooling with dynamic routing between capsules to retain local features and spatial relationships among links.Source: §3.2 · p. 6
  • Capsule-vector length encodes feature-detection probability and direction encodes feature state, while dynamic routing associates correlated local capsules with advanced capsules to capture distant-link relationships.Routing weights combine primary-capsule inputs for each advanced capsule and are refined iteratively.Source: §3.2 · pp. 6–7
  • The model uses two convolutional layers and a fully connected TrafficCaps layer: PrimaryCaps encodes local features in 8-dimensional vectors, and TrafficCaps outputs advanced 16-dimensional capsules.The first convolution extracts relations between adjacent links, while TrafficCaps captures relationships among features in the primary capsules.Source: §3.2 · p. 6

3.3. Long short-term temporal features captured by NLSTM

  • NLSTM adds depth by nesting LSTMs: each memory cell is computed by an internal LSTM unit with its own inner memory cells.This differs from stacking LSTMs, where information from a lower layer is passed to a higher layer for further filtering.Source: §3.3 · p. 8
  • The internal memories are selectively read and written through standard LSTM gates, allowing traffic events to be processed over long timescales even when irrelevant to the immediate present.The external LSTM’s cell-state update rule is changed to use the internal unit’s output.Source: §3.3 · pp. 8–9
  • NLSTM iteratively calculates traffic temporal features for prediction and is described as stable and efficient in capturing long-term dependencies.The design addresses traffic states’ long-term temporal patterns, which can include congestion lasting several hours.Source: §3.3 · pp. 8–9

3.4 Framework

  • CapsNet outputs are spread into one vector and passed to NLSTM; a fully connected layer predicts speeds for all links in an end-to-end model trained for multi-step forecasting from historical data.The model architecture is shown in Fig. 6.Source: §3.4 · p. 9, Figure 6 · p. 10

4.1. Data description

The study uses GPS traffic data from Beijing’s Second and Third Ring Roads to represent roadway traffic as images and forecast speeds at several horizons. The evaluation covers 278 links and separates data collected in June 2015 for training from data collected in August 2015 for testing.

  • GPS upload intervals varied from about 10 seconds to 1 minute, so observations were aggregated to 2 minutes to capture network traffic variation and avoid invalid zero-speed data from narrow intervals.Source: §4.1 · p. 10
  • Training data were collected June 1–31, 2015, and test data August 1–14, 2015.Source: §4.1 · p. 10
  • The evaluated Beijing network covers 278 links, including arterial roads, interchanges, and intersections, between the Second and Third Ring Roads.Source: §4.1 · p. 10
  • Traffic states are represented as 164 × 148 images at 2-minute intervals, with 15 historical time steps—30 minutes—used as model input.Source: §4.1 · p. 10
  • The 30-minute history is used to predict traffic speeds 2, 10, and 20 minutes ahead, corresponding to 1, 5, and 10 time lags.Source: §4.1 · p. 10

4.2. Implementation

The implementation combines CapsNet and NLSTM, with model parameters selected using fivefold cross-validation. The study compares the proposed model with five deep-learning baselines and evaluates prediction error using MSE and MAPE.

4.2.1. Hardware

  • The proposed CapsNet+NLSTM model was implemented in Python Keras and run on a server with eight NVIDIA GeForce Titan X GPUs, each with 12 GB RAM.Source: §4.2.1 · p. 10

4.2.2. Model parameters

  • Fivefold cross-validation selected parameters by training on four subsets and validating on the fifth, choosing the model with the lowest average validation prediction error.Source: §4.2.2 · p. 11

4.2.3. Baseline models

  • The five baselines were LSTM, NLSTM, DCNN, CapsNet, and CNN+LSTM; the CNN+LSTM baseline had approximately half the parameters of CapsNet+NLSTM.Source: §4.2.3 · p. 11

4.2.4 Evaluation metrics

  • Prediction performance was evaluated using mean squared error (MSE) and mean absolute percentage error (MAPE).Source: §4.2.4 · p. 12

4.3. Experimental results

CapsNet+NLSTM achieved the most accurate results across the evaluated prediction steps, while comparisons also found advantages for CapsNet in spatial feature extraction and NLSTM in long-sequence stability and efficiency. Visualized predictions showed fewer inaccurate links for CapsNet+NLSTM than for CNN+LSTM.

4.3.1. Comparison

4.3.2. Evaluation of CapsNet

  • The CapsNet+NLSTM visualization shows only 17 inaccurate links, compared with 83 for CNN+LSTM; links were considered inaccurate when mean absolute speed error exceeded 2 km/h.Source: §4.3.2 · p. 13
  • CNN+LSTM errors cluster on viaducts and in low-resolution areas with tightly arranged links, whereas the CapsNet-based model predicts accurately under these conditions.Source: §4.3.2 · p. 13

4.3.3. Evaluation of NLSTM

  • NLSTM MSE fluctuates less than LSTM MSE with long historical inputs for multistep speed prediction, especially at the 2-minute short-term horizon.Source: §4.3.3 · p. 14, Table 4 · p. 14
  • Stacked and nested LSTM structures have the same parameter scale of 8.8 M, and NLSTM takes little time to perform prediction.Source: §4.3.3 · p. 14
  • CapsNet outperforms CNNs in spatial feature capture, while NLSTM performs more stably and efficiently than stacked LSTMs for long time series.Source: §4.3.3 · p. 14

5. Conclusions

The conclusion presents CapsNet+NLSTM for network-level traffic forecasting, pairing spatial feature extraction with hierarchical temporal modeling. It reports the model’s best prediction performance among the baselines and promising predictions on complicated road layouts.

5. Conclusions

  • CapsNet represents roadway-network features in vectors that include segment position, direction, length, and travel speed.The approach uses the roadway network as an image to capture spatial structure and link topology.Source: §5 · p. 14
  • The framework sequentially combines CapsNet with NLSTM to capture spatial features and hierarchical temporal dependencies in traffic states.The conclusion also reports stable time-series prediction performance for NLSTM compared with traditional approaches.Source: §5 · p. 14
  • The CapsNet+NLSTM model achieves the best prediction performance compared with the baseline models.The conclusion attributes this result to its capability to capture complicated spatiotemporal traffic patterns.Source: §5 · p. 14
  • Visualized predictions indicate promise on complicated road networks with interlaced and compact links, including viaducts and side roads.Source: §5 · p. 14
  • The authors identify dynamic routing accuracy and model efficiency, as well as interpretation of learned spatiotemporal features, as areas for future work.Dynamic routing is described as a core CapsNet component.Source: §5 · p. 15
Loading 1811.04745v1…