PaperlayerView exact revision on arXiv

Source-linked AI summary

Stacked Bidirectional and Unidirectional LSTM Recurrent Neural Network for Forecasting Network-wide Traffic State with Missing Values

Zhiyong Cui, Ruimin Ke, Ziyuan Pu, Yinhai Wang

arXiv:2005.11627v1cs.LGeess.SPstat.ML

TL;DR

RNN-based traffic forecasting has not fully exploited spatiotemporal dependencies or adequately handled missing data. The paper proposes SBU-LSTM with bidirectional processing and LSTM-I, and reports superior forecasting performance and robust missing-value handling across real-world datasets.

  • Problem

    RNN-based traffic forecasting has limited exploitation of spatiotemporal predictive information and limited capability to handle missing data.

  • Method

    The paper proposes SBU-LSTM, combining stacked bidirectional and unidirectional LSTM layers, and LSTM-I, an imputation unit integrated into LSTM models.

  • Results

    The proposed stacked bidirectional LSTM models achieve superior prediction performance, while models with the imputation unit outperform state-of-the-art RNN models and compete with tensor decomposition models.

  • Takeaways & Limitations

    Bidirectional LSTM architectures and integrated imputation mechanisms support network-wide traffic prediction with missing spatial-temporal inputs across different missing-value patterns.

  • Takeaways & Limitations

    The authors identify future work using different datasets and improving spatial-feature interpretation and integration with other applications.

Abstract

from arXiv · show

Short-term traffic forecasting based on deep learning methods, especially recurrent neural networks (RNN), has received much attention in recent years. However, the potential of RNN-based models in traffic forecasting has not yet been fully exploited in terms of the predictive power of spatial-temporal data and the capability of handling missing data. In this paper, we focus on RNN-based models and attempt to reformulate the way to incorporate RNN and its variants into traffic prediction models. A stacked bidirectional and unidirectional LSTM network architecture (SBU-LSTM) is proposed to assist the design of neural network structures for traffic state forecasting. As a key component of the architecture, the bidirectional LSTM (BDLSM) is exploited to capture the forward and backward temporal dependencies in spatiotemporal data. To deal with missing values in spatial-temporal data, we also propose a data imputation mechanism in the LSTM structure (LSTM-I) by designing an imputation unit to infer missing values and assist traffic prediction. The bidirectional version of LSTM-I is incorporated in the SBU-LSTM architecture. Two real-world network-wide traffic state datasets are used to conduct experiments and published to facilitate further traffic prediction research. The prediction performance of multiple types of multi-layer LSTM or BDLSTM models is evaluated. Experimental results indicate that the proposed SBU-LSTM architecture, especially the two-layer BDLSTM network, can achieve superior performance for the network-wide traffic prediction in both accuracy and robustness. Further, comprehensive comparison results show that the proposed data imputation mechanism in the RNN-based models can achieve outstanding prediction performance when the model's input data contains different patterns of missing values.

1. Introduction

The introduction identifies unresolved challenges in RNN-based network-wide traffic forecasting: exploiting bidirectional temporal information, handling missing data, and evaluating capacity–complexity trade-offs. The paper proposes SBU-LSTM and LSTM-I to address these challenges using real-world traffic datasets.

  • Motivation: Missing traffic data commonly arises from sensor or communication failures, while separate imputation and prediction can inadequately exploit missingness patterns.Real-time imputation may also violate method assumptions and impose substantial computational cost.
  • Motivation: RNN-based traffic models typically process chronological sequences, potentially overlooking useful backward dependencies in periodic and spatially interconnected traffic states.The introduction motivates processing series in the negative direction to capture backward temporal information.
  • Motivation: Prior work lacks systematic evaluation of the trade-off between model capacity and complexity across LSTM depth, other layers, weight size, and spatial dimension.The paper notes that existing studies vary in layer count but do not fully evaluate these influential factors.
  • Proposed approach: The paper proposes SBU-LSTM, a stacked bidirectional and unidirectional LSTM architecture for network-wide traffic state prediction.The architecture is intended to reformulate how RNN variants are incorporated into traffic prediction models.
  • Proposed approach: LSTM-I adds an imputation unit that infers and fills missing spatial-temporal input values while assisting prediction.The proposed work also evaluates stacked LSTM- and BDLSTM-based models using two real-world traffic datasets.

2. Literature Review

The literature review situates traffic forecasting among neural and non-neural approaches, emphasizing imputation methods for missing spatiotemporal data and their differing abilities to model correlations and complex patterns.

  • Traffic forecasting methods: Deep learning, including feed-forward, fuzzy, and recurrent neural networks, has been widely adopted for traffic forecasting.The review notes that LSTM was previously used to forecast traffic speed.
  • Missing-data imputation: Interpolation and spline methods are simple and efficient for imputation but cannot capture variable correlations and complex patterns.The review contrasts these methods with more advanced time-series imputation approaches.
  • Missing-data imputation: Matrix factorization methods can achieve state-of-the-art prediction accuracy among non-deep-learning approaches to missing-data problems.A Bayesian temporal matrix factorization method is cited for spatiotemporal prediction with missing inputs.

3. Methodology

The methodology defines network-wide traffic forecasting with LSTM-based models, then extends LSTM with learned imputation and bidirectional processing to handle missing values and temporal dependencies. A flexible stacked architecture combines LSTM or bidirectional LSTM components for forecasting.

  • Problem formulation: Network-wide traffic states are represented across T time steps and D sensor stations, with the model mapping historical observations to the next time step.The traffic state in the tested datasets specifically refers to traffic speed.
  • Missing-data representation: A masking vector records which sensor values are missing at each time step, producing a corresponding masking data sample for the traffic-state sequence.Each masking vector has D binary elements indicating observed or missing traffic states.
  • Long Short-Term Memory: The base LSTM uses gated memory to learn long-term dependencies, taking the current input, preceding output, and preceding cell state as inputs.Its input, forget, output, and memory-cell transformations use learned weights, biases, sigmoid gates, and tanh activation.
  • LSTM with Imputation Unit: LSTM-I infers each subsequent observation from the preceding cell state and hidden state, replacing missing input elements with inferred values while also supplying the mask to the model.The imputation unit is parameterized by learnable weights and bias, and is intended to support prediction rather than necessarily recover the actual missing value.
  • LSTM with Imputation Unit: The imputation mechanism adds absolute imputation error to the prediction loss as a regularization term, allowing observed values to assess inferred values during training.The paper states that this can enhance imputation performance and has potential to improve overall prediction accuracy.
  • Bidirectional and stacked architectures: BDLSTM-I combines forward and backward LSTM-I components, so missing values can be inferred from both preceding and subsequent temporal directions.SBU-LSTM stacks flexible layers of LSTM or BDLSTM components rather than fixing the number or types of layers.

4.1. Dataset

The study evaluates models on two real-world, network-scale traffic speed datasets: LOOP-SEA from Seattle freeways and PEMS-BAY from California’s Bay Area.

  • Two real-world network-scale traffic state datasets are used for testing models.
  • LOOP-SEA contains speed data from 323 sensor stations deployed on four connected Seattle-area freeways.
  • PEMS-BAY contains speed information from 325 Bay Area sensor stations collected over six months at 5-minute intervals.

4.2. Experimental Settings

The experiments compare LSTM-based traffic forecasting models under complete and missing-data conditions using standardized training, evaluation, and masking procedures.

  • Models are compared on datasets with and without missing values, including LSTM, BDLSTM, LSTM-I, BDLSTM-I, BGCP, and GRU-D.
  • Inputs are 3-D vectors X ∈ R^b×T×D with batch size 64, sequence length 10, and sensor dimension D determined by the dataset.
  • Samples are randomized and split into training, validation, and test sets at a 6:2:2 ratio.
  • Missing-data experiments use random masking of values and non-random masking of selected time steps.
  • Prediction algorithms are evaluated using MAE, MAPE, and RMSE, with observed speed x_i compared against predicted speed x̂_i.

4.3. Experimental Results

Across the two datasets, stacked and bidirectional LSTM configurations generally outperform one-layer alternatives, with the two-layer BDLSTM achieving the lowest reported MAEs.

  • Two-layer BDLSTM achieves minimum MAEs of 2.336 on LOOP-SEA and 1.186 on PEMS-BAY.
  • One-layer models perform worst, while models with more than two layers perform worse than two-layer models.
  • BDLSTM performs better than LSTM at the same depth, and LSTM + BDLSTM outperforms BDLSTM + LSTM with equal layer counts.
  • BDLSTM-based models show stronger superiority on PEMS-BAY than on LOOP-SEA, where traffic sequences contain more irregular variations.
  • Figure 4 examines training time and MAE across input-sequence lengths for models evaluated on LOOP-SEA.

4.4. Training Time

Training time varies with model architecture and parameter count: bidirectional and deeper models require more computation than unidirectional and shallower models.

  • BDLSTM training time is nearly double LSTM training time because BDLSTM contains two LSTMs.
  • Training times for multi-layer models are nearly linearly related to the number of layers.
  • LSTM + BDLSTM and BDLSTM + LSTM combinations have training times between those of multi-layer LSTMs and multi-layer BDLSTMs.

4.5. Influential Factors of the RNN-based Model

The section evaluates how model capacity and input-sequence length affect prediction performance. Increasing parameters helps only to a point, while moderate sequence lengths perform best in the tested setting.

  • Influential factors: Model weight size and input-sequence time lags are evaluated as factors influencing prediction performance.The analysis measures both model capacity and the amount of historical input used.
  • Model capacity: In multi-layer models, weight-matrix dimensions can be customized, subject to input and output spatial dimension constraints of D.The first-layer input dimension and final-layer output dimension must match the spatial dimension D.
  • Model capacity: Increasing customized weight dimensions to 4D produces the best PEMS-BAY accuracy, but improves LOOP-SEA performance little.The results indicate that additional parameters can improve prediction capability to some extent, but accuracy does not increase indefinitely.
  • Input-sequence length: For BDLSTM+LSTM on LOOP-SEA, sequence lengths of 8, 10, and 12 yield similarly low MAEs with relatively small deviations.A sequence length of 6 produces much higher MAE and substantially larger deviation.

4.6. Dealing with Missing Values

The missing-value experiments compare recurrent forecasting and imputation models across datasets, missing patterns, and rates. Bidirectional stacked models, especially BDLSTM-I + BDLSTM, achieve the strongest forecasting results, while imputation performance depends on missing rate and method.

  • Experimental setup: Models are tested on LOOP-SEA and PEMS-BAY under random and non-random missingness from 10% to 80%.Tables 4–7 report forecasting results for the different datasets, missing scenarios, and rates.
  • Forecasting comparison: At 10% and 20% missingness, GRU-D does not outperform the other models in random scenarios, while bidirectional and stacked models perform better on LOOP-SEA.On PEMS-BAY, models with BDLSTM as the last layer perform better in both missingness scenarios.
  • Forecasting comparison: Among two-layer models, BDLSTM-I + BDLSTM achieves better prediction results than the other compared models.This advantage is reported across the missing-value forecasting comparisons.
  • Forecasting comparison: At 40% and 80% missingness, two-layer models outperform one-layer GRU-D, LSTM-I, and BDLSTM-I models.Two-layer models with a BDLSTM second layer perform better than those with an LSTM second layer, and BDLSTM-I + BDLSTM has the smallest prediction errors.
  • Forecasting comparison: Overall, non-random missingness results are close to or slightly better than random-missingness results, with the strongest advantage for BDLSTM-I + BDLSTM on PEMS-BAY.Models containing BDLSTM-I slightly outperform those with LSTM-I, and BDLSTM is reported as suitable for the final layer.
  • Data imputation comparison: BDLSTM-I + BDLSTM outperforms BGCP at relatively small missing rates, but its imputation performance decreases as the missing rate increases.BGCP imputation errors remain nearly unchanged across missing rates, reflecting its use of the whole dataset rather than short-term sequence input.

4.7. Model Interpretation and Visualization

The visualization compares ground-truth and predicted traffic states at selected sensors from LOOP-SEA and PEMS-BAY. LOOP-SEA states vary more, whereas PEMS-BAY states are smoother.

  • Model interpretation and visualization: Figure 5 compares ground truth and predicted traffic states at two LOOP-SEA and two PEMS-BAY sensing locations.The selected locations are shown in panels (a)–(d).
  • Model interpretation and visualization: LOOP-SEA traffic states show more variation, while PEMS-BAY traffic states vary smoothly.The figure uses representative sensing locations from each dataset.

5. Conclusion

The paper proposes stacked bidirectional and unidirectional LSTM models for network-wide traffic prediction and an LSTM imputation unit for missing spatiotemporal inputs. Experiments report superior stacked bidirectional performance and support further research through evaluated datasets and a published LOOP-SEA dataset.

  • Conclusion: The proposed SBU-LSTM architecture reformulates how LSTM models are incorporated into network-wide traffic state prediction.The architecture combines stacked bidirectional and unidirectional LSTM structures.
  • Conclusion: Stacked bidirectional LSTM models achieve superior prediction performance in the reported experiments.The study evaluates multiple stacked LSTM- and BDLSTM-based models to inform neural-network design for traffic prediction.
  • Conclusion: The proposed LSTM imputation unit infers and fills missing spatiotemporal input values to help improve prediction accuracy.Models using the unit outperform state-of-the-art RNN-based models and compete with tensor-decomposition-based models.
  • Conclusion: Two real-world traffic-state datasets are tested, and the LOOP-SEA dataset is published in publicly accessible repositories.The publication is intended to facilitate further traffic-prediction research.
  • Conclusion: Future work will use different datasets and improve spatial-feature interpretation while combining traffic prediction with other applications.Potential applications include non-recurring congestion detection using additional datasets.
Loading 2005.11627v1…