Source-linked AI summary
LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting
Xu Liu, Yutong Xia, Yuxuan Liang, Junfeng Hu, Yiwei Wang, Lei Bai, Chao Huang, Zhenguang Liu, Bryan Hooi, Roger Zimmermann
TL;DR
Traffic forecasting research relies on public datasets whose limited scale, temporal coverage, and metadata may not reflect practical networks or support long-term analysis. The paper introduces LargeST, a benchmark with 8,600 California sensors, 5 years of data, comprehensive metadata, and extensive analyses and baseline experiments. It uses the dataset to characterize traffic data and identify research challenges and future opportunities, while noting limits in geographic generalizability and sensor-data quality.
Problem
Existing traffic datasets often have limited network scale and temporal coverage and insufficient sensor metadata, restricting their representation, analysis, reliability, and interpretability.
Method
The paper constructs LargeST from California traffic sensors and provides 8,600 sensors, 5 years of 5-minute data, comprehensive metadata, data analysis, and benchmark experiments.
Results
LargeST provides a larger graph, longer temporal coverage, and richer node metadata than existing traffic datasets, supporting benchmarking of accuracy, efficiency, and scalability.
Takeaways & Limitations
LargeST offers a more realistic benchmark for large-scale traffic forecasting and highlights the need for simple, effective methods that remain efficient and scalable.
Takeaways & Limitations
The dataset’s generalizability is limited because analyses and experiments were conducted in California, and sensor readings contain inaccuracies and missing data.
Abstract
from arXiv · showhide
Road traffic forecasting plays a critical role in smart city initiatives and has experienced significant advancements thanks to the power of deep learning in capturing non-linear patterns of traffic data. However, the promising results achieved on current public datasets may not be applicable to practical scenarios due to limitations within these datasets. First, the limited sizes of them may not reflect the real-world scale of traffic networks. Second, the temporal coverage of these datasets is typically short, posing hurdles in studying long-term patterns and acquiring sufficient samples for training deep models. Third, these datasets often lack adequate metadata for sensors, which compromises the reliability and interpretability of the data. To mitigate these limitations, we introduce the LargeST benchmark dataset. It encompasses a total number of 8,600 sensors in California with a 5-year time coverage and includes comprehensive metadata. Using LargeST, we perform in-depth data analysis to extract data insights, benchmark well-known baselines in terms of their performance and efficiency, and identify challenges as well as opportunities for future research. We release the datasets and baseline implementations at: https://github.com/liuxu77/LargeST.
1 Introduction
Traffic forecasting advances have been driven by deep learning, but commonly used datasets may not represent practical traffic networks because of limitations in scale, temporal coverage, and metadata. LargeST addresses these issues with a large, long-duration, metadata-rich benchmark and accompanying analyses and baselines.
- Traffic forecasting supports urban planning, traffic management, and public safety, while deep learning has advanced prediction of nonlinear traffic patterns.
- Existing traffic datasets present critical issues that may obstruct further progress in traffic forecasting research.
- Current benchmarks often contain only hundreds of nodes and edges, whereas California has nearly 20,000 operational sensors, limiting their representation of real-world network scale.
- Existing datasets commonly span less than 6 months, restricting study of long-term traffic patterns and the sample availability needed to train deep models.
- LargeST contains 8,600 California sensors, 5 years of data per sensor, comprehensive metadata, data analysis, forecasting baselines, and benchmark experiments.
2 Preliminaries
Traffic forecasting predicts future target attributes from historical observations over a directed sensor graph. Common methods construct graph connectivity with a thresholded Gaussian kernel and combine graph neural networks with recurrent or temporal convolutional models.
- Traffic forecasting predicts future target attributes, such as traffic flow, from historical observations over a directed sensor graph.
- The adjacency matrix is commonly built with a thresholded Gaussian kernel based on road-network distances between sensors.
- In the adjacency construction, d_ij is the road-network distance, σ is the standard deviation of all distances, and r is the threshold.
- Deep forecasting models generally combine graph neural networks with recurrent neural networks or temporal convolutional networks to capture spatial and temporal dependencies.
3 Limitations of Existing Traffic Datasets
Existing traffic datasets are limited in graph size, temporal coverage, and node metadata. LargeST expands all three dimensions, providing a more realistic basis for studying traffic patterns, training models, and evaluating scalability.
- LargeST’s CA dataset has 8.4×–50.6× more nodes and 13.9×–729.6× more edges than predecessor datasets.
- Previous sensor graphs can be extremely sparse, with average degree around 1, limiting graph-based models’ ability to capture spatial correlations.
- LargeST covers 5 years at the same sampling rate as previous datasets, enabling analysis of seasonal trends, more recent traffic conditions, and larger training samples.
- Existing datasets often omit node metadata, reducing reliability and interpretability; LargeST provides features such as county, highway, lane count, and sensor location.
- Table 1 compares datasets using graph degree, metadata count, data points, and a common 5-minute sampling rate.
4 The LargeST Benchmark Dataset
LargeST is a large-scale California traffic benchmark designed to evaluate forecasting models across multiple graph sizes, long temporal coverage, and rich sensor metadata. Its analysis reveals regional, temporal, and metadata-related traffic patterns relevant to model design.
- Dataset Organization: LargeST provides four hierarchically structured sub-datasets for evaluating traffic forecasting models at different node scales.The dataset is sourced from California highway sensors and organized into sub-datasets with different numbers of nodes.
- Data Collection: 8,600 sensors form the California dataset, while three regional subsets cover the Greater Los Angeles, Greater Bay Area, and San Diego areas.The sensors are selected from CalTrans PeMS mainline detectors using coordinate and proximity filters.
- Data Collection: 5 years of traffic flow data from 2017 to 2021 provide 525,888 five-minute time frames, while missing values are retained for user-selected preprocessing.The dataset preserves the original data state rather than removing nodes with high missing-value rates.
- Regional Disparities: Regional traffic patterns differ in both flow levels and curve shapes, with GLA, GBA, and SD generally exceeding the statewide CA baseline.The paper attributes these disparities to differences in economic development, topography, and road planning strategies.
- Temporal Dynamics: Weekday traffic has morning and evening peaks, while weekday and weekend patterns differ substantially, highlighting time-of-day and day-of-week features.The morning peak occurs around 8 a.m. and the evening peak around 5 p.m.
- Temporal Dynamics and Metadata Characteristics: Traffic distributions shift across months, and metadata such as highway category and lane count correlates strongly with traffic flow.Morning flow has greater variance than evening flow, both peaks rise from January to May, fluctuate from June to September, and decline from October to December; more lanes are positively associated with traffic volume.
5 Experiments
Experiments evaluate representative traffic forecasting baselines on LargeST for predictive performance and computational efficiency. Results show that accuracy, scalability, and model complexity vary substantially across methods and dataset sizes.
- Experimental Setup: 12-step forecasting uses 12 historical steps, with 15-minute aggregation yielding 96 time steps per day and one year of 2019 data for experiments.The setup supports comparison over an extended future period while retaining feasibility for less efficient baselines.
- Experimental Setup: Eleven baselines span naive, temporal-only, recurrent, convolutional, attention-based, neural-ODE, and dynamic-topology approaches.The comparison includes HL, LSTM, DCRNN, AGCRN, STGCN, GWNET, ASTGCN, STTN, STGODE, DSTAGNN, DGCRN, and D2STGNN.
- Performance Comparisons: AGCRN and GWNET generally outperform their predecessors, while DGCRN and D2STGNN perform strongly on SD and GBA but do not scale to GLA and CA.Adaptive adjacency improves model capacity, whereas dynamic-topology models’ complex designs prevent scaling to the largest datasets.
- Efficiency Comparisons: LSTM is fastest, while STGCN and GWNET are generally faster and scale better than other baselines; DGCRN and D2STGNN incur long training and inference times.Only half of the selected baselines can run on CA, highlighting the computational challenge of the largest dataset.
6 Future Opportunities & Limitations
LargeST supports future traffic-forecasting research through large-scale, metadata-rich, and long-duration observations. The paper highlights feature integration, distribution-shift evaluation, simpler scalable models, and foundation-model development, while noting California-specific scope and sensor-data quality issues.
- Future Opportunities: LargeST contains 8,600 sensors with 5 years of data and comprehensive metadata, serving as a benchmark for accurate, efficient, and scalable forecasting.The dataset is positioned as a foundation for the paper’s data analysis and future research opportunities.
- Future Opportunities: Positive correlations between traffic flow and regional distribution, temporal factors, highway categories, and lane numbers motivate using spatial, temporal, and metadata features.The paper connects these observations to improving forecasting accuracy and interpretability.
- Future Opportunities: Five years including 2020 and 2021 provide a testbed for temporal distribution shifts and abrupt changes associated with extraordinary events.Researchers can use the dataset to study out-of-distribution forecasting challenges.
- Future Opportunities: Increasing model complexity improves recent accuracy but harms efficiency and scalability on larger sensor networks, motivating simple yet effective methods.The authors identify practical implementation and deployment on real-world networks as the relevant target.
- Future Opportunities: Billions of curated data points may support training foundation models for traffic forecasting or time-series forecasting.The opportunity is framed as a potential resource for future model development.
- Limitations: Generalizability is limited because data analysis and experiments were conducted in California, and sensor readings contain inaccuracies and missing data.The reported data-quality issues may arise from signal interruptions and other unforeseen circumstances.
A Dataset Documentation
LargeST is documented as a self-contained traffic forecasting dataset assembled from directly observable California PeMS data, combining sensor readings, metadata, and graph connectivity. It uses a representative deterministic sample, chronological 6:2:2 splits, and acknowledges missing or inaccurate readings and omitted high-missingness metadata.
- Dataset contents: The dataset contains traffic readings, sensor metadata, and an adjacency matrix describing connections between sensors.Metadata includes coordinates, county, PeMS district, highway, travel direction, and lane count.
- Dataset contents: 8,600 sensors are covered across five years, with a sensor graph containing 8,600 nodes and 201,363 edges.
- Sampling: The dataset is a deterministic spatial and temporal sample from California PeMS, selecting mainline sensors and five years of recent readings.The authors consider mainline sensors representative of overall system traffic conditions.
- Learning setup: Traffic forecasting is treated as a self-supervised learning task without labels or targets associated with individual instances.
- Data splits: Chronological train, validation, and test sets are provided in a 6:2:2 ratio.
- Data quality: Sensor readings may be inaccurate or missing because of unexpected factors such as signal interruptions, while city metadata is ignored because of a high missing-data rate.
- Collection process: Data are collected directly from CalTrans PeMS through its website and APIs, including sensor lists, coordinates, metadata, and historical observations.The construction process includes downloading sensor information, retrieving observations, and cleaning the data.
What (other) tasks could the dataset be used for?
The dataset is intended for broad lawful use and is currently maintained through GitHub, where its code and data are updated.
- Permitted uses: Users may apply the dataset to any task as long as the use does not violate laws.
- Access: The dataset is currently held on GitHub, and building instructions are available there; no DOI exists at present.
When will the dataset be distributed?
The dataset is distributed through the project’s GitHub instructions and is released under a noncommercial Creative Commons license, with commercial-use restrictions referring users to PeMS.
- Licensing: The benchmark dataset uses a CC BY-NC 4.0 International License, and its code uses the MIT License.
- Licensing: Commercial use is subject to additional restrictions, with users directed to the PeMS website for details.
Who will be supporting/hosting/maintaining the dataset?
The paper authors support the dataset through GitHub, where users can report issues, submit pull requests, and receive updates to the code and data.
- Support: The paper authors are responsible for supporting the dataset.
- Maintenance: The authors will actively update the code and data on GitHub.
- Versioning: Information about older dataset versions will be provided on GitHub.
- Community support: Users can report issues or bugs through GitHub and submit pull requests that the authors will actively validate.
B More Dataset Information
This section provides additional LargeST dataset documentation, including file formats, dataset statistics, node metadata, and statistics for two meta-features.
- LargeST traffic-flow data use .h5 files, adjacency matrices use .npy files, and metadata use .csv files.
- Table 5 describes the node metadata available in LargeST.
- Table 6 summarizes the statistical properties of highway categories and the number of lanes.
- Table 4 reports dataset statistics, with degree denoting average node degree and density denoting graph density.
C More Experimental Settings
The experimental setup evaluates 12 representative baselines with reproducible configurations and supports fair efficiency comparisons, while the codebase is designed for extensibility.
- The study evaluates 12 representative baselines on the LargeST benchmark dataset.
- Baseline configurations are provided in executable .sh files so users can reproduce the reported experimental results.
- Efficiency comparisons run all baseline models on the same GPU and measure training time per epoch.
- The modular codebase allows users to add custom models by following its established structure and guidelines.
- Table 5 documents the dataset's available node metadata, while Table 6 summarizes two meta-features.