RideBench: A Large-Scale Exogenous-Aware Benchmark for Ride-Hailing Time Series Forecasting

Motivation

• Ride-hailing forecasting must account for marketplace dynamics, spatial heterogeneity, external disturbances, and planning horizons that general-purpose benchmarks emphasize less systematically.Forecasts support decisions including supply-demand balancing, driver incentives, resource allocation, and event-specific operations.Source: §1 · pp. 1–2

Dataset and evaluation

• Ride-Hailing is synthesized from DiDi marketplace data and covers four years of half-hourly records across 200 spatial areas, with future-known signals for weather, holidays, and large-scale events.The benchmark evaluates over 30 representative forecasting methods under unified protocols.Source: §1 · pp. 2–3

Benchmark design

• RideBench evaluates regular week-ahead and 8-week-ahead forecasting, with scenario-specific robustness, multi-area evaluation, and long-horizon trend and first-week accuracy measures.The longest task spans 2,688 half-hourly prediction steps.Source: §1 · p. 2
• RideBench adds future-known continuous and discrete covariates, scenario-specific evaluation, representative spatial areas, and a maximum horizon of 2,688 steps to existing benchmark comparisons.These settings support assessment of robustness to external perturbations, generalization across areas, and long-horizon forecasting.Source: §1 · p. 2

Evaluation findings

• Future-known exogenous variables benefit regular week-ahead forecasting, but models still struggle with disturbance-induced changes and cannot jointly sustain pointwise accuracy, broad trends, and near-term reliability.Difficult cases include categorical holidays, sparse large-scale events, and area-dependent exogenous responses.Source: §1 · p. 3

Time series forecasting benchmarks

• Forecasting benchmarks provide shared datasets, splits, metrics, and protocols, with recent efforts addressing evaluation fairness, multiple tasks, and zero-shot forecasting.Earlier benchmarks covered domains including electricity, traffic, weather, and retail demand.Source: §2 · p. 3
• Existing benchmarks provide limited support for marketplace-driven demand, heterogeneous areas, future-known categorical variables, and long-horizon operational planning; RideBench focuses on these requirements.RideBench is positioned as a large-scale ride-hailing setting with real-world exogenous factors.Source: §2 · p. 3

Time series forecasting models

• Forecasting models span channel-independent, multivariate, exogenous-aware, and foundation-model approaches, yet many do not natively address future-known categorical signals, area-dependent responses, and long-horizon trend quality.RideBench jointly evaluates exogenous utilization, multi-area generalization, and long-horizon planning capability.Source: §2 · p. 3

3.1 Dataset Construction

• Ride-Hailing contains 14,025,600 half-hourly observations over four years and 200 areas, synthesized from DiDi marketplace data and transformed to protect sensitive operational details.The transformations anonymize identifiers and represent large-scale events with coarse-grained impact categories.Source: §3.1 · pp. 3–4
• The dataset provides nine anonymized endogenous targets, future-known weather, holiday, and event variables, and calendar indicators for hour and day of week.Weather uses an impact factor, holidays use categories, and events use two impact variables.Source: §3.1 · p. 4

3.2 Dataset Characteristics

• Ride-Hailing exhibits multi-scale periodicity, exogenous sensitivity, and heterogeneity across areas, time periods, and external conditions.Models must learn shared regularities while preserving area-specific temporal patterns in global multi-area forecasting.Source: §3.2 · p. 4, §3.2.3 · p. 5

3.2.1 Multi-scale Periodicity.

• Demand patterns vary at daily, weekly, and yearly scales, from rush-hour peaks and weekday-weekend differences to seasonal changes and long-term trends.These patterns require models to capture short-term routines and longer-term evolution.Source: §3.2.1 · pp. 4–5

3.2.2 Exogenous Effects.

• Weather disturbance is associated with higher average endogenous values and may have delayed impacts, while holiday demand changes across pre-holiday, in-holiday, and post-holiday stages.Large-scale events cause localized, time-dependent changes, with two coarse event categories representing different daytime and evening mobility effects.Source: §3.2.2 · p. 5

3.2.3 Heterogeneity.

• Area patterns differ, can evolve over years, and may respond differently to the same exogenous factor, including positive or negative holiday changes.The testbed calls for models that generalize across areas while retaining area-specific temporal patterns.Source: §3.2.3 · p. 5

4.1 Data Processing

• RideBench pools samples from all 200 areas for shared global-model training and assigns samples to chronological splits only when their full target windows fit within each split.Training uses the first three years, validation the next six months, and testing the final six months; input windows may extend into earlier periods for realistic historical context.Source: §4.1 · p. 5

4.2 Forecasting Methods

• RideBench evaluates more than 30 representative forecasting methods across five major paradigms, including statistical baselines, channel-independent models, multivariate models, and exogenous-aware models.Statistical baselines include SeasonalNaive and SeasonalMean. Discrete exogenous variables are adapted using ordinal, one-hot, or learnable categorical embeddings; interfaces are modified when needed to incorporate future-known variables.Source: §4.2 · pp. 5–6

4.3.1 Forecasting Settings.

• Week-ahead forecasting predicts 336 half-hourly steps from 1,344 historical steps, while 8-week-ahead forecasting predicts 2,688 steps from 4,704 historical steps.The longer setting examines macro-level demand trends for strategic planning, holiday operations, and resource scheduling.Source: §4.3.1 · p. 6

4.3 Comprehensive Evaluation

• Evaluation covers overall forecasting accuracy, robustness under external scenarios, and long-horizon trend quality.Scenario subsets use future-known indicators for weather disturbances, holidays, and large-scale events; long-horizon assessment adds day-level trend metrics beyond pointwise errors.Source: §4.3.2 · pp. 6–7

4.3.2 Evaluation Protocol.

• Overall accuracy uses WMAPE, MAE, and RMSE; scenario-specific metrics assess affected test periods, while long-horizon evaluation adds daily directional accuracy and correlation.Daily predictions average every 48 half-hourly steps; directional accuracy checks day-to-day change signs, and correlation measures whether the predicted daily trajectory follows the ground-truth shape.Source: §4.3.2 · p. 7

4.4 Implementation Details

• Experiments use task-specific sliding-window strides of 48 steps for week-ahead forecasting and 336 steps for 8-week-ahead forecasting.Models use MAE loss, AdamW, and a unified learning-rate search; future-known covariates are supplied through model-specific interfaces when supported.Source: §4.4 · p. 7

5.1.1 Regular Forecasting.

• Exogenous-aware models perform strongest overall and across all three regular week-ahead scenarios, with TiDE consistently strong across metrics.Endogenous-only models remain competitive in overall WMAPE and MAE, while Chronos-2 is comparable to several task-specific trained models but lags exogenous-aware models under external disturbances.Source: §5.1.1 · p. 7, Table 2 · p. 8

5.1.2 Long-Horizon Forecasting.

• No existing model dominates 8-week-ahead forecasting across pointwise error, trend quality, and first-week accuracy.Pointwise accuracy and trend quality are not strictly aligned; several channel-independent and foundation models achieve strong first-week accuracy, but week-ahead advantages from cross-variable or exogenous modeling do not necessarily transfer.Source: §5.1.2 · pp. 7–8, Table 3 · p. 9

5.1.3 Accuracy–Efficiency Trade-off.

• Task-specific deep models generally offer a more favorable accuracy–efficiency balance, while foundation models incur substantially higher inference costs.Higher computational cost is more likely to bring accuracy gains in week-ahead forecasting; this relationship is much weaker at 8 weeks, where performance also depends on capturing stable long-term trajectories.Source: §5.1.3 · pp. 8–10
• Long-horizon research should emphasize trend-aware modeling rather than relying only on greater model complexity to reduce pointwise errors.The proposed priorities differ by setting: improve pointwise accuracy and use of future-known covariates for week-ahead forecasts, and model demand directions, trajectory shapes, and stable temporal structures over long horizons.Source: §5.1.3 · p. 10

5.2.1 Exogenous Variable Utilization.

• Most exogenous-aware models improve with external variables, and TiDE achieves the largest average relative gain at 10.4%.Gains vary by scenario: sparse, heterogeneous large-scale events are harder to model, while Chronos-2 gains less on categorical holidays than on continuous weather disturbances.Source: §5.2.1 · p. 10, Figure 7 · p. 11

5.2.2 Area Scaling Study.

• With 10 target areas fixed, adding training areas from 10 to 200 improves task-specific models, while Chronos-2 is strong when training data are limited.The results support transferable shared temporal and exogenous structures across areas, and show that zero-shot foundation models become less advantaged as more domain data are available.Source: §5.2.2 · pp. 10–11, Figure 7 · p. 11

5.2.3 Lookback Study.

• Most models improve with longer lookback windows, but extending input to 98 days slightly degrades several task-specific models.The paper attributes this trade-off to potentially irrelevant or outdated context and fewer sliding-window training samples; foundation models are especially sensitive to short lookbacks and approach task-specific models around 28 days.Source: §5.2.3 · p. 11, Figure 7 · p. 11

5.2.4 Loss Function Study.

• MAE achieves the best overall performance, outperforming MSE and the other evaluated objectives in this setting.The paper relates MAE's robustness to disturbance-induced large deviations; FreDF and DBloss are competitive with MSE and Smooth L1 but still underperform MAE, which RideBench adopts as its default loss.Source: §5.2.4 · p. 11, Figure 7 · p. 11

Exogenous-driven pattern modeling

• Even TiDE only partially follows ground truth under strong external perturbations, amid heterogeneous, temporally structured effects across areas.Weather can be nonlinear or delayed, holidays have multiple phases, and large-scale events are sparse and irregular.Source: §6 · pp. 11–12, Figure 8 · p. 12
• Current exogenous-aware methods are not natively designed for future-known categorical covariates, and input adapters do not ensure semantic understanding or modeling of heterogeneous area effects.The paper characterizes one-hot encoding and learnable embeddings as pragmatic compromises rather than principled solutions.Source: §6 · p. 12

Long-horizon multi-objective balance

• No existing model dominates pointwise accuracy, trend quality, and near-term accuracy over eight weeks, while benchmarks rarely evaluate these requirements jointly.RideBench argues for forecasts supporting short-term operations and long-term planning, and for models handling exogenous variables, categorical event semantics, spatial heterogeneity, and long-horizon needs.Source: §6 · p. 12

• Future-known exogenous variables bring clear benefits in regular week-ahead forecasting, especially under external disturbance scenarios.Source: §7 · p. 12
• RideBench evaluates more than 30 representative methods using standardized processing, global multi-area training, scenario-specific evaluation, and long-horizon trend evaluation.The benchmark covers regular week-ahead and long-horizon 8-week-ahead forecasting.Source: §7 · p. 12
• Further analyses show the value of scalable multi-area training, sufficient historical context, and robust training objectives.Source: §7 · p. 12
• Current exogenous-aware models have limited ability to capture disturbance-induced pattern changes.Source: §7 · p. 12
• Long-horizon forecasting still requires a better balance among pointwise accuracy, broad trend quality, and near-term reliability.Source: §7 · p. 12
• Ride-Hailing is a synthesized dataset, and RideBench is a unified benchmark for exogenous-aware ride-hailing forecasting.The paper describes the dataset as privacy-preserving and says its unified evaluation protocols can facilitate research on deployment-oriented forecasting models.Source: §7 · p. 12