PaperlayerView exact revision on arXiv

Source-linked AI summary

SeoulMMOD: A Large-Scale Multimodal Origin-Destination Flow Benchmark

Taeyoung Yu, Seonbin Jo, Jiwon Kim, Junyoung Byun

arXiv:2610.13730v1cs.LG

TL;DR

OD forecasting supports urban transport decisions, but comprehensive and dynamic OD data are difficult to obtain. SEOULMMOD provides a multimodal, multi-year benchmark, revealing challenges in fine-scale forecasting and joint training.

  • Problem

    OD flows are difficult to observe across modes and operators, while traditional surveys and traffic-count estimates have limited coverage or update frequency and produce static matrices.

    Source: §1 · p. 1

  • Method

    SEOULMMOD combines three years of hourly flows for six modes at district and sub-district levels with contextual signals, and benchmarks 14 forecasting baselines.

    Source: §1 · p. 2

  • Results

    Spatio-temporal models that perform well at district level do not scale to sub-district forecasting, and naive joint multi-mode training does not consistently improve accuracy.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    SEOULMMOD provides a reproducible testbed for developing and evaluating models for city-scale, fine-resolution, multi-mode OD forecasting.

    Source: §5 · p. 9

    As a single-city benchmark built from mobile-network estimates rather than direct trip counts, SEOULMMOD does not establish transfer elsewhere, and estimation error may remain for visitors, phone-use patterns, or under-represented groups.

    Source: §D.5 · p. 17

Abstract

from arXiv · show

Origin-destination (OD) flow forecasting supports urban mobility decisions by predicting the volume and patterns of human movement across a city. Most existing public OD benchmarks provide only a partial view of citywide mobility, covering a single transport service or at most two modes. With partial mode coverage, observed demand changes are difficult to separate into shifts across transport modes and changes in total travel. We introduce SeoulMMOD, a large-scale public benchmark for citywide multimodal mobility in Seoul. SeoulMMOD provides three years of hourly OD flows estimating total mobility for 6 urban travel modes across 25 districts and 426 sub-districts, together with travel-time, travel-distance, calendar, rainfall, and point-of-interest (POI) signals. Using SeoulMMOD, we benchmark 14 forecasting baselines under a common protocol and analyse the effects of spatial scaling, joint versus mode-separate training, and cross-year generalization. Results show that several spatio-temporal models that work at district level do not scale to sub-district OD forecasting, and that naive joint multi-mode training does not consistently improve accuracy. Together, the dataset and benchmark establish a reproducible testbed for multi-mode, multi-year OD forecasting at the city scale.

1 Introduction

OD forecasting can support operational decisions, but complete time-varying OD flows are difficult to observe across modes and operators. SEOULMMOD addresses the limited multimode coverage of public benchmarks and evaluates forecasting across modes, spatial scales, and years.

1 Introduction

  • Household surveys and roadside counts yield static OD matrices, limiting their ability to capture hour-to-hour or day-to-day changes.Surveys have limited sample sizes and update frequencies; roadside counts have sparse coverage and non-unique solutions.Source: §1 · p. 1
  • Public benchmarks typically cover one mode, or at most two, making mode shifts difficult to distinguish from changes in total travel.The paper gives rail interruption as an example: lower subway demand could reflect shifts to other modes or cancelled trips.Source: §1 · p. 2
  • SEOULMMOD introduces an hourly, citywide OD benchmark covering all major urban transport modes.It includes six modes over three years at district and sub-district levels, with aligned trip attributes and contextual signals.Source: §1 · p. 2
  • The benchmark evaluates 14 forecasting baselines and finds that some district-level spatio-temporal models do not scale to sub-district OD forecasting.Naive joint multi-mode training does not consistently improve forecasting; the benchmark also supports cross-year generalization evaluation.Source: §1 · p. 2

2 Related Work

Prior work models OD forecasting with temporal and spatial methods and has explored joint forecasting across modes. Existing public datasets, however, are limited in mode coverage, scale, or temporal span.

2.1 OD Forecasting

  • OD forecasting models capture temporal dynamics and OD-pair dependencies using methods including LSTMs, graph convolutions, attention, and MLP-based mixing.The OD matrix is represented as a Z × Z matrix at each time step.Source: §2.1 · p. 2
  • Joint subway-bus and subway-bike forecasting has reported improvements over single-mode variants, while other studies jointly forecast zone-level mode demand.Existing multimode studies typically cover at most two modes and less than one year of data.Source: §2.1 · p. 3

2.2 Existing OD Datasets

  • Static OD datasets provide a single time-invariant matrix, which cannot represent hourly swings, weekday-weekend differences, or seasonal effects.The passage gives annual home-to-work commuting counts as an example.Source: §2.2 · p. 3
  • Dynamic datasets use time-stamped mobility records, but public benchmarks typically cover one operator, at most two modes, limited citywide OD scale, or weeks to months.These limits make finer-zone scaling and seasonal or cross-year evaluation difficult to assess.Source: §2.2 · p. 3

3 SEOULMMOD Dataset

SEOULMMOD is built from Seoul mobile-network mobility estimates and provides three years of hourly OD flows at two spatial levels. Its analyses describe temporal and mode-specific structure and document the bundled trip attributes and contextual signals.

3.1 Data Source and Collection

  • SEOULMMOD uses Seoul mobility records derived from KT LTE/5G signalling logs, with population-scaling corrections, and retains six intra-urban modes on a uniform hourly grid.It covers 2023–2025, 1,096 days, 25 districts, and 426 sub-districts after filtering for OD pairs within Seoul and reconciling sub-district identities.Source: §3.1 · pp. 3–4

3.2 Dataset Analysis

  • OD flows are uneven across space and time, with commuting peaks around 08:00 and 18:00 that disappear on holidays and monthly and yearly variation.Morning-rush net inflow distinguishes destination-heavy from origin-heavy sub-districts.Source: §3.2 · p. 4, Figure 2 · p. 4
  • Car and walk OD networks differ in spatial structure: car has long-range links (Q=0.584, 7 clusters), while walk has localized clusters (Q=0.852, 14 clusters).The analysis applies a disparity filter to extract network backbones and Louvain community detection.Source: §3.2 · p. 4, Figure 3 · p. 5

3.3 Auxiliary Data

  • Each OD record includes average travel time and distance, and higher-flow ranges differ across modes.Figure 4 relates district-level travel attributes to average hourly outflow.Source: §3.3 · p. 5, Figure 4 · p. 5
  • The daily calendar includes weekday, weekend, Korean public-holiday, and holiday-name fields; office and leisure destinations show different weekday and holiday inflow profiles.Office destinations have a weaker weekday morning peak on holidays, while leisure destinations shift toward broader daytime activity.Source: §3.3 · p. 5, Figure 4 · p. 5
  • Rainfall data provide hourly district precipitation, and rainy hours show lower walk flow but higher local-bus and subway flow.The comparison is relative to dry hours.Source: §3.3 · p. 5, Figure 4 · p. 5
  • The 2025 POI snapshot covers 20 categories across sub-districts; selected office and education examples shift from morning net inflows to later net outflows.The low-POI residential example follows the reverse pattern.Source: §3.3 · p. 5, Figure 4 · p. 5

4 Experiments

The experiments compare OD forecasting across horizons, spatial scales, training modes, and years. District-level accuracy does not guarantee sub-district scalability, and separate mode models outperform joint training in these tests.

4.1 Setup

  • The protocol forecasts 6, 12, or 24 hours from equally long histories on chronological 2024 splits; learned baselines are evaluated jointly across modes.The common setup also aligns splits, horizons, early-stopping budgets, and evaluation metrics across learned baselines.Source: §4.1 · p. 6

4.2 Multi-Horizon Performance

  • 59.8 at 6h falls to 28.8 at 24h for pairwise models’ average MAE, while STAEformer has the lowest district MAE at every horizon.Historical Average is only 6.1% behind STAEformer at 24h; explicit temporal embeddings encode recurring daily and weekly rhythms.Source: §4.2 · pp. 6–7, Table 2 · p. 7
  • At sub-district scale, ODMixer leads learned 6h and 12h results and STID leads 24h; at district 24h, HA leads three modes and STAEformer the other three.HA leads metro bus, subway, and car; STAEformer leads local bus, walk, and other.Source: §4.2 · p. 7, Table 3 · p. 8

4.3 Scalability and Computational Cost

  • At sub-district scale, spatial-interaction models exceed memory limits, while pair-independent models exceed the per-epoch time budget by an order of magnitude.District-level STID trains within minutes and trails STAEformer by only a few percent in 24h MAE; dense origin-destination activations drive memory use, while 181,476 pair sequences drive runtime.Source: §4.3 · pp. 7–8

4.4 Cross-Mode Joint Training vs One Model per Mode

  • Training separate models per mode yields lower average MAE than joint training for all four architectures.The joint setup does not explicitly model cross-mode dependencies and shares a representation across modes with different scales, sparsity, temporal profiles, and spatial structures.Source: §4.4 · p. 8, Table 4 · p. 9

4.5 Cross-Year Generalization

  • All five models have higher MAE when trained on 2024 and tested on 2025; spring is the hardest season and summer the easiest.The seasonal ordering is consistent across architectures.Source: §4.5 · pp. 8–9, Table 5 · p. 9
  • Each contextual signal lowers STID’s 24h MAE by 0.49 to 1.39, but effects at shorter horizons vary.Holiday features raise MAE at 6h and 12h; rainfall raises it at 6h but lowers it at 12h; POI counts are neutral or slightly helpful across horizons.Source: §4.5 · p. 9

5 Conclusion

SEOULMMOD exposes two challenges in city-scale, multi-mode OD forecasting: district-level models may not scale to sub-districts, and joint multi-mode training does not consistently improve accuracy.

5 Conclusion

  • Spatio-temporal models that perform well at district level do not scale to sub-district forecasting, while naive joint multi-mode training does not consistently improve accuracy.These findings motivate models that scale to fine-resolution OD pairs and use cross-mode information more directly.Source: §5 · p. 9
  • SEOULMMOD combines six-mode hourly flows with trip attributes and contextual information in a city-scale Seoul benchmark.The benchmark is presented as a reproducible testbed for developing and evaluating multi-mode OD forecasting models.Source: §5 · p. 9
  • Future model design should scale to all origin-destination pairs at fine spatial resolution and use cross-mode information more directly.Source: §5 · p. 9

Reproducibility statement

The benchmark uses a shared training and evaluation pipeline and releases the materials needed to reproduce its dataset and model runs.

Reproducibility statement

A Additional Dataset Analysis

The descriptive analysis finds mode-specific differences in flow distributions, trip volumes, distances, and spatial community structure.

A Additional Dataset Analysis

  • Mode-specific flow distributions are heavy-tailed, while trip volumes and distances vary by mode.These differences suggest distinct hub sizes and interaction ranges across modes.Source: A Additional Dataset Analysis · p. 13
  • Figure 5d shows generally low normalised mutual information between mode-specific Louvain community partitions, suggesting spatial community assignments differ across modes.Source: A Additional Dataset Analysis · p. 13

B Contextual Signal Analysis

The contextual analysis presents calendar, rainfall, and POI signals for mobility modeling, with descriptive results showing differences in demand and mode shares across conditions and associations between POIs and arrivals.

B Contextual Signal Analysis

  • The benchmark bundles calendar, rainfall, and POI signals to support context-aware extensions without re-collecting data.Source: B Contextual Signal Analysis · p. 13
  • Weekend and holiday total trip volumes are 18% and 29% lower, respectively, while subway share falls by 4.1 pp and 5.5 pp.Walking (+3.3 pp on holidays) and car (+3.0 pp) take over most of the holiday subway-share decrease, consistent with a shift from commuting to local leisure trips.Source: B Contextual Signal Analysis · p. 13
  • In heavy-rain district-hours (>10 mm/hour), walk share is 30.6% and subway share is 22.4%, versus 33.7% and 20.5% on dry district-hours.Heavy-rain cells account for 0.30% of district-hours; rainfall is released as an auxiliary feature rather than the main target.Source: B Contextual Signal Analysis · p. 13, Table 7 · p. 14
  • POI counts cover 20 categories grouped into four broad presentation groups; subway-station count correlates with subway arrival share at r=+0.458.The subway-station association remains stable under rank, density, and area-controlled variants; bus-stop count tracks local-bus share, while commerce, leisure, and education categories also show mode-share associations.Source: B Contextual Signal Analysis · pp. 13–14, Table 8 · p. 14
  • For bus stops and walk share, the raw count correlation is r=−0.284, changing to r=+0.184 under density and weakening to r=−0.097 after controlling for sub-district area.The passage attributes the raw association to sub-district size: larger areas can have many bus stops and low walk share due to longer typical trip distances; POI correlations use four variants and filter for raw |r|≥0.20 and q<0.001 across 120 pairs.Source: B Contextual Signal Analysis · p. 14

B.1 Predictive utility of contextual signals

The section tests holiday, POI, and rainfall inputs as additions to STID’s mode-flow history. All three improve 24-hour MAE, but their effects differ at shorter horizons.

B.1 Predictive utility of contextual signals

  • The variants add holiday day-type embeddings, static per-pair POI features, or separate origin and destination rainfall channels to STID.POI uses 13 district-level categories; rainfall inputs are hourly and independently projected from mode-flow channels.Source: §B.1 · p. 14
  • The reported test MAE comes from the best validation checkpoint for each variant and horizon.The variants, configurations, and build scripts are released for reproducing the table or running multi-seed experiments.Source: §B.1 · p. 14
  • At 24 hours, Holiday lowers MAE by 1.39, while POI and Rainfall each lower MAE by 0.49.Table 10 reports test MAE for the district-scale 2024 split, with each signal added to STID in isolation.Source: §B.1 · p. 14, Table 10 · p. 15
  • At h=6 and h=12, Holiday increases MAE by 0.40 and 0.48, while Rainfall changes from +1.34 MAE at h=6 to −0.70 at h=12.POI improves h=6 and h=24 but increases h=12 MAE by 0.28, so no variant improves every horizon.Source: §B.1 · p. 14

B.2 Error stratification by contextual signals

The POI-error comparison shows that absolute and relative error can characterize destination districts differently. The authors frame it as diagnostic, not causal.

B.2 Error stratification by contextual signals

  • Higher-POI destination districts have higher target volume and larger absolute errors for HA and STAEformer, but lower wMAPE.Figure 6 reports Pearson correlations for the POI-count comparisons.Source: §B.2 · p. 15
  • The POI-error comparison is diagnostic rather than causal because higher-POI districts concentrate more demand.The authors present the contextual signals as a way to inspect where errors concentrate across urban environments.Source: §B.2 · p. 15

C Benchmark Implementation Details

The appendix documents a shared evaluation protocol and model-specific implementations, then details SeoulMMOD’s data provenance, composition, and preprocessing.

C Benchmark Implementation Details

  • Learned baselines share a chronological 70/10/20 split, Adam optimization, up to 100 epochs, and validation-MAE checkpoint selection.The appendix also specifies joint prediction of all horizons and six modes, with model-specific hyperparameters and released configurations.Source: C Benchmark Implementation Details · p. 15

C.1 Spatio-temporal forecasting baselines

  • Nine spatio-temporal baselines run in BasicTS, with scaling applied to mode-flow channels while contextual channels pass through unchanged.Graph WaveNet and MTGNN are adapted to reshape outputs to [B, H, P, 6]. Figure 6 shows higher-POI districts have larger absolute errors but lower relative errors.Source: §C.1 · pp. 15–16, Figure 6 · p. 16

C.2 OD flow forecasting baselines

  • MPGCN, ODCRN, and ODMixer preserve the Z×Z OD-matrix structure rather than flattening data into OD-pair sequences.MPGCN and ODCRN use matrix-form inputs and channel projections; ODMixer folds mode channels along time and receives a previous-day window.Source: §C.2 · p. 16

C.3 Parameter-free baselines

  • HistoricalAverage maps test timestamps to a 168-bucket hour-of-week table, while LatestValue repeats the last input frame across the horizon.Source: §C.3 · p. 16

D.1 Provenance

  • SeoulMMOD is built from Seoul Metropolitan Area Living Mobility Data, jointly produced by the Seoul Metropolitan Government and KT.The released source files cover January 2023 through December 2025.Source: §D.1 · p. 16

D.2 Composition

  • The dataset covers six modes across 25 districts and 426 sub-districts, with dense benchmark tensors formed by filling absent records with zeros.The source parquet files are sparse; two sub-district boundary changes are reconciled, and the recommended evaluations include within-2024 and 2024-to-2025 splits.Source: §D.2 · pp. 16–17

D.3 Upstream Generation

  • Source flows are population-scaled estimates derived from KT subscriber trips, with residual error possible from phone-use behavior, visitors, and under-represented groups.The public source documentation describes mode assignment using transit-network matching and, for remaining trips, speed and distance.Source: §D.3 · p. 17

D.4 Our Preprocessing

  • Preprocessing filters to Seoul-to-Seoul records and six intra-urban modes, reconciles zone codes, and aggregates records to a one-hour grid.Timestamps are aligned to Asia/Seoul, and data builders convert released parquet files into benchmark tensors.Source: §D.4 · p. 17

D.5 Uses and Limitations

SEOULMMOD is intended for multi-horizon OD forecasting and context-aware mobility modeling, not individual-level tracking or re-identification. Its scope is limited to a single city and upstream mobile-network estimates, so performance does not establish transfer to other cities, operators, or zoning systems.

D.5 Uses and Limitations

  • SEOULMMOD is intended for multi-horizon OD forecasting, joint-versus-per-mode training comparisons, cross-year generalization, and context-aware modeling, while individual-level tracking and re-identification are out of scope.Source: §D.5 · p. 17
  • The single-city benchmark and upstream mobile-network estimates do not establish transfer to other cities, operators, or zoning systems, and estimation error may remain for visitors, phone-use patterns, or under-represented groups.The source applies population scaling, but this does not eliminate those possible estimation errors.Source: §D.5 · p. 17

D.6 Distribution

  • The processed SEOULMMOD release is distributed under CC BY-NC 4.0 with attribution to the original Seoul Open Data Plaza source; underlying Seoul data use KOGL Type 1.Source: §D.6 · p. 17

NeurIPS Paper Checklist

  • The authors state that the release provides instructions, preprocessing scripts, model configurations, and training and evaluation commands to support reproduction.Source: NeurIPS Paper Checklist · pp. 19–20
  • The benchmark reports one run per model and evaluation setting, without error bars or formal significance tests.Source: NeurIPS Paper Checklist · p. 20

Related papers

Loading 2610.13730v1…