PaperlayerView exact revision on arXiv

Source-linked AI summary

GAS-PFN: Graph-Aware Spatiotemporal Prior-Data Fitted Networks for Zero-Shot Traffic Forecasting

Joseph Natter, Long Vu, Yifan Zhang, Bobin Deng, Kun Suo, Rui Wu

arXiv:2610.37598v1cs.LGcs.AIcs.NE

TL;DR

Traffic forecasting must capture road-network dependencies, yet supervised graph models are commonly tied to a dataset and graph, while forecasting PFNs do not explicitly encode topology. GAS-PFN combines graph-structured synthetic priors with graph-aware in-context inference; across six benchmarks, it achieves the strongest overall zero-shot performance under limited recent history.

  • Problem

    Supervised graph models are typically trained for particular datasets and sensor graphs, while existing forecasting PFNs do not explicitly incorporate road-network topology.

    Source: Introduction · p. 1

  • Method

    GAS-PFN combines graph-coupled synthetic task pretraining with two mask-conditioned inference passes that model sensor-local dynamics and neighboring-sensor interactions.

    Source: Introduction · p. 2

  • Results

    GAS-PFN achieves the strongest overall zero-shot performance under limited recent history, with best MAE and MSE on five of six datasets at 90 minutes and lower average MSE than TimePFN and four general-purpose zero-shot forecasters.

    Source: Introduction · p. 2

  • Takeaways & Limitations

    The evaluation indicates that encoding graph structure in both the synthetic prior and inference architecture improves zero-shot forecasting under limited recent context.

    Source: Introduction · p. 2

    Zero-shot inference keeps model parameters fixed, while context examples may come from the target dataset’s chronological training split.

    Source: Problem Setting and PFN Inference · p. 4

Abstract

from arXiv · show

Traffic forecasting requires modeling complex temporal dynamics and the spatial dependencies imposed by road topology. Supervised graph-based models achieve strong performance, but are often trained for a particular dataset and sensor graph, making deployment to new or evolving networks costly. Prior-Data Fitted Networks (PFNs) offer a zero-shot alternative by learning in-context inference from synthetic task priors, which provide an explicit mechanism for encoding domain-specific assumptions. Existing forecasting PFNs, however, do not explicitly represent the graph-structured dependencies governing traffic dynamics. To combine graph-aware traffic modeling with zero-shot adaptation, we introduce the Graph-Aware Spatiotemporal Prior-Data Fitted Network (GAS-PFN). GAS-PFN incorporates sampled road topology into its synthetic task prior through random-walk diffusion, teaching the model to learn graph-coupled dynamics during pretraining. Its co-designed inference architecture uses two mask-conditioned passes through a shared Transformer backbone to separately model sensor-specific temporal dynamics and spatiotemporal interactions among neighboring sensors before integrating their representations for prediction. This design enables GAS-PFN to use target-network structure and context without parameter updates. Across six benchmarks at the 90-minute horizon, GAS-PFN reduces MSE by 34.0% on average relative to TimePFN, its closest PFN-based predecessor. It also reduces MSE by 27.7% on average relative to four recent industry-developed general-purpose zero-shot forecasters. In this comparison, all models receive 90 minutes of contiguous recent history, and any additional context is drawn from each dataset's chronological training split. These results indicate that aligning PFN priors and inference architectures with graph structure improves in-context learning for zero-shot traffic forecasting.

Introduction

Dataset-specific supervised models and general-purpose zero-shot forecasters leave a gap between transferable prediction and explicit road-network structure. GAS-PFN combines graph-structured synthetic tasks with graph-aware in-context inference, and shows strong zero-shot results under limited recent-history conditions.

Introduction

  • Supervised graph models can perform strongly but are commonly trained for a particular dataset and sensor graph, making deployment to new or evolving networks costly.The cited prior work reports nearly eight GPU-days to train on approximately one third of California’s highway network.Source: Introduction · p. 1
  • General-purpose zero-shot forecasters address transfer without task-specific retraining but do not explicitly incorporate road-network relational structure.PFNs offer in-context inference from synthetic tasks whose priors provide an explicit way to represent domain assumptions.Source: Introduction · p. 1
  • GAS-PFN pretrains on graph-coupled synthetic traffic tasks and uses two differently masked passes through a shared Transformer to model sensor-local dynamics and neighboring-sensor interactions.The branch representations are fused for prediction, with adaptation through recent measurements and target-network historical examples rather than parameter updates.Source: Introduction · p. 2
  • GAS-PFN achieves the strongest overall zero-shot performance under limited recent history, with best MAE and MSE on five of six datasets at the 90-minute horizon and lower average MSE than TimePFN and four general-purpose zero-shot forecasters.It also outperforms supervised SCPT when target-road training coverage is limited.Source: Introduction · p. 2
  • 27 of 36 MAE/MSE comparisons favor GAS-PFN across six benchmarks and three prediction horizons.The evaluation also reports gains over supervised SCPT when target-road training coverage is limited.Source: Introduction · p. 2

Related Work

Prior work spans general-purpose zero-shot forecasting, supervised graph-based traffic models and transfer-oriented methods, and PFN-based in-context forecasting. GAS-PFN distinguishes itself by co-designing graph-structured synthetic tasks and graph-aware inference.

Zero-Shot Time-Series Forecasting

  • General-purpose time-series foundation models forecast zero-shot across heterogeneous datasets, but do not explicitly encode road-network relations between channels.GAS-PFN investigates encoding domain structure directly in zero-shot pretraining and inference.Source: Zero-Shot Time-Series Forecasting · p. 2

Traffic Forecasting and Cross-Network Generalization

  • Supervised graph models use road-network structure but are generally optimized for a particular sensor graph; transfer methods instead learn representations intended to generalize to new roads.GAS-PFN adapts at inference through context examples from the target graph.Source: Traffic Forecasting and Cross-Network… · p. 3

Prior-Data Fitted Networks

  • GAS-PFN addresses the graph-topology gap in forecasting PFNs by co-designing graph-structured synthetic tasks and graph-aware in-context inference.PFN synthetic priors are an explicit means of introducing domain-specific assumptions.Source: Prior-Data Fitted Networks · p. 3

Method

GAS-PFN combines graph-coupled synthetic forecasting tasks with an inference architecture that uses graph structure and context without target-dataset parameter updates.

Method

  • GAS-PFN combines a graph-aware synthetic task prior with an inference architecture that exploits graph-structured context.The pipeline exposes the model to traffic-like temporal dynamics and graph-coupled interactions, then infers them from context at test time.Source: Method · p. 4

Problem Setting and PFN Inference

  • Zero-shot inference keeps θ fixed on the target dataset, while node-local patch tokens keep dimensionality independent of graph size.The model conditions on context examples and a query patch whose target is unobserved; context may come from the target dataset’s chronological training split.Source: Problem Setting and PFN Inference · p. 4

Spatiotemporal Prior

  • Synthetic tasks combine compositional temporal processes with connected subgraphs sampled from real road networks.The temporal base composes trend, seasonal, and stochastic components before graph-structured spatial perturbations extend it to multivariate traffic processes.Source: Spatiotemporal Prior · p. 4
  • Graph propagation, node-specific gains, and filtered noise create graph-correlated deviations while retaining shared temporal behavior.The lazy random-walk operator locally smooths signals while retaining part of their original value.Source: Spatiotemporal Prior · pp. 4–5

Graph-Aware ICL Architecture

  • Each node’s mixed context includes K recent windows and K older anchors, with recent windows time-aligned across sensors but historical anchors sampled independently.The reported setting uses K = 16, or 32 historical support patches per node.Source: Graph-Aware ICL Architecture · p. 5
  • A shared Transformer uses a self-channel mask for sensor-local temporal dynamics and a neighborhood mask for cross-sensor interactions, then a learned gate fuses the query representations.The multi-channel neighborhood combines graph connectivity with semantic similarity; auxiliary predictions keep each branch individually predictive.Source: Graph-Aware ICL Architecture · p. 5

Results

The evaluation uses six traffic benchmarks with chronological splits and reports cumulative MAE and MSE at 30-, 60-, and 90-minute horizons.

Datasets and metrics

  • Six five-minute traffic benchmarks cover speed and flow measurements across road networks of different sizes and geographic settings.METR-LA and PEMS-BAY provide speed data; PEMS03, PEMS04, PEMS07, and PEMS08 provide flow data.Source: Results · p. 6
  • Chronological 70%/10%/20% train, validation, and test splits are used across the benchmarks.Zero-valued targets are masked when computing training loss and evaluation metrics because zero commonly marks missing values in these datasets.Source: Results · p. 6
  • Cumulative MAE and MSE are reported at 30-, 60-, and 90-minute horizons, with MSE placing greater weight on large deviations.The horizons correspond to 6, 12, and 18 five-minute steps.Source: Results · p. 6

Appendix C.

GAS-PFN performs strongly across zero-shot traffic benchmarks and remains competitive when supervised target-road coverage is limited. Its synthetic prior combines temporal signals with sampled road graphs, while graph diffusion and mixed-context inference support forecasting across network settings.

Appendix C.

  • 34.0% lower average MSE than TimePFN and 27.7% lower average MSE than four general-purpose zero-shot forecasters; GAS-PFN also leads in 27 of 36 comparisons.At 90 minutes, it has the best MAE and MSE on five of six datasets; the benchmark comparison uses the better-performing prespecified context regime for each baseline.Source: Appendix C · p. 6
  • GAS-PFN remains stronger than fine-tuned TimePFN across every dataset and horizon with matched 32-patch historical-data budgets, and in most settings when TimePFN receives 64 patches.GAS-PFN receives no parameter updates in these comparisons; ablations also provide consistent evidence for contributions from the dual-branch architecture, mixed-context support, and hybrid neighborhood routing.Source: Appendix C · p. 7
  • Increasing context from none to K = 16 recent and K = 16 historical patches per node reduces 90-minute MAE by 25.4% on average.Beyond 32 total support patches per node, gains become smaller and dataset-dependent; reductions are larger on PEMS03, PEMS04, PEMS07, and PEMS08 than on METR-LA and PEMS-BAY.Source: Appendix C · p. 7, Figure 4 · p. 7
  • With SCPT trained on 10% of roads, GAS-PFN achieves lower MAE and MSE on all six datasets and reduces average MSE by 34.7%.It remains better on five of six datasets at both 20% and 30% road coverage, while SCPT generally improves as supervised coverage increases.Source: Appendix C · p. 7

A Additional Synthetic Prior Details

Temporal Base Generator

  • The temporal base composes trend, seasonal, and stochastic components, with additive or multiplicative composition followed by trend and stochastic factors.Its synthetic coordinate is measured in nominal days for seasonal phases, not as a real calendar timestamp; the resulting series are not normalized, clipped, or constrained to be nonnegative.Source: Temporal Base Generator · pp. 9–10

Road-Graph Bank and Subgraph Sampling

  • The graph bank uses OpenStreetMap connectivity and direction from 20 U.S. metropolitan areas, without benchmark traffic measurements or road attributes as diffusion weights.Each shard uses a sampled connected subgraph of 24, 36, 48, or 60 nodes, fixed across its 20,000 multivariate series.Source: Road-Graph Bank and Subgraph Sampling · p. 11

Graph-Structured Multivariate Process

  • Deterministic signal propagation uses the undirected topology, while directed adjacency contributes to graph-correlated process noise.The reported subgraphs receive explicit self-loops when the identity matrix is added; both directed and undirected random-walk matrices are row-stochastic.Source: Graph-Structured Multivariate Process · pp. 11–12

B Additional Method Details

The appendix details how GAS-PFN constructs synthetic tasks and contexts, routes graph-informed neighborhoods, constrains attention, and fuses branch representations for forecasting.

Pretraining Task Construction

  • Pretraining tasks can omit graph nodes, retain duplicate node–endpoint draws, and enforce a temporal split between context targets and query lookbacks.Overlapping windows and exact duplicate patches are permitted.Source: Pretraining Task Construction · p. 12

Tokenization and Sequence Assembly

  • Token metadata supplies node, endpoint, and context/query role for masks and routing, while separate affine projections embed context and query patches.The architecture adds no node, timestamp, context/query-type, or absolute-position embeddings.Source: Tokenization and Sequence Assembly · pp. 12–13

Context Construction

  • At target-dataset evaluation, each node receives 16 recent and 16 historical raw support patches, with the context reused across test queries and checkpoints.Pretraining instead caps context at 16 tokens per represented sensor, including recent patches, older patches, and pooled summaries.Source: Context Construction · p. 13

Graph-Representation Neighborhood Routing

  • The hard neighborhood router combines directed graph scores with content-derived similarity, then converts TopK selections into a Boolean attention mask.Routing scores do not receive forecasting-loss gradients, but learned patch embeddings keep the selected neighborhoods content-dependent.Source: Graph-Representation Neighborhood… · pp. 13–14

Exact Attention Constraints

  • The self branch attends only to causal context from the same node, while the multi-channel evaluation branch permits query attention among selected nodes at the same forecast origin.Context-to-query attention is blocked in both branches.Source: Exact Attention Constraints · p. 14

Fusion and Pretraining Objective

  • A scalar gate combines the two branch representations, and auxiliary branch losses plus a negative gate-entropy term shape pretraining.At inference, only the fused prediction is used.Source: Fusion and Pretraining Objective · pp. 14–15

Datasets, Splits, and Preprocessing

The evaluation uses retained benchmark data with chronological splits and sensor-specific scaling, alongside each dataset’s stored graph input.

  • Benchmark preprocessing retains loaded sensors and time points without interpolation, resampling, gap filling, or missing-value imputation, and supplies no calendar or timestamp features.For the PEMS datasets, feature index zero is used.Source: Datasets, Splits, and Preprocessing · p. 15
  • Chronological training, validation, and test intervals are contiguous, with windows constructed separately and the first validation and test lookbacks taken from the preceding 18 observations.Source: Datasets, Splits, and Preprocessing · p. 15
  • GAS-PFN fits a separate standardizer per sensor using training rows, then inverse-transforms predictions and targets before metric computation.Benchmark data do not use the synthetic task-level robust scaling or output clipping.Source: Datasets, Splits, and Preprocessing · p. 15

Evaluation Metrics

Evaluation computes MAE and MSE over pooled test errors, with exact-zero masking applied to raw ground truth.

  • MAE and MSE pool errors across valid test windows, forecast steps, and sensors after applying the exact-zero comparison to raw ground truth.Source: Evaluation Metrics · p. 16
  • Cross-dataset averages are unweighted arithmetic means of the six dataset-level metrics.Source: Evaluation Metrics · p. 16

GAS-PFN Configuration and Evaluation

The evaluation uses independently pretrained GAS-PFN checkpoints and fixed target-dataset support, then compares zero-shot baselines and additional supervised or fine-tuned protocols.

GAS-PFN Configuration and Evaluation

  • Three GAS-PFN checkpoints are independently pretrained for one million optimizer steps, and evaluation uses 16 aligned recent plus 16 sensor-specific historical supports per node.The frozen synthetic bank is shared across checkpoint runs, and no pooled memory tokens are used at target evaluation.Source: GAS-PFN Configuration and Evaluation · p. 16

Zero-Shot Baseline Configuration

  • Zero-shot baselines are assessed with recent-only and history-augmented context, with the lower-18-step-MAE regime selected once per baseline and dataset for all reported metrics and horizons.GAS-PFN uses one fixed 32-patch support configuration.Source: Zero-Shot Baseline Configuration · p. 16

Additional Comparison Protocols

  • The TimePFN fine-tuning comparison uses labeled-window budgets of 32 and 64 on five datasets, excluding PEMS07 because its registered configuration exceeded available GPU memory.Source: Additional Comparison Protocols · p. 17
  • The selected-road comparison evaluates SCPT with 10%–80% of roads available for training, reserving 10% for validation and the remainder as unseen test roads.Results average three paired model and road-partition seeds.Source: Additional Comparison Protocols · p. 17

Additional Zero-shot Results

The expanded comparisons show GAS-PFN remains competitive with target-specific fine-tuning and supervised forecasting, particularly when supervised road coverage is limited. They also show that adding more contiguous history does not uniformly improve zero-shot results.

Additional Zero-shot Results

  • Adding a contiguous training-tail prefix has model- and dataset-dependent effects, while GAS-PFN uses fixed graph-aware mixed context.GAS-PFN's total is 1,170 scalar input slots: 18 recent values plus 32 support patches of 36 scalar values each.Source: Additional Zero-shot Results · p. 18

TimePFN Fine-Tuning Comparison

  • GAS-PFN has the best mean in 27 of 30 comparisons; 64-patch TimePFN fine-tuning wins 90-minute MSE only on PEMS03, PEMS04, and PEMS08.The comparisons cover five datasets and three horizons; GAS-PFN uses fixed mixed context without target-specific parameter updates.Source: TimePFN Fine-Tuning Comparison · p. 18

Full SCPT Comparison

  • At 60 minutes, GAS-PFN beats SCPT12 on all six datasets at 10% road coverage and five of six at 20% and 30%.The additional horizons show the same broad pattern, while SCPT12 generally becomes more competitive as supervised road coverage increases.Source: Full SCPT Comparison · p. 18

E Ablation Tests

Frozen-checkpoint ablations assess branch fusion, historical support, and neighborhood routing. The full learned mixture performs strongly across settings, and removing historical supports consistently degrades results.

E Ablation Tests

  • The full learned mixture has the best mean in 23 of 36 settings and ranks in the top two in 35; it beats both individual experts in 34.Either learned or fixed equal fusion is best in 32 settings, supporting complementary self-channel and multi-channel representations.Source: E Ablation Tests · p. 18, Table 7 · p. 25
  • Removing historical supports degrades performance in all 36 dataset–horizon–metric settings.These frozen-checkpoint variants isolate the inference-time contribution of each component.Source: E Ablation Tests · p. 18, Table 7 · p. 25
  • Hybrid neighborhood routing outperforms graph-only routing in 33 settings, though gains are generally smaller and dataset-dependent.The ablations span six datasets and cumulative 30-, 60-, and 90-minute horizons.Source: E Ablation Tests · p. 18, Table 7 · p. 25
Loading 2610.37598v1…