PaperlayerView exact revision on arXiv

Source-linked AI summary

More Sensors Only One Field: Rethinking Continual Spatio-Temporal Forecasting

Lewei Xie, Haoyu Zhang, Jiajun Zhou, Yulong Chen, Guanxing Chen, Yu-An Huang, Hau-San Wong, Yifan Zhang, Zhi-An Huang

arXiv:2609.31325v1cs.LG

TL;DR

Continual forecasting must accommodate changing sensor layouts and evolving dynamics, while conventional graph-based methods tie representations to the current sensor layout. STFO uses a shared latent field with coordinate-based observation and query interfaces and drift-adaptive evolution; it achieves leading accuracy on the traffic streams and competitive AIR-Stream performance.

  • Problem

    Graph-based continual forecasting ties spatial representations to the current sensor layout, complicating reuse of forecasting knowledge as layouts and dynamics evolve.

    Source: §1 · pp. 1–2

  • Method

    STFO lifts sensor histories onto a shared latent grid, evolves the field with spectrally conditioned Fourier and attention updates, and queries forecasts at sensor coordinates.

    Source: §1 · p. 2, §3.2 · p. 4, §3.4 · p. 5

  • Results

    STFO-Large reduces average MAE and RMSE relative to DOL by about 8% on PEMS-Stream and 5% on CA-Stream, with leading traffic-stream accuracy and competitive AIR-Stream performance.

    Source: §4.2 · p. 7, §5 · p. 10

  • Takeaways & Limitations

    The shared latent field is supported as an effective representation for continual forecasting under evolving sensor layouts and dynamics.

    Source: §5 · p. 10

    Consistency guarantees are scoped to fixed coordinate representations and neural maps, noiseless field histories, admissible sampling refinement, and a fixed spectral band partition across grid resolutions.

    Source: §2 · p. 3, §3.2 · p. 4

Abstract

from arXiv · show

Continual spatio-temporal forecasting supports traffic management and environmental monitoring under evolving dynamics and expanding sensor networks. However, conventional graph-based continual learning methods tie forecasting representations to the current sensor layout, so sensor expansion can alter the representation of learned spatial relationships. Our key insight is that sensor expansion changes the evidence available about a process without necessarily changing the dynamics to be learned. We propose STFO (Spatio-Temporal Field Operator), which parameterizes forecasting knowledge as a shared field-evolution operator and handles changing sensor layouts through observation and query interfaces. Normalized coordinate-based aggregation lifts irregular sensor histories onto a fixed latent grid, enabling reuse of learned spatial maps across observation sets without sensor-specific parameters. To accommodate process drift, a spectral descriptor summarizes variation across spatial scales and conditions Fourier propagation and attention to adapt operator responses to the current spatial regime. Coordinate-based decoding queries the evolved field at sensor locations and combines spatial corrections with local-history predictions. Experiments on PEMS-Stream, CA-Stream, and AIR-Stream demonstrate state-of-the-art average forecasting performance. STFO-Large reduces average MAE over DOL by 8.4% on PEMS-Stream and 4.7% on CA-Stream. Our code is available at https://github.com/Xielewei/Spatio-Temporal-Field-Operator.

1 INTRODUCTION

Continual forecasting must reuse knowledge as sensor layouts and underlying dynamics evolve. STFO separates the shared spatial field from the changing observation layout while adapting field evolution to spatial drift.

1 INTRODUCTION

  • Graph-based methods define spatial computation over the current sensor set, coupling layout changes with adaptation to underlying dynamics.The proposed separation aims to keep forecasting knowledge reusable when layouts change but spatial dynamics remain similar.Source: §1 · pp. 1–2
  • On CA-Stream, expansion from 480 to 1,698 sensors reconfigures up to 72% of kNN edges, although spatial relationships need not change comparably.Figure 1 contrasts edge reconfiguration with pairwise spatial correlation.Source: §1 · p. 2, Figure 1 · p. 2
  • STFO maps changing sensor observations onto a fixed latent grid and decodes forecasts at sensor coordinates without sensor-specific parameters.Its spectral descriptor conditions Fourier propagation and state-dependent attention to adapt field evolution to spatial drift.Source: §1 · p. 2
  • The paper establishes sensor-sampling consistency under explicit assumptions and latent-grid consistency for a fixed spectral design, and separately quantifies coordinate reassignment effects.It also reports validation on three real-world datasets with substantial accuracy improvements over conventional graph-based continual learning methods.Source: §1 · p. 2
  • Figure 2 presents the continual forecasting setting and STFO pipeline, with lower panels detailing CFE, SRE, DFO, and CQD.The stages aggregate observations, summarize the spectral regime, evolve the field, and decode forecasts at sensor locations.Source: §1 · p. 2, Figure 2 · p. 3

2 PROBLEM FORMULATION

The formulation represents continual forecasting across periods with expanding, nested sensor sets. Sensors provide coordinate-indexed histories, and predictions are queried at sensor coordinates; consistency is defined under specified sampling conditions.

2 PROBLEM FORMULATION

  • Continual forecasting proceeds across periods with nested sensor sets, where each period updates parameters learned in the previous period.The sensor count is denoted by Np for period p.Source: §2 · p. 3
  • Each sensor receives coordinates in the unit square through a period-dependent mapping from static descriptors, and its history is treated as noisy field observations.The same coordinate mapping is used for encoding and querying within each period.Source: §2 · p. 3
  • For an observed query sensor, the forecasting operator uses its coordinate and history to estimate future field values at that location.The output contains estimates for the forecast horizon.Source: §2 · p. 3
  • Consistency is defined for fixed coordinates and neural maps with a noiseless field history, as sensor layouts refine under an admissible sampling condition.The formulation also references bounds for latent-grid refinement and separately analyzes coordinate reassignment.Source: §2 · p. 3

3 METHODOLOGY

STFO separates the shared forecasting field from the sensor layout by lifting histories to a fixed latent grid, evolving that field, and querying it at sensor coordinates. A spectral descriptor conditions field evolution, while the continual update reuses fixed-shape parameters across periods.

3.1 COORDINATE FIELD ENCODER

  • Coordinate-based aggregation lifts sensor histories onto a fixed latent grid, where shared maps retain a fixed G × d state as sensor layouts change.The encoder uses normalized Gaussian weights and shared position features; sensor count changes the aggregation input, not the latent state shape.Source: §3 · p. 3, §3.1 · p. 4

3.2 SPECTRAL REGIME ENCODER

  • The spectral regime encoder summarizes spatial variation through frequency centroid, spectral slope, and band-energy ratios.The descriptor captures how energy is distributed across spatial scales; its positive stabilizers support continuity under latent-field perturbations.Source: §3.2 · p. 4

3.3 DUAL-PATH FIELD OPERATOR

  • Each operator block conditions latent channels on the spectral descriptor, then combines a residual path, global Fourier propagation, and field-dependent attention.The Fourier path propagates broad spatial structure through retained modes, while attention adapts spatial interactions to current field features.Source: §3.3 · pp. 4–5

3.4 COORDINATE QUERY DECODER

  • The coordinate query decoder gathers evolved-field features at sensor locations and combines them with local-history predictions.The local-history path can express sensor-specific temporal persistence, while the field path supplies spatially informed corrections.Source: §3.4 · p. 5

3.5 CONTINUAL UPDATE AND CONSISTENCY

  • Across periods, STFO initializes from the previous model and fine-tunes shared parameters while keeping the latent grid and parameter shapes fixed.The consistency analysis bounds prediction discrepancy across sampling measures under stated bandwidth assumptions and gives O(hg) grid error for a fixed spectral design.Source: §3.5 · p. 6

4 EXPERIMENTS

Experiments evaluate STFO across expanding traffic and air-quality sensor networks, testing forecasting accuracy, component contributions, learned field patterns, and adaptation to new sensors. The results include strong traffic-stream performance and evidence of transfer to newly introduced sensors.

4 EXPERIMENTS

  • The experiments pose questions about accuracy, component contributions, efficiency, field structure, new-sensor adaptation, and sensitivity to operator design.The sensitivity study covers operator depth, grid resolution, and retained modes in Appendix I.Source: §4 · p. 6

4.1 EXPERIMENTAL SETUP

  • Experiments cover PEMS-Stream, CA-Stream, and AIR-Stream, with expanding sensor counts and continual fine-tuning from the previous-period checkpoint.PEMS-Stream grows from 655 to 871 sensors, CA-Stream from 480 to 1,698, and AIR-Stream from 1,087 to 1,202; main results average three seeded runs.Source: §4.1 · p. 6

4.2 ALL-HORIZON FORECASTING RESULTS (RQ1)

  • STFO-Large has the lowest errors across reported horizons and metrics on both traffic streams, reducing average MAE and RMSE versus DOL by about 8% on PEMS-Stream and 5% on CA-Stream.Relative gains over DOL increase with prediction horizon and are largest at horizon 12; corrected Wilcoxon tests support the overall forecasting advantage.Source: §4.2 · p. 7, Table 1 · p. 7

4.3 ABLATION STUDY (RQ2)

  • Figure 3 compares STFO-Large with variants removing spectral conditioning, either DFO path, or coordinate-feature injection, using single-seed runs.The ablations are reported for PEMS-Stream and AIR-Stream; the supplied passages do not state their outcomes.Source: §4.3 · p. 7, Figure 3 · p. 7

4.5 MECHANISM DIAGNOSTICS (RQ4)

  • On PEMS-Stream, field gains remain broadly distributed while updates show later-period local peaks; CA-Stream gains shift spatially as update magnitudes tend to decrease.Across the streams, field contributions improve predictions at many sensor locations, and AIR-Stream gains and updates show spatial structure that varies across years.Source: §4.5 · pp. 8–9
  • SRE descriptor components correlate positively and negatively with FiLM channel parameters, while channel configurations and their gain associations differ across datasets and periods.PEMS-Stream and CA-Stream show dispersed channel configurations with mixed gain correlations, whereas AIR-Stream shows tighter within-period clusters with similar correlations.Source: §4.5 · p. 9

4.6 NEW-SENSOR ADAPTATION (RQ5)

  • On AIR-Stream, STFO-Large has the lowest MAE in every existing-sensor comparison and most new-sensor comparisons under zero-shot and full adaptation.Full adaptation further reduces MAE for both new and existing sensors in every expansion period.Source: §4.6 · p. 9, Table 2 · p. 10

5 CONCLUSION

STFO combines coordinate-based lifting and querying with spectrally conditioned Fourier and attention updates in a shared latent field. The conclusion presents this formulation as supporting continual forecasting under changing sensor layouts and dynamics.

5 CONCLUSION

  • STFO combines coordinate-based lifting and querying with spectrally conditioned Fourier and attention updates in a shared latent field.The conclusion reports leading accuracy on both traffic streams and competitive performance on AIR-Stream.Source: §5 · p. 10

A RELATED WORK

  • Related work spans graph and non-graph forecasting, continual adaptation, neural operators, and coordinate-based representations, while this paper focuses on sensor discretization under distribution drift.The cited work includes approaches that organize spatial computation around observed sensors or variables.Source: A RELATED WORK · pp. 15–16

B NOTATION AND SYMBOLS

  • Table 3 summarizes the paper’s symbols and notation conventions, including typography for scalars, vectors, matrices, and selected sets or operators.Proofs use dark blue for continuum or reference objects and dark orange for discrete or empirical objects.Source: B NOTATION AND SYMBOLS · p. 16, Table 3 · p. 16

C FULL PROOFS

The proofs analyze the lift–condition–evolve–query construction under fixed maps and coordinates, treating sensor-sampling error, operator propagation, and grid refinement separately. Their sampling comparisons hold the field history, trained maps, latent grid, and common query history fixed.

C.1 OPERATOR CONVENTIONS AND ASSUMPTIONS

  • Sampling assumptions cover fixed bandwidth with convergence to a common sensor measure and local bandwidth proportional to fill distance under separation conditions.The code field and maps are assumed bounded or Lipschitz, with positive stabilizers for normalization and spectral quantities.Source: §C.1 · pp. 18–19

C.2 NORMALIZED KERNEL LIFTING AND GATHERING

  • The normalized Gaussian lift has a fixed-bandwidth error controlled by the distance between empirical and limiting sensor measures.The local-bandwidth regime instead uses a pointwise code-field reference, with interpolation neighborhoods shrinking alongside fill distance.Source: §C.1 · p. 18, §C.2 · pp. 19–20
  • Normalized Gaussian gathering is nonexpansive, and its grid-to-continuum error depends on the decoder bandwidth and grid spacing.The post-encoding map propagates lift error through its Lipschitz constant.Source: §C.2 · p. 20

C.3 STABILITY OF THE TRAINED FIELD OPERATOR

  • The attention map has a Lipschitz constant independent of grid size, and a finite operator stack remains stable under the stated assumptions.Each block uses the initial spectral descriptor; for fixed spectral design, the stability constants can be uniform over sufficiently fine grids.Source: §C.3 · pp. 21–22
  • Fourier propagation is bounded by the retained mode multipliers, while residual normalization and LayerNorm contribute to the stack’s stability bounds.The descriptor conditions each block’s modulation, and the decoder gather is nonexpansive.Source: §C.3 · pp. 21–22

C.4 SENSOR-DISCRETIZATION CONSISTENCY

The sensor-discretization analysis compares layouts under fixed coordinates, histories, trained maps, latent grid, and query. Under its sampling assumptions, layouts sharing a reference have convergent predictions; coordinate reassignment is analyzed separately.

C.4 SENSOR-DISCRETIZATION CONSISTENCY

  • Predictions converge to a sampling-independent reference under the stated sampling assumptions, with fixed-bandwidth error O(W1(νS, µ)) and local-bandwidth error O(hS).The reference uses the same trained maps, common query, and reference field across layouts in the sampling family.Source: §C.4 · pp. 22–23
  • For two layouts sharing a sampling regime and reference field, expansion stability bounds their prediction difference by the sum of their lift errors.For nested layouts in the local-bandwidth regime, the finer layout has fill distance no greater than the coarser layout’s.Source: §C.4 · p. 23
  • The analysis separates sensor-sampling contributions from coordinate-motion contributions when decomposing expansion error.Coordinate reassignment is treated for the same sensor identities with histories, bandwidths, neural maps, and grid fixed.Source: §C.4 · p. 23

C.5 FIXED-DESIGN GRID-TO-CONTINUUM CONSISTENCY

The fixed-design analysis compares a finite grid operator with its continuum counterpart while keeping the spectral design fixed. It establishes consistency through bounds on quadrature, Fourier, conditioning, attention, and propagation errors.

C.5 FIXED-DESIGN GRID-TO-CONTINUUM CONSISTENCY

  • The endpoint-grid quadrature error for a Lipschitz function is bounded in terms of grid spacing.The proof compares function values at spatial nodes with their continuum integral over associated cells.Source: §C.5 · pp. 23–24
  • Fourier coefficient and evaluation-phase errors have first-order bounds for the same finite retained modes.The phase-node offset is included in the coefficient and evaluation comparison.Source: §C.5 · p. 24
  • Normalized attention integrals incur O(hg) quadrature error, and changes in conditioned input add another first-order term.The analysis propagates these errors through the operator blocks and decoder to establish fixed-design grid consistency.Source: §C.5 · pp. 24–25

C.6 PERTURBATION STABILITY OF THE SPECTRAL STATE

The stabilized spectral descriptor is designed to change continuously with power amplitude while tracking relative spectral shape. Theorem 3 bounds descriptor changes under signed power perturbations.

C.6 PERTURBATION STABILITY OF THE SPECTRAL STATE

  • For spectra P and P′, the absolute centroid difference |α(P′)−α(P)| and band-ratio difference |ηb(P′)−ηb(P)| are each bounded by η/(1−η).The spectra are on the same frequency set.Source: §C.6 · p. 25
  • The centroid and band ratios are stabilized ratios with bounded weights and positive stabilizers, so the perturbation bound covers signed per-bin power changes.For the centroid, weights are normalized frequencies; for a band ratio, they indicate membership in the band.Source: §C.6 · p. 25
  • For positive raw powers, common amplitude scaling cancels from idealized centroid and band ratios, while the centered slope retains amplitude dependence.The idealized statistics omit additive power and ratio stabilizers and keep Dβ fixed.Source: §C.6 · p. 25
  • With stabilized power E′j = a²Pj + εs, descriptor statistics depend continuously on amplitude and approach idealized statistics as power grows relative to the stabilizers.General additive field changes induce signed per-bin power perturbations through Fourier cross terms.Source: §C.6 · p. 25

D EXPERIMENTAL SETUP DETAILS

The setup fixes model scales and coordinate handling while evaluating forecasting with shared protocols. The algorithms encode a fixed-shape field operator and continual fine-tuning, and the baselines span forecasting, neural-operator, continual-learning, and online-adaptation families.

D EXPERIMENTAL SETUP DETAILS

  • Three model scales differ only in hidden width; grid size, Fourier cutoff, attention width, coordinate encoding, and local-history skip connection are specified by dataset.Operator depth is two for PEMS-Stream and CA-Stream and one for AIR-Stream; spectral-band counts also vary by dataset.Source: D EXPERIMENTAL SETUP DETAILS · p. 26
  • Sensor coordinates blend normalized geographic coordinates and reduced attributes when available, while assignment grids are reconstructed each period and the latent field grid stays fixed.Preprocessing statistics and the PCA projection come from period one; datasets without extra attributes use geographic coordinates alone.Source: D EXPERIMENTAL SETUP DETAILS · p. 26
  • The experiments use masked MAE training, early stopping, and MAE, RMSE, and MAPE evaluation under shared data splits, horizons, and masks.Training uses AdamW with gradient clipping; validation MAE controls early stopping.Source: D EXPERIMENTAL SETUP DETAILS · p. 26

F BASELINE DESCRIPTIONS

  • The baselines cover general forecasting, fixed-mesh neural operators, continual spatio-temporal learning, and online adaptation; continual baselines retain the current sensor set in their representations.STFO instead treats expansion as changing sampling locations on a fixed latent field; neural-operator baselines use fixed meshes.Source: F BASELINE DESCRIPTIONS · pp. 26–27

E ALGORITHMS

  • Algorithm 1 lifts encoded histories to a latent field, conditions spectral and attention updates on a regime descriptor, then queries the evolved field and adds local-history predictions.The algorithm also applies residual normalization between field updates and coordinate decoding.Source: F BASELINE DESCRIPTIONS · p. 27
  • Algorithm 2 initializes from scratch in period one, then fine-tunes the prior period’s selected parameters on current-period samples with validation-based checkpoint selection.It resets AdamW state for each period and stops when validation MAE no longer improves for the patience window.Source: F BASELINE DESCRIPTIONS · p. 28

G STATISTICAL SIGNIFICANCE ANALYSIS

STFO-Large is compared with 13 baselines using corrected one-sided paired tests; reported adjusted p-values are below 0.0001 except for MAPE against DOL.

G STATISTICAL SIGNIFICANCE ANALYSIS

  • Adjusted p-values are below 0.0001 for every STFO-Large comparison except MAPE against DOL, which has pHolm < 0.01.The tests use 27 paired entries per metric with Holm correction; the authors note some entries are dependent.Source: G STATISTICAL SIGNIFICANCE ANALYSIS · p. 28

H FULL NEW-SENSOR ADAPTATION RESULTS

Under full adaptation, STFO-Large achieves the lowest errors across periods, horizons, and sensor groups on both traffic streams, and leads broadly on AIR-Stream as well. Results are reported across existing and newly introduced sensors, with zero-shot evaluation also described.

H FULL NEW-SENSOR ADAPTATION RESULTS

I HYPERPARAMETER SENSITIVITY

Hyperparameter sensitivity tests show that STFO works well with shallow operator stacks, while preferred grid resolutions and Fourier bandwidths differ across streams.

I HYPERPARAMETER SENSITIVITY

  • Depth preferences differ by stream: three layers give the lowest average MAE on PEMS-Stream, two on CA-Stream, and one on AIR-Stream.Four layers worsen performance on all three streams relative to their best settings; increasing depth does not consistently improve forecasting.Source: I HYPERPARAMETER SENSITIVITY · p. 33
  • Increasing grid points per axis from 8 to 32 substantially reduces errors on both traffic streams, especially at horizon 12.PEMS-Stream changes little beyond 32; CA-Stream continues to improve with smaller gains, while AIR-Stream benefits mainly from increasing from 8 to 16.Source: I HYPERPARAMETER SENSITIVITY · p. 33
  • Lowest average MAE occurs at M = 4 on PEMS-Stream, M = 6 on CA-Stream, and M = 2 on AIR-Stream.CA-Stream is most sensitive to the cutoff; PEMS-Stream varies little, while AIR-Stream error increases slightly as M increases.Source: I HYPERPARAMETER SENSITIVITY · p. 33
  • Across streams, STFO performs well with shallow operator stacks, while suitable grid resolutions and spectral bandwidths vary by task.Increasing hyperparameters beyond suitable settings yields limited gains or degrades performance.Source: I HYPERPARAMETER SENSITIVITY · p. 33
Loading 2609.31325v1…