PaperlayerView exact revision on arXiv

Source-linked AI summary

TRACE: Retrospective Streaming Generation of Physical Fields under Sparse Structured Sensing

Xinyu Zhang, Lihao Chen, Panqi Chen, Lei Cheng, Ting Zhang, Jianlong Li, Shikai Fang

arXiv:2608.26219v1stat.MLcs.LG

TL;DR

Sparse physical-field sensing is often structured as a stream of localized, off-grid observations or missing frames, while existing generative reconstructors largely assume fixed batch conditioning. TRACE performs approximate Bayesian inference in a continuous-coordinate latent space, fusing generative evidence with a state-space temporal prior through filtering and retrospectively smoothing past frames. Across active matter, ocean, and supernova benchmarks, it achieves the strongest overall performance, with largest gains under rapidly evolving dynamics and repeated spatial coverage.

  • Problem

    Existing generative reconstructors largely assume fixed batch evidence, whereas real sensing streams can contain localized off-grid observations and entirely missing frames.

  • Method

    TRACE converts sparse observations into generative latent evidence, fuses it with a state-space temporal prior through filtering, and applies retrospective smoothing.

  • Results

    TRACE achieves the strongest overall performance across active matter, ocean, and supernova benchmarks, with largest gains under rapidly evolving dynamics and repeated spatial coverage.

  • Takeaways & Limitations

    TRACE provides causal online estimates while later observations retrospectively refine past under-observed frames under structured sensing.

  • Takeaways & Limitations

    The temporal prior uses independent Matérn-3/2 state-space Gaussian processes for each latent dimension, with shared hyperparameters.

Abstract

from arXiv · show

Reconstructing continuous physical fields from sparse measurements is central to scientific monitoring, inverse modeling, and digital-twin construction. Generative reconstruction has recently emerged as a promising paradigm for this task by learning data-driven physical priors that complete plausible full fields from limited observations. However, existing methods largely assume fixed, batch conditioning, whereas real sensing systems often produce structured streams: probes scan local regions, instruments observe moving fields of view, and communication constraints may leave entire frames missing. We propose TRACE, a retrospective streaming generative reconstruction framework for physical fields under structured sensing. TRACE performs approximate Bayesian inference in a learned continuous-coordinate latent space, converting sparse off-grid measurements into generative latent evidence, fusing it with a state-space temporal prior through Kalman-style filtering, and refining under-observed past frames via retrospective smoothing. Experiments on active matter, ocean sound-speed fields, and supernova simulations show that TRACE matches or surpasses frame-wise generative reconstructors, offline spatiotemporal methods, and streaming data-assimilation baselines in reconstruction quality under temporally sparse and spatially localized sensing protocols.

1 Introduction

Sparse physical-field reconstruction is ill-posed because sensors provide irregular, off-grid observations, while existing generative methods largely assume fixed batch evidence. TRACE addresses structured sensing streams by combining continuous-coordinate latent generation with temporal filtering and retrospective smoothing.

  • Motivation: Sparse measurements must recover continuous fields despite irregular, off-grid readings and many full fields explaining the same observations.Generative priors turn reconstruction into conditional generation constrained by physical and observational information.
  • Research gap: Existing generative reconstructors generally assume observations are available before inference for one frame or a complete trajectory.This fixed batch view does not match mobile probes, moving field-of-view instruments, or missing frames.
  • Approach: TRACE represents each incoming frame as latent evidence in a continuous-coordinate space and fuses it with a state-space temporal prior.The framework uses a functional Tucker decoder and generative posterior sampling before temporal fusion.
  • Approach: Retrospective smoothing uses later measurements to refine frames that were missing, localized, or otherwise under-observed.TRACE separates causal online estimates from refined offline corrections.
  • Evaluation: TRACE is evaluated on structured sensing protocols spanning temporally sparse frames and spatially localized moving-window observations.The reported comparisons include frame-wise generative, offline spatiotemporal, and streaming data-assimilation methods.

2 Preliminaries

The preliminaries establish continuous low-rank field representations, diffusion-based conditional latent generation, and state-space Gaussian-process inference. Together, these components provide the representational, generative, and temporal foundations used by TRACE.

  • Continuous low-rank representations: Naive grid discretization requires O(I^K) degrees of freedom, whereas coordinate-based low-rank tensor representations provide compact continuous field models.The representation separates coordinate decoding from compact latent coefficients and supports arbitrary off-grid queries.
  • Continuous low-rank representations: Functional tensor representations use coordinate-dependent basis functions and a compact coefficient tensor to capture multilinear interactions across modes.This preserves a unified latent representation across different off-grid sensing layouts.
  • Diffusion priors: Diffusion models learn latent generative priors through denoising, while Diffusion Posterior Sampling adds measurement guidance during reverse diffusion.The guided score combines the learned prior with a forward measurement model and guidance strength λ.
  • State-space Gaussian processes: Gaussian-process inference scales cubically with trajectory length in its exact covariance formulation, motivating a state-space reformulation.Half-integer Matérn kernels admit finite-dimensional linear stochastic dynamical systems.
  • State-space Gaussian processes: Discretized Matérn state-space models yield linear-Gaussian Gauss–Markov transitions at arbitrary timestamps.Kalman filtering supports causal estimation and RTS smoothing supports retrospective refinement, both with O(N) complexity.
  • TRACE overview: TRACE combines offline continuous-coordinate and latent-diffusion pretraining with online Kalman filtering and retrospective smoothing.The overview distinguishes the offline learned components from streaming inference.

3 Method

TRACE converts irregular streaming observations into inference over a compact latent trajectory. Generative latent evidence is fused with a Matérn state-space prior by forward filtering, then future observations refine earlier latent states through smoothing.

  • Problem formulation: The problem involves noisy, sparse, off-grid measurements arriving sequentially over frame-dependent regions, including entirely unobserved timestamps.The goal is continuous field estimation at arbitrary spatiotemporal queries while permitting retrospective refinement.
  • Latent reformulation: Direct inference in physical-function space is difficult because observation layouts vary, individual frames are ill-posed, and recursive high-dimensional computation is prohibitive.These constraints motivate inference in a compact latent trajectory.
  • Latent representation: TRACE uses a Functional Tucker Model with shared continuous basis functions and frame-specific Tucker cores, enabling arbitrary off-grid decoding.Vectorized Tucker cores form fixed-dimensional latent vectors for subsequent inference.
  • Filtering and smoothing: TRACE recursively fuses per-frame generative evidence with a temporal belief through causal forward filtering, then uses backward smoothing to incorporate future observations.The inferred latent trajectory is decoded into continuous physical fields whenever observations are assimilated or queries are made.
  • Generative latent evidence: The decoder induces a frame-specific linear measurement operator that maps each latent vector to observed values.This operator provides a common interface for sparse measurements and continuous field queries.
  • Generative latent evidence: Under-informative frames are completed using a pretrained unconditional latent diffusion prior and Diffusion Posterior Sampling.Multiple observation-consistent latent samples are summarized as Gaussian evidence by moment matching.
  • Filtering and smoothing: Each latent dimension uses an independent Matérn-3/2 state-space Gaussian process whose transitions accommodate irregular arrival intervals without retraining.A tempered product of experts controls the relative weighting of temporal and generative evidence.

4 Related Work

Prior work spans continuous low-rank representations, generative physical-field reconstruction, and streaming data assimilation. TRACE combines these directions by adding continuous off-grid support and retrospective temporal inference to streaming generative reconstruction.

  • Continuous representations: Functional Tucker and related coordinate-based decompositions support continuous off-grid reconstruction through separable basis functions and compact latent structure.These methods provide the representational foundation for field modeling.
  • Generative reconstruction: Diffusion-based physical-field methods use learned priors and measurement guidance to generate observation-consistent reconstructions.Neural-field latent diffusion can synthesize coherent spatiotemporal turbulence from instantaneous sparse measurements.
  • Evaluation context: Table 1 evaluates field-domain RMSE under temporally sparse and moving-window sensing, with lower values preferred and unavailable per-frame cells marked by dashes.The protocols include Miss, contiguous blackouts, S-curves, and circular sweeps.
  • Generative reconstruction: SDIFT combines a functional Tucker latent with diffusion and batch Gaussian-process inference, but assumes the full observation horizon at inference.Its inter-frame consistency is enforced through batch posterior sampling.
  • Streaming inference: Streaming data assimilation enables sequential estimation through Kalman recursions and learned filtering variants.Most cited approaches assume gridded measurements and strictly causal forward filtering without retrospective refinement.
  • TRACE positioning: TRACE bridges the three research directions by combining continuous off-grid field representations, latent generative priors, and retrospective temporal inference.Its novelty is the integration of these components for streaming reconstruction.

5 Experiments

TRACE is evaluated on three physical-field benchmarks under temporally sparse and spatially localized sensing, with comparisons spanning offline, streaming, and temporal-inference variants. Retrospective smoothing provides the strongest gains for rapidly evolving dynamics and repeated spatial coverage.

  • Datasets: TRACE is evaluated on Active Matter, Ocean, and Supernova benchmarks spanning distinct physical systems, temporal dynamics, and spatial dimensionalities.Active Matter models rapidly evolving rod-like particle dynamics, Ocean captures moderately varying sound-speed fields, and Supernova represents comparatively slow astrophysical evolution.
  • Structured sensing protocols: The experiments test temporally sparse frames and moving local windows, requiring information to propagate across missing intervals or accumulate across partial spatial coverage.Temporally sparse sensing includes repeating missing-frame cycles and contiguous blackouts; localized sensing uses moving windows with no single-frame global coverage.
  • Temporally sparse observations: TRACE-Smoother consistently outperforms TRACE-Frame and TRACE-Filter, with the largest gains on rapidly evolving Active Matter and across temporal gaps.The RTS backward pass integrates observations arriving after missing intervals, while the causal filter degrades as gaps lengthen and dynamics accelerate.
  • Spatially localized observations: TRACE-Frame surpasses the strongest offline methods on Active Matter and Ocean, while TRACE-Smoother overtakes MMGN on Supernova after three moving-window revisits.More frequent revisits provide richer temporal evidence and lower reconstruction error as accumulated coverage compensates for limited per-frame observability.
  • Regime-structured analysis: As the stream unfolds, TRACE-Smoother can surpass offline batch methods before the complete horizon is observed, and its advantage over causal filtering widens with more severe temporal gaps.On Supernova, TRACE-Smoother also improves with revisit frequency because the SS-GP prior accumulates information across revisits to fill unobserved regions.
  • Additional analyses: Additional analyses examine dense-time random sparse controls, observation sparsity, sensor noise, hyperparameters, runtime, and qualitative examples.These analyses are reported in the appendix across all three datasets.

6 Conclusion

The paper introduces TRACE as a streaming generative framework for structured-sensing reconstruction. Across active matter, ocean, and supernova benchmarks, it achieves the strongest overall performance, especially under rapidly evolving dynamics and repeated spatial coverage.

  • Conclusion: TRACE combines continuous-coordinate latent decoding, generative posterior sampling, SS-GP temporal fusion, and RTS smoothing for physical fields under structured sensing.The framework integrates these components into a streaming reconstruction pipeline.
  • Conclusion: TRACE achieves the strongest overall performance, with its largest gains under rapidly evolving dynamics and repeated spatial coverage while remaining competitive in slowly varying regimes.The conclusion summarizes results across active matter, ocean, and supernova benchmarks.
  • Future work: Future work includes active moving-window strategies and learned physics-informed neural dynamics for latent-space field evolution from sparse observations.These directions would replace predefined trajectories or the non-parametric SS-GP prior.

A Mathematical Derivations

The derivations construct a finite-dimensional state-space representation for the Matérn prior, then connect TRACE’s tempered latent fusion to Kalman filtering and smoothing.

  • From Matérn GP to state space: The Matérn-ν Gaussian process is converted into an equivalent finite-dimensional LTI-SDE and then discretized into a Gauss–Markov chain.State augmentation collects the function and derivatives, while sampled timestamps yield discrete transition and process-noise matrices.
  • Discrete state-space construction: The stable LTI-SDE yields a unique stationary covariance through the algebraic Lyapunov equation, with discrete transitions obtained from matrix exponentials.The process-noise covariance follows from the stationary identity P∞ = AnP∞An^⊤ + Qn.
  • Matérn-3/2 specialization: For ν = 3/2, the augmented state is two-dimensional and the stationary covariance is diagonal because the function and derivative are uncorrelated.The resulting matrices provide the closed-form backbone used by the paper’s Kalman and RTS recursions.
  • Latent temporal prior: TRACE places independent Matérn-3/2 state-space Gaussian-process priors on the Tucker-core latent dimensions, producing a block-diagonal joint latent state.Each scalar latent dimension is tracked together with its time derivative, enabling per-dimension Kalman operations.
  • Tempered fusion and Kalman equivalence: At α = β = 1, tempered value-level fusion followed by the augmented-state update becomes the classical Kalman measurement update.The generative Gaussian summary qgen_n acts as a moment-matched pseudo-observation in the state-space GP regression.

B Dataset Sources and Physical Context

The paper evaluates three physical-field benchmarks spanning active matter, ocean sound-speed reanalysis, and supernova temperature evolution, alongside dataset-specific architecture settings.

  • Architecture configuration: Table 2 reports per-dataset Tucker multilinear rank R and vectorised-core latent dimension d.These configuration quantities characterize the architecture used for each dataset.
  • Physical benchmark suite: The benchmark suite covers active-nematic dynamics, layered Pacific sound-speed fields, and three-dimensional supernova temperature evolution.These datasets provide distinct physical contexts for testing continuous-field reconstruction.
  • Active Matter: Active Matter uses 256×256 scalar frames with T=24, 900 training trajectories, 28 held-out test trajectories, and a fixed 10% training mask.The data come from The Well’s active_matter collection.
  • Ocean: Ocean uses five-layer 5×38×76 frames with T=24, 950 training trajectories, 50 held-out test trajectories, and a fixed 10% training mask.The fields are derived from the HYCOM operational reanalysis product.
  • Supernova: Supernova uses 64^3 frames with T=16, 370 training trajectories, 26 held-out test trajectories, and a fixed 15% training mask.The data come from The Well’s supernova_explosion_64 collection.

C Implementation and Hyperparameters

TRACE uses a frozen continuous-coordinate latent representation and pretrained diffusion prior, then performs streaming inference with DPS-based evidence injection, SS-GP fusion, causal filtering, and RTS smoothing. Experiments use fixed inference settings and dataset-specific trajectory and sensing configurations.

  • Inference settings: Inference uses S=20 posterior samples, N=100 reverse-diffusion steps, σobs=0.05, β=1, and diagonal Gaussian fusion.The sample count is selected at the accuracy plateau of the Gaussian moment summary.
  • Representation and pretraining: TRACE represents fields with a functional Tucker basis and per-trajectory cores, then freezes the basis for latent diffusion prior learning.The basis is trained with masked off-grid reconstruction and temporal total variation before latent cores are used to train the EDM denoiser.
  • Temporal prior: TRACE fuses generative latent evidence with a Matérn-3/2 SS-GP prior, where α controls temporal-prior trust and β remains fixed at 1.The default α is 0.5, with Supernova using α=0.3 temporally and α=0.7 for the spatial moving-cube regime.
  • Evaluation configuration: Evaluation includes S-curve and circular moving-window trajectories with local density ρloc=15%, plus dataset-specific windows for Active Matter, Ocean, and Supernova.The trajectory patterns are adapted to each dataset's dimensionality and physical domain.
  • Baselines: Baselines share the frozen basis where applicable, while LRTFR, SDIFT, MMGN, and DBF use adaptations designed for off-grid sparse reconstruction.MMGN receives linearly interpolated latent codes on unobserved frames, and methods are evaluated on identical scattered masks and held-out trajectories.
  • Streaming inference: The streaming procedure propagates latent states forward, injects DPS evidence when observations exist, carries missing frames through prediction, and applies an RTS backward pass.The resulting filtering and smoothed posteriors support continuous prediction at arbitrary coordinates and times.

E.5 Streaming Filters That Failed to Reproduce

The reproduced streaming-filter comparisons expose practical failure boundaries for EnSF and LD-EnSF under the paper's high-dimensional physical-field settings. TRACE is compared against baselines using common masks and named main-text variants.

  • EnSF: EnSF frequently produces NaNs in direct field space because its ensemble cannot represent the posterior at Active Matter and Ocean dimensionalities.Increasing the ensemble from 50 to 2,000 particles does not improve RMSE beyond approximately mean-field performance.
  • EnSF: EnSF remains near mean-field RMSE despite larger ensembles: approximately 1.15 on Active Matter and 1.03 on Ocean.The passage reports 1.147→1.225 on Active Matter as the ensemble grows, and identifies the outcome as an empirical reproduction failure rather than a definitive causal diagnosis.
  • LD-EnSF: LD-EnSF cannot run its assimilation stage because the learned neural-ODE latent forward model fails to fit the data.On Active Matter, validation relative error stagnates at approximately 1.0 after 855 trajectories and remains approximately 0.71 even in a two-trajectory overfit check.
  • Comparison protocol: The main comparison names TRACE-Frame, TRACE-Filter, and TRACE-Smoother alongside LRTFR, MMGN, SDIFT, and DBF.TRACE-Frame is unavailable on unobserved frames, whereas TRACE-Filter is causal and TRACE-Smoother includes RTS smoothing.

F.2 Temporally Sparse Observations: Additional Results

Additional experiments show that TRACE-Smoother remains strongest for shorter temporal gaps and that performance is relatively insensitive to tempering across tested values. The main limitation is concentrated in longer unobserved intervals.

  • Boundary of performance: The supplementary shorter-gap and higher-density experiments confine TRACE's reported boundary to the longer unobserved intervals of the main experiments.This conclusion is stated explicitly for the additional results.
  • Shorter gaps: TRACE-Smoother attains the lowest RMSE in every Miss-1 column across all three datasets and observation densities.Miss-1 observes every other frame, with ρ∈{1%, 3%}.
  • Length-scale sensitivity: A single Matérn length-scale ℓ≈5 is near-optimal across sparse regimes, whereas full observation prefers ℓ≈1.The optimum shifts with missing level, but sparse settings share a broad plateau near ℓ≈5.
  • Tempering sensitivity: RMSE varies by under 3% as α changes from 0.3 to 1.0, with only mild degradation at α=1.5.The α=1 curve is the untempered Kalman update, and α=0.5 is fixed as the default.

F.5 Robustness: Observation Noise and Sparsity

TRACE-Smoother remains robust as measurements become noisier or sparser, while temporal accumulation and retrospective smoothing provide the largest benefits when observations are limited or frames are missing.

  • Observation noise: 0.184 at σ=0.3 is TRACE-Smoother’s Active Matter RMSE, the best result at the highest tested noise level.All variants degrade gracefully, while low-noise performance is indistinguishable.
  • Observation sparsity: RMSE decreases monotonically as the observation rate ρ increases, with TRACE-Smoother ahead of TRACE-Filter at every tested rate.The largest relative gain occurs at the sparsest setting, ρ=0.5%.
  • Coverage accumulation: TRACE-Smoother improves monotonically with sensor revisits, whereas the per-frame INR baseline is essentially flat in revisit count.It dominates at every loop on Active Matter and Ocean, and overtakes the Supernova baseline from the second revisit onward.
  • Runtime and complexity: Diagonal fusion is O(d) in compute and memory, while full-covariance fusion is O(d^3) in compute and O(d^2) in memory.The full-covariance variant exhausts memory beyond d≈5000; the RTS pass adds one O(T) recursion.
  • Missing-frame recovery: 21%: Active Matter Blk-10 RMSE falls from 0.780 to 0.612 when retrospective smoothing uses later evidence.The gain is concentrated on missing frames; observed-frame estimates differ negligibly between TRACE-Filter and TRACE-Smoother.
  • Probabilistic reliability: TRACE-Smoother’s calibration error is ECE 0.058 versus 0.220 for the per-frame ensemble.The posterior remains somewhat overconfident at higher credible levels.

F.9 Qualitative Reconstructions

Qualitative reconstructions show that TRACE-Smoother uses temporal and spatially accumulated evidence to recover fields across missing intervals and moving-window coverage, while revealing a failure boundary when local evidence is too weak.

  • Temporally sparse: blackout: TRACE-Smoother bridges contiguous blackouts by propagating evidence from both ends, whereas TRACE-Filter loses structure inside the missing interval.The Active Matter and Ocean blackouts cover frames 7–16; the Supernova blackout covers frames 3–12.
  • Spatially localized: moving window: TRACE-Frame collapses outside the current window because it lacks the accumulated spatial history used by the streaming methods.The qualitative columns compare TRACE-Smoother, TRACE-Filter, TRACE-Frame, SDIFT, MMGN, and LRTFR.
  • Failure case: When a narrow window leaves the field under-determined, TRACE-Filter propagates an erroneous estimate and TRACE-Smoother cannot recover it.This failure occurs when per-frame evidence is too weak for the generative prior to be informative.
  • Temporally sparse: missing frames: In the Miss pattern, TRACE-Smoother recovers coherent spatial structures throughout missing intervals, while offline baselines produce temporally inconsistent transitions.The Ocean comparison includes TRACE-Filter, SDIFT, MMGN, DBF, and LRTFR.
  • Spatially localized: moving window: During a circular moving-window sweep, TRACE-Smoother carries previously swept regions into a full-field estimate.The Active Matter visualization spans nine frames over one lap; per-frame baselines collapse outside the current window.
Loading 2608.26219v1…