PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative Refinement Learning for Temporal Interpolation in Scientific Machine Learning

Pu Ren, Michael W. Mahoney, N. Benjamin Erichson

arXiv:2610.36751v1cs.LGcs.AImath-phmath.MPstat.ML

TL;DR

Sparse simulation snapshots and limited observation frequency leave intermediate scientific fields unavailable, while simple averaging may misrepresent evolving structures. iFLEX combines a weighted estimate from two observations with a conditional diffusion model that learns its residual. Across three datasets it outperforms evaluated baselines, and residual learning improves on direct generation in a controlled turbulence comparison.

  • Problem

    Simulations and observations may lack the temporal resolution needed by downstream tasks, and linear averaging can misrepresent moving or changing physical structures.

    Source: §1 · p. 1, §4.1 · p. 8

  • Method

    iFLEX linearly weights two endpoint fields and uses conditional diffusion, conditioned on both observations and temporal information, to generate a residual correction.

    Source: §1 · p. 3, §4.3 · p. 11

  • Results

    Across NSKT, Shanghai radar precipitation, and SST, iFLEX achieves lower reconstruction error than evaluated baselines; at Re = 36000, residual generation yields 10.48% RFNE versus 13.73% for direct generation.

    Source: §1 · p. 3, §7 · p. 21

  • Takeaways & Limitations

    For systems represented in training data, iFLEX can estimate missing fields between recorded observations without rerunning the underlying simulation.

    Source: §7 · p. 21

    Reconstruction assumes observations are close enough to constrain relevant evolution; events occurring entirely between observations may leave no endpoint evidence.

    Source: §6 · p. 20

Abstract

from arXiv · show

Computationally expensive simulations and real-world observations often provide data at a lower temporal resolution than downstream tasks require. Averaging observations at fixed grid points can introduce errors, and motion estimates alone do not determine how the physical field values change. In this work, we study whether learning to correct a simple numerical interpolation improves the reconstruction of intermediate scientific fields. Our model, interpolative FLow EXpert (iFLEX), first estimates the missing field by taking a weighted average of the two observations at each grid point, giving more weight to the observation closer to the requested time. During training, subtracting this estimate from an observed intermediate field gives the residual that a conditional diffusion model learns to generate. Both observations condition the model, together with the target time, the interval between observations, and available physical parameters. To reconstruct an unobserved intermediate field, the generated residual is added to the weighted average. We evaluate iFLEX on Navier-Stokes Kraichnan turbulence, Shanghai radar precipitation, and sea surface temperature. Across these datasets, iFLEX achieves lower reconstruction error than the evaluated baselines, including optical-flow and diffusion models developed for video frame interpolation in computer vision. In a controlled turbulence comparison at a Reynolds number of 36,000, learning the residual reduces relative Frobenius norm error from about 13.7 for direct field generation to 10.5, with the diffusion backbone and training setup held fixed. Code is available at https://github.com/paulpuren/iflex.

1 Introduction

iFLEX reconstructs intermediate scientific fields by correcting a linear estimate with a learned residual. It achieves lower reconstruction error than the evaluated baselines across three datasets.

1 Introduction

  • Temporal interpolation addresses missing times in simulations and observations, where storage limits or measurement frequency can leave records too sparse for downstream needs.Unlike forecasting, it uses observations from both ends of the interval to constrain intermediate states.Source: §1 · pp. 1–2
  • Fixed-grid averaging can misplace or weaken moving structures, while motion estimates alone do not capture changes in their physical values.Figure 2 illustrates closer reproduction of vortex structures by iFLEX than Super SloMo in the shown examples.Source: §1 · p. 2, Figure 2 · p. 3
  • iFLEX estimates the intermediate field by linearly weighting both observations, then uses conditional diffusion to generate and add the residual correction.Its conditioning includes both observed fields, requested time, interval length, and available physical parameters.Source: §1 · pp. 3–4
  • Across NSKT, Shanghai radar precipitation, and SST, iFLEX achieves lower reconstruction error than the evaluated baselines, including video-interpolation methods.In a controlled NSKT comparison, residual learning further improves on direct field generation with the diffusion setup held fixed.Source: §1 · pp. 3–4
  • At Re = 36000, iFLEX's RFNE is 10.48%, versus 13.73% for direct field generation under matched architecture and training settings.The comparison isolates the effect of learning the residual rather than generating the full intermediate field.Source: §1 · p. 3

2 Related Work

Prior work addresses missing scientific data through spatial reconstruction, forecasting, super-resolution, and temporal interpolation, alongside video-frame methods and scientific generative models. This paper positions iFLEX as a correction to linear interpolation for reconstructing scientific fields between observations.

2 Related Work

  • Scientific reconstruction spans sparse-sensor recovery [Erichson et al., 2020, Fukami et al., 2021b], forecasting, super-resolution, and downscaling, while this work targets intermediate states between observed times.FLINT and HyperFLINT use learned motion estimation for scientific temporal reconstruction.Source: §2 · p. 4
  • Video interpolation methods use optical flow, motion trajectories, direct generation, or residual diffusion, but physical fields also change in their underlying values.The paper evaluates video methods on scientific fields and tests learned correction of numerical interpolation.Source: §2 · p. 5
  • Scientific diffusion work includes forecasting, reconstruction, and downscaling; iFLEX builds on FLEX [Erichson et al., 2025] and adapts residual diffusion to two-endpoint interpolation.The approach uses a weighted average as its initial estimate and compares residual learning with direct field generation under matched settings.Source: §2 · p. 5

3 Problem Formulation: Temporal Interpolation from Two Observed States

The problem is to reconstruct a physical field at an unobserved time between two observed states. The formulation conditions on both endpoint fields and temporal information, while recognizing that endpoints may not uniquely reveal the intervening evolution.

3 Problem Formulation: Temporal Interpolation from Two Observed States

  • Given fields observed at t0 and t1, the goal is to reconstruct the unknown field uτ at an intermediate time.The endpoint observations can constrain the reconstruction, unlike forecasting from earlier observations alone.Source: §3 · pp. 5–6
  • Reconstruction accuracy depends on temporal spacing: short-lived events occurring entirely between observations may leave no evidence in either endpoint.The formulation assumes observations are sufficiently close to constrain the relevant evolution.Source: §3 · p. 6
  • The model conditions on relative position τ and interval duration ∆t because identical interpolation ratios can span different amounts of physical evolution.Available physical parameters, such as Reynolds number, can also be included as conditioning information λ.Source: §3 · p. 6
  • Training examples contain both endpoints and the known intermediate target, whereas inference uses endpoint observations and their associated conditions.A conditional distribution represents possible intermediate states when the available observations do not uniquely determine them; linear interpolation provides a reference for correction.Source: §3 · p. 6

4 Method

iFLEX combines a linear estimate from two observed fields with a conditional diffusion model that learns a correction. It conditions that correction on endpoint features and the requested temporal and physical settings, then adds it to the estimate.

4 Method

  • iFLEX uses linear interpolation as an initial estimate and a conditional diffusion model to learn its correction from both observations.The reconstructed field is obtained by adding the generated residual to the linear estimate.Source: §4 · p. 7, Figure 3 · p. 7

4.1 Linear Interpolation and Residual Learning

  • Linear interpolation is inexpensive but can misplace or weaken moving and deforming structures, so training learns the difference between the estimate and the known intermediate field.The residual correction can alter structure location and shape even when applied pointwise.Source: §4.1 · pp. 7–8

4.2 Learning the Residual with Diffusion

  • A conditional diffusion model learns to recover the residual from noisy versions, using endpoint fields and physical conditions while training on multiple noise levels.The denoiser predicts diffusion velocity with an ℓ1 objective, using a FLEX encoder-transformer-decoder backbone.Source: §4.2 · pp. 8–9

4.3 Conditioning on Observations and Physical Variables

  • The denoiser combines separate endpoint spatial features with scalar conditions for target position, interval duration, physical parameters, and diffusion noise.FiLM applies scalar-conditioned channel-wise scale and shift, while learned gates inject endpoint features.Source: §4.3 · pp. 9–11

4.4 Training and Reconstruction at Arbitrary Times

  • Training varies the gap between endpoint observations and samples an intermediate target, supplying both its relative position and the physical interval duration.The two temporal quantities are distinct because the same interpolation ratio can represent different durations of physical evolution.Source: §4.4 · pp. 11–12
  • At inference, residual generation starts from Gaussian noise and uses three DDIM-style denoising steps before adding the correction to the linear estimate.Multiple target times can be processed together, but their independently generated fields have no explicit temporal-consistency constraint.Source: §4.4 · p. 12

5 Experiments

The experiments evaluate iFLEX across turbulence, radar precipitation, and sea surface temperature, then examine error across time, temporal gaps, inference cost, and design choices. Across these settings, iFLEX improves on the evaluated baselines, while reconstruction difficulty and computational cost vary by dataset and interval.

5.1 Reconstruction Across Different Physical Systems

  • Across NSKT, radar precipitation, and SST, iFLEX achieves lower reconstruction error than evaluated baselines, including video-interpolation methods.The comparisons include Super SloMo and EDEN on NSKT, and show RFNE falling from 1.94% to 1.81% against linear interpolation on SST.Source: §5.1 · pp. 13–15, Table 1 · p. 14, Table 2 · p. 15
  • iFLEX has the lowest RFNE at every reported Reynolds number, including unseen values, but the main sweep does not establish accuracy across independent realizations.Variation across additional trajectories is relatively small at Re = 8000 and 12000, but substantially larger at Re = 32000.Source: §5.1 · p. 14

5.2 How Reconstruction Error Changes Within an Interval

  • Errors often peak near the interval midpoint; iFLEX maintains the lowest error across evaluated steps on NSKT, radar, and SST.The curves need not be symmetric, and methods are closer together on SST, where temporal changes are smaller.Source: §5.2 · p. 16, Figure 4 · p. 17

5.4 Reconstruction Across Different Temporal Gaps

  • At radar gaps T = 4 and T = 12, iFLEX RFNE is 21.08% and 40.42%, versus 36.85% and 55.02% for linear interpolation; it also beats baselines at NSKT gaps T = 12, 16, and 24.The NSKT training gaps span T ∈{5, . . . , 20}, so T = 24 is beyond the training range.Source: §5.4 · p. 17, Figure 6 · p. 19

5.3 Spatial Structure, Spectral Content, and Inference Cost

  • At Re = 12000, iFLEX's 4.11% RFNE requires 35.39 ms per field, versus Super SloMo's 13.20% and 3.54 ms; the spectrum diagnostic is windowed vorticity power.The latency comparison matters when reconstructing many fields; the spectrum is not a direct measure of energy conservation.Source: §5.3 · p. 16, Figure 5 · p. 18

5.5 Ablation Study

  • Variable-gap training yields 10.48% RFNE versus 10.75% for fixed T = 20; explicit average and difference features improve RFNE outside the training Reynolds-number range.The feature-fusion effect depends on evaluation regime; performance is similar for unseen Reynolds numbers within the training range.Source: §5.5 · pp. 19–20
  • At Re = 36000, residual generation achieves 10.48% RFNE versus 13.73% for direct-field generation under the same backbone and training setup.This comparison changes only the quantity generated by diffusion, isolating the learning target in this evaluated setting.Source: §5.5 · p. 18, Figure 7 · p. 20
  • At Re = 36000, removing the second endpoint raises RFNE to 20.44% and removing Δt raises it to 12.71%, versus 10.48% for full iFLEX.The one-endpoint comparison changes both the available observations and learning target; the Δt ablation retains the other model components.Source: §5.5 · p. 19, Figure 7 · p. 20

6 Limitations

Reconstruction depends on whether the two observations constrain the intermediate evolution, an assumption that becomes more restrictive with longer intervals or faster-changing systems. The evaluation also leaves limits around the interpolation reference, sample coverage, and temporal or physical consistency.

Observation constraints

  • Reconstruction errors generally increase as observation gaps widen, and short-lived events occurring entirely between observations may be unrecoverable from the endpoints.The authors assume observations are close enough to constrain relevant evolution; flood-prediction applications require validation at the downstream task's temporal resolution.Source: §6 · p. 20

Interpolation reference

  • The authors lack a theoretical characterization of when linear interpolation is preferable to other references; a more accurate starting estimate may help over longer gaps but adds computational cost.The cost would need to be justified by improvements in final reconstruction.Source: §6 · pp. 20–21

Evaluation scope

  • Evaluation covers few independent realizations, and the held-out SST region establishes geographic transfer only within the evaluated dataset, not arbitrary regions or conditions.The Reynolds-number sweep uses one held-out trajectory per parameter value, with two-trajectory evaluations at three Reynolds numbers; flow difficulty varied across realizations.Source: §6 · p. 21

Temporal and physical consistency

  • Individual-field accuracy does not ensure smooth sequences, accurate temporal derivatives, governing-equation compliance, conservation, or calibrated uncertainty.These properties and downstream derived quantities need additional evaluation beyond the reconstruction metrics reported.Source: §6 · p. 21

7 Conclusions

The conclusions present iFLEX as a way to correct a simple interpolation with conditional diffusion, with controlled turbulence results supporting residual learning and improvements also reported for radar and SST. The section identifies follow-up questions about better references and temporally consistent sequences, alongside the dataset and implementation scope.

7 Conclusions

  • At Re = 36000, residual generation reduces RFNE from 13.73% for direct field generation to 10.48% and raises SSIM from 0.827 to 0.869 under the same architecture, observations, conditioning, and training procedure.The comparison supports the residual formulation in this evaluated setting; the linear estimate itself has 43.96% RFNE.Source: §7 · p. 21
  • The proposed formulation linearly interpolates the two observed fields, then uses conditional diffusion to generate a residual correcting structure position, shape, and values.The conclusion describes this as correcting a simple numerical estimate of the missing field.Source: §7 · p. 21
  • RFNE falls from 47.78% to 32.69% for Shanghai radar and from 1.94% to 1.81% for SST, spanning cases where the initial interpolation is substantially erroneous or relatively accurate.The authors state iFLEX can estimate missing fields for systems represented in the training data without rerunning the simulation.Source: §7 · p. 21
  • Future work proposes comparing learned or equation-based initial estimates across temporal gaps and jointly generating neighboring states to study derived quantities such as temporal derivatives.The reference comparison would weigh easier correction against added computational cost.Source: §7 · pp. 21–22

A Implementation Details

  • The appendix describes dataset specifications, evaluation metrics, training configurations, spectral diagnostics, and inference-time measurements as reproducibility and interpretation details.It points to Section 4 for the reconstruction method itself.Source: A Implementation Details · p. 27

A.1 Datasets

  • The evaluation covers simulated NSKT vorticity, Shanghai radar precipitation, and SST; NSKT uses 256 × 256 crops from a 2048 × 2048 simulation and supplies Reynolds number as a condition.Radar and SST use 128 × 128 grids without a Reynolds-number condition; each sample pairs two observed fields with a known intermediate state.Source: §A.1 · p. 27

A.2 Evaluation Metrics

The appendix defines field-level metrics and a turbulence spectrum diagnostic, then details model configurations, training and inference, and illustrative reconstructions. It also cautions that the spectrum diagnostic does not test equation satisfaction or kinetic-energy conservation.

A.2 Evaluation Metrics

  • RFNE measures field-value error relative to target magnitude, while Pearson correlation measures agreement in spatial variation but not magnitude.The evaluation averages RFNE, PC, and SSIM over target states using one stochastic reconstruction per target.Source: §A.2 · pp. 27–28
  • SSIM compares local field structure using means, variances, and covariance; higher values indicate closer structural agreement.Predicted and observed fields are normalized to a common range before SSIM is computed.Source: §A.2 · p. 28
  • The turbulence spectrum diagnostic compares vorticity-fluctuation power across scales, using a Hann window and weighted-mean subtraction to reduce crop-boundary effects.The resulting spectrum does not measure kinetic-energy conservation or verify Navier–Stokes compliance.Source: §A.2 · pp. 28–29

A.3 Model and Experiment Configurations

  • The supervised U-Net predicts intermediate states from endpoint fields with target-time and physical-parameter conditioning; Super SloMo estimates optical flow, while EDEN generates intermediate fields in a learned latent representation.Table 4 reports Adam at 10^-4 for U-Net across all three datasets; Table 5 reports Adam for Super SloMo, at 6 × 10^-5 on NSKT and 10^-4 on Shanghai radar and SST.Source: §A.3 · p. 29, Table 4 · p. 30, Table 5 · p. 30
  • iFLEX uses endpoint fields and scalar conditions including target position, endpoint separation, and available physical parameters; a three-step DDIM-style sampler generates the residual added to the linear estimate.Training samples vary endpoint gaps; the shared architecture is adjusted for data resolution and configuration.Source: §A.3 · pp. 30–31

B Additional Reconstruction Examples

  • Inference timing is reported per generated field and amortized over batch size and target count; the appendix also presents individual turbulence, radar, and SST reconstructions.Sequential cost is estimated as K times per-frame cost, while each requested iFLEX target still requires three denoising steps.Source: §A.3 · pp. 30–31, B Additional Reconstruction Examples · p. 31

C Refining Linear Interpolation Across Temporal Gaps

The radar gap study finds that iFLEX reduces linear-interpolation error at all tested observation gaps, while both methods become less accurate as the gap grows. The section also compares reconstructions across radar, turbulence, and SST examples and notes that events with little endpoint evidence remain difficult to recover.

C Refining Linear Interpolation Across Temporal Gaps

  • At T = 4, iFLEX reduces radar RFNE from 36.85% for linear interpolation to 21.08%; at T = 12, it reduces RFNE from 55.02% to 40.42%.The table reports improvement at all three tested gaps.Source: C Refining Linear Interpolation Across… · p. 31, Table 8 · p. 35
  • Reconstruction error increases with the observation gap for both methods, and larger gaps leave more intervening evolution to infer.The study evaluates T ∈ {4, 8, 12} intermediate states at corresponding normalized positions.Source: C Refining Linear Interpolation Across… · pp. 31, 35
  • At Re = 36000, iFLEX more closely reproduces illustrated vortex structures and achieves 10.48% RFNE versus 22.87% for Super SloMo over the corresponding test set.The aggregate errors are not measurements of the individual displayed fields.Source: Figure 8 · p. 32
  • For radar, iFLEX retains localized precipitation structures more closely at middle target times than EDEN or Super SloMo; aggregate RFNE is 32.69% for iFLEX, 56.54% for EDEN, and 36.77% for Super SloMo.Source: Figure 9 · p. 33
  • On the held-out South Pacific test set, iFLEX achieves 1.81% RFNE and 0.87 SSIM, while the displayed methods show similar broad temperature patterns but differing local gradients.The scores summarize the full evaluation set rather than the pictured examples.Source: Figure 10 · p. 34
  • Across all three datasets, training uses Adam with a learning rate of 10^-4, while training duration and batch size vary by dataset.Source: Table 7 · p. 35
  • Events that leave little evidence at either endpoint remain difficult to reconstruct reliably, even when the model learns precipitation-change patterns.Source: C Refining Linear Interpolation Across… · p. 35

D Comparison with Supervised Temporal Interpolation

The section compares iFLEX with an implemented supervised interpolation baseline, PIT-UNet. iFLEX has lower RFNE and higher SSIM at all three unseen Reynolds numbers, though the comparison does not isolate residual learning.

D Comparison with Supervised Temporal Interpolation

E Ablation of Mean and Difference Features

The ablation tests whether presenting endpoint features with explicit mean and difference combinations changes reconstruction. Their effect varies by Reynolds number, with better performance in the tested extrapolation settings but no uniform advantage.

E Ablation of Mean and Difference Features

  • Explicit mean and difference features add no information beyond the endpoint features, but make relationships between observations directly available to the fusion module.Source: E Ablation of Mean and Difference… · p. 36
  • At Re = 36000, complete fusion achieves 10.48% RFNE versus 11.26% for endpoint-only fusion.The complete representation performs better at the three evaluated Reynolds numbers outside the training range, while endpoint-only fusion is slightly better at two unseen values within it.Source: E Ablation of Mean and Difference… · p. 36
  • The comparison holds the observed fields, linear reference, diffusion objective, network, training data, optimization, and sampling fixed while changing how endpoint features are presented.Source: E Ablation of Mean and Difference… · p. 36
  • The results support a limited conclusion: endpoint representation can affect generalization to different physical parameters, but the advantage is not uniform across Reynolds numbers.Source: E Ablation of Mean and Difference… · p. 36

F Interpretation of Conditional Sampling

The section cautions that stochastic sample variation is not automatically calibrated physical uncertainty. The reported experiments evaluate field agreement, not uncertainty calibration or coherent joint evolution across independently generated target times.

F Interpretation of Conditional Sampling

  • Different generated samples may reflect alternative reconstructions given incomplete observations, not calibrated physical uncertainty.The NSKT crop omits the complete flow field, and observational records may omit relevant changes or contain measurement noise.Source: F Interpretation of Conditional Sampling · p. 36
  • The experiments do not establish whether sample variation represents reconstruction-error frequency or magnitude, or separate observation uncertainty from model uncertainty.Source: F Interpretation of Conditional Sampling · pp. 36–37
  • Because target times are generated independently, sampling does not enforce coherent joint evolution across consecutive reconstructed states.Assessing calibration and temporal coherence would require experiments beyond the reported field reconstruction metrics.Source: F Interpretation of Conditional Sampling · p. 37

G Complete NSKT Evaluation Across Reynolds Numbers

Across the evaluated Reynolds numbers, iFLEX achieves the lowest RFNE and highest SSIM, including when Reynolds number is outside the training range. Conditioning baseline models on Reynolds number affects them differently, but iFLEX retains the lowest RFNE.

G Complete NSKT Evaluation Across Reynolds Numbers

H Reynolds-Number Conditioning for Baseline Models

  • EDEN receives a learned embedding of normalized Reynolds number through its global conditioning pathway, while Super SloMo receives it as a broadcast constant input channel.The remaining architecture of each baseline is unchanged.Source: H Reynolds-Number Conditioning for… · pp. 38–39
  • At Re = 36000, iFLEX records 10.48% RFNE versus 23.67% for conditioned EDEN and 24.26% for conditioned Super SloMo.Across tested cases, Reynolds-number conditioning reduces EDEN RFNE by approximately 7.2%–19.4% relative, while the tested Super SloMo condition increases it by approximately 6.1%–8.3%.Source: H Reynolds-Number Conditioning for… · pp. 39–40, Table 14 · p. 40

I Evaluation Across Additional Test Realizations

A second held-out trajectory was evaluated at Re = 8000, 12000, and 32000 using the same trained checkpoint. The two realizations differed more in reconstruction error at Re = 32000, but two trajectories provide only an initial check of variation across initial conditions.

I Evaluation Across Additional Test Realizations

  • For each Reynolds number, trajectory-level RFNE averages across samples and target times; the reported standard deviation is between trajectories, not spatial crops.The additional evaluation uses a fixed trained checkpoint, with two trajectories per value.Source: I Evaluation Across Additional Test… · p. 40, Table 15 · p. 41
  • With only two trajectories per Reynolds number, this evaluation provides an initial check rather than a precise estimate of variation across initial conditions.The added trajectories are at Re = 8000 and 12000 with seed 1000, and Re = 32000 with seed 987; the original test seed is 3407.Source: I Evaluation Across Additional Test… · p. 40, Table 15 · p. 41
  • The two evaluated trajectories show greater reconstruction-error variation at Re = 32000 than at Re = 8000 or 12000.At Re = 32000, the two realizations appear to differ substantially in reconstruction difficulty.Source: I Evaluation Across Additional Test… · p. 40, Table 15 · p. 41
Loading 2610.36751v1…