PaperlayerView exact revision on arXiv

Source-linked AI summary

Benchmarking Generative Models for Weather Data Assimilation on Real Station Observations

Ruizhe Huang, Qidong Yang, Jonathan Giezendanner, Sherrie Wang

arXiv:2610.00728v1cs.LGcs.AI

TL;DR

Reanalysis depends on computationally costly data assimilation, and generative DA has lacked controlled evaluation on real station observations. This paper benchmarks generative and classical methods under fixed conditions, finding generative methods outperform 3D-Var and full-gradient guidance outperform alternative conditioning strategies.

  • Problem

    Real-station generative DA has lacked controlled comparisons, despite data assimilation's substantial computational cost in reanalysis pipelines.

    Source: §1 · pp. 1–2

  • Method

    The benchmark holds the architecture, dataset, and evaluation protocol fixed while comparing generative and classical DA on MADIS stations across CONUS.

    Source: §1 · p. 2, §7 · p. 15

  • Results

    Generative methods achieve 35.7% vs. 33.3% RMSE improvement over ERA5 against 3D-Var; full-gradient guidance outperforms alternatives, while diffusion and flow matching reach parity.

    Source: §7 · pp. 15–16

  • Takeaways & Limitations

    Flow Guidance is the practical choice among equally accurate options, using half the memory of Diffusion-SDA and requiring only one hyperparameter to tune.

    Source: §7 · p. 16

    Without an explicit ERA5 background, generative estimates far from stations can be less accurate than ERA5 for temperature and dewpoint in the sparse benchmark.

    Source: §6.1 · p. 14

Abstract

from arXiv · show

Weather reanalysis products rely on computationally intensive numerical weather predictions followed by data assimilation that corrects the forecast toward observations. Deep generative models offer a cheaper alternative that shifts much of this cost from inference to offline training. However, existing generative approaches have been evaluated on synthetic observations or under different datasets and evaluation schemes, making it unclear which design choices actually improve real-world data assimilation. We present the first controlled benchmark of generative weather data assimilation on real weather station observations. Using 11,849 NOAA MADIS stations across the contiguous United States and four weather variables, we evaluate methods while holding the dataset, observation operator, and deep learning architecture fixed. The benchmark compares the major design choices, including diffusion versus flow matching, pixel versus latent-space formulations, and multiple inference-time conditioning strategies, against a classical 3D-Var baseline. The benchmark reveals three clear conclusions. First, learned generative priors outperform the Gaussian prior of 3D-Var (35.7% vs. 33.3% RMSE reduction over ERA5) despite using no ERA5 background field at inference. Second, full-gradient guidance consistently outperforms stop-gradient and initial-noise optimization. Third, other choices provide little measurable benefit: diffusion and flow matching perform nearly identically under matched conditions, and latent-space variable mixing does not help. We further evaluate both dense and sparse station settings and find advantages from generative AI and full-gradient guidance more pronounced under sparsity. Together, these results identify which components of generative weather data assimilation improve performance on real station observations and establish a standardized benchmark for future work.

1 Introduction

Reanalysis is costly, and classical data assimilation relies on assumptions that can pass model biases into weakly observed regions. This paper benchmarks generative methods on real station observations and compares their design choices under controlled conditions.

1 Introduction

  • Reanalysis is computationally expensive, while classical DA assumes Gaussian, unbiased errors and linearized dynamics; weak observations can leave model systematic errors in the analysis.ERA5 produces only 6–9 days of reanalysis per day of wall-clock time, even on ECMWF’s high-performance computing facility.Source: §1 · p. 1
  • Generative DA has lacked controlled comparisons on real, off-grid stations, leaving open how methods and design choices compare under a fixed dataset and protocol.Existing evaluations often use synthetic masked observations or ERA5 as a reference, which cannot test whether generative DA corrects ERA5 biases.Source: §1 · p. 2
  • 35.7% vs. 33.3% RMSE reduction over ERA5: the best generative method outperforms 3D-Var without an explicit background field, with a wider margin under sparse observations.The margin grows from 2.4 percentage points on the dense benchmark to 7.9 percentage points on the sparse benchmark.Source: §1 · p. 2
  • With the backbone, data, and protocol fixed, Flow Guidance outperforms stop-gradient guidance and initial-noise optimization, while flow matching and diffusion perform nearly identically.Flow matching reaches 35.7% RMSE improvement versus 35.4% for diffusion; the guidance advantage is especially pronounced under sparse observations.Source: §1 · p. 2
  • A controlled ablation finds little measurable benefit from several design choices, including latent-space variable mixing.The ablations vary guidance schedule, stop-gradient approximation, terminal extrapolation, solver, and operating space one at a time.Source: §1 · p. 2

2 Related Work

Related work spans deterministic point-estimate methods and generative approaches that sample analysis posteriors. Generative conditioning methods differ in whether they depend on observation layout and in how they inject observations during inference.

Related work

  • Deterministic DA networks replace iterative variational minimization with a forward pass but return point estimates, whereas generative models sample from the posterior and provide uncertainty directly.Some deterministic methods need a separate mechanism, such as an ensemble, to quantify uncertainty.Source: §2 · p. 3

Observation-layout-dependent conditioning; Observation-layout-independent conditioning

  • Layout-dependent conditioning requires retraining when observation coverage changes, while layout-independent methods inject observations at inference and can handle any layout with one pretrained prior.The benchmarked methods use layout-independent conditioning through guidance, optimization, or inpainting.Source: §2 · p. 3

Guidance-based methods

  • Guidance methods steer each sampling step using an observation-loss gradient; full-gradient variants backpropagate through the denoiser or velocity network and continue from the corrected state.Full-gradient methods also differ in how strongly they weight the observation gradient across noise levels.Source: §2 · p. 3

Guidance-based methods; Optimization-based methods

  • Stop-gradient methods differentiate with respect to the field rather than backpropagating through the model, while optimization methods adjust initial noise to minimize observation-space loss.Stop-gradient variants differ in how sampling resumes; D-Flow differentiates through the full ODE solve during initial-noise optimization.Source: §2 · p. 3

Inference-time inpainting

  • Inference-time inpainting replaces observed grid points during denoising but cannot assimilate off-grid station observations.It requires on-grid observations of state variables, so the paper does not benchmark it.Source: §2 · p. 3

Latent-space formulations

  • Latent-space methods apply guidance or optimization in an autoencoder’s compressed representation, where prior work argues inter-variable relationships may be absorbed and covariance may be approximately diagonal.Prior latent-space methods had not tested variable mixing or compression depth.Source: §2 · p. 3

3 Methods

The methods pair a classical variational baseline with learned diffusion and flow-matching priors, then condition frozen generative models on observations at inference. The paper also compares full-gradient and stop-gradient formulations of that conditioning.

3.1 Classical Baseline: 3D-Var

  • 3D-Var minimizes a background-plus-observation cost, using ERA5 as the background and tuned covariance settings for its analysis.The generative methods use no background field.Source: §3.1 · p. 4

3.2 Generative Priors: Diffusion Models and Flow Matching

  • Diffusion perturbs clean data with scaled Gaussian noise and predicts clean states, while flow matching interpolates noise and data and learns a velocity field.Both learned priors use the same U-Net backbone and ERA5 training fields.Source: §3.2 · p. 4

3.3 Inference-Time Conditioning

  • At inference, the conditioning methods steer a frozen generative prior toward observations by approximating their likelihood from a denoised state.Methods differ in guidance strength, gradient computation, and which sampling state they modify.Source: §3.3 · p. 5

3.3.1 Diffusion-SDA: Score-Based Data Assimilation

  • Diffusion-SDA modifies the diffusion denoiser using an observation-mismatch gradient that backpropagates through the denoiser network.Its guided denoiser is substituted into the predictor-corrector sampler.Source: §3.3.1 · p. 5

3.3.2 Flow Guidance

  • Flow Guidance subtracts the observation-loss gradient from the velocity field and integrates the guided ODE from Gaussian noise to the data endpoint.The terminal extrapolation estimates the clean output, and the loss measures its observation mismatch.Source: §3.3.2 · p. 5, §3.3.1 · p. 5

Guidance schedule.

The remaining methods vary how observation gradients enter sampling and whether assimilation operates in pixel or learned latent space. Their designs range from optimizing initial noise to adapting classical and flow-based methods through an autoencoder.

Guidance schedule.

  • Dropping the velocity-network Jacobian makes stop-gradient guidance cheaper, but removes the learned spatial spreading of station residuals provided by full-gradient guidance.The guided ODE can also use an Euler-plus-corrector sampler.Source: Guidance schedule · p. 6

3.3.3 D-Flow

  • D-Flow optimizes the initial Gaussian noise through the full ODE to reduce observation mismatch, regularizing noise toward a typical Gaussian set.It returns the iterate with the lowest validation physical RMSE.Source: §3.3.3 · p. 6

3.4 Latent-Space Formulation

  • The latent-space formulation tests whether dimensionality reduction and decoder structure affect assimilation quality, varying variable interaction and compression depth.The study applies latent-space formulations to both flow matching and 3D-Var.Source: §3.4 · p. 6

3.4.1 Autoencoder

  • A convolutional VAE encodes physical states into latent representations and decodes them, with grouped convolutions isolating variables and standard convolutions mixing them.Six configurations sweep compression depth across both variable-mixing settings.Source: §3.4 · p. 6

3.4.2 Latent 3D-Var

  • Latent 3D-Var sets the background covariance to identity in latent space and evaluates observation mismatch after decoding, leaving the decoder to supply spatial structure.The method follows the autoencoder setup proposed for testing latent-space 3D-Var.Source: §3.4.2 · p. 6, §3.4.1 · p. 6

3.4.3 Latent Flow Matching

  • The latent flow model is trained on interpolations from Gaussian noise to encoded states, then samples by integrating its latent ODE and decoding the result.Source: §3.4.2 · p. 6

3.4.4 Latent Flow Guidance

  • Latent Flow Guidance applies Flow Guidance in latent space, with observation loss evaluated after decoding so gradients pass through the decoder.Source: §3.4.3 · p. 7, §3.4.4 · p. 7

3.5 Summary of Methods

  • The method overview contrasts classical costs, per-step diffusion or flow guidance, terminal-estimate optimization, and guidance in an autoencoder latent space.FlowDPS iterates optimization on the terminal estimate, while D-Flow updates initial noise through the full ODE.Source: Figure 2 · p. 7, §3.5 · p. 7

4 Experimental Setup

The experiments use ERA5 fields and real MADIS station observations, with fixed station splits and shared evaluation conditions. Dense and sparse benchmarks support comparisons across differing observation availability.

4.1.1 ERA5 Reanalysis

  • Hourly ERA5 grids cover the contiguous United States at 0.25° resolution from 2016–2023, with four wind, temperature, and dewpoint variables standardized using training statistics.Source: §4.1 · p. 7, §4.1.1 · p. 8
  • The 2023 MADIS dataset contains 11,849 spatially heterogeneous stations, split once into fixed train, validation, and test sets; a random 1,000-station subset defines the sparse benchmark.The dense split has 8,294/1,777/1,778 stations; the sparse split has 700/150/150.Source: §4.1.1 · p. 8

4.1.3 Temporal Splits and Evaluation Snapshots

  • The flow model is trained on 2017–2022 ERA5, validated on 2016, and used for 2023 assimilation; 360 daily sampled snapshots remain after outage exclusions.The same trained model serves both benchmarks because ERA5 fields do not depend on the station network.Source: §4.1.2 · p. 8, §4.1.3 · p. 8

4.2 Evaluation Protocol

  • Methods are assessed by held-out-station RMSE across shared snapshots using ERA5 as the baseline and a common bicubic observation operator.Reported measures include absolute RMSE, percentage reduction relative to ERA5, and computational cost.Source: §4.2 · p. 8
  • Performance is also reported by test-station distance to the five nearest observed training stations, exposing variation that a station-wide average can mask.The same distance binning is used in dense and sparse benchmarks.Source: §4.2 · pp. 8, 10

5 Results

Across dense and sparse station benchmarks, generative methods match or outperform 3D-Var, while results vary with conditioning strategy, station distance, and region.

5.1 Dense Benchmark: Generative Methods Outperform 3D-Var

  • 35.7% test improvement for Flow Guidance and 35.4% for Diffusion-SDA put both about 2 pp ahead of 3D-Var on the dense benchmark.The methods also have similar training–test gaps: 13.9 pp for Flow Guidance and 12.7 pp for Diffusion-SDA.Source: Table 3 · p. 10, §5.1 · p. 10
  • Latent-space Flow Guidance does not beat pixel space on the dense benchmark; the best latent configuration is 1.4 pp less accurate and about 10% slower.At S=2, separate variables match mixing while using four times fewer parameters.Source: §5.1 · pp. 10–11
  • The Langevin corrector adds 5.1 pp to Diffusion-SDA on average, entirely through temperature and dewpoint gains; wind changes stay within ±0.3 pp.All methods improve wind more than temperature and dewpoint.Source: §5.1 · p. 11

5.2 Sparse Benchmark: The Generative Advantage Widens

  • With 1,000 stations, average test improvements range from 7.6% to 22.4%; Flow Guidance and Diffusion-SDA remain close at 21.7% and 21.5%.Flow Guidance with the Euler-128 PC sampler is the best single configuration at 22.4%.Source: §5.2 · p. 11, Table 4 · p. 11

5.3 Spatial Propagation of Corrections

  • On the dense benchmark, improvement is about 40% within 10 km and decays with distance, remaining positive to roughly 120 km.That reach is about four times ERA5’s roughly 30 km grid spacing.Source: §5.3 · p. 12
  • At 50–60 km, generative methods retain about 25% improvement versus about 17% for 3D-Var, despite comparable performance within 10 km.These are dense-benchmark results.Source: §5.3 · p. 12
  • At 80–100 km, dense-benchmark generative methods improve wind by more than 24%, versus less than 15% for temperature and less than 20% for dewpoint.Under sparse observations, beyond about 150 km, generative temperature and dewpoint improvements turn negative while wind improvements remain positive.Source: §5.3 · p. 12, Figure 4 · p. 12

5.4 Regional Case Studies

  • In the Rocky Mountains, all methods reduce ERA5’s cold bias; at Great Lakes shoreline stations, generative methods reach about 2.4 m/s RMSE versus 2.81 m/s for 3D-Var.The Great Lakes’ lake-interior fields cannot be directly verified because there are no observations over the lakes.Source: §5.4 · pp. 12–13, Figure 5 · p. 13

6 Discussion

Learned priors outperform the ERA5-centered 3D-Var background, while full-gradient guidance is the strongest tested conditioning strategy. Sparse observations magnify generalization differences, and pixel-space Flow Guidance offers a practical accuracy–cost balance.

6.1 Learned Priors Provide More Effective Regularization

  • Flow Guidance and Diffusion-SDA outperform 3D-Var on every variable in the dense benchmark, with larger gains under sparse coverage and no ERA5 field at inference.At Great Lakes shoreline stations, they reach 2.39 m/s RMSE, a 53.0% reduction, versus 3D-Var’s 2.81 m/s RMSE and 44.8% reduction.Source: §6.1 · p. 13, Figure 6 · p. 14
  • Generative methods retain more improvement farther from stations, and learned priors may accommodate regional shifts that 3D-Var’s ERA5 penalty can suppress.This flexibility may help explain their 6–8 pp advantage over 3D-Var in the Rocky Mountains case.Source: §6.1 · pp. 13–14
  • Under sparse coverage, generative temperature and dewpoint estimates far from stations become less accurate than ERA5 because they lack a background fallback.Conditioning on ERA5 is proposed as a possible fallback while retaining learned-prior flexibility.Source: §6.1 · p. 14

6.2 Full-Gradient Guidance is the best Conditioning Strategy

  • Full-gradient guidance is the only tested conditioning strategy that beats 3D-Var on both dense and sparse benchmarks.Its denoiser Jacobian lets station residuals update grid locations beyond those directly connected by the observation operator.Source: §6.2 · p. 14
  • Stop-gradient guidance fits assimilated stations more closely but predicts test stations less accurately, especially when sparse; additional updates can worsen overfitting.D-Flow has poor fit even at assimilated stations and requires roughly an order of magnitude more memory and time than Flow Guidance.Source: §6.2 · pp. 14–15

6.3 Sparse Observations Amplify Differences in Generalization

  • Sparse coverage widens train–test gaps and leaves stop-gradient guidance farther behind full-gradient guidance.Isolated stations are easier to fit, while predictions between stations depend more on how methods spread corrections.Source: §6.3 · p. 15

6.4 Pixel-Space Flow Guidance Offers the Best Practical Trade-off

  • Flow Guidance matches Diffusion-SDA’s accuracy with a simpler sampler, while diffusion’s corrective updates substantially improve temperature and dewpoint.With predictor-only sampling, flow matching performs better than diffusion, whose deficit lies mainly in temperature and dewpoint.Source: §6.4 · p. 15
  • Pixel-space Flow Guidance is the practical default; latent space can match its sparse-coverage accuracy at lower cost, while variable mixing adds little benefit.Pixel space retains higher accuracy under dense coverage; Flow Guidance uses roughly half Diffusion-SDA’s peak memory and one tuning parameter.Source: §6.4 · p. 15

7 Conclusion

The paper establishes a controlled benchmark of generative and classical data assimilation on real station observations. Generative methods outperform 3D-Var without an ERA5 inference background, and full-gradient guidance is the strongest tested conditioning choice, though the evaluation remains limited in scope.

7 Conclusion

  • The controlled benchmark covers 11,849 MADIS stations across CONUS, four surface variables, and 360 snapshots, with architecture, dataset, and evaluation protocol fixed.It compares flow matching, diffusion, and classical baselines.Source: §7 · p. 15
  • Generative methods improve RMSE over ERA5 by 35.7%, versus 33.3% for 3D-Var, without an ERA5 background field at inference; full-gradient guidance outperforms the tested alternatives.Diffusion and flow matching reach parity on both benchmarks, while latent-space assimilation and inter-variable mixing do not improve accuracy.Source: §7 · pp. 15–16
  • The study is limited to four near-surface variables from CONUS surface stations and evaluates deterministic RMSE against ERA5 and 3D-Var, not operational regional analyses.The prior also lacks temporal consistency, is confined to ERA5’s distribution and grid, and has no fallback far from stations.Source: §7 · p. 16
  • Future work could add ERA5 fallback conditioning, temporal priors, sub-grid structure, additional observation types, ensemble-calibration metrics, and operational-analysis comparisons.The proposed comparisons include RTMA/URMA and HRRR.Source: §7 · p. 16

A Architecture and training details

The appendix records the shared model and training setup, selects bicubic interpolation based on validation performance, and relates wind-versus-temperature guidance behavior to normalized ERA5 errors.

A Architecture and training details

  • The shared U-Net has about 3.8 M parameters; models train on z-scored ERA5 from 2017–2022, validate on 2016, and test on 2023.Latent runs fit a variational autoencoder first; assimilation runs use a single RTX 8000 GPU.Source: A Architecture and training details · p. 19

A.1 Interpolation scheme comparison

  • Bicubic slightly improves Flow Guidance validation RMSE (1.43 vs. 1.44), while bilinear marginally improves ERA5 baseline RMSE (2.21 vs. 2.22).The same bicubic observation operator is used for baseline evaluation and all assimilation methods.Source: §A.1 · p. 20, Table 5 · p. 20

A.2 Normalized-space ERA5 baseline

  • Normalized ERA5 baseline error is about 2.4× larger for wind than for temperature and dewpoint, leaving more residual signal for wind guidance corrections.The passage attributes the recurring asymmetry to the normalized baseline-error ratio.Source: §A.2 · p. 20, Table 6 · p. 20

B Generative priors and samplers

The section describes diffusion and flow-matching generative priors, their samplers, and observation-guidance choices. In the guidance-schedule ablation, constant guidance gives the best average test improvement.

B Generative priors and samplers

  • In the generative setting, the learned prior is the trained distribution pθ(x), sampled from base noise and decoupled from any specific observation or background field.The section obtains flow matching as a special case of the EDM framework.Source: B Generative priors and samplers · p. 20

B.1 Forward process and training

  • The diffusion framework trains a denoiser across noise levels; its training setup uses VE scaling, while sampling uses VP scaling.The loss weighting is chosen so each noise level contributes equally to the training gradient.Source: §B.1 · pp. 20–21

B.2 Score and sampling

  • The trained denoiser implicitly defines a score used for stochastic SDE or deterministic probability-flow ODE sampling, with the ODE integrated from noise to data.The predictor-corrector sampler adds Langevin corrector steps that project samples toward the support of pσ.Source: §B.2 · p. 21

B.3 Flow matching as a special case

  • Flow matching linearly interpolates between Gaussian noise and clean data, trains a velocity field, and samples by solving the resulting ODE.The velocity field implicitly defines a score through its relation to the denoiser.Source: §B.3 · pp. 21–22

B.4.1 Predictor-corrector sampler

  • The diffusion predictor-corrector sampler combines a deterministic exponential-integrator predictor with stochastic Langevin corrections, costing N(1 + K) network evaluations.Its discretization uses a cosine noise schedule and uniform time steps.Source: §B.4 · p. 22, §B.4.1 · p. 22

B.4.2 Flow-matching solvers

  • Flow-matching sampling uses adaptive Dormand–Prince, fixed-step Euler, or Euler with a Langevin corrector; fixed-step Euler uses exactly N network evaluations.The corrector adds K steps and raises the total to N(1 + K) evaluations.Source: §B.4.2 · p. 23

C.1 Likelihood-approximation taxonomy

  • Guided sampling steers trajectories toward observations using a Gaussian likelihood centered on a denoised estimate; this work uses the SDA covariance form.The SDA form avoids per-step linear solves through a closed-form covariance approximation.Source: §C.1 · p. 24, Table 7 · p. 24

C.2.1 Guidance schedule

  • 35.7% test improvement makes the constant schedule best, ahead of SDA-derived (34.6%), cosine-increasing (34.4%), and cosine-decreasing (32.5%) schedules.Removing guidance near the clean endpoint causes the largest drop (−3.3 pp); removing it near the noisy endpoint costs −0.8 pp, while middle windows change accuracy by at most 0.1 pp.Source: §C.2.1 · p. 24, Table 8 · p. 25, Figure 7 · p. 25

C.2.2 Stop-gradient approximation

The ablations compare gradient approximations, terminal extrapolation, solvers, and Langevin correction. Full-gradient guidance leads the stop-gradient alternatives, while the corrector contributes differently for diffusion and flow matching.

C.2.2 Stop-gradient approximation

  • 35.7% to 32.2% is the test-improvement drop with stop-gradient guidance, which saves 8× memory and about 5× time.The accuracy degradation is roughly uniform across variables on the dense benchmark and concentrates in temperature and dewpoint under sparsity.Source: §C.2.2 · p. 26, Table 9 · p. 26
  • FlowDPS peaks at 28.2% test improvement at K=20, then falls to 24.4% at K=100; neither stop-gradient variant approaches full-gradient Flow Guidance.Increasing terminal-space iterations tends to overfit observation noise on real station data.Source: §C.2.2 · p. 26, Table 9 · p. 26

C.2.3 Terminal extrapolation

  • 35.8% versus 35.7% test improvement shows essentially no average gain from RK4 two-step terminal extrapolation over the single-step estimate.The small temperature and dewpoint gains are offset by wind penalties, while memory rises about 8× and runtime about 28×.Source: §C.2.2 · p. 26, Table 10 · p. 26, §C.2.3 · p. 27

C.2.4 ODE solver choice

  • The Langevin corrector raises Flow Guidance Euler-128 improvement from 32.8% to 34.9%, but raises Diffusion-SDA from 30.3% to 35.4%.The asymmetry is larger under sparsity: +2.1 pp for flow matching versus +8.5 pp for diffusion.Source: §C.2.4 · p. 27, Table 11 · p. 27

D The role of the Langevin corrector

  • Flow matching has the more accurate predictor but weaker corrector, while diffusion has the weaker predictor but stronger corrector, yielding comparable overall accuracy.The Langevin corrector projects samples toward the learned data manifold, counteracting off-manifold drift from observation-loss gradients.Source: D The role of the Langevin corrector · pp. 27–28

D.1 Perturbation analysis

The perturbation analysis separates the effects of score quality and trajectory. It finds that the diffusion score enables stronger suppression, especially near the terminal time.

D.1 Perturbation analysis

  • ≈0.09 versus ≈0.14 mid-trajectory perturbation amplification shows that Hybrid PC suppresses random perturbations more than Flow PC on the same flow trajectory.The flow-derived score limits the stable Langevin step to τ ≈ 0.02, versus τ = 0.1 for the diffusion score.Source: §D.1 · p. 28, Figure 8 · p. 29
  • With the diffusion score fixed, Hybrid PC and Diffusion PC have similar mid-trajectory attenuation (≈0.09 vs. ≈0.07), so trajectory is not the primary factor there.Near t = 1, Diffusion PC retains stronger suppression than Hybrid PC.Source: §D.1 · p. 28, Figure 8 · p. 29
  • Near t = 1, the VP cosine schedule yields an approximately 4× larger effective Langevin step than the flow-matching schedule, preserving stronger perturbation suppression.A small noise floor σmin prevents the corrector from becoming completely inert at the final steps.Source: §D.1 · p. 28, Figure 8 · p. 29

E Latent-space study

The latent-space study finds little benefit from autoencoder variable mixing, while compression depth affects accuracy and cost. The additional spatial and regional analyses describe how performance varies by variable and station coverage.

E Latent-space study

  • Best improvement falls from 34.3% at S=2 to 32.8% at S=3 and 31.2% at S=4, with losses concentrated in temperature and dewpoint.Mean reconstruction MAE rises about 20× from S=2 to S=4.Source: E Latent-space study · p. 30, Figure 9 · p. 30
  • Pixel-space Flow Guidance improves RMSE by about 1.4 pp over the best S=2 latent configuration at roughly 0.9× the wall-clock, while latent space uses less peak memory.Pixel space uses 3.05 GB versus 1.85 GB for the latent configuration; latent compression is useful when memory is the binding constraint.Source: E Latent-space study · pp. 30–31, Figure 9 · p. 30

E.1 Autoencoder reconstruction MAE

  • 34.34% versus 34.26% shows that variable-mixing and variable-isolated autoencoders reach essentially identical Flow Guidance accuracy at S=2.The G=4 configuration uses 4× fewer autoencoder parameters; deeper compression at S=4 yields better wind but worse temperature and dewpoint reconstruction with G=1.Source: E Latent-space study · pp. 30–31, §E.1 · p. 31, Table 13 · p. 31

F.1 Per-variable spatial decay

  • Wind improvements decay more slowly with distance than temperature and dewpoint; beyond about 150 km in the sparse benchmark, negative improvements come from temperature and dewpoint while wind gains remain positive past 200 km.The per-variable pattern is consistent with the roughly 2.4× normalized-space signal imbalance.Source: §F.1 · p. 31, Figure 10 · p. 32, Figure 11 · p. 32

F.2 Additional regional case studies

  • In the dense Los Angeles basin, Flow Guidance and Diffusion-SDA reduce T2 RMSE by 36.1% and 35.7%, versus 36.9% for 3D-Var; in Florida, their u10 reductions are 71.8% and 71.0%, versus 65.9%.The regional cases align with a small generative-method gap where stations densely cover a region and a larger gap where substantial areas are ungauged.Source: Figure 12 · p. 33, §F.2 · p. 33, Figure 13 · p. 34

F.3 Full benchmark in physical units

With ensemble size controlled at M=1, Flow Guidance and Diffusion-SDA still outperform D-Flow. The full-year evaluation also checks whether the 360-snapshot results are representative.

F.3 Full benchmark in physical units

  • At M=1, Flow Guidance and Diffusion-SDA improve over ERA5 by 34.0% and 33.4%, respectively, versus 30.5% for D-Flow.This control supports the authors’ conclusion that D-Flow’s lower accuracy reflects noise-space optimization rather than an ensemble-size disadvantage.Source: §F.3 · p. 34
  • Table 14 reports dense-benchmark absolute RMSE in physical units, from which the main-text percentage improvements over ERA5 are computed.Wind is reported in m s−1 and temperature and dewpoint in K.Source: Table 14 · p. 34, §F.3 · p. 34

F.4 Full-year benchmark

  • Across 8,664 valid hourly snapshots in 2023, test-station improvements match the 360-snapshot averages within 0.1 percentage points on average and 0.3 points for every variable.The evaluation uses the dense benchmark protocol.Source: §F.4 · p. 35, Table 15 · p. 35
  • Both benchmarks retain 360 snapshots, averaging 9,818 available stations per dense snapshot and 833 per sparse snapshot.Snapshots are filtered using split-specific minimum station thresholds; the sparse thresholds are scaled from the dense benchmark.Source: §F.4 · pp. 34–35
Loading 2610.00728v1…