PaperlayerView exact revision on arXiv

Source-linked AI summary

Spatio-temporal Latent Denoising Diffusion Probabilistic Models for Reduced-order Modeling of Parametrized Dynamical Systems

Michiel Nikken, Nicolò Botteghi, Federico Califano, Silke Glas

arXiv:2609.17768v1math.NAcs.NA

TL;DR

High-dimensional full-order models can make real-time and multi-query simulation costly. The paper proposes a POD-autoencoder latent DDPM that generates parameter-conditioned spatio-temporal trajectories, and reports accurate, temporally consistent predictions for unseen parameters and times beyond training.

  • Problem

    High-dimensional FOMs can make real-time and multi-query simulation computationally prohibitive, motivating reduced-order surrogates.

    Source: §1 · pp. 1–2

  • Method

    DDP-ROM combines POD and an autoencoder for latent reduction with a parameter-conditioned DDPM that generates spatio-temporal trajectories from an initial state.

    Source: §1 · p. 2, §5 · p. 16

  • Results

    On fluid flow around an obstacle, DDP-ROM was 400 times faster than the FOM and produced temporally consistent predictions for unseen parameters in interpolation and extrapolation tasks.

    Source: §5 · p. 16

  • Takeaways & Limitations

    The authors conclude that DDP-ROM is effective for parameterized dynamical systems and that its probabilistic predictions can support uncertainty-aware ROMs.

    Source: §5 · p. 16

    Direct DDPM application to high-dimensional data remains computationally demanding because inference requires many neural-network evaluations whose cost grows with data dimensionality.

    Source: §1 · p. 2

Abstract

from arXiv · show

Many scientific problems require accurate modeling of complex physical phenomena, such as fluid dynamics or climate modeling. These phenomena often result in high-dimensional and thus computationally expensive computational models that can limit their application to real-time and multi-query problems. Model-order reduction (MOR) is an approach that seeks to approximate full-order models (FOMs) using reduced-order models (ROMs), trading a minor reduction in accuracy for a major reduction in computational cost. In this work, we propose a non-intrusive MOR method using generative machine learning by means of denoising diffusion probabilistic models (DDPMs) for generating solutions of the dynamical systems under different instances of their parameters. Unlike conventional DDPMs, which often operate purely in the spatial domain, we aim to generate spatio-temporal solutions to improve quality and temporal coherence. In addition, we embed the DDPM in a latent space obtained by sequentially applying proper orthogonal decomposition and an autoencoder to reduce the data dimensionality and the computational cost of the DDPM. We test our approach on a parametrized 2D fluid flow around an obstacle. The numerical experiments demonstrate that our latent DDPM can (i) produce accurate and temporally coherent solutions, (ii) achieve strong generalization capabilities to scenarios involving unseen parameter values, and (iii) extrapolate in time beyond the training horizon.

1 Introduction

High-dimensional full-order models can make real-time and multi-query simulation costly, motivating accurate, efficient reduced-order surrogates. The paper introduces a non-intrusive, spatio-temporal latent DDPM approach for forecasting parameterized systems.

1 Introduction

  • High-dimensional FOMs can make real-time and multi-query tasks computationally prohibitive, motivating ROMs that reduce complexity while retaining essential dynamics.Examples include optimal control, uncertainty quantification, and digital twins.Source: §1 · pp. 1–2
  • Non-intrusive MOR can build ROMs from system data when the FOM or governing PDE is unavailable, while neural networks can learn nonlinear state representations and reduced dynamics.Existing approaches include operator inference, dynamic mode decomposition, sparse identification, neural operators, and generative models.Source: §1 · p. 2
  • Direct DDPM application to high-dimensional data remains computationally demanding because inference requires many neural-network evaluations whose cost grows with data dimensionality.DDPMs progressively add noise and learn a reverse denoising process for generating samples.Source: §1 · p. 2
  • DDP-ROM embeds diffusion in a POD-plus-autoencoder latent space and generates parameter-conditioned spatio-temporal trajectories, fixing the encoded initial state during denoising.The generated latent trajectory is decoded through the autoencoder and POD to produce a full-space prediction.Source: §1 · pp. 2–3, Figure 1 · p. 3
  • The authors report that DDP-ROM produces stable time-extrapolative predictions, outperforms a POD DL-ROM baseline, and can estimate uncertainty from its probabilistic predictions.The method is tested on parameterized fluid flow around an obstacle.Source: §1 · pp. 2–3

2 Preliminaries

The preliminaries establish the parameterized dynamical-system setting and the dimensionality-reduction and diffusion-model components used by the proposed ROM.

2.1 Problem Setting

  • The problem setting treats spatially discretized nonlinear parameterized PDEs as high-dimensional initial-value problems whose solutions are sampled as snapshots across times and parameter instances.The snapshot collection forms the data used to construct the reduced representation.Source: §2.1 · pp. 3–4

2.2 POD-AE Dimensionality Reduction

  • POD projects snapshots onto an orthogonal linear basis, and an autoencoder then provides a nonlinear encoder-decoder mapping to a lower-dimensional latent representation.The two-stage POD-AE reduction addresses settings where linear approximation spaces may be inefficient, including advection-dominated problems.Source: §2.2 · pp. 4–5

2.3 Denoising Diffusion Probabilistic Models

  • A DDPM learns a reverse Markov process that progressively denoises Gaussian noise into structured data, alongside a forward process that adds noise to data.The forward noise schedule is chosen so that the distribution approaches a standard normal after the prescribed diffusion steps.Source: §2.3 · p. 5

2.3.1 Training Objectives

  • The simplified noise-prediction objective trains a neural network to estimate the noise injected during diffusion rather than directly predicting the reverse-process mean.The formulation uses the closed-form forward process and samples a diffusion-step index during training.Source: §2.3.1 · pp. 5–6

2.3.2 Conditional Generation

  • Classifier-free guidance conditions generation by supplying conditioning variables to the diffusion model, avoiding the auxiliary classifier required by classifier guidance.Training randomly drops the conditioning input, and the guided noise prediction trades off conditional and unconditional predictions.Source: §2.3.2 · p. 6
  • Inpainting is another conditioning mechanism: the reverse process preserves known data regions while generating completions for unobserved regions.Source: §2.3.2 · p. 6

3 Spatio-temporal Latent Denoising Diffusion Reduced-order Model

DDP-ROM reduces snapshots to a latent space and models complete trajectories there with a parameter-conditioned diffusion process. At inference, it fixes the initial state, denoises a chosen horizon, and decodes the trajectory.

3 Spatio-temporal Latent Denoising Diffusion Reduced-order Model

  • DDP-ROM combines low-dimensional latent-space diffusion with conditioned spatio-temporal generation from an initial state and parameter vector, avoiding autoregressive rollouts.The design is intended to reduce reverse-diffusion cost and improve temporal consistency.Source: §3 · p. 6

3.1 Offline and Online Phases

  • The offline phase first constructs a POD basis, then trains an autoencoder on POD coefficients and uses the resulting latent snapshots to form trajectories for DDPM training.The DDPM learns the conditional relation between parameter values and latent trajectories using classifier-free conditioning.Source: §3.1 · p. 7
  • Online prediction projects the initial condition into latent space, samples a Gaussian latent trajectory, and fixes its initial state at every reverse-diffusion step.The generated latent snapshots are decoded and mapped through the POD basis to recover full-space predictions.Source: §3.1 · p. 7
  • The prediction horizon H can differ from the training trajectory length, allowing DDP-ROM to generate trajectories beyond the training horizon.The paper attributes this capability to a noise-prediction network convolutional along the trajectory's time dimension.Source: §3.1 · p. 7

4 Numerical Experiments

The experiments evaluate DDP-ROM against POD DL-ROM on fluid flow around an obstacle, including unseen parameters and times beyond the training horizon. They also examine prediction uncertainty across repeated samples.

4.1 Fluid Flow Around Obstacle

  • The test uses 800 training trajectories of 10 s and 125 unseen-parameter test trajectories of 20 s to assess parameter generalization and time extrapolation.The fluid-flow model has 80,592 velocity degrees of freedom.Source: §4.1 · pp. 8–9
  • Every DDP-ROM outperforms all POD DL-ROMs at almost all latent dimensions on total test-set prediction error for unseen parameters and time extrapolation.Table 1 compares feedforward, convolutional, and invertible autoencoder architectures; the supplied passages do not give its cell values.Source: Table 1 · p. 10, §4.1 · p. 10
  • At t > 10 s, DDP-ROM's prediction error is 0.15 versus 0.47 for DL-ROM on one test trajectory, while average DDP-ROM error remains stable during extrapolation.DL-ROM error rises rapidly beyond 10 s, although it may outperform some DDP-ROMs within the training horizon when its latent space is sufficiently large.Source: §4.1 · pp. 10, 12
  • For a low-velocity test case, DDP-ROM predicts velocity features well but struggles on the domain's right side after 10 s because the preceding transient has not reached that region.The compared test parameters produce the largest mean relative errors across the test trajectories.Source: §4.1 · p. 11

4.1.1 Uncertainty quantification with DDP-ROM

  • Across 10 predictions, regions with greater standard deviation tend to coincide with higher mean absolute error, and uncertainty is larger for the slower-flow transient case.The standard deviation and mean absolute error do not match exactly; the slower-flow case also lacks the full transient in the training horizon.Source: §4.1.1 · p. 13, Figure 11 · p. 14, Figure 12 · p. 14
  • For inv-AE DDP-ROM, larger latent dimensions produce a larger spread in relative error over time.The comparison covers latent dimensions 2, 5, and 10.Source: §4.1.1 · p. 13, Figure 15 · p. 15, Figure 16 · p. 15

Related papers

Loading 2609.17768v1…