PaperlayerView exact revision on arXiv

Source-linked AI summary

DiffusionPDE: Generative PDE-Solving Under Partial Observation

Jiahe Huang, Guandao Yang, Zichen Wang, Jeong Joon Park

arXiv:2406.17763v2cs.LGcs.AIcs.CVcs.NAmath.NA

TL;DR

PDE methods often assume complete observations, despite sparse measurements being common and challenging for real-world applications. DiffusionPDE jointly models coefficients and solutions and guides denoising with observations and PDE constraints. The paper reports improved sparse-observation results across forward and inverse PDE tasks, with limitations in dynamic-PDE coverage and sampling speed.

  • Problem

    Existing PDE approaches often rely on complete observations, although real-world measurements are commonly sparse.

    Source: §1 · p. 1

  • Method

    DiffusionPDE learns the joint distribution of PDE coefficients and solutions, then guides diffusion denoising with sparse observations and PDE constraints.

    Source: §1 · pp. 1–2, §5 · p. 10

  • Results

    Across sparse-observation forward and inverse PDE tasks, DiffusionPDE outperforms existing learning-based methods; on bounded Navier–Stokes, relative error falls from 12.0% to 2.7% versus GraphPDE with 1% observed points.

    Source: §1 · p. 2, §4.4 · p. 8

  • Takeaways & Limitations

    A single model can recover coefficients and solutions from sparse observations on either side without retraining.

    Source: §4.4 · pp. 8–9

    DiffusionPDE currently solves slices of 2D dynamic PDEs, and its sampling procedure is slow.

    Source: §5 · p. 10

Abstract

from arXiv · show

We introduce a general framework for solving partial differential equations (PDEs) using generative diffusion models. In particular, we focus on the scenarios where we do not have the full knowledge of the scene necessary to apply classical solvers. Most existing forward or inverse PDE approaches perform poorly when the observations on the data or the underlying coefficients are incomplete, which is a common assumption for real-world measurements. In this work, we propose DiffusionPDE that can simultaneously fill in the missing information and solve a PDE by modeling the joint distribution of the solution and coefficient spaces. We show that the learned generative priors lead to a versatile framework for accurately solving a wide range of PDEs under partial observation, significantly outperforming the state-of-the-art methods for both forward and inverse directions.

1 Introduction

PDE solvers commonly assume complete observations, although real measurements are often sparse, limiting their use on real-world cases. DiffusionPDE jointly models PDE coefficients and solutions, using sparse observations and PDE constraints to guide reconstruction; experiments report improved performance on sparse-observation forward and inverse tasks.

1 Introduction

  • Existing solvers and learning-based approaches rely on complete information, while real measurements are often sparse, making real-world application challenging.The paper notes that this limitation confines existing methods primarily to synthetic simulations with full scene configurations.Source: §1 · p. 1
  • DiffusionPDE models the joint distribution of coefficients and solutions, then guides denoising with sparse observations and PDE constraints to reconstruct both.The framework is designed for forward and inverse problems under highly partial observations, typically 1–3% of total information.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • Across tested static and temporal PDEs, DiffusionPDE outperforms state-of-the-art learning-based methods on sparse-observation forward and inverse problems and is comparable with full observations.The paper reports tests on Darcy Flow, Poisson, Helmholtz, Burgers’, and Navier–Stokes equations.Source: §1 · p. 2

2 Related Works

Related work spans numerical and learned forward solvers, inverse PDE methods, and diffusion models. The paper positions its approach at the intersection of these lines, focusing on inverse solving with partial observations.

Forward PDE Solvers

  • Numerical methods such as Finite Element and Boundary Element methods can be computationally expensive or difficult to set up, while neural operators learn mappings from PDE parameters to solutions [3].PINNs [16] use PDE constraints as self-supervised losses but are difficult to scale because they require network optimization.Source: §2 · p. 2

PDE inverse problem

  • Inverse PDE methods include extensions of numerical solvers, inverse PINNs, and learned operators, but these methods generally assume full observations [36] [37] [38] [39] [40] [42].Generative-prior approaches using graph neural networks address partial observations, but the cited works had not demonstrated high-resolution PDE solving [43] [8].Source: §2 · p. 3

Diffusion models

  • Diffusion models support guided generation and have been applied to PDE data, spectra, and sparse-observation 2D flows, but inverse solving under partial observation remains underexplored [46–54].Source: §2 · p. 3

3 Methods

The methods train a diffusion model on joint coefficient and solution data, then guide denoising with observations and PDE constraints. This single-network framework handles forward and inverse recovery from sparse observations on either field or both.

3.1 Overview

  • A pre-trained diffusion model learns the joint distribution of PDE coefficients and solutions, with sparse observations and the PDE function guiding iterative denoising to recover full data.The method targets forward and inverse problems under uncertainty, using observations from either or both spaces.Source: §3.1 · p. 3

3.2 Prelimary: Diffusion Models and Guided Diffusion

  • Guided diffusion modifies score-based denoising with measurement guidance; the described DPS approach uses a likelihood approximation under a Gaussian sparse-measurement assumption.The diffusion model progressively adds Gaussian noise in its forward process and learns a reverse denoising process.Source: §3.2 · pp. 3–4

3.3 Solving PDEs with Guided Diffusion

  • For dynamic systems, the method recovers the initial condition and solution at a specified time from sparse observations on either field or both; Burgers’ experiments also model states across all timesteps.For static systems, coefficient and solution fields satisfy the PDE and boundary constraint; the observation and PDE losses encode those constraints during guidance.Source: §3.3 · pp. 4–5
  • Darcy Flow examples show recovery of both coefficient and solution fields from sparse observations on the coefficient, solution, or both.Source: Figure 2 · p. 5
  • One network learns the joint coefficient–solution distribution, and physics-informed DPS guides inference to satisfy sparse observations and the PDE.The method unifies forward and inverse operators and accommodates arbitrary sparsity patterns without retraining.Source: §3.3 · p. 5

4 Experiments

The experiments evaluate DiffusionPDE across varied PDE settings and sparse-observation tasks, comparing it with learning-based solvers in forward and inverse problems. They report stronger sparse-observation results, with analyses of guided sampling, temporal recovery, and observation conditions.

4.1 PDE Problem Settings

  • Experiments cover static and temporal PDE settings, including Helmholtz, non-bounded and bounded Navier–Stokes, and Burgers’ equation.For non-bounded Navier–Stokes, the model learns the joint distribution of w0 and wT; the bounded setting uses velocity and pressure with boundary conditions.Source: §4.1 · pp. 5–6

4.2 Dataset Preparation and Training

  • Training uses 50,000 data pairs per PDE; the joint diffusion models are trained on three A40 GPUs for approximately four hours.For Burgers’ equation, the model is trained on 50,000 samples, and five of 128 spatial points are used as sensors across time.Source: §4.2 · p. 7

4.3 Baseline Methods

  • The compared neural PDE baselines are generally trained on full observations; GraphPDE is additionally compared on bounded inverse Navier–Stokes, while CFG-based methods and OFormer are evaluated on unbounded Navier–Stokes.The authors report that PINO, DeepONet, PINNs, and FNO do not demonstrate operation on partial observations; attempts to train them on partial observations produced degenerate outcomes.Source: §4.3 · pp. 7–8, Figure 4 · p. 8

4.4 Main Evaluation Results

  • DiffusionPDE outperforms PINO [4], DeepONet, PINNs, and FNO [3] in both directions under sparse observations; for bounded Navier–Stokes, it reduces GraphPDE [8] error from 12.0% to 2.7%.The bounded Navier–Stokes comparison uses 1% observed points. Table 1 reports sparse-observation relative errors for solutions or final states and coefficients or initial states.Source: Table 1 · p. 7, §4.4 · p. 8, Figure 5 · p. 9
  • DiffusionPDE jointly recovers coefficients and solutions from sparse observations on either space or both, without retraining.The Darcy Flow experiment reports low errors when observations are on the coefficient space, solution space, or both.Source: §4.4 · pp. 8–9

4.5 Advantage of Guided Sampling

  • With 500 observations, guided sampling achieves lower relative errors than Diffusion using CFG in both forward and inverse unbounded Navier–Stokes problems.Shu et al.’s autoregressive optimized CFG has approximately 13% final-state error; OFormer’s relative errors are around 17% and 23%, higher than DiffusionPDE’s.Source: §4.5 · p. 9

4.6 Recovering Solutions Throughout a Time Interval

  • DiffusionPDE recovers all Burgers’ equation time steps from continuous observations at five sparse sensors, with a test relative error of 2.68%.Baselines were trained on neighboring snapshot pairs to provide continuous observations.Source: Figure 4 · p. 8, §4.6 · p. 9

4.7 Additional Analysis

  • PDE guidance reduces Helmholtz relative error for u from 9.3% to 0.6% and for a from 13.2% to 9.4%; errors decrease with more observations and become insensitive above 3%.DiffusionPDE is also reported robust to grid and non-uniformly concentrated sampling patterns, and supports continuous coordinates through bilinear interpolation without compromising accuracy.Source: §4.7 · pp. 9–10

5 Conclusion and Future Work

DiffusionPDE uses generative priors to jointly model coefficient and solution spaces, guiding denoising with sparse observations and PDE constraints. The work reports gains from combining PDE and observation losses, while identifying scope, accuracy, and sampling-speed limitations.

5 Conclusion and Future Work

  • DiffusionPDE learns the joint distribution of coefficient or initial-state and solution or final-state spaces, then guides denoising with sparse measurements and PDE constraints.The conclusion presents this as a step toward a general PDE-solving framework using generative models.Source: §5 · p. 10
  • The current model solves slices of 2D dynamic PDEs; full time intervals, accuracy in unconstrained spaces, and faster sampling remain future work.The paper identifies all three as directions for further research.Source: §5 · p. 10

B.1 Static PDEs

  • For static Darcy Flow, Poisson, and Helmholtz data, coefficients are generated from Gaussian random fields and solutions use second-order finite differences.The resulting coefficient and solution grids are 128 × 128, with a mollifier enforcing no-slip solution boundaries.Source: §B.1 · p. 15

B.2 Non-bounded Navier-Stokes Equation; B.3 Bounded Navier-Stokes Equation

  • Non-bounded Navier–Stokes data use a Gaussian-random-field initial condition and a pseudo-spectral solver, while bounded cases use Difftaichi’s Marker-and-Cell method.The non-bounded simulation uses Fourier transforms; the bounded setup uses pressure-Poisson solves and predictor-corrector updates on a grid with a random cylinder.Source: §B.2 · p. 15, §B.3 · p. 15

B.4 Burgers’ Equation

  • Burgers’ equation data use Gaussian-random-field initial conditions, a spectral solver, and a one-second simulation with 127 additional timesteps.The final space has resolution 128 × 128.Source: §B.4 · p. 15

C Guided Sampling Details

  • Sparse-observation sampling uses observation-loss guidance for the first 80% of iterations, then introduces PDE loss and reduces observation-loss weight tenfold.The weighting adjustment shifts the primary guidance toward the PDE dynamics.Source: C Guided Sampling Details · p. 16

D Improvement in Prediction through PDE Loss Term

  • Adding PDE loss to observation-loss guidance significantly reduces errors in both coefficients or initial states and solutions or final states.The reported visual comparison covers Darcy Flow, Poisson, and Helmholtz and finds combined guidance more accurate than observation loss alone.Source: D Improvement in Prediction through… · p. 16, Figure 8 · p. 17

E Additional Results on All PDEs with Sparse Observation

With sparse observations, DiffusionPDE is reported to outperform the compared methods across several forward and inverse PDE tasks. Additional tests report similar performance under changed Darcy coefficients and lower-viscosity settings.

E Additional Results on All PDEs with Sparse Observation

F Solving Forward and Inverse Problems with Full Observation

With full observations, DiffusionPDE and PINO generally outperform the other methods, and DiffusionPDE performs best on static PDEs. The reported errors for DiffusionPDE are below 10% across the discussed PDE classes and both problem directions.

F Solving Forward and Inverse Problems with Full Observation

G Training Baselines Methods on Partial Inputs

The authors train baseline models on full observations because sparse inputs hinder physics-informed PDE-loss computation and degrade other baselines. Optimizing baseline outputs reduces errors, but their overall error remains much higher than DiffusionPDE.

G Training Baselines Methods on Partial Inputs

  • PINO, DeepONet, PINNs, and FNO are trained on full observations because sparse inputs impede PDE-loss computation for physics-informed models and yield poor results for DeepONet and FNO.A partially trained DeepONet produces similar test outputs across inputs and generalizes poorly to observation distributions unlike its training distribution.Source: G Training Baselines Methods on… · pp. 20–21, Figure 14 · p. 21

H Baseline Optimization

  • Optimizing baseline outputs reduces errors and smooths solutions, but their errors remain much higher than DiffusionPDE’s.The authors suggest noisy derivatives of a and u may make the optimization difficult.Source: H Baseline Optimization · p. 21

I Standard Deviation of DiffusionPDE Experiment Results

DiffusionPDE shows relatively low standard deviations across tests, with full observations improving stability and more complex inverse problems showing greater variability.

I Standard Deviation of DiffusionPDE Experiment Results

J Runtime Analysis

Inference cost is lower than that of Shu et al. [54] for Navier–Stokes, with the advantage growing as more time steps are used; optimization-based full-observation methods are slower.

J Runtime Analysis

K Robustness of DiffusionPDE

DiffusionPDE maintains useful reconstructions across noisy and varying sparse observations, and its errors generally decrease as observations increase. The reported results also show performance across resolutions and some prediction uncertainty from different initial noise.

K Robustness of DiffusionPDE

L Solving Forward and Inverse Problems with Different Numbers of Observations

  • Errors decrease as observations increase; u is recovered well with about 2% observation points, while both a and u have 1%–10% errors with about 6% for most PDE families.The results cover forward and inverse problems with 100, 300, 500, or 1,000 observations; performance becomes insensitive to observation count above 3%.Source: L Solving Forward and Inverse Problems… · pp. 24–25, Figure 21 · p. 25

M Solving Forward and Inverse Problems across Varied Resolutions

N Comparison with Other Baselines

The RBF kernel’s reported errors rise substantially for inverse problems, while U-Net errors vary across input-point settings and are higher than the diffusion model’s reported error for Navier–Stokes.

N Comparison with Other Baselines

Loading 2406.17763v2…