PaperlayerView exact revision on arXiv

Source-linked AI summary

InverseBench: Benchmarking Plug-and-Play Diffusion Priors for Inverse Problems in Physical Sciences

Hongkai Zheng, Wenda Chu, Bingliang Zhang, Zihui Wu, Austin Wang, Berthy T. Feng, Caifeng Zou, Yu Sun, Nikola Kovachki, Zachary E. Ross, Katherine L. Bouman, Yisong Yue

arXiv:2503.11043v2cs.LG

TL;DR

Scientific inverse problems are difficult because they infer unknown sources from indirect, noisy observations under ill-posed and complex physical models. InverseBench systematically evaluates plug-and-play diffusion-prior methods across such problems, finding generally strong and forward-model-robust performance but important sensitivity to PDE constraints and out-of-distribution sources.

  • Problem

    Scientific inverse problems infer unknown sources from indirect, noisy observations and are challenging because of ill-posedness, complex physics, and unknown measurement noise.

  • Method

    InverseBench is a comprehensive framework for systematically and extensibly evaluating plug-and-play diffusion-prior approaches.

  • Results

    Plug-and-play diffusion-prior methods generally outperform conventional baselines and remain robust across forward models, with average changes of 0.48dB PSNR and 0.016 SSIM across models.

  • Takeaways & Limitations

    The benchmark identifies stability conditions for forward-model queries and inference speed as important directions for improving plug-and-play diffusion-prior methods.

  • Takeaways & Limitations

    Methods can be biased toward the diffusion prior when the unknown source lies outside its distribution, and PDE-based forward models can cause numerical instability and hyperparameter sensitivity.

Abstract

from arXiv · show

Plug-and-play diffusion priors (PnPDP) have emerged as a promising research direction for solving inverse problems. However, current studies primarily focus on natural image restoration, leaving the performance of these algorithms in scientific inverse problems largely unexplored. To address this gap, we introduce \textsc{InverseBench}, a framework that evaluates diffusion models across five distinct scientific inverse problems. These problems present unique structural challenges that differ from existing benchmarks, arising from critical scientific applications such as optical tomography, medical imaging, black hole imaging, seismology, and fluid dynamics. With \textsc{InverseBench}, we benchmark 14 inverse problem algorithms that use plug-and-play diffusion priors against strong, domain-specific baselines, offering valuable new insights into the strengths and weaknesses of existing algorithms. To facilitate further research and development, we open-source the codebase, along with datasets and pre-trained models, at https://devzhk.github.io/InverseBench/.

1 INTRODUCTION

InverseBench addresses the limited evaluation of plug-and-play diffusion priors in scientific inverse problems by benchmarking them across five physics-driven domains. It compares 14 PnP diffusion-prior algorithms with domain-specific baselines and reports task-dependent accuracy–efficiency trade-offs.

  • Inverse problems infer unknown sources from indirect, noisy observations and are difficult because of ill-posedness, complex physics, and unknown measurement noise.
  • PnP diffusion-prior methods decompose inference into a diffusion-model prior and a forward-model likelihood for fitting observations.
  • InverseBench evaluates five scientific inverse problems—optical tomography, black hole imaging, medical imaging, seismology, and fluid dynamics—with physics-based forward models that can be highly nonlinear and difficult to evaluate.
  • The benchmark compares 14 representative PnP diffusion-prior algorithms across tasks and establishes strong domain-specific baselines for comparison with traditional methods.
  • PnP diffusion-prior methods generally perform strongly with suitable training data and appropriate tuning, but sensitivity increases for constrained forward models and sources outside the prior distribution.
  • InverseBench is modular and open-sources its codebase, datasets, and pre-trained models to support scalable evaluation and further research.

2 PRELIMINARIES

Inverse problems infer unknown sources from noisy observations through potentially ill-posed forward models, motivating regularization with diffusion priors. PnPDP methods combine a learned prior with observation likelihood information through several algorithmic strategies.

  • 2.1 INVERSE PROBLEMS: Inverse problems use observations y generated from an unknown source z through a noisy forward model G, then seek an inverse mapping G†.The forward model represents a scientific measurement or sensing device.
  • 2.1 INVERSE PROBLEMS: Ill-posedness, unknown measurement noise, and complex physics make inverse problems difficult and necessitate regularization beyond simple local smoothness.Diffusion priors are attractive because they can capture global or anisotropic properties.
  • 2.2 DIFFUSION MODELS: Diffusion models learn score functions for high-dimensional data distributions and generate samples by solving a reverse-time denoising stochastic differential equation.The forward process adds Gaussian noise, while the backward process denoises it.
  • 2.3 PLUG-AND-PLAY DIFFUSION PRIORS FOR INVERSE PROBLEMS: PnPDP methods use a diffusion model or denoising network as a plug-and-play prior while the forward model supplies observation-likelihood information.Their target posterior is expressed as p(x|y) ∝ p(x)p(y|x).
  • 2.3 PLUG-AND-PLAY DIFFUSION PRIORS FOR INVERSE PROBLEMS: Existing PnPDP approaches include guidance-based, variable-splitting, variational-Bayes, and sequential-Monte-Carlo methods.These categories differ in how they combine diffusion sampling with forward-model information.

3 INVERSEBENCH

INVERSEBENCH evaluates diffusion-prior algorithms on five scientific inverse problems with diverse forward-model structures and computational constraints. The benchmark spans optical scattering, MRI, black hole imaging, full waveform inversion, and Navier–Stokes-based fluid dynamics.

  • 3 INVERSEBENCH: INVERSEBENCH covers five scientific inverse problems: linear inverse scattering, compressed sensing MRI, black hole imaging, full waveform inversion, and Navier–Stokes dynamics.The problems span optical microscopy, medical imaging, astronomy, seismology, and fluid dynamics.
  • Linear inverse scattering: Linear inverse scattering recovers permittivity contrast from scattered light using Green’s-function models and a first-Born linearization.The formulation tests algorithms designed specifically for linear problems.
  • Compressed sensing MRI: Compressed sensing MRI reconstructs images from subsampled multi-coil Fourier measurements, while retaining complex-domain and fine-detail challenges.The parallel-imaging formulation uses coil sensitivity maps and measurement noise.
  • Black hole imaging: Black hole imaging uses noisy VLBI visibilities and nonlinear closure quantities, with additional super-resolution and phase-ambiguity challenges.These properties can produce a non-convex forward model and multiple posterior modes.
  • Full waveform inversion: Full waveform inversion recovers subsurface compressional-wave velocity from observed wavefields, but free-surface observations yield non-unique solutions and repeated forward solves are expensive.The conventional adjoint-state method formulates FWI as local optimization requiring a sufficiently accurate initialization.
  • Navier–Stokes equation: Navier–Stokes inversion recovers initial vorticity from noisy partial observations after evolution under a viscous incompressible-flow solution operator.The operator lacks a closed-form solution and differentiating through thousands of solver steps is challenging.

4 EXPERIMENTS

The experiments evaluate diffusion priors across five scientific inverse problems using shared pretrained models, multiple accuracy and efficiency metrics, and domain-specific baselines. Results show strong average performance and robustness, but sensitivity to PDE stability, initialization, computational cost, and prior-distribution mismatch.

  • Experimental setup: The benchmark covers black hole imaging, full waveform inversion, linear inverse scattering, compressed sensing MRI, and Navier–Stokes, with diffusion priors pretrained separately for each problem.The same diffusion checkpoint is used across plug-and-play methods for each problem to support fair comparison.
  • Evaluation metrics: PnPDP methods are compared using PSNR, SSIM, observation consistency, relative ℓ2 error for PDE problems, computational-efficiency metrics, and an aggregate ranking score.The ranking score averages per-metric algorithm ranks within each problem.
  • Main findings: Given a suitable diffusion-prior training dataset, PnPDP methods generally outperform conventional baselines across the scientific inverse problems.The comparison reflects the stronger prior information incorporated by diffusion models.
  • Main findings: Well-initialized classic optimization methods can outperform PnPDP methods in full waveform inversion, although PnPDP methods rely much less on initialization.Randomly or constantly initialized baselines are outperformed, whereas heavily blurred ground-truth initialization gives classic methods an advantage.
  • Main findings: For closed-form forward models, DAPS and PnP-DM tend to be more accurate but require more forward-model queries, higher computational cost, and more careful tuning.The larger hyperparameter spaces of these methods contribute to their tuning burden.
  • Main findings: PDE-based forward models reverse the closed-form trend: DAPS and PnP-DM can be numerically unstable, while PnPDP recovery is biased toward the diffusion prior for out-of-distribution sources.Small step sizes may ignore measurements, larger ones may cause failure, and unfamiliar sources can inherit prior-specific features.
  • Main findings: As measurement sparsity increases, the average performance gain of top PnPDP methods over baselines generally grows across three problems.Greater sparsity makes the inverse problem more ill-posed while widening the observed performance gap.
  • Main findings: PnPDP methods show smaller average performance changes across forward models than end-to-end approaches: 0.48dB PSNR and 0.016 SSIM versus 9.58dB and 0.21.Traditional baselines change by 1.62dB PSNR and 0.027 SSIM under the same comparison.

5 DISCUSSION

The discussion identifies stability-aware forward-model querying and inference speed as key challenges for PnPDP methods. It also points to robustness against model error and prior mismatch as further research directions.

  • Research challenges: Current PnPDP methods do not account for forward-model stability conditions, causing degraded performance and numerical instability.This is especially consequential for PDE-based problems, where violating input constraints can produce meaningless solutions.
  • Research challenges: Inference speed remains a major weakness because almost all PnPDP methods are less computationally efficient than conventional baselines.The discussion identifies substantial room for optimization.
  • Research opportunities: Robustness to model error and prior mismatch are highlighted as additional research directions.

A.1 TABLES OF MAIN RESULTS

The appendix organizes the main results across five scientific inverse problems and reports reconstruction, consistency, and computational-complexity metrics. Tables include both aggregate results and numerical-stability annotations where applicable.

  • Main results: Tables 3–8 report results for linear inverse scattering, compressed sensing MRI, black hole imaging, full waveform inversion, and Navier–Stokes inversion.The reported metrics include PSNR, SSIM, chi-squared, and relative ℓ2 error, depending on the problem.
  • Compressed sensing MRI: Compressed sensing MRI results include mean and standard deviation over 94 test cases, including ×4-acceleration generalization with raw measurements.
  • Numerical stability: Full waveform inversion and Navier–Stokes tables mark cases excluded because of numerical instability.The FWI table also identifies Gaussian-blurred initialization for marked results.
  • Computation: Table 9 summarizes the metrics used to capture each algorithm’s computational complexity.Figure 6 separately defines runtimes for forward models, diffusion models, and their gradient evaluations; Navier–Stokes forward gradients are unavailable under black-box access.

A.2 EXTENDED EVALUATION OF CS-MRI

The extended evaluation examines compressed sensing MRI reconstruction quality beyond generic image metrics, while documenting the physical and preprocessing setups used for related inverse-scattering and MRI experiments.

  • Diagnostic evaluation: Compressed sensing MRI quality is assessed with a pathology detector because high PSNR and SSIM do not always indicate clinically reliable reconstructions.The detector is a medium-size YOLOv11 model fine-tuned on fully sampled images with fastMRI+ pathology annotations.
  • Linear inverse scattering: The linear inverse-scattering setup models a permittivity distribution using transmitters, receivers, scattered fields, and a first-Born-approximation forward model.The discretized system uses Green’s-function matrices and precomputed SVD to support plug-and-play methods for linear inverse problems.
  • Compressed sensing MRI: MRI experiments use raw multi-coil fastMRI knee k-space data, ESPIRiT coil sensitivities, 320×320 preprocessing, and MVUE magnitude images as ground truth.The data represent sparse-measurement recovery in compressed sensing MRI.

B.3 BLACK HOLE IMAGING

Black hole imaging uses sparse, noisy VLBI measurements and nonlinear closure quantities, producing a non-convex inverse problem with systematic multimodal solutions. The section also describes the forward-model construction and uncertainty metrics used for evaluation.

  • Black hole imaging: VLBI measurements sample spatial Fourier frequencies through telescope-pair visibilities, while atmospheric and instrumental errors corrupt amplitudes and phases.Closure quantities are introduced to reduce the impact of these calibration errors.
  • Measurement model: Closure phase cancels visibility phase errors, while log closure amplitudes help remove amplitude gain errors and retain inferential information.Because these quantities are nonlinear transformations of visibilities, the resulting forward model is non-convex.
  • Evaluation: Black hole imaging data consistency is evaluated with a χ2 statistic using estimated standard deviations for closure phase, log closure amplitude, and flux.
  • Multi-modal observation: Sparse and non-convex measurements create multimodal behavior, allowing samples that fit observations well and have high prior likelihood while differing substantially from the true image.DAPS and PnP-DM discover two systematic modes illustrated in Figure 7.

B.6 PRETRAINED DIFFUSION MODEL DETAILS

The paper trains diffusion models using established UNet-based pipelines and provides a model card for the pre-trained models.

B.7.1 PROBLEM-SPECIFIC BASELINES

The benchmark uses domain-specific baselines spanning black hole imaging, waveform inversion, scattering, MRI, and fluid dynamics, with systematic hyperparameter tuning.

  • Black hole imaging: Black hole imaging uses SMILI and eht-imaging as baseline methods.The test dataset is preprocessed to match their default hyperparameters.
  • Full waveform inversion: Full waveform inversion compares LBFGS and Adam, with specified iteration, update-step, line-search, and learning-rate settings.The traditional method is initialized from a smoothed ground-truth version.
  • Linear inverse scattering: Linear inverse scattering includes FISTA-TV with batch size B = 20 and τ = 5 × 10−7.
  • Compressed sensing multi-coil MRI: Compressed sensing multi-coil MRI uses Wavelet+ℓ1, TV, Residual UNet, and E2E-VarNet baselines.Traditional methods are fine-tuned by hyperparameter search, while end-to-end models use Adam with learning rate 1 × 10−4 until convergence.
  • Navier-Stokes equation: Navier-Stokes experiments include EKI with 2048 particles and 500 update steps, plus DPS-fGSG and DPS-cGSG.The latter methods use forward and central Gaussian smoothed-gradient estimation.
  • 50-100 Bayesian-optimization iterations are used for fast forward models after coarse grid search narrows the search space.The tuning strategy combines grid search, Bayesian optimization, and early termination on a validation dataset.

C FUTURE DIRECTION

The paper identifies future benchmark directions involving forward-model mismatch, prior mismatch, and multimodal posteriors, while documenting selected diffusion-model hyperparameters.

  • Robustness to forward model mismatch: Future work should evaluate robustness when the assumed forward model differs from the actual one.The current study considers only exact and explicit forward models, including noisy or imperfect representations as a practical extension.
  • Robustness to prior mismatch: Prior robustness depends on task ill-posedness and the balance between the diffusion prior and observations.Highly ill-posed black hole imaging requires a more in-distribution prior, while methods using more data-gradient steps are more robust to out-of-distribution priors in linear inverse scattering.
  • Multi-modal posterior: The benchmark focuses on unimodal ground truth, leaving systematic evaluation of multimodal posterior recovery as an open problem.Real-world inverse problems may have multiple plausible solutions given the observations.
  • Table 12 lists the hyperparameter search spaces and final choices for diffusion-model algorithms across all five inverse problems.The reported choices come from the hybrid search strategy described in Appendix B.7.2.
Loading 2503.11043v2…