PaperlayerView exact revision on arXiv

Source-linked AI summary

Active Diffusion-Based Inference for Ill-Posed Inverse Problems under Incomplete Priors

Jitao Xu, Nobuo Sato, Yaohang Li

arXiv:2608.27080v1stat.MLcs.AIcs.LG

TL;DR

Ill-posed inverse problems become especially difficult when the true parameters lie outside a solver’s fixed training domain, because generative models may extrapolate without signaling misspecification. The paper trains conditional diffusion models and uses posterior uncertainty, with active domain augmentation, to discover missing support; demonstrations include a toy problem and QCD inference, while ensemble diagnostics address unreliable single-model variance.

  • Problem

    Generative inverse solvers commonly assume known fixed parameter domains, although true parameters may lie outside them and unsupported extrapolation may go unflagged.

  • Method

    The method trains a conditional diffusion model and iteratively augments its training domain using posterior analysis and uncertainty-guided forward simulations.

  • Results

    The framework recovers out-of-domain QCD parameters within three posterior standard deviations, with posterior uncertainties contracting to approximately 0.03–0.05.

  • Takeaways & Limitations

    The active diffusion framework discovers relevant parameter regions and supports reliable inverse inference without prior knowledge of valid parameter domains.

  • Takeaways & Limitations

    A single diffusion model’s posterior variance can be unreliable because out-of-domain extrapolation may produce narrow, overconfident posteriors.

Abstract

from arXiv · show

Many scientific and engineering applications require estimating unknown parameters from experimentally observable data -- an inverse problem that is inherently challenging due to nonlinearity, noise, and ill-posedness. In this paper, we propose an active diffusion-based inverse problem solver. A DM is trained to learn the mapping between the parameter space and the observable space. By iteratively detecting and correcting model misspecification through posterior uncertainty, the method discovers and learns the correct region of parameter space, even when initial training bounds exclude the true parameters. This provides a principled, Bayesian justification for adaptive domain augmentation and ensures robust inference for inverse problems under incomplete prior knowledge. We demonstrate the effectiveness of our inverse solver for a toy inverse problem with infinite solutions, and for the parameterization of the quantum correlation functions to event observables in a Quantum Chromodynamics analysis of nucleon structure.

1 Introduction

Inverse problems recover parameters from observables but are often ambiguous, unstable, and noise-sensitive. The paper proposes an active diffusion-based solver that expands its training domain when uncertainty indicates missing support.

  • Motivation: Inverse problems are ill-posed because mapping observables back to parameters can be ambiguous, unstable, noise-sensitive, and computationally challenging.The forward mapping is well-posed but may itself be computationally costly.
  • Generative inversion: Generative inverse solvers learn probability distributions over parameters and observables, enabling inference through conditional sampling.The paper situates diffusion models alongside INNs, VAIM, and normalizing flows.
  • Motivation: Generative inverse solvers assume the admissible parameter space is known, but true parameters may lie outside fixed training bounds.Out-of-domain extrapolation can produce misleading confidence because expressive models do not inherently signal unsupported regions.
  • Proposed approach: The proposed active diffusion solver detects model misspecification through posterior analysis and augments training with forward simulations in uncertain regions.A Bayesian justification is provided for adaptive domain augmentation.
  • Validation: The method is demonstrated on a toy inverse problem with infinitely many solutions and on QCD parameterization linking quantum correlation functions to event-level observables.The QCD application concerns nucleon structure.

2 Related Work

Prior work established data-driven and generative approaches for probabilistic inverse problems, including diffusion models. However, most existing methods assume fixed, known parameter domains, motivating adaptive support expansion.

  • Classical approaches: Classical inverse solvers use assumptions or regularization, such as sparsity or low dimensionality, to constrain nonunique solution spaces.Many classical methods are also application-specific and depend on detailed domain knowledge.
  • Data-driven approaches: Deep neural networks learn mappings between observables and parameters and can improve accuracy and computational efficiency over traditional numerical methods.Prior work also uses neural networks as learned regularizers and for extracting prior knowledge.
  • Generative approaches: Generative models provide probabilistic inversion by explicitly learning parameter and observable distributions, with MDNs representing multimodal posteriors under distributional assumptions.The passage identifies generative modeling as a probabilistic framework for inverse problems.
  • Diffusion models: Diffusion models have been applied to inverse problems in image reconstruction and scientific inference.These applications condition the generative process on observed data.
  • Open challenge: Most existing diffusion-based approaches assume a known, fixed admissible parameter domain and may extrapolate without reliably signaling misspecification.Addressing this limitation remains an open challenge.
  • This work: This work introduces adaptive domain expansion through posterior inconsistencies and ensemble disagreement, addressing incomplete prior knowledge.The training support is iteratively augmented during learning.

3 Methods

The method models inverse inference as conditional diffusion sampling from observables to parameters. An active loop expands the initially guessed domain when it may exclude the true solution region.

  • Problem formulation: The inverse problem seeks unknown parameters θ from measured observables y using a forward model F.The target is a parameter vector consistent with the observed data.
  • Problem formulation: Because F−1 is often analytically intractable and nonunique, the method adaptively expands the training support to capture parameters consistent with observations.This addresses an unknown admissible parameter domain.
  • Diffusion model: A conditional diffusion model pϕ(θt | y, t) approximates the parameter distribution and conditional distribution p(θ | y).Parameters are sampled from an initial guessed domain and synthetic observables are generated with the forward model.
  • Diffusion model: The diffusion model represents a joint distribution with observables supplying conditioning information during reverse diffusion.For fixed y⋆, reverse-time sampling approximates the posterior over parameters.
  • Posterior inference: The posterior mean ˆθ and covariance Σθ summarize the inferred parameter distribution.These quantities support subsequent uncertainty-based diagnostics.

3.2 Parameter-Space Uncertainty Estimation as a Diagnostic

Posterior variance is used as a diagnostic for missing parameter-space support. If the true parameters lie outside the initial domain, inferred covariance can become large.

  • Out-of-domain inference: When the true parameters fall outside the initial domain Θ0, the diffusion model may extrapolate beyond its training support.The method treats this situation as a potential source of diagnostic uncertainty.
  • Diagnostic: For out-of-domain parameters, the trace of posterior covariance tends toward a large value, Tr(Σθ) → large.This formalizes inflated posterior variance under domain mismatch.

3.3 Active Learning Loop

The active-learning loop infers parameters, evaluates posterior uncertainty, and expands the training domain when uncertainty exceeds a problem-specific threshold. It regenerates forward-model data and retrains the diffusion model until the posterior is sufficiently supported.

  • The procedure begins with an initial parameter domain, generates training data through the forward model, and trains the diffusion model.
  • Given target observable y⋆, reverse diffusion produces inferred parameter samples and their posterior covariance.
  • When uncertainty exceeds threshold τ, inferred samples become proposal points identifying missing training support.τ should reflect observation noise and desired accuracy.
  • The expanded domain receives new synthetic parameter–observable samples, after which the diffusion model is fine-tuned on the augmented dataset.
  • Inference, uncertainty evaluation, domain expansion, and retraining repeat until uncertainty falls below the threshold.

3.4 Theoretical Justification

The theoretical justification treats posterior covariance as a diagnostic for missing training support. When the true parameter region lies outside the current domain, the method expands toward informative estimates and the diffusion posterior contracts around that region.

  • Under standard assumptions, reverse diffusion samples from the true posterior, while missing training support inflates uncertainty.
  • Σθ increases when θ⋆ lies outside the current domain, so posterior covariance diagnoses missing support.
  • When θ⋆ is outside Θk, the MAP estimate identifies the nearest parameter-space region for additional training samples.
  • Expanding Θk to include the MAP estimate reduces epistemic uncertainty and minimizes the posterior discrepancy measured by KL divergence.
  • The sequence of expanded domains leads to diffusion-posterior contraction around the true parameter region.

3.5 Practical Limitations of Posterior-Covariance Diagnostics

Posterior variance from a single diffusion model is not consistently reliable for detecting model misspecification outside the training domain.

  • A diffusion model may extrapolate overconfidently outside its training domain, producing narrow posterior variance and premature convergence.

3.6 Ensemble-based Inference

The ensemble strategy uses disagreement among independently trained diffusion models to detect unsupported parameter regions more robustly than single-model posterior variance. Divergent predictions trigger new simulations, domain expansion, and retraining until agreement is reached.

  • Multiple diffusion models are trained independently with different random initializations to form an inference ensemble.
  • Within the training region, properly trained models converge to similar conditional parameter distributions because the data constrain the inverse mappings.
  • Outside the training domain, model disagreement signals that the inverse mappings are less constrained by data.
  • Ensemble disagreement can be quantified using the variance of posterior means or direct pairwise distributional similarities and divergences.
  • When ensemble predictions diverge, new parameter samples and forward simulations are generated, the models are retrained, and the loop continues until agreement.

4 Results

The toy and QCD experiments show that diffusion-based inference is reliable in-domain but can become uncertain and inconsistent outside fixed training support. Active domain expansion corrects this misspecification and recovers out-of-domain parameters.

  • 4.1 A Toy Problem: The toy inverse problem has infinitely many solutions on a circle, providing a test of whether the diffusion model captures the full posterior.For observable z*, solutions lie on a circle centered at the origin.
  • 4.1 A Toy Problem: For z*=4 within the training domain, four models consistently recover the posterior along the radius-2 circle, with mean Wasserstein distance W̄=0.12.The models produce samples uniformly distributed along the target circle and closely matching theoretical marginal densities.
  • 4.1 A Toy Problem: For z*=25 outside the training domain, independent models disagree and scatter predictions across parameter space, with mean Wasserstein distance W̄=3.00.This reflects extrapolation beyond the initial domain r∈[0,2].
  • 4.1 A Toy Problem: After active learning, the toy-model predictions converge on the radius-5 posterior, reducing mean Wasserstein distance to W̄=0.19.Adaptive domain expansion restores agreement across the four independently trained models.
  • 4.3 QCD Parameter Inference Results: In-domain QCD inference recovers all six parameters with posterior standard deviations between 0.03 and 0.06.The posteriors concentrate tightly around the true values within the training support.
  • 4.3 QCD Parameter Inference Results: For QCD parameters outside the training bounds, four models show larger uncertainties and elevated ensemble disagreement σens ranging from 0.04 to 0.18.Posterior disagreement provides an indicator that predictions are unsupported by the training distribution.
  • 4.3 QCD Parameter Inference Results: Active learning recovers the out-of-domain QCD parameters, placing every ground-truth value within three standard deviations of its posterior mean.Posterior uncertainties contract to approximately 0.03–0.05 even though the inferred posteriors lie outside the original support.

5 Discussion

The discussion relates the active diffusion framework to shooting methods and shows that active learning restores out-of-domain performance across several generative inverse solvers, at the cost of additional forward simulations and retraining.

  • Relation to Shooting Method: The active diffusion framework parallels shooting methods by using forward-model feedback to refine unknown parameters, while representing uncertainty with a full posterior rather than deterministic residuals.The DM acts as a surrogate inverse mapper, and posterior uncertainty or ensemble disagreement signals insufficient coverage of the current parameter domain.
  • Comparison with Other Generative Models: All fixed-domain inverse solvers degrade substantially out of domain, whereas adding active learning restores near in-domain accuracy for MDN, INN, RealNVP-style NF, and DM-based methods.The comparison uses training z ∈[0, 4], with in-domain testing at z*= 4 and out-of-domain testing at z*= 25.
  • Computational Cost and Practical Considerations: The framework’s principal computational costs are expanded forward-model evaluations and generative-model retraining or fine-tuning.In the QCD proxy experiment, forward simulation takes approximately 2.6 seconds per training step, compared with approximately 0.03 seconds for the DM update.

6 Conclusion

The paper concludes that active diffusion-based inference can adaptively recover relevant parameter regions when prior domains are incomplete, with validation on toy and QCD problems. Future work targets higher-dimensional models, improved efficiency, and real experimental data.

  • 6 Conclusion: The framework adaptively expands and refines the training support, correcting out-of-domain failures when true parameters are excluded from the initial domain.It was validated on a toy inverse problem with infinitely many solutions and a QCD quantum-correlation-function parameterization task.
  • 6 Conclusion: Fixed-domain training remains sufficient when a large valid parameter domain is known and computationally affordable; the proposed method addresses unknown, effectively unbounded, or high-dimensional domains.This defines the complementary scope in which adaptive domain discovery is needed.
  • 6 Conclusion: Future work will scale the method to higher-dimensional parameter spaces and more sophisticated forward models.The authors also plan surrogate forward modeling and physics-informed constraints to improve efficiency and stability.
  • 6 Conclusion: The authors plan to evaluate the approach on real experimental data containing data, systematic, and model uncertainties.They identify this as a step toward practical active generative inference with uncertainty quantification for large-scale scientific inverse problems.
Loading 2608.27080v1…