PaperlayerView exact revision on arXiv

Source-linked AI summary

FlowDPS: Flow-Driven Posterior Sampling for Inverse Problems

Jeongsol Kim, Bryan Sangwoo Kim, Jong Chul Ye

arXiv:2503.08136v1cs.CVcs.AIcs.LG

TL;DR

Inverse problem solving has been extensively developed for diffusion models but remains insufficiently examined for flow-based models. FlowDPS extends posterior sampling to flows by decomposing the flow ODE into clean-image and noise components, integrating data consistency and stochasticity, and outperforming existing solvers across four linear inverse problems without additional training.

  • Problem

    Flow-based inverse problem solving lacks a rigorous posterior-sampling framework despite extensive development of diffusion-based inverse solvers.

  • Method

    FlowDPS derives a flow-version of Tweedie’s formula, applies the likelihood gradient to clean-image estimation, adds stochastic noise to noise estimation, and integrates with latent flow models.

  • Results

    FlowDPS outperforms existing LDM-based and flow-based solvers across four linear inverse problems using flow-based Stable Diffusion 3.0.

  • Takeaways & Limitations

    FlowDPS provides a general flow-based posterior-sampling framework for high-resolution linear inverse problems without requiring additional training.

  • Takeaways & Limitations

    LDM-based inverse solvers cannot be naively applied to flow-based settings, and computing the exact likelihood gradient is computationally prohibitive.

Abstract

from arXiv · show

Flow matching is a recent state-of-the-art framework for generative modeling based on ordinary differential equations (ODEs). While closely related to diffusion models, it provides a more general perspective on generative modeling. Although inverse problem solving has been extensively explored using diffusion models, it has not been rigorously examined within the broader context of flow models. Therefore, here we extend the diffusion inverse solvers (DIS) - which perform posterior sampling by combining a denoising diffusion prior with an likelihood gradient - into the flow framework. Specifically, by driving the flow-version of Tweedie's formula, we decompose the flow ODE into two components: one for clean image estimation and the other for noise estimation. By integrating the likelihood gradient and stochastic noise into each component, respectively, we demonstrate that posterior sampling for inverse problem solving can be effectively achieved using flows. Our proposed solver, Flow-Driven Posterior Sampling (FlowDPS), can also be seamlessly integrated into a latent flow model with a transformer architecture. Across four linear inverse problems, we confirm that FlowDPS outperforms state-of-the-art alternatives, all without requiring additional training.

1. Introduction

Inverse problems seek meaningful reconstructions from noisy, information-losing measurements, but flow-based models have lacked a rigorous posterior-sampling framework. FlowDPS addresses this gap by decomposing the flow ODE into clean-image and noise components, integrating data consistency and stochasticity, and achieving strong reconstruction results without additional training.

  • Inverse problems recover a true signal from measurements produced by a forward model that may introduce noise, distortion, or information loss.
  • Because inverse problems are ill-posed, multiple signals can satisfy the measurement equation, requiring constraints for unique and meaningful reconstruction.
  • Diffusion- and latent-diffusion-based solvers guide posterior sampling or exploit generative priors, but their effectiveness often decreases when extended to flow-based models.
  • Existing flow-based solvers provide limited insight into their relationship with posterior sampling for inverse problems.
  • FlowDPS derives a flow-version of Tweedie’s formula to split the flow ODE into clean-image and noise-estimation components.
  • FlowDPS incorporates the likelihood gradient into clean-image estimation, adds stochastic noise to the noise component, and integrates with latent flow models.
  • FlowDPS is validated across linear inverse problems with flow-based Stable Diffusion 3.0 and is reported to outperform existing LDM-based and flow-based solvers.

3. Main Contribution

FlowDPS derives posterior sampling for flow models by decomposing the flow ODE into clean-image and noise estimates, then applying data consistency and stochastic-noise updates to the appropriate components. The method supports latent flow models and uses adaptive guidance to balance measurement consistency with generative detail.

  • 3.1. Decomposition of Flow ODE: The flow velocity is decomposed through Tweedie estimates of the denoised signal and noisy source, yielding a corresponding decomposition of the flow ODE.The denoised and noisy estimates are conditional expectations, and the ODE can be solved with Euler integration.
  • 3.1. Decomposition of Flow ODE: FlowDPS replaces deterministic noise updates with stochastic noise mixing while retaining the flow trajectory through generalized DDIM-style coefficients.The update uses C1(t) and C2(t) derived from the flow schedule and step size.
  • 3.2. Posterior Sampling via Flow Models: Posterior sampling adds a likelihood gradient to the flow velocity, but the intractable conditional likelihood gradient is approximated using a DPS-style procedure.The posterior score decomposes into likelihood and prior terms, motivating likelihood-gradient guidance.
  • 3.3. Comparison with other methods: FlowDPS applies the data-consistency gradient to the clean-image component rather than the full intermediate state, enabling adaptive step sizing and stochastic-noise enhancements.This contrasts with FlowChef, which guides xt as a whole.
  • 3.3. Comparison with other methods: The guidance weight decreases during denoising, emphasizing likelihood maximization early for structure and unconstrained flow sampling later for high-frequency details.The paper reports that early flow stages primarily determine overall structure, while later stages refine fine details.
  • 3.4. Latent FlowDPS: FlowDPS extends to latent flow models by operating on encoder-decoder latent codes, supporting higher pixel resolutions and foundational generative models.The latent formulation is intended to reduce memory and time costs while enabling conditioning such as text.

4. Experiments

Experiments evaluate FlowDPS on three 768×768 datasets across four noisy linear inverse problems and compare it with diffusion- and flow-based solvers. FlowDPS achieves strong reconstruction quality, while ablations examine interpolation, stochastic noise, NFE, and CFG scale.

  • Experimental setup: FlowDPS is evaluated on AFHQ, FFHQ, and DIV2K using PSNR, SSIM, FID, and LPIPS across four linear inverse problems.The tasks are average-pooling and bicubic super-resolution, Gaussian deblurring, and motion deblurring.
  • Quantitative and qualitative results: FlowDPS outperforms existing methods across all datasets and linear inverse problems without additional training.The evaluation uses noisy measurements with σn = 0.03.
  • Quantitative and qualitative results: LDM-based solvers perform suboptimally with flow-based Stable Diffusion 3.0, producing degraded outputs on challenging 768×768 reconstructions.Their pixel-fidelity metrics can be high despite poor perceptual quality, reflected by FID and LPIPS.
  • Quantitative and qualitative results: FlowChef and FlowDPS improve metrics over LDM-based solvers, but FlowDPS alone consistently provides state-of-the-art image fidelity and perceptual quality in qualitative comparisons.FlowChef fails to achieve acceptable visual quality under challenging degradations.
  • Ablation study: Mixing stochastic noise removes artifacts caused by deterministic noise alone, while NFE = 28 and CFG scale λ = 2.0 suffice for high-quality results.These findings come from stochastic-noise, NFE, and CFG ablations.

5. Conclusion

The conclusion presents FlowDPS as a general flow-based inverse problem solver that combines data consistency with generative quality through a decomposition of the flow ODE. It reports superior results on degraded high-resolution linear inverse problems and frames the method as both a practical solver and a framework for understanding posterior sampling.

  • 5. Conclusion: FlowDPS decomposes the flow ODE into clean-image and noise components to integrate data consistency while preserving generative quality.The method is presented as a general flow-based inverse problem solver.
  • 5. Conclusion: FlowDPS yields superior results on various linear inverse problems, including high-resolution images with severe degradations.The conclusion also describes the framework as offering a theoretical perspective and a practical method.
  • 5. Conclusion: The framework clarifies posterior sampling in flow-based generative models and is intended to support further work across applications.

6. Derivation of DDIM form

This derivation rewrites the flow update in a DDIM-like one-step form. The resulting expression uses time-dependent coefficients for its deterministic and stochastic components.

  • 7. Derivation of DDIM form: The stochastic component of Eq. (14) is isolated before deriving the one-step update.
  • 7. Derivation of DDIM form: The one-step update has the same form as DDIM but uses different coefficients C1(t) and C2(t).The passage gives C1(t) = a_t + ȧ_t dt = a_{t+dt}.

7. Proofs

The proofs derive flow-model denoising and noisy estimates, then analyze the score function and conditional flow under affine-subspace and uniform-distribution assumptions. The derivation connects these assumptions to projection-based expressions involving the clean-data manifold.

  • Tweedie formula: Proposition 1 derives denoised and noisy estimates conditioned on x_t through a flow-version of Tweedie’s formula.
  • Affine conditional flow: Under an affine clean-data manifold with a uniform distribution, Proposition 2 relates the conditional flow to the orthogonal projection P_M.The assumption that the clean images lie uniformly on subspace M is stated explicitly.
  • Score function: The score-function proof seeks a closed-form expression under stated distributional assumptions and derives intermediate densities and Jacobian terms.
  • Conditional flow: The proof represents x_t through the conditional flow velocity after integrating over x_0.
  • Score function: With a uniform-distribution assumption and σ → ∞, c_t tends to zero, simplifying the resulting expression.

8. Implementation details

The implementation reworks diffusion-based inverse solvers for flow models and fixes common solver settings for comparison. Stable Diffusion 3.0 is used as the baseline with a time-scheduler shift factor of 4.0.

  • Implementation with flow models: Diffusion-based baselines are reimplemented for flow models, with Euler’s method used for methods whose updates depend on intermediate samples.PSLD, ReSample, and related implementations are adapted to the flow setting for fair comparison.
  • Implementation with flow models: ReSample uses hard data consistency at every Euler step because its original staged configuration performs poorly with few ODE steps.A skip step size of 1 produces the best empirical comparison setting.
  • Hyper-parameter setting: Stable Diffusion 3.0 is the baseline model for all implementations, with the time-scheduler shift factor set to 4.0.This common configuration controls the comparison across solvers.
  • Hyper-parameter setting: PSLD uses η = 1.0, γ = 0.1, and 200 NFEs, following its original settings.The listed hyperparameters reproduce the baseline configuration used for comparison.

9. Ablation Study

The ablations examine adaptive guidance, stochastic noise, and classifier-free guidance in FlowDPS. The method emphasizes measurement consistency early and preserves high-frequency detail later.

  • Analysis on βt and γt: The interpolation coefficient γ controls the trade-off between data-consistent and unconstrained clean estimates during sampling.Using γ = 1 causes later-stage trajectory deviation, so the final algorithm selects γ = σt.
  • Analysis on βt and γt: −βt rapidly decreases to zero, implying larger likelihood-gradient steps early in sampling and a gradual return toward the unconstrained flow trajectory.Early guidance aligns samples with measurements, while later denoising compensates for deviations and refines details.
  • CFG scale and NFE: Higher NFEs require larger CFG scales λ to produce high-frequency details, while excessively high λ can guide samples in the wrong direction.Low λ produces blurry results; Figure 7 compares λ values 0.0, 2.0, and 7.5 at 28, 50, and 100 NFEs.

10. Runtime Comparison

Runtime comparison shows that Jacobian computation makes PSLD slowest, whereas several alternatives avoid it. LatentDAPS remains slower than FlowDPS because it repeatedly solves the flow ODE and Langevin dynamics.

  • Runtime Comparison: PSLD takes the longest time because it computes the transformer denoiser’s Jacobian.ReSample, FlowChef, and FlowDPS avoid this computation through Jacobian approximations and are relatively efficient.
  • Runtime Comparison: LatentDAPS is slower than FlowDPS because repeated flow-ODE solves and Langevin-dynamics iterations are needed for sufficient performance.Backpropagation through the decoder is the main computational bottleneck for the other compared methods.

11. Additional Results

Additional experiments extend FlowDPS to high-resolution images, the FLUX backbone, and AFHQ and FFHQ qualitative comparisons. These results support applicability across resolutions, backbones, datasets, and inverse problems.

  • High-resolution experiments: Stable Diffusion 3.0’s latent flow representation supports inverse-problem experiments above 1K pixels and across non-square aspect ratios.Motion deblurring is demonstrated on 1024×1408 DIV2K crops using image descriptions extracted by LLaVA.
  • Inverse problem solving with FLUX: FlowDPS is designed for general affine conditional flows and is demonstrated with both Stable Diffusion 3 and FLUX.The framework is intended to construct inverse solvers across multiple flow models.
  • Inverse problem solving with FLUX: FlowDPS effectively solves inverse problems with FLUX 1.0-schnell on DIV2K super-resolution and deblurring examples.The FLUX experiments use 10 data-consistency iterations, with step sizes 7.0 for super-resolution and 12.0 for deblurring.
  • Additional Qualitative comparisons: Qualitative comparisons across four inverse problems on AFHQ and FFHQ show consistently promising reconstructions that outperform all baselines.Figures 11–18 cover average-pooling and bicubic x12 super-resolution, Gaussian deblurring, and motion deblurring.
Loading 2503.08136v1…