Livin' on a Prior: Likelihood Score Approximation for Inverse Problems

1 INTRODUCTION

• Non-invertible or noisy measurements make recovery ill-posed, so a prior supplies information absent from the observations.Examples span image restoration and audio tasks such as denoising and dereverberation.Source: §1 · p. 1
• A Bayesian decomposition separates the reusable clean-data prior from the observation-specific likelihood induced by each measurement process.The posterior combines the likelihood p(y | x0) with the prior p(x0).Source: §1 · p. 1
• Known observation models support posterior sampling with a pretrained prior, while unknown models motivate learning from paired clean–observation examples.The known-model approach requires access to the observation model; adapting a pretrained model can be more efficient than training a conditional model from scratch when paired data are scarce.Source: §1 · p. 2
• The paper introduces Likelihood Score Approximation (LSA), which keeps a pretrained unconditional model fixed and learns an observation-conditioned likelihood score from paired data.LSA covers unknown degradations learned from few examples and known operators trained with self-generated data, and supports score- or flow-based models, deterministic or stochastic sampling, and post-training prior swapping.Source: §1 · p. 2

2.1 GAUSSIAN PROBABILITY PATH, SCORE AND VELOCITY

• A Gaussian stochastic interpolant defines a continuous path from clean data to Gaussian noise, with score and velocity fields describing density geometry and transport.The score and velocity representations can be converted, and posterior and prior fields differ by the likelihood score or its time-scaled velocity counterpart.Source: §2.1 · pp. 2–3

2.2 OBSERVATION-CONDITIONED VECTOR FIELDS

• Conditioning the probability path on observation y yields posterior vector fields, whose likelihood-score term is generally intractable and can instead be learned from paired data.The sample-wise score and velocity targets remain unchanged by conditioning; y is an additional conditioning input rather than part of the regression target.Source: §2.2 · p. 3

2.3 DECOMPOSING POSTERIOR VECTOR FIELDS INTO PRIOR AND LIKELIHOOD TERMS

• LSA keeps the pretrained prior fixed and adds a separately learned observation-conditioned likelihood-score approximation, converting the prior output when its native parameterization differs.The prior and LSA share neither parameters nor intermediate features and are combined by adding their outputs.Source: §2.3 · p. 4

2.4 TRAINING THE LIKELIHOOD SCORE APPROXIMATION

• LSA approximates the likelihood score up to the frozen prior-model error, and a replacement prior can be swapped after training without retraining LSA.A replacement prior must use the same α_t and σ_t schedules, and swapping does not generally preserve the exact posterior field.Source: §2.4 · pp. 4–5

2.5 DETERMINISTIC AND STOCHASTIC POSTERIOR SAMPLING

• Posterior sampling starts from the Gaussian endpoint and proceeds toward the data endpoint; setting sampler stochasticity to zero recovers the probability-flow ODE.With exact posterior fields, different stochasticity choices define different sampling dynamics with the same posterior marginals.Source: §2.5 · p. 5

3 RELATED WORK

• LSA takes only (x, y, t) and applies to score and velocity parameterizations, whereas DEFT conditions on prior- and operator-derived features.For inpainting, DEFT also receives the observation mask.Source: §3 · p. 5
• CompFlow identifies the conditional–unconditional velocity residual with a noised likelihood gradient, but uses it for training-free composition of conditional generators.Source: §3 · p. 5
• ControlNet, LoRA, and related approaches adapt pretrained models with limited supervision, motivating their use as paired-data baselines.Prior work reports ControlNet use with as few as 1k conditional pairs and a ControlNet-based estimator trained on 25 images.Source: §3 · p. 5
• Figure 3 compares ImageNet-512 8× super-resolution FID across 25–1k real pairs and a self-generated setting, alongside LSA qualitative results.Source: Figure 3 · p. 6

4 EXPERIMENTAL SETUP

• Experiments span three speech tasks, multiple ImageNet-256 restoration tasks, and ImageNet-512 8× super-resolution, testing both few-pair unknown-degradation and self-generated known-operator regimes.The ImageNet-256 low-data setting uses 128 fixed pairs, about 0.01% of ImageNet training data; speech uses fixed 1- or 10-minute subsets.Source: §4 · p. 6

4.1 MODELS AND BASELINES

• LSA trains only its likelihood-score network while keeping the unconditional prior fixed; speech uses score coordinates and the main image experiments use velocity coordinates.The priors include speech NCSN++ models and pretrained PixelDiT models; paired-data baselines include ControlNet and LoRA, with other comparisons matched to each benchmark regime.Source: §4.1 · pp. 6–7

4.2 INFERENCE, HYPERPARAMETER SELECTION, AND EVALUATION

• Deterministic probability-flow ODE sampling is the default, with validation-selected hyperparameters and optional time-dependent LSA scaling and restart-based refinement.A restart re-noises the restored estimate and follows a new conditional trajectory using the same observation; its extra evaluations count toward NFE.Source: §4.2 · pp. 7–8
• Evaluation reports PSNR, SSIM, LPIPS, and FID for images, and complementary quality, perceptual, and signal measures for speech.The ImageNet-256 Gaussian-noise table covers 100 images at σy = 0.05 and marks the best result in bold.Source: §4.2 · p. 8, Table 2 · p. 8

5.1 SPEECH INVERSE PROBLEMS

• In low-data speech enhancement, LSA outperforms ControlNet and direct posterior training, especially on FAD and DNN-based MOS; full-data gains over direct training are less pronounced.Out-of-domain VoiceBank-DEMAND performance degrades, and swapping only the prior to Urgent or VoiceBank mitigates that degradation.Source: §5.1 · p. 8
• For bandwidth extension, LSA beats ControlNet and direct posterior training across reported metrics with 1 or 10 minutes of pairs; for phase retrieval, it exceeds other full-data approaches even with 1 minute.With self-generated data for known corruption, LSA reaches PESQ 4.55 and FAD 0.011 in phase retrieval.Source: §5.1 · p. 8

5.2 IMAGE INVERSE PROBLEMS

• LSA achieves considerably lower FID, modestly better LPIPS and SSIM, comparable PSNR, and up to several orders of magnitude fewer evaluations than DPS-based image baselines.The known forward operator is used to train LSA on self-generated pairs; the compared baselines do not use paired data.Source: §5.2 · pp. 8–9
• With 128 fixed pairs, LSA surpasses all low-data ImageNet-256 baselines in FID and LPIPS across tasks, sometimes approaching its self-generated counterpart.Figure 4 compares paired methods trained on the same 128 pairs and includes LSA trained with self-generated samples.Source: §5.2 · p. 9, Figure 4 · p. 9
• On ImageNet-512 8× super-resolution, 100 real pairs let LSA reconstruct better than the best-performing CSE method using 1,000 pairs; LSA also performs better in the self-generated setting.Figure 3 reports FID across paired-data budgets and includes qualitative LSA results.Source: §5.2 · p. 9

6 CONCLUSION

• LSA keeps a pretrained unconditional prior fixed and approximates the observation-dependent likelihood score from paired data.The conclusion reports effectiveness with as little as minutes of paired speech or 25 paired ImageNet examples.Source: §6 · p. 9
• When the degradation operator is known, LSA can train on self-generated pairs and substantially reduce inference cost relative to diffusion posterior-sampling approaches.Source: §6 · p. 9
• The framework applies to score and velocity formulations, supports deterministic and stochastic posterior sampling, and allows prior replacement at inference without retraining LSA.Source: §6 · p. 9

A CONDITIONAL VECTOR FIELDS AND LSA REGRESSION

• The appendix collects derivations for conditional regression, score–velocity conversion, and LSA objectives.Source: A CONDITIONAL VECTOR FIELDS AND LSA… · p. 13

A.1 CONDITIONAL REGRESSION FROM PAIRED SAMPLES

• Conditioning paired samples on an observation changes the clean endpoint to its posterior distribution while leaving the sample-wise score and velocity targets unchanged.The resulting posterior velocity averages sample-wise velocities over trajectories compatible with both the perturbed state and observation.Source: §A.1 · p. 13

A.2 SCORE-VELOCITY CONVERSION AND POSTERIOR-PRIOR DECOMPOSITION

• The posterior–prior score difference is the likelihood score, while the velocity difference carries the same score scaled by κt.The conditional path can also be represented through clean-data prediction fields.Source: §A.2 · p. 14

A.3 SCORE-VELOCITY CONVERSION AND POSTERIOR-PRIOR DECOMPOSITION

• At the population optimum, LSA recovers the posterior field minus the frozen prior, and compatible score and velocity losses correspond under time-dependent weighting.The appendix also states that the assembled predictor has the same population optimum as ordinary conditional score or flow regression.Source: §A.3 · p. 15

B.1 LINEAR GAUSSIAN PROBABILITY PATH

• Experiments use Xt = (1 − t)X0 + ctϵ, with c = 1 for images and c = 0.5 for speech; ρ = 0 gives deterministic sampling and ρ = 1 the KL-motivated reverse-SDE coefficient.Source: §B.1 · p. 16

B.2 DATASETS AND OBSERVATION PROTOCOLS

• Protocols cover three EARS speech tasks and ImageNet restoration tasks, with both limited paired-data budgets and self-generated training when the degradation operator is known.Speech subsets contain 1 or 10 minutes of paired data; ImageNet-256 evaluation uses 100 images, and low-data training uses 128 fixed pairs per task. Self-generated training uses prior samples degraded on the fly by the known operator.Source: §B.2 · pp. 16–17

B.3 MODELS AND TRAINING DETAILS

• The default LSA is an NCSN++ model trained from scratch that receives (x, y, t) and shares no parameters or intermediate features with the frozen prior.Speech uses an NCSN++ score prior, while image experiments use pretrained PixelDiT priors in their native velocity parameterization.Source: §B.3 · p. 17

B.4 INFERENCE REFINEMENTS

• Inference uses a time-dependent LSA weight as guidance-like control, and optional restart refinement adds field evaluations that are included in reported NFE.Because the LSA contains prior-model error as well as the likelihood score, scaling it does not correspond purely to likelihood tempering; inference settings are selected on held-out validation data.Source: §B.4 · pp. 19–20

C.1 LOW-DATA IMAGENET-256 RESULTS

• The ImageNet-256 table reports NFE, PSNR, SSIM, LPIPS, and FID across tasks using 128 real pairs, with bold indicating the best result for each task.Source: Table 6 · p. 20
• Figure 5 presents ImageNet-256 blur-restoration references, observations, and LSA reconstructions.Source: Figure 5 · p. 21
• Figure 6 presents ImageNet-256 results for super-resolution, inpainting, and HDR restoration.Source: Figure 6 · p. 22

D IMAGENET-512 SUPER-RESOLUTION RESULTS

• ImageNet-512 8× super-resolution is reported for 25, 50, 100, or 1k real pairs and self-generated pairs, with the best available value marked within each budget.The caption does not provide the table's metric values or identify which method is best.Source: Table 7 · p. 23

E GAUSSIAN DEBLURRING ABLATION

• For Gaussian deblurring with 128 pairs, the parameterization ablation reports validation-selected sampling starts and initializations, plus Gaussian-endpoint sampling for velocity and data prediction.The endpoint setting is undefined for score parameterization because of its endpoint singularity.Source: Table 8 · p. 23
• The Gaussian-deblurring backbone ablation compares NCSN++ and PixelDiT with Gaussian initialization, using both validation-selected starts and the Gaussian endpoint.The accompanying ablation tables report parameterization and architecture comparisons, and Figure 7 shows training dynamics.Source: Table 9 · p. 23, E GAUSSIAN DEBLURRING ABLATION · p. 23
• Figure 7 tracks Gaussian-deblurring learning curves for PixelDiT-prior LSA trained with 128 real pairs or self-generated data, measuring training time on one A100 GPU.Source: Figure 7 · p. 24

• The figure compares ControlNet, direct posterior training, and LSA using 1 minute, 10 minutes, or the full paired training set on three EARS test utterances.Source: Figure 8 · p. 25
• Each utterance’s shared top rows show the clean reference and low-pass-filtered observation before the method outputs.Source: Figure 8 · p. 25
• Spectrogram magnitudes are measured relative to each utterance’s clean-reference peak, with frequency shown in kHz.Source: Figure 8 · p. 25

G PRIOR-SWAPPING ABLATION

• Prior swapping without LSA retraining keeps performance broadly comparable, with a substantial LSD improvement when the speech-enhancement prior changes from NCSN++ to ADM.The figure also reports ImageNet-256 Gaussian-deblurring metric changes when earlier PixelDiT checkpoints replace the training prior.Source: Figure 9 · p. 26

H STOCHASTIC-SAMPLING ABLATION

• Mild stochasticity generally improves bandwidth-extension performance relative to ODE sampling across 1-minute, 10-minute, and full paired-data settings.The SDE values of ρ are 0.18, 0.09, and 0.13, respectively.Source: Figure 10 · p. 26

I INFERENCE REFINEMENT ABLATION

• The inference-refinement ablation varies restart refinement and the LSA guidance-weight schedule while keeping the 1-minute speech-enhancement model fixed.Source: Table 10 · p. 27