Diffusion Editing with Soft Mask: Pixel Level Redo of Image and Video with Adjustable Strength

1 Introduction

• Soft masks express spatially varying redo strength, from preserving a reference to fully re-synthesizing selected regions.The paper frames this as a continuous alternative to the hard preserve-or-change choice of binary masks.Source: §1 · p. 2
• Diffusion models' global denoising does not naturally support spatially varying soft redo, motivating reference-guided sampling from p(x | x0, r).The mask r specifies per-location redo strength for a given reference image x0.Source: §1 · p. 2
• Training-based approaches make continuous per-pixel supervision expensive, while training-free methods may require heavy gradients or produce inferior results.The paper also motivates a universal tool that works across diverse models without model-specific fine-tuning.Source: §1 · p. 2
• SoftPaint combines the Unified Redo Score with Diffusion-Damped Langevin to provide gradient-free, memory-efficient soft-mask editing across diffusion backbones.The sampler is described as applicable to flow, ODE-based, and SDE-based models, and supports image and video editing.Source: §1 · p. 2

2 From Binary Mask to Soft Mask

• Binary inpainting hard-replaces known pixels with a noised reference and generates the missing regions, but this does not naturally extend to continuous masks.Vanilla diffusion inpainting uses a pretrained unconditional model and blends its sample with the forward-diffused reference at each step [18] [19].Source: §2 · p. 3
• Naive pixel-space interpolation can mix meaningfully different clean images, producing blur and outputs that remain too close to the reference at intermediate redo strengths.As a result, intermediate mask values may fail to produce proportional redo.Source: §2 · p. 3
• The soft-inpainting target q(x | xref₀, r), parameterized by continuous pixel-wise r ∈ [0,1]^(H×W), must be a valid probability density, yield perfect reconstruction at r = 0, and reduce to free generation at r = 1.Source: §2 · p. 3
• SoftPaint introduces the Unified Redo Score and a Langevin-based sampler to meet the soft-mask formulation challenge.Figure 2 describes r as tuning the balance between preservation and redo.Source: Figure 2 · p. 3, §2 · p. 4

3.1 Unified Redo (UR) Score

• A time- and pixel-dependent blending weight shifts reference influence over sampling; smaller r keeps it higher, while larger r moves it toward zero sooner.The weight can exceed 1 to amplify the reference constraint.Source: §3.1 · pp. 4–5
• Gaussian-softening the reference lets each pixel deviate from exact reconstruction in a controlled way, with variance set by the soft mask.The paper uses η = 2, which makes the clipped variance approximately linear in r.Source: §3.1 · pp. 4–5
• The UR score combines the pretrained model score with an analytic reference term, smoothly covering exact reconstruction and free generation across mask strengths.The construction also applies to rectified flow models.Source: §3.1 · pp. 4–6

3.2 Diffusion-Damped Langevin (DDL)

• At each diffusion step, DDL runs a short inner Langevin sampler driven by the UR score before the denoising step.The inner chain starts from the current noisy sample and uses only a few updates per diffusion step.Source: §3.2 · p. 6
• Because naive Euler discretization is unstable under strong guidance, DDL uses a damped exponential integrator to make the inner sampler stable.Its update uses the inner-loop step size and Gaussian noise.Source: §3.2 · p. 6

4 Experiments

• The evaluation uses FlowEdit and DAVIS-Edit, compares SoftPaint with Differential Diffusion and Naive Mixing, and anchors quality comparisons at matched CLIP-Text.The image experiments use Qwen-Image; the video experiments use CogVideoX-5B and Wan2.2.Source: §4 · p. 6, §4.1 · p. 7, §4.2 · p. 8

4.1 Soft-Mask Image Editing

• At matched CLIP-Text, SoftPaint provides the smoothest image-edit control and achieves lower LPIPS, higher PSNR, and higher CLIP-IQA than DD, with gaps widening at stronger edits.Naive Mixing stays near the source until r>0.8 and then overshoots; Qwen-Image-Edit is reported at CLIP-T 30.2 and LPIPS 0.306.Source: Figure 4 · p. 7, §4.1 · p. 7
• With equal 60-NFE budgets, SoftPaint has lower LPIPSedit and higher PSNRedit than DD at weak, medium, and strong matched-CLIP-T anchors.SoftPaint uses 20 outer steps and two inner model evaluations per step; DD is also run for 60 outer steps.Source: §4.1 · p. 7
• At the medium CLIP-Text anchor of 33.0 ± 0.2, SoftPaint improves all four reported quality metrics over DD, including at an equal 60-NFE budget.The evaluation uses four vertical zones with redo strengths r∈{1, 0.75, 0.5, 0.25}.Source: §4.1 · p. 7, Table 1 · p. 8

4.2 Soft-Mask Video Editing

• Across Wan2.2 and CogVideoX, SoftPaint achieves lower LPIPS, higher PSNR, and lower VFID than DD at matched CLIP-Text, with larger gaps at mid-to-strong edit strengths.Naive Mixing barely shifts CLIP-Text for r<0.8, then overshoots at r=1.0 as quality metrics collapse.Source: §4.2 · p. 8

4.3 Qualitative Comparison

• At r=0.8, representative Qwen-Image edits preserve unmasked regions verbatim while realizing the requested semantic changes inside the mask.The examples cover diverse FlowEdit prompts.Source: §4.3 · p. 8
• On the CogVideoX train edit, SoftPaint progresses from localized change at r=0.4 to full transformation at r=1.0, while DD shows missing structure and seams at stronger settings.At r=0.8, the train’s original shape is largely maintained while its carriage changes to wood.Source: §4.3 · p. 8

5 Related Works

• Diffusion editing spans fine-tuned instruction methods, plug-in spatial conditioning, image-prompt adaptation, multi-task frameworks, and reference-conditioned models.Recent proprietary models support multireference and text-image-to-image editing with strong semantic preservation.Source: §5 · p. 8
• Existing image and video editing models operate at the semantic level and offer limited control over how strongly each individual pixel is edited.This motivates continuous, pixel-level edit-strength control through a soft mask.Source: §5 · pp. 8–9
• Many training-free methods target binary masks and use gradient-based optimization or line search, while Differential Diffusion supports soft masks but reduces to naive replacement on binary masks.SoftPaint is presented as a continuous-soft-mask, gradient-free, memory-efficient alternative compatible with modern ODE-based backbones.Source: §5 · p. 9

6 Conclusion and Limitations

• Experiments show improved edit controllability and a better quality-preservation trade-off for SoftPaint’s image and video editing.The method is described as training-free and model-agnostic.Source: §6 · p. 10
• Each inner Langevin iteration adds a network evaluation, making generation roughly 2×–5× slower than standard backbone sampling.The text identifies interactive editing and large-scale deployment as settings where this latency can be a practical limitation.Source: §6 · p. 10
• The work may benefit creative workflows, content restoration, and controllable-generation research, but could also enable deceptive edits of real scenes or people.Source: §6 · p. 10

A Additional Tables

• The additional tables cover matched-CLIP-T comparisons, including DAVIS-Edit video editing and Qwen-Image FlowEdit image editing.Table 2 uses specified SoftPaint redo-strength anchors on Wan2.2 and CogVideoX, with nearby measured strengths selected for DD.Source: A Additional Tables · p. 13, Table 2 · p. 13

B Derivation of the DDL Update

• The UR score’s stiff linear component makes naive Euler–Maruyama discretization require very small step sizes; explicit Euler steps are fragile when the coefficient grows.The coefficient (1 − ᾱ_t)^−1 becomes large when 1 − ᾱ_t is small.Source: B Derivation of the DDL Update · p. 13
• The drift is split into a linear damping term and a remainder, with damping increasing with guidance factor λβ_t(r) for stability at large λ.This damping rule is described as a simplified variant of damped Langevin dynamics introduced in LanPaint [34].Source: B Derivation of the DDL Update · pp. 13–14
• The supplemental Qwen-Image tables report image-editing comparisons at matched NFE and matched CLIP-T, with Naive Mixing unmatched at the medium anchor.Table 3 compares FlowEdit at equal NFE; Table 4 uses SoftPaint anchors and nearby measured CLIP-T matches for DD and Naive Mixing.Source: Table 3 · p. 14, Table 4 · p. 14
• Treating the remainder as approximately constant over a short interval yields an Ornstein–Uhlenbeck transition that uses one score evaluation per step and remains stable for arbitrarily large ∆τ.The text contrasts this bounded trajectory under strong guidance with standard Langevin updates using overly large explicit steps.Source: B Derivation of the DDL Update · p. 14

C Ablation Studies

• The video experiments use 50 denoising steps for CogVideoX-5B and 40 for Wan2.2, following each backbone’s official default schedule.Source: C Ablation Studies · p. 14
• The ablation varies Langevin step size s and inner-update count n on Wan2.2 over DAVIS-Edit, averaging metrics across r ∈ {0.2, 0.4, 0.6, 0.8, 1.0}.Source: C Ablation Studies · p. 15
• The image-editing experiments use the same defaults, n=2 and s=0.2, without per-domain retuning.Source: C Ablation Studies · p. 15

C.1 Inner-loop hyperparameters

• Increasing n from 0 to 2 improves PSNR by +1.05 and lowers LPIPS by 0.0053; n=2, s=0.2 is the reported quality-compute sweet spot.Increasing n from 3 to 5 adds +0.43 PSNR while raising inner-loop cost 2.5×; increasing s from 0.05 to 0.2 yields +0.65 PSNR, with diminishing returns beyond 0.2. Figure 8 varies n with s=0.2 fixed and s with n=2 fixed; metrics are averaged across five r values.Source: Figure 8 · p. 15, §C.1 · p. 15
• The tested defaults are n=2 and s=0.2, and the figure notes that they are also used for image editing without retuning.Source: Figure 8 · p. 15, §C.1 · p. 15

C.2 Runtime

• On Qwen-Image at 1024 × 1024 with 20 diffusion steps, DD averages 10.2 s/image and SoftPaint with n=2 averages 29.4 s/image in the measured denoising loop.The measurement excludes model loading, encoder passes, VAE decoding, and post-processing; peak GPU memory matches standard forward-only inference.Source: §C.2 · p. 15

C.3 Redo-aware warm start

• The warm start initializes extrapolation-regime pixels from the noised reference and shortens burn-in without changing the inner Langevin chain’s stationary distribution.Without it, the chain starts at a low-likelihood point and, with n=2, leaves residual noise at mask boundaries.Source: §C.3 · pp. 15–16
• At r=0.8, SoftPaint improves editability and LPIPS and VFID relative to warm-start-only, while warm start substantially improves editability over naive mixing.The ablation uses the CogVideoX backbone and 50 denoising steps.Source: §C.3 · p. 16, Table 5 · p. 16

C.4 Binary-mask inpainting: isolating the inner-loop operator

• SoftPaint yields lower LPIPS and FID than the RePaint-style inner loop on both masks at both tested budgets in binary-mask inpainting.The comparison uses matched inner-loop budgets; Replace is a 20-step reference with a different NFE.Source: §C.4 · p. 16, Table 6 · p. 16

C.5 Choice of exponent η in Σ(r; η)

• On Flow Edit Benchmark with Qwen-Image, η=2 makes clipt approximately linear in mask value r, yielding a more predictable mapping to redo strength than η=4.Figure 9 compares the effective redo strength for η=2 and η=4.Source: §C.5 · p. 16, Figure 9 · p. 17

D Derivation of the Expectation Form for the Reference Factor

• Completing the square in the Gaussian smoothing variable and integrating it out yields a reference-factor exponent with the same functional form as the expression in Eq. (17).Source: D Derivation of the Expectation Form… · pp. 17–18

• For Dalmatian-to-cheetah editing, the figure compares Naive Mixing, DD, and SoftPaint across r values from 0.1 to 1.0.The mask inset appears at top right; the source and target prompts describe a dog and cheetah on a white background.Source: Figure 10 · p. 18
• The pink-to-orange flower example uses the same layout as the Dalmatian-to-cheetah comparison.The source prompt describes a bouquet of pink, red, and white flowers; the target changes these to orange, yellow, and white.Source: Figure 11 · p. 18
• The dog-to-wolf video figure compares SoftPaint and DD at r=0.4, 0.8, and 1.0 against original DAVIS-Edit frames at r=0.Source: Figure 12 · p. 19