PaperlayerView exact revision on arXiv

Source-linked AI summary

DecFlowEdit: Self-Localized Flow-based Image Editing via Guidance Decoupling

Zheyuan Zhan, Can Wang, Jiawei Chen, Chun Chen, Siwei Lyu, Zeyu Zheng, Defang Chen

arXiv:2609.34237v1cs.CV

TL;DR

Default CFG in FlowEdit can cause background leakage, while removing CFG improves localization but weakens semantic edits. DecFlowEdit extracts a prior from w/o-CFG velocity differences and uses it to reweight default-CFG updates. Across three flow-based backbones, it improves preservation, including a 61–73% reduction in structure distance relative to FlowEdit.

  • Problem

    Asymmetric default CFG can weaken cancellation of source and target velocities in unchanged regions, allowing residual updates to leak into the background.

    Source: §3 · p. 4

  • Method

    DecFlowEdit aggregates w/o-CFG velocity differences into a spatial prior and uses it to reweight default-CFG editing updates.

    Source: §1 · p. 2

  • Results

    Across FLUX, SD3, and SD3.5, DecFlowEdit reduces structure distance by approximately 61–73% relative to raw FlowEdit, with a 2–3% reduction in CLIP similarity.

    Source: §5.2 · p. 6

  • Takeaways & Limitations

    Across three flow-based backbones, separating localization from semantic editing improves preservation while maintaining a better balance between semantic fidelity and structure preservation.

    Source: §6 · pp. 9–10

    The temporal aggregation prior is less suitable for global style changes because it can be content-dependent and non-uniform rather than covering the image uniformly.

    Source: G LIMITATIONS · p. 23

Abstract

from arXiv · show

Flow-based image editing (FlowEdit) enables inversion-free semantic changes through the difference between source and target velocities. In this paper, we observe that FlowEdit's default classifier-free guidance (CFG) configuration, with asymmetric source and target scales, causes substantial background leakage. Matching these guidance scales, for example by removing CFG, improves edit-relevant localization but severely degrades editability. To get the best of both worlds, we propose DecFlowEdit, which decouples the optimal guidance scales for localization and for editing in flow-based generative models. In particular, DecFlowEdit first extracts an edit-relevant prior by temporally aggregating velocity differences evaluated without CFG, and then uses this prior to reweight the original updates under default CFG. Our method remains training-free and inversion-free, requiring neither external spatial masks nor attention manipulation. Experiments on PIE-Bench across FLUX, SD3, and SD3.5 show that DecFlowEdit improves background preservation, reducing structure distance by approximately 61 to 73 percent and background LPIPS by 68 to 80 percent relative to FlowEdit at comparable editing fidelity.

1 INTRODUCTION

Training-free image editing offers flexibility but must balance semantic change against preservation of edit-irrelevant content. DecFlowEdit separates localization from editing by extracting a spatial prior without CFG and applying it to default-CFG updates.

1 INTRODUCTION

  • Training-free editing uses pretrained generative models without task-specific optimization, but stronger semantic changes can modify regions outside the intended edit.The goal is to realize a user-specified change while preserving edit-irrelevant content.Source: §1 · p. 1
  • FlowEdit avoids inversion and attention manipulation, yet spatially global updates can accumulate residual differences in non-edit regions and cause background leakage.Its velocity differences are typically stronger in edit-relevant regions, but may remain elsewhere.Source: §1 · p. 1
  • Asymmetric default CFG weakens cancellation between source and target velocities in unchanged regions, while matching scales without CFG improves localization but weakens semantic edits.Thus the guidance configuration that helps locate edits differs from the one that enables effective editing.Source: §1 · p. 2
  • DecFlowEdit aggregates velocity differences without CFG into a spatial prior, then uses that prior to reweight default-CFG updates.It requires no training, inversion, external masks, or attention manipulation; Figure 1 qualitatively compares edits and preserved content.Source: §1 · p. 2, Figure 1 · p. 2

2 RELATED WORK

Prior training-free editing approaches preserve content through inversion, internal attention or feature interventions, or explicit spatial localization. DecFlowEdit instead derives localization from FlowEdit’s velocity differences and uses it to constrain edits.

2 RELATED WORK

  • Inversion-and-regeneration methods map a source image to noise before regenerating it under a target condition, while attention and feature interventions inject source information into generation.The latter require access to internal model architectures and can increase inference complexity.Source: §2 · pp. 2–3
  • Attention and feature interventions can be sensitive to layer and timestep choices, add implementation and inference overhead, and require backbone-specific redesign.Large source–target semantic gaps can also introduce artifacts when preserved patterns conflict with target semantics.Source: §2 · p. 3
  • FlowEdit builds an inversion-free path from target- and source-conditioned velocity differences, but default CFG can spread updates beyond the intended region.DecFlowEdit uses more localized w/o-CFG responses to form a prior that constrains default-CFG updates.Source: §2 · p. 3
  • Localization approaches use attention maps, user masks, segmentation models, or prediction differences; prediction-based masks can be unstable, while DecFlowEdit temporally aggregates FlowEdit responses.The resulting soft prior provides spatial control without attention manipulation, external segmentation, or user-provided masks.Source: §2 · p. 3

3 BACKGROUND

Flow matching represents generation as a trajectory guided by a conditional velocity field. FlowEdit uses paired source and target states and integrates their velocity difference, avoiding inversion, but CFG choices can leave residual updates in unchanged regions.

3 BACKGROUND

  • Flow matching learns a time-dependent conditional velocity field that maps data samples toward a prior, typically a standard Gaussian, along a linear interpolation trajectory.The field is trained to predict the difference between the prior and data samples.Source: §3 · p. 3
  • Inversion-and-regeneration maps the source image to a shared noise latent and regenerates from it under the target prompt, but can substantially change image details.The shared latent provides only coarse structural control.Source: §3 · pp. 3–4
  • FlowEdit pairs source and target states at each timestep and integrates the difference between their conditional velocities as a direct, inversion-free editing path.Shared noise promotes cancellation of similar responses in unchanged regions and helps preserve source content.Source: §3 · p. 4
  • Strong asymmetric CFG can mismatch source and target responses in unchanged regions, weakening cancellation and allowing residual updates to leak into the background.Figure 2 depicts the contrasting velocity-difference maps and editing outcomes under w/o CFG and default CFG.Source: §3 · p. 4, Figure 2 · p. 4

4 METHOD

DecFlowEdit separates the guidance used to find edit regions from the guidance used to make semantic changes. It extracts a soft spatial prior under w/o CFG and applies that prior to default-CFG updates.

4 METHOD

  • The method first examines CFG’s different effects on localization and editing, then extracts a prior without CFG and uses it to constrain default-CFG updates.The method comprises prior extraction and prior-guided editing.Source: §4 · p. 4

4.1 DECOUPLING LOCALIZATION FROM EDITING

  • Without CFG, velocity differences stay concentrated on the goat but barely change the image; default CFG replaces it with a horse but also changes the neighboring cat.Prior-guided editing combines the localized response with default-CFG strength, replacing the goat while preserving the cat and other regions.Source: §4.1 · p. 5

4.2 PRIOR EXTRACTION W/O CFG

  • The prior stage aggregates middle-schedule w/o-CFG velocity differences, takes their norm, smooths the energy map, and converts the response into a soft spatial prior.A schedule with approximately one quarter as many sampling steps yields comparable editing and preservation performance across FLUX, SD3, and SD3.5.Source: §4.2 · p. 5

4.3 PRIOR-GUIDED EDITING WITH DEFAULT CFG

  • The editing stage re-evaluates velocity differences under default CFG and applies the same spatial prior at every step, retaining larger updates within the edit region and suppressing smaller ones outside it.The spatial weight is broadcast across channels, and weighting is inserted before each FlowEdit update.Source: §4.3 · pp. 5–6

5 EXPERIMENTS

DecFlowEdit improves background preservation over FlowEdit across three backbones, while experiments show that guidance settings favor different roles: localization or semantic editing. Comparisons of velocity differences and spatial priors further assess how this separation works and where its gains hold.

5.1 EXPERIMENTAL SETUP

  • PIE-Bench evaluation uses 700 samples and reports structure, background-preservation, and CLIP-based fidelity metrics, with controlled FlowEdit comparisons on FLUX, SD3, and SD3.5.The prior uses w/o CFG (1, 1), while editing follows default CFG; the soft prior uses threshold τ = 0.55 and softness s = 0.08.Source: §5.1 · p. 6

5.2 MAIN RESULTS ON PIE-BENCH

  • DecFlowEdit reduces structure distance by approximately 61–73% versus raw FlowEdit across FLUX, SD3, and SD3.5.These preservation gains accompany a 2–3% reduction in CLIP similarity; quarter-length prior extraction retains strong content preservation and competitive text alignment.Source: §5.2 · p. 6, Table 1 · p. 7

5.3 CFG AFFECTS LOCALIZATION AND EDITING

  • Without CFG yields the highest IoU but lowest CLIP, whereas default CFG yields the highest CLIP but lowest IoU.Priors extracted without CFG reduce background LPIPS by approximately 39–50% relative to default-CFG priors, while edited-region CLIP differs by less than 0.13%.Source: §5.3 · p. 7

5.4 UNDERSTANDING GUIDANCE-DEPENDENT LOCALIZATION

  • On FLUX, the non-edit/edit velocity-magnitude ratio is 0.5175 without CFG versus 0.8007 with default CFG; the temporal Cos margin is 0.12740 vs. 0.04984.The spatial ratio shows greater contrast between edit and non-edit regions without CFG; the larger temporal Cos margin indicates more selective retention of edit-region differences.Source: §5.4 · pp. 8–9

5.5 COMPARISON OF SPATIAL PRIORS

  • Compared with other internal localization methods, DecFlowEdit’s prior has higher IoU, lower background LPIPS and structure distance, and competitive edited-region CLIP.Against Auto VLM→SAM3, it has higher edited-region CLIP and lower structure distance at similar background LPIPS, despite lower mask IoU.Source: §5.5 · p. 9

6 CONCLUSION

DecFlowEdit separates spatial localization from semantic editing to improve spatial control in flow-based image editing. Across three flow-based backbones, the authors report a better balance between semantic fidelity and structure preservation.

6 CONCLUSION

  • DecFlowEdit is training-free and inversion-free, improving spatial control in flow-based image editing.It does not modify the underlying generative model or require additional supervision.Source: §6 · pp. 9–10
  • Without CFG favors localization, whereas default CFG favors effective semantic editing.DecFlowEdit exploits this distinction to improve preservation of edit-irrelevant content.Source: §6 · p. 9
  • Across three flow-based backbones, the separation yields more reliable editing trajectories and a better balance between semantic fidelity and structure preservation.The authors suggest internal flow-based editor dynamics can provide useful spatial signals for controlling edits.Source: §6 · p. 10

A ADDITIONAL ANALYSIS OF GUIDANCE-DEPENDENT LOCALIZATION

The appendix explains why guidance mismatch can leave residual velocity updates in unchanged regions, then measures how guidance affects spatial and temporal contrast. Across three backbones, w/o CFG yields clearer edit/non-edit magnitude contrast, and on FLUX it yields larger temporal directional and cancellation gaps.

A.1 INTUITION FOR GUIDANCE-DEPENDENT LOCALIZATION

  • Default CFG can leave residual velocity differences on unchanged objects because asymmetric source and target guidance strengths yield different responses despite shared semantics.Paired states share sampled noise, allowing common-content components to partially cancel when the editing state remains close to the source in non-edit regions.Source: §A.1 · p. 12

A.2 WITHIN-STEP SPATIAL CONTRAST

  • Switching from w/o CFG to default CFG increases R from 0.5175 to 0.8007 on FLUX, 0.5985 to 0.8334 on SD3, and 0.5903 to 0.8409 on SD3.5.The sample-averaged Rt is higher under default CFG at every actual update step: 24/24 on FLUX and 33/33 on each SD backbone.Source: §A.2 · p. 13

A.3 TEMPORAL DIRECTIONAL CONTRAST

  • On FLUX, edit/non-edit gaps are larger under w/o CFG than default CFG for both Cos (0.12740 vs. 0.04984) and TCR (0.08470 vs. 0.03135).The gaps indicate stronger regional contrast in directional consistency and retained magnitude after vector aggregation, supporting w/o CFG for prior extraction.Source: §A.3 · p. 13, Table 3 · p. 13

B MASK EXTRACTION BASELINES

The appendix compares localization-mask baselines for FLUX editing, including velocity-disagreement maps, attention-derived masks, and an external VLM-to-segmentation pipeline. It also details threshold selection and evaluation handling for empty masks.

Mask signals and thresholding

  • DiffEdit-Single and DiffEdit-Multi use single-step and multi-step source–target velocity disagreement, while the proposed prior aggregates FlowEdit velocity differences aligned with its editing operator.On 556 local tasks, threshold 0.5 gives IoUs of 0.0702 and 0.1228 for the two FLUX DiffEdit variants; at τ = 0.15, their IoUs are 0.3368 and 0.3615.Source: B MASK EXTRACTION BASELINES · pp. 14–15

Attention-based masks

  • The appendix includes attention-based masks from Prompt-to-Prompt and MasaCtrl, which aggregate selected-token attention maps and apply method-specific thresholds.Prompt-to-Prompt exports LocalBlend's hard mask; the MasaCtrl-style FLUX mask comes from joint-attention maps in double-stream transformer blocks.Source: B MASK EXTRACTION BASELINES · pp. 14–15

External segmentation baseline

  • Auto VLM→SAM3 generates a source-side object or part phrase for segmentation, retaining instance probabilities for downstream editing.When SAM3 returns no instance above confidence 0.5, the evaluation uses an all-one weight map; segmentation and editing use different mask treatments.Source: Figure 6 · p. 14, B MASK EXTRACTION BASELINES · p. 15

C IMPLEMENTATION DETAILS

The appendix specifies implementation settings and examines how spatial reweighting and prior parameters affect editing and preservation. Spatial reweighting outperforms uniform scaling in background LPIPS at comparable edited-region CLIP, while threshold choice trades semantic change against preservation.

C IMPLEMENTATION DETAILS

  • The prior-extraction stage uses w/o CFG (1, 1), while prior-guided editing uses default CFG and the backbone-specific schedule; full extraction costs approximately 2× and short extraction 1.25× a single FlowEdit edit.The full variant uses the same number of sampling steps for both stages; Ours-short uses about one quarter as many extraction steps.Source: C IMPLEMENTATION DETAILS · pp. 15–16

D PARAMETER SENSITIVITY AND EDITING–PRESERVATION TRADE-OFFS

  • The parameter study varies update strength, prior threshold, softness, and aggregation window to characterize editing–preservation trade-offs rather than maximize every metric at one setting.Unless stated otherwise, it uses τ = 0.55 and s = 0.08, with default CFG during prior-guided editing.Source: D PARAMETER SENSITIVITY AND… · p. 16

D.1 SPATIAL REWEIGHTING VERSUS UNIFORM SCALING

  • At λ = 1, both spatial reweighting and uniform scaling recover raw FlowEdit; at comparable CLIP, the controls are compared rather than at equal λ.Uniform scaling attenuates every location and channel, whereas spatial reweighting targets low-prior locations.Source: §D.1 · p. 16
  • At comparable edited-region CLIP, spatial reweighting achieves substantially lower background LPIPS than uniform update scaling.This indicates spatial selectivity contributes to preservation gains beyond uniformly reducing editing strength.Source: §D.1 · p. 16, Figure 7 · p. 17

D.2 THRESHOLD: LOCALIZATION AND DOWNSTREAM EDITING

  • Among displayed thresholds, τ = 0.40 yields the highest mask IoU on all three backbones, while increasing τ lowers both CLIP and background LPIPS.The authors use τ = 0.55 as a preservation-oriented operating point, not a universal optimum.Source: §D.2 · p. 17

D.3 SOFTNESS: EDITING CHANGES DESPITE AN UNCHANGED BINARY MASK

The appendix shows that softness changes final editing despite an unchanged binary mask, while tested aggregation windows produce only small performance differences. It also treats ground-truth-mask results as diagnostic references and documents CFG settings and evaluation protocols.

D.3 SOFTNESS: EDITING CHANGES DESPITE AN UNCHANGED BINARY MASK

  • Increasing softness s lowers both edited-region CLIP and background LPIPS in the FLUX sweep, even though the binary mask at threshold 0.5 is unchanged.At τ = 0.55, s values from 0.04 to 0.16 give identical IoU (0.4150), precision (0.6130), and recall (0.6293).Source: §D.3 · pp. 17–18, Figure 9 · p. 18

D.4 AGGREGATION-WINDOW SENSITIVITY

  • The small localization and editing differences across tested aggregation windows support stability of the w/o-CFG spatial signals and robustness to window choice.The prior-extraction trajectory executes all 24 FLUX updates for every window setting.Source: §D.4 · p. 19, Table 5 · p. 19

D.5 GROUND-TRUTH-MASK DIAGNOSTIC

  • Ground-truth-mask editing reduces CLIP from 22.71 to 22.41, and its preservation scores are diagnostic references rather than fair automatic-method baselines.The ground-truth mask also defines the background evaluation region, giving this variant an oracle advantage.Source: §D.5 · p. 19, Table 6 · p. 19

E CFG CONFIGURATIONS: LOCALIZATION AND EDITING

  • Direct-editing CFG comparisons include w/o CFG (1, 1), each backbone's default guidance, and symmetric guidance settings, with images generated without a spatial mask.The comparisons use the same input images and paired noise draws within each backbone.Source: §E.1 · p. 19, Table 7 · p. 20

E.1 EXPERIMENTAL SETTINGS

  • The evaluation uses 556 local tasks for IoU and background metrics, while displayed CLIP and SSIM use 700 tasks.The appendix reports results across FLUX, SD3, and SD3.5 using their standard FlowEdit settings.Source: §E.1 · p. 20

E.2 ADDITIONAL RESULTS ON FLUX AND SD3.5

Across CFG sweeps, removing CFG improves localization but weakens editing, while extracting priors without CFG and editing with default CFG improves preservation. Global norm matching does not recover default-CFG editing quality, and multi-seed tests retain preservation gains.

E.2 ADDITIONAL RESULTS ON FLUX AND SD3.5

  • Across FLUX and SD3.5, w/o CFG has the highest prior IoU but lowest edited-region CLIP, while default CFG has the highest CLIP and lowest IoU.Increasing symmetric guidance raises CLIP while reducing IoU.Source: Figure 10 · p. 20, §E.2 · p. 20

E.3 EFFECT OF PRIOR-EXTRACTION CFG

  • Background LPIPS falls 50.0%, 41.1%, and 39.4% on FLUX, SD3, and SD3.5 with w/o CFG priors; edited-region CLIP changes +0.013%, −0.121%, and −0.103%.Prior-guided editing uses default CFG in both configurations on the same 556 local-edit tasks.Source: Table 8 · p. 21, §E.3 · p. 21

E.4 PER-STEP GLOBAL NORM MATCHING

  • Global norm matching raises edited-region CLIP over w/o CFG but remains below default CFG on all three backbones, while structure distance is higher.SD3 and SD3.5 also worsen on all four background metrics, FLUX results are mixed, and the norm-matched edits show severe blur.Source: Table 9 · p. 21, §E.4 · pp. 21–22

F MULTI-SEED MAIN RESULTS

  • Across five random seeds, DecFlowEdit consistently improves background and structure preservation over FlowEdit, with small standard deviations and competitive semantic-editing metrics.The repeated experiments use the PIE-Bench protocol and report mean and standard deviation across runs.Source: Table 10 · p. 22, F MULTI-SEED MAIN RESULTS · p. 22

G LIMITATIONS

DecFlowEdit is most reliable for local edits with a clear spatial anchor; object-add and global style edits challenge its localization prior. The added prior-extraction pass also increases inference time.

G LIMITATIONS

  • The method is most reliable for local edits when the source image contains a clear editing target or spatial anchor.Temporally aggregated velocity differences can accumulate edit-driven responses around that region to form a selective prior.Source: G LIMITATIONS · p. 22
  • Object-add edits often lack a source-side counterpart, so the prior may diffuse or attach to semantically related existing regions instead of the new object's intended location.Source: G LIMITATIONS · pp. 22–23
  • Global style changes are outside the method's scope because its prior is content-dependent and non-uniform; the paper recommends raw FlowEdit for globally consistent transformations.Source: G LIMITATIONS · p. 23
  • Approximately 2× the sampling steps are needed for the additional prior-extraction pass, or 1.25× with the short schedule.The pass requires no training, optimization, external segmentation, or attention manipulation.Source: G LIMITATIONS · p. 23

H BROADER IMPACT

  • Improved editing may benefit creative production and user control, but more reliable tools may also be misused to create deceptive visual content.Source: H BROADER IMPACT · p. 23

I ASSETS AND LICENSES

  • The work introduces neither a new dataset nor a pretrained generative model; its released asset is an anonymized implementation with reproduction instructions.Source: I ASSETS AND LICENSES · p. 24

Related papers

Loading 2609.34237v1…