PaperlayerView exact revision on arXiv

Source-linked AI summary

ProEdit: Inversion-based Editing From Prompts Done Right

Zhi Ouyang, Dian Zheng, Xiao-Ming Wu, Jian-Jian Jiang, Kun-Yu Lin, Jingke Meng, Wei-Shi Zheng

arXiv:2512.22118v1cs.CV

TL;DR

Existing inversion-based visual editing injects source information to preserve consistency, but this can interfere with instructed edits. ProEdit changes attention and latent handling to reduce that influence, and experiments report state-of-the-art performance across varied editing tasks.

  • Problem

    Source-information injection in inversion-based editing can interfere with instructed changes to subject attributes such as color, pose, and number.

    Source: §1 · p. 2

  • Method

    ProEdit combines KV-mix, which mixes source and target K/V features in edited regions, with Latents-Shift, which injects random noise into the source distribution there.

    Source: §1 · p. 2

  • Results

    Experiments report state-of-the-art performance across varied editing tasks while preserving non-edited content.

    Source: §1 · p. 2

  • Takeaways & Limitations

    ProEdit is described as plug-and-play for integration into a wide range of existing inversion and editing methods.

    Source: §1 · p. 2

Abstract

from arXiv · show

Inversion-based visual editing provides an effective and training-free way to edit an image or a video based on user instructions. Existing methods typically inject source image information during the sampling process to maintain editing consistency. However, this sampling strategy overly relies on source information, which negatively affects the edits in the target image (e.g., failing to change the subject's atributes like pose, number, or color as instructed). In this work, we propose ProEdit to address this issue both in the attention and the latent aspects. In the attention aspect, we introduce KV-mix, which mixes KV features of the source and the target in the edited region, mitigating the influence of the source image on the editing region while maintaining background consistency. In the latent aspect, we propose Latents-Shift, which perturbs the edited region of the source latent, eliminating the influence of the inverted latent on the sampling. Extensive experiments on several image and video editing benchmarks demonstrate that our method achieves SOTA performance. In addition, our design is plug-and-play, which can be seamlessly integrated into existing inversion and editing methods, such as RF-Solver, FireFlow and UniEdit.

1. Introduction

Inversion-based editing preserves source content by injecting source information during sampling, but that reliance can obstruct instructed changes to subject attributes. ProEdit addresses this tension with attention- and latent-level changes, and experiments report strong editing performance across tasks.

1. Introduction

  • Existing inversion-based methods reinject source information during sampling to preserve source fidelity, but this can hinder changes to attributes such as color, pose, and number.These methods begin from inverted source latents and use the target prompt to guide sampling.Source: §1 · p. 2
  • Global attention injection can overemphasize source attributes and weaken text guidance, while source-distribution initialization can pull sampling toward reconstruction.The paper identifies both attention injection and latent initialization as sources of excessive source information.Source: §1 · p. 2
  • ProEdit introduces KV-mix and Latents-Shift to reduce source influence while preserving background consistency and non-edited content.KV-mix mixes source and target K/V features in edited regions while retaining source features elsewhere; Latents-Shift injects random noise into the source distribution of edited regions.Source: §1 · p. 2
  • Experiments report state-of-the-art performance across varied editing tasks, with preserved non-edited content, plug-and-play integration, and strong attribute editing.The paper describes integration into a wide range of existing inversion and editing methods.Source: §1 · p. 2

2. Related Work

The related work traces visual generation and editing from diffusion-based approaches toward flow models and training-free inversion methods. It positions ProEdit as addressing the editing impact of inversion strategies, a concern that recent flow-based work has overlooked.

2.1. Text-to-Visual generation

  • Flow models such as FLUX and HunyuanVideo [25] use MMDiT and probability-flow ODEs to simulate paths between distributions for faster, fewer-step generation.These text-to-image and text-to-video models also support editing by generating target visuals from source visuals and modified prompts.Source: §2.1 · pp. 2–3

2.2. Text-driven Editing

  • Text-driven editing spans training-based methods [27], inversion-based methods, sampling-based methods, and attention-based methods.Attention-based editing has also expanded to video editing.Source: §2.2 · p. 3
  • Recent flow-based inversion work improves solvers [18] and joint attention [56], but overlooks how inversion strategies affect editing content.ProEdit addresses that impact from attention and latent-distribution perspectives without selecting specific attention heads, layers, or block types.Source: §2.2 · p. 3

3. Method

ProEdit addresses source-image influence in inversion-based editing through attention control and latent shifting while preserving background consistency. Its method moves from flow-model preliminaries and an analysis of source injection to two editing mechanisms and an overall sampling pipeline.

3.1. Preliminaries

  • Flow matching learns a velocity field along a straight trajectory, and reversing that trajectory yields the inversion process used for visual reconstruction and editing.The forward and reverse processes are numerically solved by discretizing their ODEs.Source: §3.1 · p. 3

3.2. Rethinking the Inversion-Sampling Paradigm

  • The investigation attributes editing failures to global attention injection and latent distribution injection, which retain source information that can compete with target-prompt guidance.Global source attention can carry source attributes into edits, while the inverted latent retains source attributes that hinder editing when source and target prompts differ substantially.Source: §3.2 · p. 4

3.3. KV-mix

  • KV-mix combines source and target visual attention features in edited regions while retaining full source-feature injection outside them.The edited-region mask is derived from attention maps, and the mixing strength δ controls the mix and the level of non-edited-content preservation.Source: §3.3 · pp. 4–5

3.4. Latents-Shift

  • Latents-Shift uses random noise to shift the inverted-noise distribution in edited regions, with β controlling the fusion ratio.The edited-region mask is inherited from KV-mix.Source: §3.4 · p. 5

3.5. Overall

  • The pipeline inverts the source image, derives an editing-region mask, shifts the inverted noise, and applies KV-mix during target-prompt sampling.The listed evaluation includes text-driven image editing on PIE-Bench and a quantitative Color Editing comparison; the supplied passages give no table scores.Source: §3.5 · pp. 5–6, Table 1 · p. 6, Table 2 · p. 6

4. Experiments

Experiments evaluate ProEdit on text-driven image and video editing and ablate its components. The reported results indicate improved editing while retaining non-edited content.

4.1. Setup

  • The evaluation covers 700 PIE-Bench images across 10 editing types and 55 video-editing pairs, with ProEdit integrated into several flow-based methods.Image experiments use 15 sampling steps; video experiments use 25.Source: §4.1 · pp. 6–7

4.2. Text-driven Image Editing

  • ProEdit with UniEdit achieves state-of-the-art performance in both source-content preservation and editing quality on text-driven image editing.Across flow-based inversion methods, ProEdit improves image-editing results; qualitative comparisons also report consistency in non-editing content.Source: §4.2 · p. 7

4.3. Text-driven Video Editing

  • ProEdit improves flow-based inversion results on video editing, with high-quality edits and spatial and temporal consistency.Table 3 reports average scores across videos for each metric.Source: §4.3 · p. 7

4.4. Ablation Study

  • KV-mix improves CLIP similarity, and Latents-Shift further enhances it; the modules work synergistically to improve editing results.Among tested attention-feature combinations, KV achieved the best performance for both background consistency and editing quality.Source: §4.4 · p. 8

5. Conclusion

The conclusion identifies excessive source-image information injection as a trade-off in existing flow-based editing. ProEdit combines attention- and latent-level modules, and experiments report high background consistency alongside excellent editing quality.

5. Conclusion

  • Existing flow-based methods sacrifice editing quality to maintain background consistency when they inject excessive source-image information.The identified issue arises from the inverted-latent-with-global-injection strategy.Source: §5 · p. 9
  • ProEdit is a training-free method that addresses source-information injection through KV-mix and Latents-Shift.The modules address the issue from attention and latent perspectives.Source: §5 · p. 9
  • Experiments show ProEdit integrates into existing flow-based inversion methods while achieving high background consistency and excellent editing quality.Source: §5 · p. 9

A. Extracting Mask From Attention Map

The mask is extracted from the last Double block’s attention map at a low-noise timestep. It is expanded to cover coarse boundaries, which supports semantic alignment and smoother edges.

A. Extracting Mask From Attention Map

  • The editing-region mask comes from the last Double block’s attention map, selected for its association between relevant text and image regions and lower memory consumption.Extraction uses the first inversion step or last sampling step, when images are least affected by noise and have the best text-to-image correlation.Source: A. Extracting Mask From Attention Map · p. 12
  • A one-step outward diffusion expands the downsampled mask to cover coarse pixel-space boundaries, helping smooth edges and avoid artifacts.The paper states that this relatively coarse mask is sufficient for semantic alignment.Source: A. Extracting Mask From Attention Map · p. 12
  • The target object can be identified by the editing-object noun or by an externally provided mask for more flexible control.Source: A. Extracting Mask From Attention Map · p. 12

B. Implementation Details

Implementation uses fixed fusion settings for image and video editing, applies feature fusion across Double and Single blocks, and tunes injection steps and baseline hyperparameters.

B. Implementation Details

  • The mix strength δ is 0.9 and fusion ratio β is 0.25 for image and video editing.The paper describes these settings as balancing source-content preservation and editing performance and achieving the best editing results, respectively.Source: B. Implementation Details · p. 12
  • Feature fusion injection is applied at each timestep to all Double and Single blocks.Source: B. Implementation Details · p. 12
  • The authors fine-tune attention-fusion injection steps and adjust official baseline implementations’ hyperparameters for satisfactory results.Source: B. Implementation Details · p. 12

C. Quantitative Results of Attention Feature Combination Effect

The section compares attention-feature combinations and reports that the KV combination was selected for its results in background consistency and editing quality.

C. Quantitative Results of Attention Feature Combination Effect

  • The quantitative comparison evaluates different attention-feature combinations in the fusion injection mechanism.The table marks the best and second-best results in bold and underline, respectively.Source: C. Quantitative Results of Attention… · p. 12, Table 5 · p. 12
  • Masks extracted from either the last Double block or all blocks effectively segment the editing region for an orange editing target.Source: Figure 8 · p. 12
  • The KV combination showed satisfactory results in both background consistency and editing quality, motivating its adoption for KV-mix.The comparison covers different attention-feature combinations in the fusion injection mechanism.Source: C. Quantitative Results of Attention… · p. 12, Table 5 · p. 12

D. More Qualitative Results for Image Editing

The qualitative image results show cases where ProEdit achieves semantically consistent edits when other inversion-based methods fail or produce insufficient edits or inconsistent results.

D. More Qualitative Results for Image Editing

E. More Qualitative Results for Video Editing

The video results report temporal consistency and preservation of original motion patterns. The instruction-editing examples use Qwen3-8B to let users direct edits with instructions.

E. More Qualitative Results for Video Editing

F. Editing by Instruction

Loading 2512.22118v1…