Source-linked AI summary
Multi-History-Step SDE Inversion for Image Editing with Superior Regional Awareness
Haiyan Wei, Yunlong Wang, Huaibo Huang, Zhenan Sun, Kunbo Zhang
TL;DR
Training-free diffusion editing methods can reconstruct images faithfully, but existing approaches remain inefficient, have limited plasticity, and struggle to preserve unedited regions. MIEdit combines predictor–corrector multi-history-step SDE inversion with inversion-time semantic masking; experiments report consistent gains, especially on challenging structural edits.
Problem
Existing training-free diffusion editing approaches remain inefficient, exhibit limited plasticity, and struggle to preserve unedited regions accurately.
Source: §1 · p. 2Method
MIEdit uses predictor–corrector multi-history-step SDE inversion, an optimized inference-time noise schedule, and CFG-derived semantic angle masks generated during inversion.
Source: §1 · pp. 2–3Results
Experiments and EditEval++ demonstrate consistent gains for MIEdit, especially on challenging structural edits.
Source: §5 · p. 15Takeaways & Limitations
The authors suggest MIEdit’s approach has promising extensions to video and 3D generation.
Source: §5 · p. 15
Abstract
from arXiv · showhide
In recent years, diffusion stochastic differential equation (SDE) inversion and inversion-free methods have become prevalent for training-free image editing, as they can achieve faithful reconstruction without tuning. However, existing approaches remain inefficient, exhibit limited plasticity, and struggle to accurately preserve unedited regions. To address these issues, we propose MIEdit, a training-free editing framework based on SDE inversion. MIEdit introduces a predictor-corrector multi-history-step scheme to achieve superior editing quality with fewer steps. We further mitigate heterogeneity and conflict between the multi-conditioned noise residuals and gradient terms during sampling, improving stability and editing plasticity under large edits. MIEdit also includes Inversion-Time Automatic Semantic Angle Masking (IASM); it leverages classifier-free guidance to automatically generate semantic angle masks during inversion and applies them throughout the sampling process for regional constraints, without extra user inputs. We additionally construct EditEval++ (30 fine-grained tasks, 1,000+ image-text-mask triplets) for comprehensive evaluation; experiments show that MIEdit outperforms state-of-the-art techniques. Project page: https://whywwwzzzg.github.io/MIEdit/.
1 Introduction
MIEdit targets inefficiency and limited plasticity in diffusion-inversion editing, while also addressing weak preservation of unedited regions. It combines multi-history-step SDE inversion with a predictor–corrector scheme, inference-time schedule changes, and automatic semantic masking.
1 Introduction
- Existing ODE inversion can require tuning or repeated network evaluations for reconstruction consistency, while SDE inversion methods often use low-order samplers and can lack stability or editing plasticity.Both ODE- and SDE-based methods remain weak at preserving non-edited regions; existing mask strategies may require extra inputs or a separate branch.Source: §1 · p. 2
- MIEdit combines predictor–corrector multi-history-step SDE inversion with a modified inference-time noise schedule to improve editing quality and plasticity, including in large-edit scenarios.The schedule is reported to mitigate artifacts in naive multi-history-step inversion.Source: §1 · pp. 2–3
- IASM generates semantic angle masks during inversion and uses them throughout generation for regional constraints without extra inputs or inference branches.It uses classifier-free guidance and a direction-based metric for fine-grained mask control.Source: §1 · p. 3
- MIEdit reformulates DDPM inversion within a unified SDE framework and introduces a two-stage predictor–corrector, multi-history-step inversion-and-generation method.The contribution list identifies improved efficiency and generation quality as aims of the method.Source: §1 · p. 3
- EditEval++ expands the original seven tasks to 30 fine-grained tasks with 1,000+ image–text–mask triplets, and experiments on PIE-Bench and EditEval++ demonstrate MIEdit’s advantages.The benchmark is described as featuring more diverse scenes and descriptions.Source: §1 · p. 3
2 Related Work
Prior image-editing work uses diffusion inversion to reconstruct inputs under new prompts, while localized editing methods seek to change selected regions and preserve the rest. The literature includes ODE and SDE inversion approaches, alongside mask-based strategies with input or control limitations.
2.1 Diffusion Inversion for Image Editing
- DDIM and flow inversion map images to noise or intermediate states, but faithful reconstruction can require optimization or multiple function evaluations per step [2].DDPM inversion instead recovers noise vectors for reconstruction, and related inversion-free approaches use differential corrections to mitigate ODE error accumulation [21] [51].Source: §2.1 · p. 3
- MIEdit draws on high-order SA-Solver, whose arbitrary-order SDE formulation includes DDPM and SDE-DPM-Solver++ as low-order cases.The paper states that its method achieves higher efficiency and quality with exact reconstruction.Source: §2.1 · p. 3
2.2 Precisely Localized Image Editing
- Localized editing methods use user-provided masks or infer masks from attention maps, but may require blended-word prompts, identify subjects rather than precise regions, or add inference overhead [13] [51] [8] [28] [41].Some automated methods also cannot reliably provide high-quality control throughout the sampling trajectory.Source: §2.2 · p. 4
3 Methods
MIEdit frames image editing as SDE inversion, using multi-history-step sampling and inversion-derived masks to guide generation. Its methods address reconstruction, editing plasticity, and control over edited regions.
3.1 Diffusion SDE and Its Inversion
- In the SDE formulation, inversion constructs latent codes so the reverse trajectory reconstructs the given image.The codes include the initial random state and random drives injected throughout generation.Source: §3.1 · p. 4
3.2 Naive SA-Inversion
- The SDE inversion scheme stores Wiener-process increments as per-step noise terms, which can reconstruct the input and support editing under a new prompt.The multi-history-step sampler uses preceding model outputs as well as the current output for interpolation.Source: §3.2 · p. 5
3.3 Granting Higher Plasticity to Multi-History-Step SDE Inversion
- Multi-history inversion can distort latent statistics and destabilize large edits; a power-interpolated β schedule lowers noise at matched timesteps to improve plasticity and reduce artifacts while retaining reconstruction.The latent-scale deviation tends to become more pronounced as the number of historical steps increases.Source: §3.3 · pp. 6–7
3.4 Inversion-Time Automatic Semantic Masking
- IASM generates masks during inversion from lower-noise latents, providing region guidance through early generation without external conditions.The masks are intended to remain effective even when inversion and generation are weakly consistent.Source: §3.4 · p. 8
- IASM sets the inversion negative prompt to the target prompt, using conditional prediction differences to identify regions of semantic conflict for editing.Small differences indicate source–target agreement; large differences indicate regions supported by the source but not the target.Source: CFG for Masking · p. 9
- IASM computes masks at selected high-confidence inversion steps, aligns them with corresponding sampling steps, and uses an averaged mask at higher-noise steps to improve stability.The method uses lower-noise representations where structures and details are clearer.Source: Temporal Scheduling of Masks · p. 9
4 Experiments
Experiments compare MIEdit with training-free editing baselines on PIE-Bench and EditEval++, reporting stronger preservation and editing fidelity. Ablations examine sampling history, noise scheduling, and mask localization.
4.1 Setup
- EditEval++ expands EditEval from 7 to 30 fine-grained tasks and from 150 to 1,160 samples, with masks for each sample.Its tasks span object-level, scene-level, and compositional and reasoning editing.Source: §4.1 · p. 11, Figure 5 · p. 11
- Evaluation uses PIE-Bench preservation and CLIP edit-success metrics, EditEval++ human-preference EditScore, and function evaluations for efficiency.The comparison follows baseline protocols; one NFE is one forward pass on a single sample.Source: §4.1 · pp. 10–11
4.2 Comparison with Previous Editing Methods
- Qualitative comparisons show prompt-following edits with high fidelity and better preservation of non-target content.The section presents these comparisons in Fig. 6.Source: §4.2 · p. 11, Figure 6 · p. 12
- The reported method outperforms all PIE-Bench baselines in background preservation and CLIP-measured instruction fidelity, and its SD3 version achieves the highest EditScore, 4.30, on EditEval++.On EditEval++, it also has higher CLIP scores while better preserving background; among SD3-based methods, it is fastest at 1.8 seconds per image.Source: §4.2 · p. 12
4.3 Ablation Study and Analysis
- With sampling steps fixed, 2 prediction histories and 3 correction histories achieve the best editing performance; the power-interpolated β schedule is reported stable for large edits.The linear β schedule tends to introduce artifacts in large-scale editing.Source: §4.3 · pp. 12–13, Figure 7 · p. 14, Table 3 · p. 14
- IASM localizes editing regions better than the compared attention and sampling-generated masks; temporal scheduling and semantic-angle masks outperform magnitude-based masks.The comparisons use an SD3.5-based SA-Inversion pipeline.Source: §4.3 · p. 13, Figure 8 · p. 14, Table 3 · p. 14
5 Conclusion
MIEdit combines high-order SDE inversion with automatic semantic-angle masks for training-free image editing. Experiments and EditEval++ report gains, especially on challenging structural edits, while the authors identify video and 3D generation as possible extensions.
5 Conclusion
- MIEdit is a training-free image-editing framework based on high-order SDE inversion.Its predictor–corrector multi-history scheme enables exact reconstruction with fewer steps, while an optimized inference noise schedule improves stability and quality for large edits.Source: §5 · p. 15
- IASM derives semantic-angle masks during inversion from CFG feature discrepancies and constrains regions throughout sampling without extra inputs.The conclusion describes these masks as automatic.Source: §5 · p. 15
- Experiments and EditEval++ show consistent gains, especially on challenging structural edits, and the authors suggest video and 3D generation as extensions.Source: §5 · p. 15