PaperlayerView exact revision on arXiv

Source-linked AI summary

SpotEdit: Selective Region Editing in Diffusion Transformers

Zhibin Qin, Zhenxiong Tan, Zeqing Wang, Songhua Liu, Xinchao Wang

arXiv:2512.22323v1cs.CVcs.AI

TL;DR

Image editing often changes only a small region, but existing approaches regenerate all regions at every denoising step. SpotEdit selectively identifies stable regions and reuses condition-image features; it reports speedups on two benchmarks while maintaining comparable quality. Its discussion also reports fidelity degradation from static KV caching as edited-region features evolve.

  • Problem

    Most image edits affect small regions, yet methods regenerate every region at each timestep, spending computation on unchanged areas and potentially causing subtle artifacts.

    Source: §1 · p. 1, §2.2 · p. 3

  • Method

    SpotEdit is a training-free framework that uses perceptual similarity to identify stable regions for feature reuse and adaptively fuses reference features to maintain contextual coherence.

    Source: §1 · p. 2, §4.1 · p. 5

  • Results

    SpotEdit reports 1.7× speedup on imgEdit-Benchmark and 1.9× on PIE-Bench++ while maintaining quality comparable to the original model.

    Source: §1 · p. 2

  • Takeaways & Limitations

    SpotEdit selectively updates edited regions while reusing stable-region features, with experiments reporting speedups and preserved editing precision and background consistency.

    Source: §1 · p. 2, §6 · p. 8

    Static KV pairs can become misaligned with evolving edited-region features, and the ablation reports clear degradation in editing fidelity.

    Source: §4.2 · p. 5

Abstract

from arXiv · show

Diffusion Transformer models have significantly advanced image editing by encoding conditional images and integrating them into transformer layers. However, most edits involve modifying only small regions, while current methods uniformly process and denoise all tokens at every timestep, causing redundant computation and potentially degrading unchanged areas. This raises a fundamental question: Is it truly necessary to regenerate every region during editing? To address this, we propose SpotEdit, a training-free diffusion editing framework that selectively updates only the modified regions. SpotEdit comprises two key components: SpotSelector identifies stable regions via perceptual similarity and skips their computation by reusing conditional image features; SpotFusion adaptively blends these features with edited tokens through a dynamic fusion mechanism, preserving contextual coherence and editing quality. By reducing unnecessary computation and maintaining high fidelity in unmodified areas, SpotEdit achieves efficient and precise image editing.

1. Introduction

Image editing often changes only a small region, yet existing methods regenerate the whole image, spending computation on unchanged areas and risking artifacts. SpotEdit selectively reuses stable-region features while SpotFusion blends contextual features, aiming for faster editing with preserved fidelity.

1. Introduction

  • Existing editing methods denoise every region even when only a small area needs modification, wasting computation and potentially introducing artifacts in unchanged areas.This motivates asking whether every region must be regenerated during editing.Source: §1 · p. 1
  • SpotSelector identifies stable, non-edited regions by measuring perceptual similarity between reconstructed latents and condition-image latents through VAE decoder layers.Regions below a perceptual-distance threshold are classified as non-edited, without manual masks.Source: §1 · p. 2
  • SpotFusion adaptively blends condition-image features with edited tokens, relying more on the reference early and shifting toward the current estimate as denoising progresses.The mechanism is intended to preserve temporal coherence and contextual consistency without additional computation for unedited regions.Source: §1 · p. 2
  • SpotEdit reports 1.7× and 1.9× speedups on imgEdit-Benchmark and PIE-Bench++, respectively, while maintaining quality comparable to the original model.Qualitative results are reported to preserve non-edited regions and produce clean, localized edits.Source: §1 · p. 2

2. Related works

Precise image-editing methods evolved from control and inversion approaches to reference-feature and mask-based strategies, followed by DiT methods that accept instructions without manual masks. Across these approaches, full-image regeneration remains common; existing acceleration methods do not selectively bypass computation for preserved regions.

2.1. Precise image editing

  • ControlNet adds external control signals to U-Nets, while inversion-based methods add noise to a condition image and denoise it under a target prompt.The noise-addition and denoising procedure can cause unwanted global changes.Source: §2.1 · p. 2
  • KV-injection methods preserve reference features during denoising, whereas mask-based methods restrict computation to explicitly marked editable regions and retain unedited pixels.Mask-based control offers fine-grained region control but limits flexibility and real-world applicability.Source: §2.1 · p. 3
  • DiT editing models jointly process condition images and noise, enabling instruction-based edits without manually provided masks, but the approaches described still regenerate the entire image at every denoising timestep.Source: §2.1 · p. 3

2.2. Acceleration of efficient editing

  • Full-token acceleration methods reuse or approximate features across timesteps but treat all spatial positions uniformly, and aggressive acceleration can disproportionately harm semantically important edited regions.The cited discussion characterizes the resulting speed gains as offering limited practical benefit for editing when fidelity loss is noticeable.Source: §2.2 · p. 3
  • Token-space methods such as ToCa, DUCA, and RAS reduce timesteps or compress representations, but do not explicitly skip computation for non-edited regions.The section identifies selective updating of edited tokens while bypassing preserved regions as a form of region-aware acceleration these methods do not achieve.Source: §2.2 · p. 3

3. Preliminary

The preliminary section describes conditional image generation as a learned flow from a source distribution to a target image distribution. Rectified Flow uses a linear path between endpoints, and inference approximates the reverse process to transport noise into an image.

3. Preliminary

  • Flow Matching learns a deterministic, time-dependent velocity field that transports samples from a source distribution, typically Gaussian noise, to a target distribution of real images.The condition contains the reference image and the instruction describing the desired edit.Source: §3 · p. 3
  • Rectified Flow assumes linear interpolation between source and target endpoints, yielding a constant target velocity along the path.The endpoints are sampled from the data distribution and a Gaussian source distribution, respectively.Source: §3 · p. 3
  • Inference approximates the reverse process by stepping from X1 toward X0 under condition C, progressively transforming the initial noise sample into the final image.Source: §3 · p. 3

4. Methodology

SpotEdit selectively skips computation for stable, non-edited regions while preserving context for regenerated tokens. SpotSelector routes tokens by perceptual stability, and SpotFusion reuses and blends cached features over denoising steps.

4. Methodology

  • SpotEdit is a training-free framework that skips redundant computation while preserving editing quality and high fidelity.It pairs SpotSelector, which detects non-edited regions, with SpotFusion, which supplies temporally consistent context to regenerated tokens.Source: §4 · pp. 3–4

4.1. SpotSelector

  • SpotSelector compares early reconstructed latents with the condition image using a token-wise perceptual score, then routes stable tokens away from full DiT computation.The LPIPS-like score uses VAE decoder features; non-edited tokens are replaced with their condition-image counterparts before decoding.Source: §4.1 · pp. 4–5

4.2. SpotFusion

  • SpotFusion addresses the loss of contextual contribution and temporal mismatch that can arise when non-edited tokens or their KV features are simply removed or statically cached.It gradually blends cached non-edited features toward reference features across blocks and timesteps, using their observed trajectory alignment as motivation.Source: §4.2 · p. 5
  • Partial attention computes queries only for regenerated tokens and prompt tokens, while cached non-edited and condition-image KV features retain their contextual influence.This concentrates computation on active queries while preserving contextual coherence.Source: §4.2 · p. 6

5. Experiments

Experiments evaluate SpotEdit’s editing quality and efficiency across two benchmarks, with ablations examining token fusion, condition caching, and reset. Results also test preservation qualitatively and transfer to Qwen-Image-Edit.

5.1. Settings

  • Experiments use PIE-Bench++ partial-edit subsets and single-turn imgEdit subsets, evaluating editing quality and inference efficiency against caching and precise-editing baselines.Runs use one NVIDIA H200 GPU, 50 steps, and 1024 × 1024 resolution.Source: §5.1 · pp. 6–7

5.2. Results analyse

  • On imgEdit-Benchmark, SpotEdit reaches a 1.67× speedup; on PIE-Bench++, it achieves 18.73 PSNR, 0.792 SSIMc, and a 1.95× speedup.The reported overall comparison finds comparable or improved quality while accelerating inference.Source: Table 1 · p. 7, §5.2 · p. 7
  • SpotEdit earns the highest imgEdit-Benchmark VL score, 3.77, including 4.41 on Replace and 2.65 on Compose, with a -0.14 drop from original inference.The results cover eight instruction types.Source: Table 2 · p. 7, §5.2 · p. 7

5.3. Ablation study

  • Without periodic reset, speedup rises from 1.95× to 2.25×, but PSNR drops by 1.6 dB and DISTS increases by 0.018.The reset refreshes cached tokens to limit accumulated numerical error, with negligible overhead.Source: Table 3 · p. 7, §5.3 · p. 8

5.2. Results analyse

  • Figure 5 contrasts prior methods’ background changes or color inconsistency with SpotEdit’s faithful preservation of unedited areas and accurate edits.Qualitative examples also show localized regeneration across diverse instructions.Source: §5.2 · p. 7, Figure 5 · p. 8

5.4. Additional results on Qwen-Image-Edit

  • On Qwen-Image-Edit, SpotEdit achieves 1.59× acceleration with +0.01 PSNR and -0.01 DISTS on imgEdit, and 1.72× with +0.03 SSIMc, +1.08 PSNR, and -0.01 DISTS on PIE-Bench.The comparison is presented in Table 4.Source: Table 4 · p. 8, §5.4 · p. 8

5.3. Ablation study

  • Adaptive token fusion preserves background fidelity and edit quality, whereas Naive Skip and Static Token Fusion introduce artifacts or inconsistency.Figure 6 presents the qualitative ablation.Source: §5.3 · p. 7, Figure 6 · p. 8
  • Full condition caching is faster than caching only non-edited tokens: 1.95× versus 1.24×, while the uncached variant has slightly higher PSNR, 19.15 versus 18.73.The full-cache approach reuses condition-image and unedited-region features.Source: §5.3 · p. 7, Table 5 · p. 8

6. Conclusion

The conclusion presents SpotEdit as a training-free, region-aware approach to instruction-based image editing. It selectively processes edited tokens and reports speedups while preserving editing precision and background consistency.

6. Conclusion

  • SpotEdit accelerates instruction-based image editing with a training-free, region-aware framework guided by “edit what needs to be edited.”It avoids uniformly denoising all image tokens by isolating edited regions at token level.Source: §6 · p. 8
  • SpotSelector removes non-edited tokens from processing and restores them directly from the condition image.Source: §6 · p. 8
  • SpotFusion refines feature reuse for contextual alignment, and experiments report speedups while preserving editing precision and background consistency.Source: §6 · p. 8

A. Pseudocode of SpotEdit

The pseudocode specifies SpotEdit’s inputs and output, initializes a condition cache, and outlines transformer processing of active tokens.

A. Pseudocode of SpotEdit

  • The algorithm takes instruction P, condition-image latent Y, initial noise X_T, total steps T, initial-stage steps K_init, threshold τ, and a time schedule; it outputs an edited image.Source: A. Pseudocode of SpotEdit · p. 9
  • SpotEdit initializes a condition cache containing K_Y and V_Y before its denoising loop.Source: A. Pseudocode of SpotEdit · p. 9
  • Within each transformer block, the pseudocode constructs queries for active tokens by combining Q_P and Q_Ati, then constructs keys and values.Source: A. Pseudocode of SpotEdit · p. 9

23 Identify final non-edited regions

The appendix labels SpotSelector’s pseudocode as LPIPS-like perceptual scoring and specifies its inputs and output regions.

23 Identify final non-edited regions

  • SpotSelector’s pseudocode is titled “LPIPS-like Perceptual Scoring.”Source: §23 · p. 9
  • Its stated inputs include conditional image latent Y, VAE decoder shallow layers L, spatial dimensions, and patch size p.Source: §23 · p. 9
  • The pseudocode outputs a regenerated region and a non-edited region.Source: §23 · p. 9

B. Compatibility with Existing Acceleration Methods

SpotEdit accelerates spatially by skipping non-edited regions, complementing accelerators that operate along other dimensions. Its reconstruction of edited-token attention states supports full-token and reuse-based accelerators on the edited subgraph.

B. Compatibility with Existing Acceleration Methods

  • SpotEdit accelerates along the spatial dimension, which the authors describe as orthogonal and complementary to temporal, feature, and attention acceleration.They state these effects can be combined additively for further speed improvements.Source: B. Compatibility with Existing… · p. 9

B.1. General compatibility with full-token computation accelerators

  • SpotFusion reconstructs full attention states for edited tokens, forming a closed, condition-complete subgraph where full-context accelerators can operate.The authors state SpotEdit restricts the spatial domain without changing the functional form of DiT computation, supporting compatibility with temporal accelerators.Source: §B.1 · pp. 9–10

B.2. Compatibility with TeaCache and TaylorSeer

  • SpotEdit integrates TeaCache and TaylorSeer without modification because the reconstructed edited subgraph supports their reuse-based acceleration operations.TeaCache reuses cached residuals, while TaylorSeer applies local Taylor-series predictions to edited tokens; non-edited tokens continue using cached features.Source: §B.2 · p. 10

B.3. Experimental results

Experiments report that SpotEdit combined with TeaCache and TaylorSeer improves efficiency while preserving editing quality. The discussion explains why decoder-feature perceptual scores are preferred to latent ℓ2 distance for selecting stable regions.

B.3. Experimental results

  • Both TeaCache and TaylorSeer integrations remain stable, improve efficiency, and preserve editing quality.The cited tables compare models on imgEdit-Benchmark and PIE-Bench++, but the supplied passages do not state their cell values.Source: §B.3 · p. 10, Table S1 · p. 10, Table S2 · p. 10

C. Discussion between ℓ2-distance and LPIPS-like score

  • Latent ℓ2 distance can inflate similarity responses to low-frequency brightness shifts and miss high-frequency edits, causing both types of token misclassification.The examples describe incorrectly preserving a spaceship that should be removed and unnecessarily regenerating background tokens, with resulting background degradation.Source: C. Discussion between ℓ2-distance and… · p. 10, Figure S1 · p. 10
  • SpotSelector's LPIPS-like score measures distances between VAE decoder features, capturing high-frequency discrepancies and fine-grained spatial patterns for region selection.The authors say this perceptually aligned descriptor addresses the spectral bias of ℓ2 distance.Source: C. Discussion between ℓ2-distance and… · p. 11

D. More Visualization Results

Additional qualitative examples cover diverse instruction-based editing tasks and are described as maintaining high fidelity in non-edited regions.

D. More Visualization Results

Loading 2512.22323v1…