Source-linked AI summary
RegionCache: Semantic-Aware Region Reuse for Efficient Multi-Turn Image Generation
Peizheng Li, Xin Ai, Hanyuan Liu, Qiange Wang, Yanfeng Zhang
TL;DR
Multi-turn image editing repeatedly recomputes expensive DiT inference even when most image regions and prompt semantics persist across turns. RegionCache aligns shared prompt semantics with regions through cross-attention and adaptively reuses cached diffusion states, achieving consistent end-to-end speedups while preserving editing quality. Its scope remains bounded by prior acceleration methods’ inability to capture cross-turn redundancy and their potential quality degradation under aggressive approximation.
Problem
Existing DiT-based editing pipelines recompute full-image denoising across turns, while prior acceleration methods overlook cross-turn semantic redundancy or risk quality degradation.
Method
RegionCache aligns shared semantics across consecutive prompts with spatial regions using cross-attention, then selectively reuses cached hidden states with an adaptive reuse schedule.
Results
RegionCache significantly reduces redundant attention computation and achieves consistent end-to-end speedups while preserving editing quality.
Takeaways & Limitations
Semantic stability across editing turns can support region-level state reuse while retaining high editing fidelity.
Takeaways & Limitations
Existing acceleration methods are primarily designed for single-pass generation and can fail to reduce cross-turn redundancy or degrade quality under aggressive approximation.
Abstract
from arXiv · showhide
Real-world image generation often involves multi-turn editing, where users iteratively modify small regions while most image content remains unchanged. However, existing diffusion transformer (DiT)-based editing pipelines recompute the entire image at every turn, causing substantial redundant computation. Existing DiT acceleration methods further ignore semantic correspondence across prompts, leading to unnecessary recomputation or unsafe reuse that harms editing quality. To address this, we propose RegionCache, a semantic-aware reuse framework for multi-turn image editing that selectively reuses diffusion states from unchanged regions. RegionCache detects reusable regions through semantic overlap between consecutive prompts and cross-attention localization, and adopts an adaptive reuse schedule based on prompt similarity and contextual consistency. Experiments on PixArt-alpha demonstrate that RegionCache achieves 1.43x--2.55x end-to-end speedup while maintaining comparable image quality. Code is available at https://github.com/hebutBryant/RegionCache.
1 Introduction
Multi-turn editing repeatedly reruns expensive DiT inference even though consecutive edits usually change localized regions and preserve much of the prompt semantics. RegionCache addresses this redundancy by aligning shared prompt semantics with spatial regions and adaptively reusing cached diffusion states.
- Motivation: 81.05%–92.70% unchanged regions are shared on average across consecutive turns in five real-world image-editing datasets.The measured overlap indicates that edits are typically spatially localized.
- Motivation: Repeated prompt semantics typically correspond to unchanged image regions, creating an opportunity to reuse intermediate diffusion states across turns.Spatial alignment lets the system avoid full-image attention recomputation and focus updates on regions implied by the prompt delta.
- Motivation: Existing DiT acceleration methods target single-pass inference and do not directly address redundancy arising across successive editing turns.Their decisions rely on intra-run characteristics such as denoising-step similarity or token importance.
- RegionCache: RegionCache identifies reusable regions by aligning shared semantics across consecutive prompts and localizing their spatial support through cross-attention.It combines prompt-aligned region caching with a semantic-aware region-reusing policy.
- RegionCache: RegionCache adaptively sets reuse depth: semantically consistent edits permit aggressive reuse, while larger semantic changes trigger conservative reuse.The policy is designed to preserve editing fidelity while reducing unnecessary recomputation.
2 Related Work
Prior diffusion-editing work improves controllability, consistency, or single-run efficiency, but generally leaves full-image inference unchanged across multi-turn edits. Training-based approaches also require additional data and optimization, while existing acceleration methods can fail to capture cross-turn redundancy or degrade quality through aggressive approximation.
- Diffusion Transformer Inference Acceleration Methods: DiT inference is expensive because iterative denoising and quadratic self-attention operate over latent image patches.This computational cost motivates acceleration methods for diffusion inference.
- Diffusion Transformer Inference Acceleration Methods: Intermediate-representation reuse methods exploit smooth feature evolution across denoising steps to skip redundant computation.DeepCache reuses representations at steps with limited feature changes, while related methods introduce delta-based caching.
- Diffusion Transformer Inference Acceleration Methods: Attention-reduction methods lower runtime by identifying tokens that contribute less to generation or by reusing attention selectively across diffusion stages.Examples include TGATE, RAS, and ToCa.
- Diffusion Transformer Inference Acceleration Methods: Existing DiT acceleration methods primarily target single-pass generation and either miss cross-turn redundancy or cause noticeable quality degradation when applied to multi-turn editing.Their reuse decisions depend on intra-run characteristics such as step-wise similarity or token importance.
- Multi-turn Image Editing: Multi-turn editing methods iteratively refine an image while preserving appropriate aspects across turns, using training-based or training-free approaches.Training-based methods require extra data and optimization, whereas training-free methods modify pretrained-model inference behavior.
- Multi-turn Image Editing: Output- or attention-guidance methods improve editing quality and consistency but still perform full-image denoising and attention computation at every step.Repeated full-image inference therefore remains the dominant source of multi-turn editing latency.
3 RegionCache
RegionCache accelerates multi-turn DiT editing by identifying unchanged semantic regions and reusing their hidden states, while recomputing edited regions. It combines cross-attention localization with adaptive timestep scheduling to balance reuse and editing fidelity.
- 3 RegionCache: RegionCache combines prompt-aligned region caching with semantic-aware reuse scheduling to reduce redundant computation across editing turns.The framework exploits both region-level and timestep-level reuse.
- 3.2 Prompt-aligned Region Caching: Cross-attention maps align newly introduced or modified prompt tokens with spatial image regions, defining which image tokens are affected by an edit.Persistent-semantic regions outside the edit region are treated as reusable.
- 3.2 Prompt-aligned Region Caching: RegionCache reuses hidden states from unchanged regions across turns, allowing self-attention and MLP computation to be skipped without altering model execution.A cache pool stores hidden states along the diffusion trajectory, while affected regions are recomputed and updated.
- 3.2 Prompt-aligned Region Caching: Region-level sparse attention updates active regions while retrieving inactive-region hidden states from the cache and merging them into the complete representation.Only active-region queries participate in attention computation; keys and values remain shared across regions.
- 3.3 Semantic-aware Region Reusing: Similarity above threshold τs permits reuse throughout diffusion, whereas semantic changes restrict reuse to the first ρT steps; the implementation sets ρ = 0.2.Early denoising steps mainly establish coarse structure, while later steps refine semantic details and appearance.
- 3.3 Semantic-aware Region Reusing: The reuse schedule balances aggressive computation reuse for semantically stable regions with conservative refreshes for larger semantic edits.Cosine similarity between corresponding pre-edit and post-edit token embeddings measures semantic change.
4 Experiment
Experiments evaluate RegionCache on unified multi-turn editing benchmarks against multi-turn frameworks and adapted single-turn acceleration methods. RegionCache reduces latency and attention computation while maintaining competitive or superior editing and image quality, with larger attention benefits at higher resolutions.
- Comparison with Multi-turn Image Editing Frameworks: RegionCache completes a MagicBrush editing sequence in 11.11 s, yielding 1.6× speedups over Stable Flow and P-2-P and a 2.9× speedup over RAG.Stable Flow and P-2-P recompute full-image self-attention, while RAG adds explicit region-aware control overhead.
- Comparison with Multi-turn Image Editing Frameworks: RegionCache achieves the highest CLIPtxt score of 0.3204 and comparable CLIPimg and CLIPdir scores among multi-turn editing frameworks.The reported results indicate strong prompt alignment, visual consistency, and editing-direction coherence across turns.
- Comparison with Single-turn Acceleration Methods: RegionCache reduces total inference latency to 2.53 s, a 1.57× speedup over the 3.98 s full PixArt-α baseline, while reducing cumulative self-attention time to 0.55 s, a 2.89× reduction.It also outperforms ToCa and TGATE in both total inference time and cumulative attention cost under the unified multi-turn setting.
- Comparison with Single-turn Acceleration Methods: DeepCache achieves a higher total inference speedup of 1.81×, but its aggressive diffusion-step skipping substantially worsens FID and IR relative to RegionCache.The comparison exposes a speed–editing-quality trade-off for step-skipping reuse.
- Comparison with Single-turn Acceleration Methods: RegionCache attains the lowest FID and highest CLIP, IR, and PICK metrics in Table 2, indicating strong semantic alignment and fine-grained editing fidelity.The paper attributes this advantage to reusing hidden states from regions whose semantics persist across turns.
- Runtime Breakdown Analysis: Across resolutions, retrieval and hidden-state loading remain below 0.3 s at 2048×2048, while full-attention cost exceeds 6 s, so savings outweigh added overhead.The runtime breakdown shows that RegionCache targets the dominant computation with minor additional cost.
- Sensitive Study: At 15 diffusion steps, attention speedup increases from approximately 2.7× at 512×512 to 4.7× at 1024×1024 and exceeds 6× at 2048×2048.Self-attention latency rises from 0.31 s to 7.39 s across these resolutions, making attention savings increasingly consequential.
5 Conclusion
RegionCache reuses diffusion states for semantically stable image regions across editing turns and recomputes only regions affected by new edits. Experiments show reduced redundant attention computation and consistent end-to-end speedups while preserving editing quality.
- 5 Conclusion: RegionCache aligns repeated prompt semantics with image regions through cross-attention and caches hidden states where attention computation dominates.An adaptive reuse schedule controls how long cached reuse remains valid along the diffusion trajectory.
- 5 Conclusion: RegionCache selectively reuses cached hidden states for semantically stable regions and recomputes regions affected by new edits.This design targets cross-turn redundant computation in multi-turn Diffusion Transformer editing.
- 5 Conclusion: Experiments demonstrate reduced redundant attention computation, consistent end-to-end speedups, and preserved editing quality.