Source-linked AI summary
EditCrafter: Tuning-free High-Resolution Image Editing via Pretrained Diffusion Model
Kunho Kim, Sumin Seo, Yongjun Cho, Hyungjin Chung
TL;DR
High-resolution image editing with pretrained diffusion models is limited by fixed training resolutions and can suffer from repetition and structural artifacts under naive patch-wise editing. EDITCRAFTER combines tiled inversion with NDCFG++ to edit high-resolution images without tuning or optimization, and reports improved alignment, human preference, and artifact avoidance.
Problem
Pretrained diffusion-based editing methods are difficult to apply beyond 512x512 or 1024x1024 training resolutions, while naive patch-wise editing can cause object repetition and unrealistic structures.
Method
EDITCRAFTER uses tiled inversion to preserve high-resolution input identity and NDCFG++ to guide editing from the inverted latent.
Results
EDITCRAFTER outperforms baseline approaches in human preference and image-text alignment while avoiding visible seams and unwanted object repetition.
Takeaways & Limitations
The tuning-free and optimization-free pipeline supports high-resolution image editing across resolutions using pretrained diffusion models.
Abstract
from arXiv · showhide
We propose EditCrafter, a high-resolution image editing method that operates without tuning, leveraging pretrained text-to-image (T2I) diffusion models to process images at resolutions significantly exceeding those used during training. Leveraging the generative priors of large-scale T2I diffusion models enables the development of a wide array of novel generation and editing applications. Although numerous image editing methods have been proposed based on diffusion models and exhibit high-quality editing results, they are difficult to apply to images with arbitrary aspect ratios or higher resolutions since they only work at the training resolutions (512x512 or 1024x1024). Naively applying patch-wise editing fails with unrealistic object structures and repetition. To address these challenges, we introduce EditCrafter, a simple yet effective editing pipeline. EditCrafter operates by first performing tiled inversion, which preserves the original identity of the input high-resolution image. We further propose a noise-damped manifold-constrained classifier-free guidance (NDCFG++) that is tailored for high resolution image editing from the inverted latent. Our experiments show that the our EditCrafter can achieve impressive editing results across various resolutions without fine-tuning and optimization.
1. Introduction
Pretrained T2I diffusion models enable strong text-based editing but remain tied to low training resolutions, making high-resolution editing difficult. EDITCRAFTER addresses this with tiled inversion and tailored guidance, achieving improved alignment and avoiding common patch-wise artifacts.
- Motivation: Pretrained T2I diffusion models are generally constrained to 512 × 512 or 1024 × 1024 training resolutions.These limits restrict their direct use for arbitrary high-resolution image editing.
- Motivation: High-resolution editing requires preserving fine-grained details and structural coherence, but pretrained-model resolution limits are insufficient for real-world applications.The problem affects domains such as digital content creation and industrial design.
- Motivation: Independent patch-wise editing can produce partial objects, prompt-mismatched content, and repeated structures when the same prompt is applied across patches.A patch may contain only part of an object or lack the object described by the prompt.
- EDITCRAFTER: EDITCRAFTER uses tiled inversion to obtain high-resolution latent representations that preserve the original input information without tuning.The method analyzes patch-wise inversion reconstruction to make the latent suitable for editing.
- EDITCRAFTER: NDCFG++ guides high-resolution latents with a single text prompt, while experiments report stronger human preference and image-text alignment than baseline approaches.Qualitative results also report target-object modification without visible seams or unwanted repetition.
2. Related Work
Related work extends pretrained diffusion models toward high-resolution generation and editing through direct training, patch-wise methods, kernel dilation, attention control, and inversion. EDITCRAFTER targets patch boundary artifacts and repetition that remain challenges in prior high-resolution editing.
- High-resolution Generation: Direct high-resolution training can address resolution limits but requires substantially greater model complexity, training data, and computational resources.These costs motivate approaches that retain fixed-size pretrained models.
- High-resolution Generation: Panoramic diffusion methods use latent merging or patch-wise joint processes, but seamless patch integration and structural coherence remain difficult.Latent averaging can smooth transitions, yet prior work still reports seam artifacts.
- High-resolution Generation: Kernel-dilation methods provide training-free high-resolution generation, while FouriScale reduces repetitive patterns and structural distortions across scales.EDITCRAFTER adapts high-resolution generation approaches for image editing.
- Diffusion-Based Editing: Attention control guides edited details, whereas inversion maps source images into latent space to support editing while maintaining image consistency.These are complementary directions within pretrained diffusion-model editing.
- High-resolution Editing: CSD performs synchronous patch-wise high-resolution editing, but boundary artifacts remain a challenge due to patch-wise sampling and pretrained-model dependencies.EDITCRAFTER combines patch-wise editing with high-resolution generation to pursue artifact-free results.
3. Method
EDITCRAFTER first converts arbitrary-size inputs into edit-friendly high-resolution latents through tiled DDIM inversion, then applies re-dilated sampling and NDCFG++ for text-guided editing. The design addresses identity preservation, high-resolution processing, and object repetition.
- 3.3. High-resolution Sampling Process with Kernel Dilation: The reverse process uses a re-dilated noise estimator because fixed-size diffusion models cannot directly handle high-resolution latents during sampling.The framework adapts a high-resolution generator based on dilated convolutions while retaining pretrained parameters.
- 3.2. Tiled DDIM Inversion: Tiled DDIM inversion partitions a high-resolution image into training-resolution tiles and separately inverts them to preserve input identity.The tile latents are concatenated into a high-resolution inverted latent, with CFG guidance set to ω = 0 during inversion.
- 3.2. Tiled DDIM Inversion: Concatenating the inverted tile latents provides a suitable initial point for the subsequent reverse diffusion process regardless of input image size.This enables DDIM inversion for arbitrary input dimensions.
- 3.3. High-resolution Sampling Process with Kernel Dilation: Patch-wise joint reverse processes can propagate one prompt across patches and cause unintended object repetition.This occurs when patches contain only partial object representations or when the object is absent from a patch.
- 3.3. High-resolution Sampling Process with Kernel Dilation: Standard CFG with ω = 7.5 may fail to preserve source-image information during editing because it was designed for generation.The paper therefore introduces a guidance mechanism tailored to editing rather than relying directly on high CFG settings.
- 3.3. High-resolution Sampling Process with Kernel Dilation: NDCFG++ interpolates guidance with a small λ ∈ [0, 1], making estimates less likely to deviate from the data manifold.It is designed to control detailed information while preserving object-level information during sampling.
4. Experiments
Experiments evaluate EditCrafter across resolutions and compare it with patch-wise and low-resolution editing baselines, including qualitative, quantitative, user, and ablation studies.
- Experiment Setup: Experiments use SD 2.1 and SDXL 1.0 at 4×, 8×, and 16× pixel scales beyond their training resolutions.Evaluated resolutions reach 2048×2048 for SD 2.1 and 4096×4096 for SDXL on a single RTX 4090.
- Experiment Setup: The evaluation dataset contains 150 prompt-image pairs spanning square, wide, and panoramic high-resolution images.It includes 30 images with manually selected prompts for each resolution, with editing prompts created by replacing object or background descriptions.
- Qualitative Evaluation: Compared with CSD, EditCrafter produces more faithful, prompt-aligned edits while preserving intricate original-image details across 4×, 8×, and 16× settings.CSD frequently generates repetitive objects, adheres more to the source at higher resolutions, and produces white patch-boundary grids.
- Quantitative Results: 72.61% of user-study cases favored EditCrafter over CSD among 112 participants.The study collected 25 responses, including five vigilance tasks, and measured human preference for the editing outputs.
- Ablation Study: Removing NDCFG++ degrades text-to-image alignment, human-preference scores, and CLIP text-image matching in the 16× SD 2.1 ablation.Qualitatively, NDCFG++ helps position the edited object’s head consistently with the source object’s location and identity.
5. Conclusion
EDITCRAFTER is a tuning-free and optimization-free high-resolution editing pipeline built on pretrained diffusion models. Its tiled inversion and NDCFG++ design achieves consistently superior performance across high-resolution editing methods.
- EDITCRAFTER performs high-resolution image editing without tuning or optimization using pretrained diffusion models.
- The pipeline combines tiled inversion with NDCFG++ to support high-resolution editing.
- The method demonstrates consistently superior performance across high-resolution image editing methods.
- The paper reports extensive quantitative and qualitative evaluation of high-resolution image editing.
A. Implementation Details
The implementation details compare reverse diffusion processes and specify sampling settings for EditCrafter and ScaleCrafter across multiple resolutions and pretrained models.
- DDIM sampling uses T = 50 steps for the reverse processes.
- For ×4 editing, τ = 10, while ×8 and ×16 editing use τ = 37.
- The reported settings apply to both SD 2.1 and SDXL 1.0.
- The implementation follows ScaleCrafter’s re-dilated convolution configurations for each resolution.
- The reverse-diffusion procedures initialize differently: ScaleCrafter requires a noisy latent zT, whereas EditCrafter uses an inverted latent z∗.
- EditCrafter’s reverse process applies NDCFG++ and decodes the final latent into the output image.
B. Effect of Classfier-Guidance Scale
The study examines small classifier-guidance scales and finds that increasing λ improves prompt conformity, while λ = 0.5 balances editing adherence with identity preservation.
- λ = 0 produces a reconstruction that does not exactly replicate the original image but provides a foundation for later editing.
- As λ increases, edited images progressively conform more closely to the specified edit prompt.
- Higher guidance scales enhance alignment with generated modifications and facilitate more precise, controlled image editing.
- λ = 0.5 is used in all experiments as a balance between prompt adherence and preserving the original identity.
- Figures A7 and A8 examine the effect of CFG scale, including λ in 4 × SD 2.1.
C. User Study
The user study collected preference statistics from 112 vigilance-test-passing Amazon MTurk participants evaluating text-based edits against source-image detail preservation.
- Preference statistics were collected from 112 participants who passed vigilance tests on Amazon MTurk.
- Participants selected which displayed outcome better applied the requested edit while preserving most source-image details.
- The study question followed the formulation used in prior work [33].
D. Quantitative Evaluation of Low-Resolution Editing Combined with Super-Resolution
EditCrafter is quantitatively compared with low-resolution editing followed by super-resolution, achieving the highest reported ImageReward and CLIPScore and stronger human preference.
- EDITCRAFTER achieves the highest scores on both ImageReward and CLIPScore against ProxEdit + StableSR and InfEdit + StableSR.InfEdit + StableSR attains high HPSv2 scores but fails to recover intricate details after resizing.
- 61.12% of evaluators preferred EDITCRAFTER over InfEdit + StableSR.The comparison included 124 participants and five vigilance tasks.
- 92.38% of evaluators preferred EDITCRAFTER over ProxEdit + StableSR.The comparison included 117 participants and five vigilance tasks.
- The user study asked participants to choose the editing outcome most faithful to both the original image and the textual instruction.
F. More Qualitative Comparisons
The qualitative comparison includes the original image, CSD, and EDITCRAFTER outputs.
- The comparison presents the Original Image, CSD, and EDITCRAFTER results side by side.