Source-linked AI summary
SeedEdit: Align Image Re-Generation to Image Editing
Yichun Shi, Peng Wang, Weilin Huang
TL;DR
Instructional image editing seeks to make generated images more controllable, but current approaches face unstable reconstruction and re-generation or limited editing-pair data. SeedEdit aligns a text-to-image model into an image-conditioned editor using diverse generated pairs and iterative fine-tuning. It reports higher editing scores than open-source baselines on both benchmarks and higher image similarity on HQ-Edit.
Problem
Instructional image editing remains less developed than image generation and understanding, while existing approaches face instability in reconstruction and re-generation or scarce, tool-limited editing-pair data.
Source: §1 · p. 1Method
SeedEdit treats a text-to-image model as a weak editor, creates diverse image pairs, and iteratively aligns an image-conditioned diffusion model toward image consistency.
Source: §1 · p. 1, §2 · p. 3, §2.1 · p. 3Results
SeedEdit achieves significantly higher editing scores than open-source baselines on both benchmarks and higher CLIP image similarity on HQ-Edit.
Source: §3.2 · p. 5Takeaways & Limitations
The reported results support revising text-to-image-generated images as a first step toward editing arbitrary in-the-wild images.
Source: §3.2 · p. 5On the out-of-domain Emu Edit benchmark, generated-image quality is described as unsatisfying for all methods, including SeedEdit.
Source: §3.2 · p. 5
Abstract
from arXiv · showhide
We introduce SeedEdit, a diffusion model that is able to revise a given image with any text prompt. In our perspective, the key to such a task is to obtain an optimal balance between maintaining the original image, i.e. image reconstruction, and generating a new image, i.e. image re-generation. To this end, we start from a weak generator (text-to-image model) that creates diverse pairs between such two directions and gradually align it into a strong image editor that well balances between the two tasks. SeedEdit can achieve more diverse and stable editing capability over prior image editing methods, enabling sequential revision over images generated by diffusion models.
1 INTRODUCTION
Instructional image editing aims to make generated content more controllable, but the paper describes the task as lagging behind image generation and understanding. SeedEdit addresses instability in reconstruction and re-generation, as well as the scarcity and quality limits of editing-pair datasets, by aligning a text-to-image model toward image editing.
1 INTRODUCTION
- Instructional editing is presented as a way to make generated content more controllable, while the technology is said to lag behind image generation and understanding.The paper frames editing as an intersection of image generation and image understanding.Source: §1 · p. 1
- Training-free editing combines reconstruction and re-generation techniques, but instability in both can accumulate errors and undermine consistency with the input or target description.The passage names DDIM inversion, test-time fine-tuning, and attention control as examples.Source: §1 · p. 1
- Data-driven editing is constrained by scarce, difficult-to-collect high-quality image pairs, while datasets generated with existing tools inherit those tools’ limitations.The passage cites Prompt-to-Prompt and in-painting as data-creation tools.Source: §1 · p. 1
- SeedEdit gradually aligns a text-to-image model toward a balance between image reconstruction and re-generation, producing an editor for instructions or descriptions.The framework first generates diverse pairwise data spanning those directions, then aligns an image-conditioned diffusion model.Source: §1 · p. 1
2 SEEDEDIT
SeedEdit treats a text-to-image model as a weak editor and progressively aligns it to retain re-generation ability while improving image consistency. It creates diverse editing pairs, uses a causal image-conditioned diffusion architecture, and iteratively fine-tunes the model on newly generated, labeled, and filtered data.
2 SEEDEDIT
- SeedEdit distills and aligns a text-to-image model treated as a weak editor, seeking to retain re-generation capability while improving image consistency.The paper identifies scarcity of pairwise image data as the core difficulty.Source: §2 · p. 3
2.1 T2I MODEL FOR EDITING DATA GENERATION
- Diverse training pairs come from a pre-trained text-to-image model combined with varied re-generation techniques and parameters, followed by filtering for training and alignment.The paper says prompt-to-prompt and attention-control approaches alone produce limited types of pairs; Figure 3 compares aligned editing with naive re-generation on prompt alignment and image similarity.Source: §2.1 · p. 3, Figure 3 · p. 3
2.2 CAUSAL DIFFUSION MODEL WITH IMAGE INPUT
- The causal image-conditioned diffusion model uses shared-parameter branches for input and output images, reusing self-attention to communicate through intermediate features.The authors report better performance on geometric deformation tasks and fewer new parameters; dropping the input branch recovers the original text-to-image model, enabling mixed editing and text-to-image training.Source: §2.2 · pp. 3–4, Figure 4 · p. 4
2.3 ITERATIVE ALIGNMENT
- After initial training, the model generates another data set that is labeled and filtered for fine-tuning, with rounds repeated until the metrics stop improving.The iterative alignment is intended to address the initial model’s limited success rate on diverse tasks.Source: §2.3 · p. 4
3 EXPERIMENTS
Across the reported benchmarks, SeedEdit achieves higher editing scores than open-source baselines and higher image similarity on HQ-Edit. The experiments focus on revising text-to-image-generated images, while treating Emu Edit as an out-of-domain test; qualitative comparisons also report better handling of ambiguous instructions and a balance of precise edits with fewer unintended changes.
3.1 BENCHMARK AND METRICS
- SeedEdit is evaluated on HQ-Edit and Emu Edit, with the latter treated as out of domain because its real-scene local edits differ from the training data.HQ-Edit contains 293 Dalle3-generated images; Emu Edit contains 535 real in-the-wild inputs, and the authors mainly rely on HQ-Edit to assess application potential.Source: §3.1 · p. 4
- Evaluation uses CLIP Direction Score for prompt alignment, CLIP image similarity for consistency, and GPT as an alternative evaluator of editing success.Source: §3.1 · p. 4
3.2 IMAGE EDITING COMPARISON
- SeedEdit has significantly higher editing scores than open-source baselines on both benchmarks, and higher CLIP image similarity on HQ-Edit.The table caption identifies Table 1 as a quantitative evaluation on image-editing benchmarks; the passage does not provide the cell values.Source: §3.2 · p. 5, Table 1 · p. 5
- On Emu Edit, all methods’ generated-image quality is described as unsatisfying, although SeedEdit’s quantitative scores are comparable to or better than the original Emu Edit method.The authors characterize revising text-to-image images as a possible first step before editing arbitrary in-the-wild images.Source: §3.2 · p. 5
- Qualitative HQ-Edit examples show SeedEdit handling ambiguous instructions and fine-grained edits with a higher success rate.The comparison includes baselines; the passage states the qualitative outcome but gives no numeric success rate.Source: §3.2 · p. 5
- Against commercial tools, SeedEdit is described as making more precise edits that follow instructions, while DALLE3 and Midjourney introduce more unintended content changes.An internal user study indicated a strong preference for SeedEdit’s results.Source: §3.2 · p. 5
4 CONCLUSION
SeedEdit progressively aligns a pre-trained text-to-image diffusion model for image editing, combining image consistency with prompt alignment and reporting superior results over existing methods.
4 CONCLUSION
- SeedEdit yields superior results over existing methods by a large margin.The conclusion reports this overall experimental outcome.Source: §4 · p. 5
- The framework adapts a pre-trained text-to-image diffusion model to editing through iterative data generation and fine-tuning.It uses a causal diffusion model conditioned on both images and text.Source: §4 · p. 5
- The conclusion includes example comparisons on HQ-Edit and Emu Edit, with method labels shown for each benchmark.The HQ-Edit examples label Instruct-Pix2Pix, MagicBrush, UltraEdit, and SeedEdit (SDXL); the Emu Edit examples label Instruct-P2P, MagicBrush, UltraEdit, Emu Edit, and SeedEdit.Source: Figure 5 · p. 6, §4 · p. 6, Figure 6 · p. 6