Source-linked AI summary
SeedEdit 3.0: Fast and High-Quality Generative Image Editing
Peng Wang, Yichun Shi, Xiaochen Lian, Zhonghua Zhai, Xin Xia, Xuefeng Xiao, Weilin Huang, Jianchao Yang
TL;DR
Instructive image editing needs scalable, high-quality data that preserves real-image content while following complex instructions. SeedEdit 3.0 combines heterogeneous data curation, meta-information, joint training, and reward-based learning, achieving a strong multi-metric trade-off with faster inference. Its scope remains bounded by synthetic-to-real domain differences and limitations of current VLM detail partitioning.
Problem
Instructive editing requires large, high-quality image-editing data, but heterogeneous sources contain quality variation, caption errors, and synthetic-to-real domain gaps.
Method
SeedEdit 3.0 combines multi-source data curation, multi-granularity meta-information, recaptioning, joint T2I-editing training, and diffusion-reward learning.
Results
SeedEdit 3.0 achieves the best trade-off across instruction following, content preservation, and image quality, while taking 15 s versus 50 s per query for GPT-4o.
Takeaways & Limitations
The resulting system improves real-image editing, face and identity preservation, text editing quality, prompt understanding, and dynamic motion within a high-performance editing system.
Takeaways & Limitations
Current VLM-based models are weaker at detail partition than expert reward models, and synthesized data retains a real-image performance gap.
Abstract
from arXiv · showhide
We introduce SeedEdit 3.0, in companion with our T2I model Seedream 3.0, which significantly improves over our previous SeedEdit versions in both aspects of edit instruction following and image content (e.g., ID/IP) preservation on real image inputs. Additional to model upgrading with T2I, in this report, we present several key improvements. First, we develop an enhanced data curation pipeline with a meta-info paradigm and meta-info embedding strategy that help mix images from multiple data sources. This allows us to scale editing data effectively, and meta information is helpfult to connect VLM with diffusion model more closely. Second, we introduce a joint learning pipeline for computing a diffusion loss and reward losses. Finally, we evaluate SeedEdit 3.0 on our testing benchmarks, for real/synthetic image editing, where it achieves a best trade-off between multiple aspects, yielding a high usability rate of 56.1%, compared to SeedEdit 1.6 (38.4%), GPT4o (37.1%) and Gemini 2.0 (30.3%).
1 Introduction
SeedEdit 3.0 addresses the challenge of building robust instructive editing models from heterogeneous data by combining meta-information, diverse curation, and model improvements. It achieves a strong multi-metric trade-off against commercial and prior systems while improving speed.
- Growing T2I model scale makes data quantity and quality increasingly important for instructive editing of real images.
- Heterogeneous editing datasets vary in image quality, edit diversity, and noise, making unified data utilization crucial for robust general-purpose editing.
- Meta-information with labels or captions at multiple granularities helps the diffusion model distinguish datasets and balance their mixture.
- The curation pipeline combines multiple sources, including internal T2I and SeedVLM data, to support high-resolution editing while preserving facial identity and hair texture.
- SeedEdit 3.0 achieves the best trade-off across instruction following, content preservation, and image quality against SeedEdit 1.6, Gemini 2.0, and GPT-4o.
- 15 s vs. 50 s per query, with SeedEdit 3.0 significantly faster than GPT-4o while maintaining its multi-metric trade-off.
2 Related Work
Prior instructive editing research spans training-free diffusion control, retrained editing models, unified generation-editing architectures, and increasingly sophisticated dataset construction. The central data challenge is obtaining scalable, diverse, high-quality image-editing pairs with accurate instructions.
- Instruct Editing Methods: Training-free diffusion editing is fast and low-cost but suffers from inferior content preservation and low editing accuracy.
- Instruct Editing Methods: Training-based methods retrain diffusion models on synthesized editing datasets and later improve instruction-image interaction through new architectures.
- Instruct Editing Methods: Unified frameworks jointly model text and images within a single transformer to learn shared representations for generation and editing.
- Dataset Creation: A major dataset-creation challenge is scaling high-quality image-editing pairs with corresponding instructions across diverse editing types.
- Dataset Creation: Synthetic data can be diverse and scalable but remains biased toward underlying generative models, creating a performance gap between synthetic and real images.
3 Approach
SeedEdit 3.0 curates editing data from diverse synthetic and real-image sources, then uses multi-granularity meta-information to combine them while reducing task ambiguity and domain mismatch.
- 3.1 Data Sources: The curation pipeline combines synthesized data, editing-specialist workflows, traditional editing operators, and video-frame sources for real and synthetic editing coverage.These sources support high-resolution editing, specialist capabilities, realistic operations, and broader real-image generalization.
- 3.1 Data Sources: Importance sampling broadens synthesized-data coverage across important and long-tail editing classes and subjects.The strategy targets broader input and edit sample spaces.
- 3.1 Data Sources: Synthesized data can be biased toward the generated-image domain, creating a performance gap between real images and synthetic data.The curation strategy addresses this gap by organizing multiple datasets.
- 3.2 Data Merging: The pipeline uses task labels, re-captioned instructions, and pixel-level editing tags to combine heterogeneous editing data.The labels distinguish data sources, recaptioning describes image differences and similarities, and tags encode local editing and preservation attributes.
- 3.2 Data Merging: Recaptioning decomposes editing-description generation into identifying image differences and similarities, then producing captions or instructions from them.The paper reports improved accuracy and more detailed descriptions from this decomposition.
- 3.2 Data Merging: Forward and backward editing operations are used after recaptioning, filtering, and alignment to balance and cover the curated data.The approach leverages all datasets in both editing directions.
3.3 Models
The model combines a VLM for high-level semantics with a causal diffusion network for fine-grained image details, connected through task and tag conditioning.
- 3.3 Models: SeedEdit 3.0 uses a VLM for high-level semantic information and a causal diffusion network that captures fine-grained image details.A connector aligns editing intent, including task type and editing tags, with the diffusion model.
- 3.3.1 Models Architecture: Replacing Seedream 2.0 with Seedream 3.0 enables approximately 1024×1024 native generation without a refiner.The upgrade benefits input-detail preservation and bilingual text and character-level editing.
- 3.3.1 Models Architecture: Independent task and tag embeddings help the model distinguish different dataset properties compared with prompt-based injection.Classifier-free guidance can optionally be applied to further improve performance.
3.4 Model Training.
Training combines pretraining and fine-tuning with multi-resolution handling, reward-aware diffusion objectives, and joint T2I data to improve editing quality and generalization.
- 3.4 Model Training: Training uses pretraining on collected image pairs followed by fine-tuning to stabilize editing performance.Fine-tuning resamples high-quality, high-resolution data selected by filter models and human filters.
- 3.4 Model Training: NaViT batches varying resolutions, while resolution grouping and dynamic token lengths support progressive low-to-high-resolution training.The pipeline is adapted to the dataset’s broader range of aspect ratios and resolutions.
- 3.4 Model Training: Joint reward-model training supplements diffusion loss for high-value attributes such as face identity, detailed structures, and aesthetics.The rewards are applied conditionally according to the editing context.
- 3.4 Model Training: Reward application is instruction-dependent; facial-identity preservation is unnecessary when the instruction requests a face change.The reward models are evaluated when the output can be reliably estimated at the relevant timestep and context.
- 3.4 Model Training: Current VLM-based models perform worse than expert reward models because they are not good at detail partition.The authors suggest improved VLM image-detail understanding could eventually enable replacing or merging the reward models.
- 3.4 Model Training: Joint training with T2I data improves high-resolution editing and helps preserve the model’s original T2I ability for editing generalization.The motivation is that editing data quality is lower than that of the best T2I datasets.
3.5 Inference Efficiency
SeedEdit3.0 improves inference efficiency through tailored generative paths, CFG distillation, unified noise references, adaptive timestep sampling, and operator optimization. The framework supports few-step high-fidelity sampling while reducing inference and training costs.
- Inference framework: A tailored generative path assigns each sample its own trajectory instead of routing all examples through one fixed Gaussian-prior path.The design targets overlapping probability-space paths that introduce randomness, weaken fine-grained control, and destabilize generation.
- Inference framework: CFG distillation encodes guidance scale as a learnable embedding, enabling guided outputs in one forward pass and approximately two times faster inference.The method avoids the nearly doubled inference cost of separate conditional and unconditional evaluations at each timestep.
- Inference framework: A single predicted noise reference remains constant across timesteps, aligning denoising and reducing the total number of sampling steps without compromising fidelity.The paper states that theoretical analysis shows this design maximizes the joint likelihood of the forward and reverse processes.
- Training efficiency: Adaptive timestep sampling uses SSD and a lightweight module to focus updates on timesteps yielding the greatest loss reduction.This concentrates computation on informative intervals rather than sampling timesteps uniformly.
- Efficiency results: The method converges faster and reduces training cost, while few-step sampling matches or outperforms baselines using up to 75 function evaluations across key quality metrics.The reported metrics include aesthetic quality, text–image alignment, and structural accuracy.
- Implementation optimization: Operator optimization through kernel fusion, memory coalescing, and low-bit quantization improves operator performance and reduces memory usage.Certain operators more than double their performance relative to the original implementations.
4 Experiments
The experiments evaluate SeedEdit3.0 on real and generated images across diverse editing operations using automatic and human metrics. Results show strong preservation, alignment, and user satisfaction, with faster processing than GPT-4o.
- Evaluation setup: The test sets contain a few hundred real and generated images spanning stylization, addition, replacement, deletion, camera motion, object shifts, and scene changes.The varied operations are intended to reflect general user usage rather than a narrow set of cases.
- Evaluation setup: Evaluation combines CLIP image similarity, CLIP direction score, GPT scores, and human ratings of instruction response, content preservation, and image quality.Human evaluation uses a 0–5 scoring standard across three aspects.
- Quantitative comparisons: SeedEdit3.0 significantly improves over previous SeedEdit versions and outperforms Gemini and Step1x on both plotted metrics.The comparisons use GPT mean score against CLIP image similarity and face similarity, with SeedEdit points varied by image and text CFG.
- Human evaluation and qualitative comparisons: SeedEdit3.0 yields the highest user satisfaction rate and processes an image in 10–15s, compared with 50–60s for GPT-4o.Qualitative comparisons further emphasize face, foreground, detail preservation, and alignment.
- Qualitative comparisons: Qualitative comparisons use single CFG settings and further support the reported conclusions, while direct comparison with Step1X is difficult for real-image quality, editing-intention following, and understanding.The limitation applies specifically to direct comparison with the open-source Step1X model.
5 Conclusion
The report presents SeedEdit3.0 as an improved real-image editing system with stronger preservation, editing quality, prompt understanding, and dynamic-motion handling. Its data curation and joint learning pipeline support scalable editing data and image consistency for real-world applications.
- Conclusion: SeedEdit3.0 significantly improves previous versions in real-image performance, face and ID preservation, text editing quality, prompt understanding, and dynamic motion.The conclusion lists these improvements as key aspects of the system’s performance.
- Conclusion: An efficient data curation pipeline scales editing data, while joint learning with diffusion and reward losses enhances image consistency.The conclusion identifies image consistency as particularly important for real-world applications.
A Other Contributors
The report acknowledges team members who contributed to evaluation, model acceleration, and deployment engineering.
- Other Contributors: Contributors supported data evaluation, model acceleration engineering, and model deployment engineering.The acknowledgments name team members associated with each contribution area.
B Ethical Claims
The authors restrict the technique to common industry tasks for ethical purposes and caution against unwanted violent or sexual-content scenarios. They also recommend careful examination and synthetic labeling because the approach may inherit T2I biases and limitations.
- The technique is intended for common industry tasks and should not be applied to unwanted violent or sexual-content scenarios.
- Because the approach may inherit T2I-model biases and limitations, synthesized images or models should be carefully examined and presented as synthetic.