Source-linked AI summary
Step1X-Edit: A Practical Framework for General Image Editing
Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, Guopeng Li, Yuang Peng, Quan Sun, Jingwei Wu, Yan Cai, Zheng Ge, Ranchen Ming, Lei Xia, Xianfang Zeng, Yibo Zhu, Binxing Jiao, Xiangyu Zhang, Gang Yu, Daxin Jiang
TL;DR
Image editing still has a performance and reproducibility gap between proprietary multimodal systems and open-source methods. Step1X-Edit addresses this with a high-quality data pipeline, an MLLM-conditioned diffusion architecture, and the real-world GEdit-Bench; it outperforms open-source baselines by a substantial margin and approaches leading proprietary models.
Problem
Open-source image editing systems lag closed-source multimodal models in generalization, edit accuracy, and generated-image quality, while existing datasets are limited in quality or diversity.
Method
Step1X-Edit combines an MLLM that processes reference images and editing instructions with a connector and DiT-style diffusion decoder, trained on a large high-quality editing dataset.
Results
Step1X-Edit outperforms existing open-source editing algorithms by a substantial margin and approaches leading proprietary models on GEdit-Bench.
Takeaways & Limitations
The publicly released model and data-generation pipeline provide resources intended to reduce the open-source/closed-source performance gap and support further image-editing research.
Abstract
from arXiv · showhide
In recent years, image editing models have witnessed remarkable and rapid development. The recent unveiling of cutting-edge multimodal models such as GPT-4o and Gemini2 Flash has introduced highly promising image editing capabilities. These models demonstrate an impressive aptitude for fulfilling a vast majority of user-driven editing requirements, marking a significant advancement in the field of image manipulation. However, there is still a large gap between the open-source algorithm with these closed-source models. Thus, in this paper, we aim to release a state-of-the-art image editing model, called Step1X-Edit, which can provide comparable performance against the closed-source models like GPT-4o and Gemini2 Flash. More specifically, we adopt the Multimodal LLM to process the reference image and the user's editing instruction. A latent embedding has been extracted and integrated with a diffusion image decoder to obtain the target image. To train the model, we build a data generation pipeline to produce a high-quality dataset. For evaluation, we develop the GEdit-Bench, a novel benchmark rooted in real-world user instructions. Experimental results on GEdit-Bench demonstrate that Step1X-Edit outperforms existing open-source baselines by a substantial margin and approaches the performance of leading proprietary models, thereby making significant contributions to the field of image editing.
1 Introduction
Step1X-Edit targets the gap between proprietary and open-source image editing by combining multimodal instruction understanding with diffusion decoding and user-grounded evaluation. It is trained on broad, high-quality editing data and reportedly approaches leading proprietary performance.
- Instruction-based editing requires nuanced semantic understanding, precise region localization, and image-fidelity preservation, which text-encoder diffusion designs often struggle to achieve for subtle or compositional edits.
- Closed-source multimodal systems perform strongly across diverse editing scenarios, while open-source methods remain behind in generalization, edit accuracy, and generated-image quality.
- The training data pipeline organizes common instructions into 11 categories and generates over 1 million high-quality image-instruction pairs spanning manipulation, attributes, layout, and stylization.
- Step1X-Edit combines an MLLM with a DiT-style diffusion architecture to process reference images and editing prompts into a target-image latent condition.
- Step1X-Edit outperforms existing open-source baselines by a large margin and approaches leading proprietary models such as GPT-4o.
- GEdit-Bench is curated from real-world user editing needs to provide authentic and comprehensive evaluation across diverse editing scenarios.
2 Related Work
Related work spans autoregressive and diffusion-based control, modular instruction-guided editing, and increasingly unified multimodal architectures. Persistent limitations in general-purpose editability and architectural integration motivate Step1X-Edit.
- Autoregressive models incorporate spatial and pixel-level guidance, including edges, segmentation masks, and depth maps, to enable localized and structured control.
- Diffusion models became dominant for high-fidelity synthesis through latent-space scaling and DiT architectures that improve generalization, image quality, and knowledge capacity.
- Early instruction-based editors commonly use modular pipelines in which MLLMs generate prompts, spatial cues, or synthetic instruction-image pairs for diffusion editing.
- Recent methods improve instruction fidelity through multimodal interaction, task-aware routing, fine-grained editing, and unified architectures covering diverse visual tasks.
- Joint vision-language training in systems such as Gemini and GPT-4o reflects a shift toward tightly integrated, instruction-driven editing frameworks.
- Existing approaches often remain task-specific, lack incremental editing and fine-grained correspondence, and couple instruction understanding with generation only shallowly.
3 Step1X-Edit
Step1X-Edit is built from a large, filtered, multilingual editing dataset and an MLLM–connector–DiT framework. Its pipeline covers 11 editing categories and combines semantic conditioning with detailed visual features.
- 3.1 Data Pipeline: Current editing datasets are constrained by scale or quality, motivating a large-scale, high-quality dataset tailored to image editing.
- 3.1 Data Pipeline: Web-crawled examples are categorized into 11 distinct editing categories designed to cover most practical image-editing requirements.
- 3.1 Data Pipeline: The data pipeline generates over 20 million instruction-image triplets and retains more than 1 million high-quality triplets after MLLM and human filtering.
- 3.1 Data Pipeline: The pipeline uses task-specific procedures for subject, background, color, material, text, motion, portrait, style, and tone edits, combining automated models with human validation where described.
- 3.1.2 Caption Strategy: Annotations use recursive refinement, style-aligned examples, and bilingual Chinese-English labeling to improve consistency, format control, accessibility, and multilingual evaluation.
- 3.2 Our Method: The model passes the reference image and instruction through an MLLM, restructures embeddings with a connector, and supplies multimodal features and global guidance to a downstream DiT.
- 3.2 Our Method: Token concatenation balances responsiveness to editing instructions with preservation of fine-grained image details during multimodal conditioning.
4 Benchmark and Evaluation
GEdit-Bench evaluates image editing systems on de-identified, real-world English and Chinese instructions using automated quality metrics and user preference studies. Step1X-Edit outperforms open-source counterparts across evaluation axes and achieves comparable subjective quality to state-of-the-art systems.
- GEdit-Bench: GEdit-Bench contains 606 testing examples collected from real-world user editing instances and organized into 11 instruction categories.The benchmark evaluates open-source systems alongside GPT-4o and Gemini2 Flash.
- GEdit-Bench: A de-identification protocol uses visually and semantically similar public image alternatives while preserving the original editing intent.When alternatives are unavailable, editing instructions are modified to maintain fidelity to the original intent.
- Quantitative Evaluation: VIEScore measures Semantic Consistency, Perceptual Quality, and Overall Score on a 0-to-10 scale, with GPT-4o and Qwen2.5-VL-72B used for automatic evaluation.English and Chinese instructions are paired for each benchmark image.
- Quantitative Evaluation: Step1X-Edit outperforms open-source models across 11 evaluation axes and is comparable to Gemini2 Flash and Doubao, including on Chinese instructions.It also surpasses Gemini2 Flash and GPT-4o on some style-change and color-alteration axes, without requiring editing masks.
- User Study: A blinded study with 55 participants ranks outputs from four methods using five quality levels mapped to numerical scores.Overall user preference is computed by averaging scores across editing tasks.
- User Study: Step1X-Edit achieves comparable subjective quality to other state-of-the-art approaches, while Gemini2 Flash receives especially high preference for identity preservation.The comparison is reported in Table 4 and Figure 10.
5 Conclusion
The paper presents Step1X-Edit as a publicly released general image editing algorithm trained with a large-scale, high-quality data-generation pipeline. Its MLLM-diffusion integration substantially outperforms existing open-source editing algorithms on GEdit-Bench.
- Conclusion: Step1X-Edit is a general image editing algorithm that will be publicly released to foster further research.The paper frames the release as a contribution to the image editing community.
- Conclusion: The proposed data-generation pipeline produces large-scale, high-quality editing triples containing a reference image, editing instruction, and target image.These triples are used to train the model.
- Conclusion: Step1X-Edit integrates a Multimedia Large Language Model with a diffusion-based image decoder and substantially outperforms existing open-source editing algorithms on GEdit-Bench.The conclusion attributes the evaluation to the collected GEdit-Bench.
Contributors and Acknowledgments
The paper distinguishes core contributors from contributors based on their duration and extent of involvement in developing Step1X-Edit.
- Contributor Categories: Core contributors participated in Step1X-Edit development throughout the entire process.The paper lists Shiyu Liu, Yucheng Han, Peng Xing, and other core contributors.
- Contributor Categories: Contributors worked on early versions or contributed part-time.The paper provides a separate contributor list for this category.