Source-linked AI summary
Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generation
Junyan Ye, Dongzhi Jiang, Zihao Wang, Leqi Zhu, Zhenghao Hu, Zilong Huang, Jun He, Zhiyuan Yan, Jinghua Yu, Hongsheng Li, Conghui He, Weijia Li
TL;DR
Open-source unified image-generation models still lag GPT-4o, while real-world datasets underrepresent rare scenarios and provide noisy text-image supervision. The paper introduces Echo-4o-Image, fine-tunes Bagel into Echo-4o, and proposes GenEval++ and Imagine-Bench; Echo-4o performs strongly and the dataset transfers across models.
Problem
Open-source unified multimodal models still show a significant quality gap from GPT-4o, especially in instruction alignment.
Method
The paper builds Echo-4o-Image from GPT-4o synthetic images covering rare scenarios and controllable supervision, then fine-tunes Bagel and introduces two evaluation benchmarks.
Results
Echo-4o achieves notable gains across mainstream, complex-instruction, imaginative, and multi-reference generation evaluations, while Echo-4o-Image consistently improves other foundation models.
Takeaways & Limitations
Echo-4o-Image addresses limitations of real-world datasets and transfers effectively across different foundation models.
Takeaways & Limitations
The paper identifies image editing as a future dataset-extension area and broader model fine-tuning as future validation.
Abstract
from arXiv · showhide
Recently, GPT-4o has garnered significant attention for its strong performance in image generation, yet open-source models still lag behind. Several studies have explored distilling image data from GPT-4o to enhance open-source models, achieving notable progress. However, a key question remains: given that real-world image datasets already constitute a natural source of high-quality data, why should we use GPT-4o-generated synthetic data? In this work, we identify two key advantages of synthetic images. First, they can complement rare scenarios in real-world datasets, such as surreal fantasy or multi-reference image generation, which frequently occur in user queries. Second, they provide clean and controllable supervision. Real-world data often contains complex background noise and inherent misalignment between text descriptions and image content, whereas synthetic images offer pure backgrounds and long-tailed supervision signals, facilitating more accurate text-to-image alignment. Building on these insights, we introduce Echo-4o-Image, a 180K-scale synthetic dataset generated by GPT-4o, harnessing the power of synthetic image data to address blind spots in real-world coverage. Using this dataset, we fine-tune the unified multimodal generation baseline Bagel to obtain Echo-4o. In addition, we propose two new evaluation benchmarks for a more accurate and challenging assessment of image generation capabilities: GenEval++, which increases instruction complexity to mitigate score saturation, and Imagine-Bench, which focuses on evaluating both the understanding and generation of imaginative content. Echo-4o demonstrates strong performance across standard benchmarks. Moreover, applying Echo-4o-Image to other foundation models (e.g., OmniGen2, BLIP3-o) yields consistent performance gains across multiple metrics, highlighting the datasets strong transferability.
1 Introduction
The paper argues that GPT-4o-generated synthetic images address coverage and supervision gaps in real-world datasets. It introduces Echo-4o-Image, Echo-4o, and two benchmarks designed to improve image-generation training and evaluation.
- Motivation: Synthetic images complement rare surreal-fantasy and multi-reference scenarios underrepresented in real-world datasets.They also provide clean backgrounds and controllable long-tail attribute combinations for instruction-following supervision.
- Dataset: Echo-4o-Image is a 180K-scale GPT-4o-generated dataset covering surreal scenes, multi-reference generation, and complex instruction-following tasks.The dataset contains 38K surreal-fantasy, 73K multi-reference, and 68K complex instruction-following samples.
- Benchmarks: GenEval++ increases instruction complexity, while Imagine-Bench evaluates imaginative generation through fantasy fulfillment, identity preservation, and aesthetic quality.These benchmarks target score saturation and limited evaluation of imaginative content.
- Model and results: Fine-tuning Bagel on Echo-4o-Image yields Echo-4o, which achieves state-of-the-art performance across multiple benchmarks.The dataset also consistently improves other backbones, including OmniGen2 and BLIP3o, demonstrating transferability.
2 Echo-4o-Imgae
Echo-4o-Image is constructed as a large synthetic dataset spanning surreal fantasy, multi-reference generation, and complex instruction following. Its pipeline uses GPT-4o to create diverse inputs, images, and aligned training descriptions.
- Dataset overview: Echo-4o-Image contains approximately 179,000 samples across three task types: 38K surreal fantasy, 73K multi-reference, and 68K complex instruction execution.The surreal-fantasy and multi-reference subsets target rare or underrepresented data.
- Surreal fantasy: The fantasy subset transforms common object concepts through attribute shifts, hybridization, spatiotemporal anomalies, and multi-object interactions.GPT-4o constructs imaginative instructions and generates visually coherent samples.
- Multi-reference generation: The multi-reference subset combines two to four images from diverse categories into outputs guided by GPT-4o-generated interaction instructions.Instructions explicitly reference image indices to reduce ambiguity and improve alignment.
- Instruction following: Complex instruction-following data systematically combines color, position, count, and size attributes in multi-object prompts.Synthetic images provide cleaner backgrounds and controllable long-tail compositions than typical real-world data.
- Alignment refinement: When generated images and prompts misalign, the pipeline rewrites the text to match the image instead of discarding the sample.For example, a description specifying four watches is corrected when the image contains three.
3 Echo-4o
The paper fine-tunes the unified multimodal model Bagel on Echo-4o-Image to produce Echo-4o. The training uses text-to-image and multi-reference data to improve both capabilities.
- Training: Echo-4o training uses all text-to-image and multi-reference data, with flow-matching loss computed exclusively on the output image.All model components except the VAE are trained for 24,000 steps at a learning rate of 2e-5.
4 GenEval++ & Imagine-Bench
The paper introduces GenEval++ and Imagine-Bench to evaluate instruction fidelity and imaginative image generation more accurately and challengingly. The benchmarks use GPT-4.1-based scoring and richer creative or compositional criteria.
- Motivation: Existing instruction-following benchmarks suffer from detector errors, limited semantic diversity, and score saturation around 0.8–0.9.These issues reduce the accuracy and discriminative power of automatic evaluation.
- GenEval++: GenEval++ uses GPT-4.1 to check Object, Counts, Color, Position, and Size criteria across 280 high-complexity instructions.A result is correct only when all checklist conditions are satisfied.
- Imagine-Bench: Imagine-Bench evaluates surreal and imaginative generation through fantasy fulfillment, identity preservation, and aesthetic quality.Its creative instructions require models to alter conventional attributes while preserving invariant identity features.
- Scoring: Imagine-Bench computes its final score as 0.8 × min(Fantasy Fulfillment, Identity Preservation) + 0.2 × Aesthetic Quality.GPT-4.1 scores each dimension from 0 to 10 and provides reasoning.
5 Experiments
Echo-4o performs strongly across instruction-following, imaginative, and multi-reference generation tasks. Echo-4o-Image also transfers consistently to other unified multimodal models, while outperforming another GPT-4o-derived dataset on instruction following.
- Instruction-Following Image Generation: Echo-4o achieves 0.89 on GenEval and 86.07 on DPG-Bench, outperforming prior unified and diffusion-based models.
- Instruction-Following Image Generation: 8.5% improvement on GenEval over Bagel demonstrates stronger instruction-following performance after training on Echo-4o-Image.The reported gain is attributed to pure synthetic data and its clean-background supervision.
- Surreal and Imaginative Image Generation: Echo-4o attains the best open-source performance on Imagine-Bench, benefiting from fantasy-oriented data beyond real-world image distributions.Qualitative results show closer adherence to fantasy instructions while preserving object characteristics.
- Multi-Reference Image Generation: Echo-4o achieves the best open-source results under both MULTIPLE and SCENE settings on OmniContext, surpassing OmniGen2 and Bagel.It also outperforms OmniGen2 in instruction adherence and reference-image fidelity with two or three references.
- Transferability and Dataset Comparison: Echo-4o-Image consistently improves BLIP-3-o, Bagel, and OmniGen2 across GenEval, GenEval++, DPG-Bench, and OminiContext.The gains span instruction understanding, fantasy synthesis, and multi-reference generation.
- Transferability and Dataset Comparison: Bagel reaches 0.895 on GenEval with Echo-4o-Image versus 0.838 with ShareGPT-4o-Image and 0.820 before fine-tuning.A similar advantage is reported on GenEval++, while ShareGPT-4o-Image still improves aesthetic alignment.
6 Conclusion
The paper presents Echo-4o-Image, a GPT-4o-generated synthetic dataset, and Echo-4o, a model developed by fine-tuning on it. It also introduces GenEval++ and Imagine-Bench and reports transfer across foundation models, while identifying future expansion to image editing and broader model coverage.
- Echo-4o-Image is a large-scale GPT-4o-generated synthetic dataset used to enhance unified multimodal generation models through Echo-4o.
- The paper introduces GenEval++ and Imagine-Bench for more comprehensive and challenging evaluation of image generation.
- Future work will extend the dataset to image editing and fine-tune a broader range of models, including FLUX.
7 Related Works
Related work spans diffusion-based image generation, synthetic-data transfer from stronger models, and increasingly instruction-focused image-generation evaluation. Existing benchmarks still have accuracy limitations, while systematic evaluation of imagination remains underexplored.
- Diffusion models, including Stable Diffusion and DALL·E, have demonstrated strong text-to-image generation capabilities.
- Synthetic data generated by stronger models has been widely used to improve weaker models across language and multimodal understanding tasks.
- Image-generation evaluation has expanded from visual-quality metrics such as FID, IS, and KID toward instruction-following assessment.
- Existing benchmarks can suffer from detector and CLIP-model accuracy limitations, while systematic evaluation of imagination and creative generation remains underexplored.