ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generation

• GPT-4o-Image offers strong instruction alignment, semantic coherence, and photorealism across text-to-image and text-and-image-to-image generation, while remaining proprietary and opaque.Existing open-source models such as Janus-Pro still fall significantly behind.Source: §1 · p. 2
• ShareGPT-4o-Image contains 45K text-to-image and 46K text-and-image-to-image prompts with 91K GPT-4o-generated image outputs.The dataset was curated for diversity and quality and covers varied styles, grounded visual reasoning, instruction-following, and visual aesthetics.Source: §1 · p. 2
• Janus-4o is developed by fine-tuning Janus-Pro on ShareGPT-4o-Image and supports both text-to-image and text-and-image-to-image generation.This adds image-to-image capability to a predecessor described as text-to-image-only.Source: §1 · p. 3
• Janus-4o improves over Janus-Pro by 4 points on EvalGen and 1.6 points on DPG-Bench, while achieving strong open-source image-editing performance.Human evaluations also report stronger preference for Janus-4o outputs.Source: §1 · p. 3
• The release of ShareGPT-4o-Image and Janus-4o is intended to foster open research in instruction-aligned multimodal image generation.Source: §1 · p. 3

• ShareGPT-4o-Image comprises 45k text-to-image pairs and 46k instruction-guided image-editing triplets.The data-generation process uses Gemini-Pro-2.5 for text synthesis tasks.Source: §2 · p. 3
• The text-to-image collection combines prompt-first generation with image-first grounding to cover controlled attributes and realistic visual content.Source: §2.1 · p. 3

2.1 Text-to-Image Data

• The prompt-first pipeline samples attributes from a six-dimensional space and composes them into natural-language prompts before GPT-4o-Image generates paired images.The attribute vocabulary includes 1,000 ImageNet object categories.Source: §2.1 · pp. 3–4
• The image-first pipeline uses diverse real-world images from ALLaVA and generates detailed prompts describing their visual content.These pairs expose the dataset to language describing naturally occurring scenes.Source: §2.1 · p. 4

2.2 Instruction-Guided Image Editing Data

• The editing pipeline defines 14 tasks across five high-level categories, selects source images and tasks, and synthesizes editing instructions.Source: §2.2 · p. 4
• GPT-4o-Image executes each synthesized instruction on the source image, producing 46k triplets spanning common image-editing scenarios.Source: §2.2 · p. 4

• Janus-4o is built on Janus-Pro through fine-tuning on ShareGPT-4o-Image, with both generation tasks jointly trained.Source: Figure 4 · p. 4, §3 · p. 4, §3.3 · p. 5

3.1 Text-to-Image Fine-Tuning

• For text-to-image training, tokenized prompts and codebook-mapped target-image patches are embedded and passed to the LLM for autoregressive image-token prediction.Source: §3.1 · p. 4
• 10% of text-prompt tokens are randomly masked during text-to-image fine-tuning to encourage pixel-level dependency modeling.Source: §3.1 · p. 5

3.2 Text-and-Image-to-Image Fine-Tuning: Unlocking New Capabilities

• Text-and-image-to-image training combines an input image’s semantic embedding and tokenized representation with prompt tokens to generate target-image tokens.This design addresses Janus-Pro’s lack of native text-and-image-to-image support.Source: §3.2 · p. 5
• 50% of input-image tokens are randomly masked during editing fine-tuning to reduce overfitting to the input image.Source: §3.2 · p. 5

• Training jointly samples 45K text-to-image and 46K text-and-image-to-image examples for three epochs, completing in 6 hours on one 8×A800 GPU machine.Source: §3.3 · p. 5

Evaluation Setup

• The evaluation covers GenEval and DPG-Bench for text-to-image generation, ImgEdit-Bench for image-conditioned editing, and a human study of model preferences.The human study compares Janus-4o with Janus-Pro-7B and UltraEdit using real Twitter examples.Source: §4.1 · pp. 5–6

Text-to-Image Performance

• 4 points on GenEval and 1.6 points on DPG-Bench represent Janus-4o’s improvements over Janus-Pro in text-to-image generation.Janus-4o also achieves 80% overall accuracy on GenEval and a score of 85.71 on DPG-Bench.Source: §4.2 · p. 6

Text-and-Image-to-Image Performance

• 3.26 on ImgEdit-Bench surpasses Step1X-Edit and ImgEdit-E1, with particularly strong results in Motion Change and Style Transfer.Janus-4o reaches this performance using only 91K training samples, although Janus-Pro lacked text-and-image-to-image support.Source: §4.2 · p. 7

Human Evaluation

• Janus-4o’s human-evaluation outputs align more closely with preferences than previous models across text-to-image and text-and-image-to-image tasks.It shows higher visual quality than Janus-Pro for text-to-image and improved instruction-following for image modifications.Source: §4.2 · p. 7, Figure 5 · p. 7

5 conclusion

• ShareGPT-4o-Image is presented as a large-scale dataset covering GPT-4o capabilities in text-to-image and text-and-image-to-image generation.The dataset is described as the first large-scale dataset of this kind.Source: §5 · p. 7, §A.1 · p. 13
• Janus-4o generates high-quality images from text alone or from combined image and text inputs.The model supports both text-to-image and text-and-image-to-image generation.Source: §5 · p. 7
• 6 hours on an 8×A800 GPU machine yields notable performance gains with minimal compute.The conclusion attributes this efficiency to MLLM-based autoregressive image generation.Source: §5 · p. 7

A.2 Datasets for Image Generation

• MS-COCO and TextCaps provided early natural-language captions aligned with images for text-to-image generation.These datasets are described as early image-text pair resources.Source: §A.2 · p. 13
• Later datasets collected larger-scale image-text pairs from the internet to enhance generative-model performance and generalization.The passage frames this as a subsequent development after early image-text datasets.Source: §A.2 · p. 13
• Multimodal LLM research enabled image understanding by aligning visual features with text using paired image-text data and pretrained image encoders.Image generation is presented as a natural extension of this multimodal development.Source: §A.3 · p. 13

B.1.1 Text-to-Image Dimensions

• Table 4 defines the dimensions and sub-dimensions used to describe text-to-image data.The table is the stated source for the detailed text-to-image dimensions.Source: Table 4 · p. 14, §B.1.1 · p. 14
• Figure 6 presents the distribution of text-to-image examples across categories.The passage identifies Figure 6 as the visual summary of category distributions.Source: Figure 6 · p. 14, §B.1.1 · p. 14
• Object-dimension sub-dimensions are available through ImageNet rather than being detailed in Table 4.This is the stated exception to the table’s coverage.Source: §B.1.1 · p. 14

• Table 5 lists the categories and subcategories for text-and-image-to-image tasks.Source: §B.1.2 · p. 14, Table 5 · p. 15
• Figure 7 presents the distribution of text-and-image-to-image categories.Source: §B.1.2 · p. 14, Figure 7 · p. 15
• The sampling appendix describes selecting the number of object categories k from 1 to 100 with weighted random sampling.Source: §B.2 · p. 16

• An exponential-decay distribution assigns decreasing probability to larger category counts, favoring scenarios with fewer categories.Source: §B.2.1 · p. 16
• λ controls the decay rate, with larger λ producing faster decay.Source: §B.2.1 · p. 16
• The distribution is illustrated in a figure showing its exponential-decay characteristics.Source: §B.2.1 · p. 16, Figure 8 · p. 16

• ShareGPT-4o-Image contains synthesized outputs from proprietary GPT-4o-Image, whose internal workings remain opaque.Source: E Ethical Considerations and Societal… · p. 19
• GPT-4o-Image outputs may inherit or amplify biases related to gender, race, age, occupation, and other demographic attributes.Source: E Ethical Considerations and Societal… · p. 19
• The dataset includes no post-hoc filtering for perceived bias because subjective filtering could introduce new biases.Source: E Ethical Considerations and Societal… · p. 19
• LLM-generated document and descriptive text may be misconstrued or reflect unintentional training-data biases despite generic, fictional prompting.Source: E Ethical Considerations and Societal… · p. 19
• The release is intended for academic research, while harmful content, stereotypes, misinformation, and rights infringement are explicitly discouraged.Source: E Ethical Considerations and Societal… · p. 19
• The authors encourage systematic bias audits and shared responsibility for investigating and mitigating potential societal harms.Source: E Ethical Considerations and Societal… · p. 19