PaperlayerView exact revision on arXiv

Source-linked AI summary

FireRed-Image-Edit-1.0 Technical Report

Super Intelligence Team, Changhao Qiao, Chao Hui, Chen Li, Cunzheng Wang, Dejia Song, Jiale Zhang, Jing Li, Qiang Xiang, Runqi Wang, Shuang Sun, Wei Zhu, Xu Tang, Yao Hu, Yibo Chen, Yuhao Huang, Yuxuan Duan, Zhiyi Chen, Ziyuan Guo

arXiv:2602.13344v2cs.CVeess.IV

TL;DR

FireRed-Image-Edit targets the reproducibility and practical-evaluation gaps in instruction-based image editing. It combines large-scale curated data, multimodal diffusion-transformer training, specialized alignment mechanisms, and a broad benchmark, achieving leading open-source performance and competitive results against proprietary systems.

  • Problem

    Proprietary systems offer limited interpretability and reproducibility, while conventional metrics do not fully capture practical image-editing quality.

  • Method

    The paper develops an end-to-end diffusion-transformer framework using curated data, efficient staged training, specialized rewards and consistency constraints, and practical benchmark design.

  • Results

    The model achieves state-of-the-art performance among open-source systems and competitive results against proprietary systems across REDEdit-Bench, ImgEdit, and GEdit.

  • Takeaways & Limitations

    The results support a carefully engineered combination of data curation, training methodology, and evaluation as an effective alternative to brute-force scaling.

Abstract

from arXiv · show

We present FireRed-Image-Edit, a diffusion transformer for instruction-based image editing that achieves state-of-the-art performance through systematic optimization of data curation, training methodology, and evaluation design. We construct a 1.6B-sample training corpus, comprising 900M text-to-image and 700M image editing pairs from diverse sources. After rigorous cleaning, stratification, auto-labeling, and two-stage filtering, we retain over 100M high-quality samples balanced between generation and editing, ensuring strong semantic coverage and instruction alignment. Our multi-stage training pipeline progressively builds editing capability via pre-training, supervised fine-tuning, and reinforcement learning. To improve data efficiency, we introduce a Multi-Condition Aware Bucket Sampler for variable-resolution batching and Stochastic Instruction Alignment with dynamic prompt re-indexing. To stabilize optimization and enhance controllability, we propose Asymmetric Gradient Optimization for DPO, DiffusionNFT with layout-aware OCR rewards for text editing, and a differentiable Consistency Loss for identity preservation. We further establish REDEdit-Bench, a comprehensive benchmark spanning 15 editing categories, including newly introduced beautification and low-level enhancement tasks. Extensive experiments on REDEdit-Bench and public benchmarks (ImgEdit and GEdit) demonstrate competitive or superior performance against both open-source and proprietary systems. To support future research, our code, models, and benchmark suite are publicly available at https://github.com/FireRedTeam/FireRed-Image-Edit/ .

1. Introduction

FireRed-Image-Edit addresses barriers in image generation and editing by optimizing data, architecture, training efficiency, and evaluation. Its framework scales to roughly 100 million diverse pairs and introduces a benchmark aimed at practical editing utility.

  • The current ecosystem combines opaque proprietary systems with open-source models increasingly scaled to tens of billions of parameters.
  • FireRed-Image-Edit is an end-to-end framework that optimizes data curation, architecture design, training efficiency, and inference optimization.
  • Approximately 100 million diverse image-text pairs support text-to-image generation, multi-image synthesis, and instruction-based editing.
  • Distributed timestep sampling, logit-normal loss weighting, and exponential moving average jointly refine convergence and robustness.
  • REDEdit-Bench evaluates editing across instruction alignment, background preservation, and photographic fidelity to better reflect practical deployment.

2. Data

The authors construct a large, balanced multimodal dataset and apply controlled synthesis plus hierarchical filtering to improve coverage, diversity, and training-data quality. The resulting pipeline combines real and produced editing pairs across broad generation and editing categories.

  • The corpus contains 1.6 billion samples: 900 million text-to-image pairs and 700 million image editing pairs.
  • After cleaning and selection, text-to-image and image-to-image data are approximately balanced at a 1:1 ratio.
  • Text-to-image data covers Nature, Design, and People, while image-to-image data covers Semantic, Stylistic, and Structural Editing.
  • Data Filtering: Hierarchical deduplication removes global, pair-level, and fine-grained near-duplicates using image and multimodal similarity measures.
  • Data Filtering: Photometric, statistical, artifact, perceptual-quality, and AIGC filters remove visual noise, invalid content, low-quality samples, and synthetic images.
  • The production engine creates paired editing data through Instructional Control, Structured Control, and Model-free Template-based Synthesis.

3. Model Training

The training system combines multimodal diffusion-transformer modeling with variable-resolution batching, stochastic instruction alignment, and staged optimization. These components target efficient distributed training, robust conditioning, and progressive editing capability.

  • The architecture uses MMDiT blocks with VAE-encoded visual inputs and Qwen VL processing reference images and textual instructions.
  • Training Efficiency Optimization: Collate Shuffle & Drop randomly permutes or drops reference images and dynamically updates prompts to improve robustness against input permutations.
  • Training Efficiency Optimization: A Multi-Condition Aware Bucket Sampler groups batches by aspect ratio and input-image count while constraining visual sequence length.
  • Training Efficiency Optimization: The sampler minimizes aggregate cropping across predefined resolution buckets to preserve spatial layouts and reduce padding-induced waste.
  • System-Level Efficiency and Stability: Offline VLM embeddings, FSDP, gradient checkpointing, BF16 training, and HSDP reduce memory and inter-node synchronization demands.
  • Progressive Training: Training progresses from pretraining and continued pretraining through supervised fine-tuning and DPO reinforcement learning to the final DiffusionNFT model.

3.3. Pre-training

Pre-training builds broad visual and semantic capability through large-scale, diverse data, adaptive resolution handling, and staged diffusion training strategies.

  • Pre-training prioritizes visual vocabulary and world knowledge through data scale and semantic diversity rather than aesthetic refinement.
  • Dynamic bucket sampling groups similar aspect ratios, reducing padding waste while preserving spatial layout integrity across arbitrary resolutions.
  • Inclusive web-data filtering retains effective samples, including rare but semantically valuable concepts, despite long-tail quality variation.
  • Progressive timestep sampling initially emphasizes high-noise steps to prioritize global semantic structures, layouts, and low-frequency image components.
  • Training uses unified sampling across text-to-image, single-image, and multi-image tasks, with broad aspect-ratio coverage to support diverse input modalities and canvas dimensions.
  • Dense captions and synthetic data improve coverage of underrepresented domains, long-tail vocabulary, rare objects, nuanced textures, and intricate scenarios.
  • Cluster-based balancing samples semantic groups uniformly so common objects and niche artistic styles receive sufficient training exposure.

3.5. Supervised Fine-Tuning (SFT)

Supervised fine-tuning narrows the broad pre-training distribution toward high-quality, instruction-aligned image editing while using conservative optimization to preserve generalization.

  • SFT uses high-resolution, human-filtered imagery with instruction-following captions and structured prompts to enforce precise text-image alignment.
  • A smaller learning rate refines high-frequency details and photorealistic attributes without disrupting previously learned capabilities.
  • Model weight averaging combines capabilities across optimization steps, mitigating checkpoint-specific biases and improving balanced generalization.

3.6. Reinforcement Learning with Human Feedback (RLHF)

The reinforcement-learning stage addresses DPO instability and improves text-editing rewards by reinforcing high-quality samples and incorporating layout-aware, continuous feedback.

  • Standard DPO can exhibit double degradation, with Win Diff and Lose Diff increasing together as chosen-sample quality declines during negative-sample avoidance.
  • Asymmetric Gradient Optimization anchors alignment on Positive Sample Reinforcement and adds SFT regularization to prioritize high-quality distributions.
  • Setting 𝜔 > 1 amplifies Win Diff gradients so optimization emphasizes high-fidelity chosen samples over unconstrained rejection of negative samples.
  • Automated instruction evolution and Mix-Policy construction combine VLM-generated requests with positive samples from expert branches and filtered negative samples.
  • DiffusionNFT performs online reinforcement learning on the forward process using optimality probabilities and weighted flow-matching errors instead of paired preference data.
  • Continuous soft scores from CoT-based VLM rewards and ensemble averaging smooth the reinforcement signal for more stable optimization.
  • Layout-aware OCR reward evaluates character correctness together with position and scale, penalizing missing, misaligned, or oversized characters.
  • Semi-hard mining targets competent but high-variance edits because random sampling often allocates capacity to already-comfortable examples.

3.7. Consistency Loss

The consistency-loss design targets identity drift in human-centric editing by applying identity constraints during structural formation and tapering them during fine-detail refinement.

  • Mean squared reconstruction losses can preserve low-level pixels yet miss semantic identity, producing identity drift in human-centric editing.
  • The denoising process establishes identity during the early high-noise stage, while later low-noise steps focus on fine-grained texture refinement.
  • Dynamic weighting makes the identity loss decay with noise level so identity anchoring tapers as the model transitions toward pixel-level refinement.
  • The identity loss extracts and normalizes face regions using landmark-derived transformations and a one-step denoised estimate.
  • For multiple subjects, the loss averages cosine distances across independently aligned faces and uses a pre-trained face-recognition backbone.

3.8. Training Strategy.

The training strategy improves diffusion optimization by coordinating timestep sampling, loss weighting, and parameter averaging to enhance coverage, convergence, and robustness.

  • Together, the strategy coordinates global timestep distribution and parameter convergence to support training stability and data efficiency.
  • Distributed Stratified Timestep Sampling partitions the diffusion horizon across GPU ranks to improve global uniformity in noise coverage.Synchronized rotation prevents rank-specific overfitting during distributed training.
  • Logit-Normal Loss Weighting concentrates gradient contributions on semantically critical intermediate timesteps while suppressing negligible extremes.
  • Exponential Moving Average averages parameters across converging iterations instead of relying on a single final checkpoint.This smooths transient optimization fluctuations and improves robustness against distribution shifts in real-world editing scenarios.

4. REDEdit-Bench

REDEdit-Bench is a bilingual, multi-category benchmark designed to evaluate instruction fidelity and preservation quality across practical image-editing tasks.

  • 1,673 bilingual Chinese–English edit pairs span 15 structured editing categories in REDEdit-Bench.The benchmark is reported to have a balanced distribution across task categories.
  • REDEdit-Bench is described as larger and broader in evaluation design than widely used open-source editing benchmarks.Its key attributes are compared with other open-source benchmarks in Table 2.
  • The benchmark is constructed from over 3,000 real-world images covering landscapes, architecture, objects, animals, and portraits.Professional staff write editing instructions, and multiple experts review the resulting pairs.
  • REDEdit-Bench emphasizes Prompt Compliance, Visual Naturalness, and Physical & Detail Coherence in evaluation.Gemini 3 Flash is used as the automated evaluator, and FireRed-Image-Edit achieves SOTA performance among open-source models.
  • OCR and VLM Judge jointly evaluate text editing through character and word accuracy, style preservation, visual consistency, and over-editing.

5. Performance and Evaluation

FireRed-Image-Edit is evaluated through human assessment, figures, and multiple benchmarks, showing strong instruction following, consistency preservation, and open-source competitiveness across editing tasks.

  • Blind human evaluation assesses editing outputs along Prompt Following and Consistency Preservation dimensions.Annotators view randomized outputs without model identities to reduce positional and brand bias.
  • FireRed-Image-Edit leads on Prompt Following and Consistency, significantly outperforming LongCat and Qwen-Image-Edit-2511 while remaining competitive with commercial systems.
  • The model obtains the highest Consistency Preservation score among compared methods, preserving non-edited content while modifying intended regions.
  • General Editing: FireRed-Image-Edit achieves state-of-the-art performance on ImgEdit, surpassing both open-source and closed-source models.
  • General Editing: On GEdit, FireRed-Image-Edit outperforms existing open-source models in instruction adherence and visual fidelity evaluation.
  • General Editing: On REDEdit-Bench, the model ranks highest among open-source systems for complex instruction-based editing with preservation constraints.
  • General Editing: Visualizations show accurate localization, physically plausible interactions, and stable visual quality for object, attribute, hybrid, and restoration edits.
  • Text-Centric Editing: Text-centric evaluation finds leading open-source performance across content accuracy, font-style preservation, visual consistency, and unnecessary modification control.The evaluation uses five dimensions and applies the OCR and VLM Judge pipeline.

6. Conclusion

FireRed-Image-Edit combines large-scale data engineering, efficiency-oriented training, and a broad benchmark to deliver strong image-editing performance across public evaluations.

  • FireRed-Image-Edit is a diffusion transformer whose contributions span data engineering, training methodology, and evaluation.
  • Over 100 million high-quality samples are distilled from approximately one billion multi-source image–text pairs and balanced between generation and editing.The curation pipeline uses cleaning, stratification, auto-labeling, and two-stage filtering.
  • The training framework combines bucket sampling, Stochastic Instruction Alignment, multi-stage training, DPO optimization, Diffusion NFT, Consistency Loss, and EMA.These components are presented as improving efficiency, robustness, controllability, and convergence stability.
  • REDEdit-Bench spans 15 structured categories and 1,673 bilingual samples, while experiments on REDEdit-Bench, ImgEdit, and GEdit show strong open-source and competitive proprietary-model performance.

7. Authors

The paper lists its core contributors and additional contributors alphabetically.

  • Core and additional contributors are listed alphabetically.

Annotation Prompt

The annotation prompt defines a two-step workflow for image captioning and analysis, followed by a structured JSON output.

  • Step 1: Image Caption: Step 1 requests a natural, descriptive image caption with object attributes, spatial relationships, environment details, and exact visible-text transcription.
  • Step 2: Image Analysis: Step 2 classifies image type and visual style while recording watermarks and distracting anomalies.
  • Output Format: The output format organizes caption, image properties, subjects, environment, and quality-related fields in JSON.
Loading 2602.13344v2…