PaperlayerView exact revision on arXiv

Source-linked AI summary

MIRA: Multimodal Iterative Reasoning Agent for Image Editing

Ziyun Zeng, Hang Hua, Jiebo Luo

arXiv:2511.21087v2cs.CV

TL;DR

Complex image-editing instructions remain difficult for diffusion editors to follow consistently. MIRA uses an iterative loop to select atomic edits from visual feedback, and it improves open-source editors’ semantic consistency and perceptual quality, narrowing the gap with proprietary systems.

  • Problem

    Diffusion-based editors struggle with complex instructions involving composition, contextual cues, or referring expressions, limiting faithful instruction-guided editing.

    Source: §1 · p. 1

  • Method

    MIRA is a lightweight vision-language agent that predicts atomic edits iteratively from the image, instruction, and intermediate editing state, with SFT and GRPO training.

    Source: §1 · pp. 1–3

  • Results

    MIRA improves semantic consistency and perceptual quality; with Flux.1-Kontext, GPT-SC increases by 4.98%.

    Source: §5.2.1 · p. 7, §6 · p. 9

  • Takeaways & Limitations

    MIRA’s iterative multimodal reasoning narrows the performance gap between open-source editors and proprietary systems such as GPT-Image and Nano-Banana.

    Source: §1 · p. 2, §6 · p. 9

Abstract

from arXiv · show

Instruction-guided image editing offers an intuitive way for users to edit images with natural language. However, diffusion-based editing models often struggle to accurately interpret complex user instructions, especially those involving compositional relationships, contextual cues, or referring expressions, leading to edits that drift semantically or fail to reflect the intended changes. We tackle this problem by proposing MIRA (Multimodal Iterative Reasoning Agent), a lightweight, plug-and-play multimodal reasoning agent that performs editing through an iterative perception-reasoning-action loop, effectively simulating multi-turn human-model interaction processes. Instead of issuing a single prompt or static plan, MIRA predicts atomic edit instructions step by step, using visual feedback to make its decisions. Our 150K multimodal tool-use dataset, MIRA-Editing, combined with a two-stage SFT + GRPO training pipeline, enables MIRA to perform reasoning and editing over complex editing instructions. When paired with open-source image editing models such as Flux.1-Kontext, Step1X-Edit, and Qwen-Image-Edit, MIRA significantly improves both semantic consistency and perceptual quality, achieving performance comparable to or exceeding proprietary systems such as GPT-Image and Nano-Banana.

1. Introduction

Complex image-editing instructions expose limits in one-shot editors and static orchestration. MIRA instead selects atomic edits step by step, using visual feedback to improve editing outcomes.

1. Introduction

  • Diffusion editors degrade on compositional, multi-object, or context-dependent instructions, while proprietary systems also struggle with fine-grained consistency and controllability.The introduction argues for systems that can reason, adapt, and revise rather than execute a single prompt.Source: §1 · p. 1
  • Prompt-refinement approaches remain static, while agentic systems use decomposition and visual feedback but often depend on large, computationally intensive toolchains.MIRA is introduced as a lightweight, plug-and-play alternative to static orchestration.Source: §1 · p. 1
  • At each iteration, MIRA observes the original image, instruction, and intermediate state, then predicts an atomic edit or a stop decision.The editor executes the action, and the resulting image feeds back into the next reasoning step.Source: §1 · pp. 1–2
  • MIRA improves semantic consistency and perceptual quality, often enabling open-source editors to match or surpass proprietary systems.The paper attributes these gains to iterative reasoning and feedback-driven tool use.Source: §1 · p. 2
  • MIRA-EDITING provides 150,000 paired examples, and its two-stage SFT-plus-GRPO training uses a composite reward model for edit quality and instruction fidelity.The reward couples an image-editing backbone with an image-editing reward model.Source: §1 · pp. 2–3

2. Related Works

Prior work advances instruction-guided editing through direct text-to-edit models and multimodal reasoning. The literature also moves from static prompt refinement toward agentic editing frameworks.

2.1. Instruction-guided Image Editing

  • Instruction-guided editing progressed from manually prompted diffusion methods to data-driven text-to-edit systems and models using language or vision-language reasoning for contextual control.Open-source editors advanced controllability, while the passage describes proprietary systems as state of the art in fidelity, responsiveness, and adaptability.Source: §2.1 · p. 3

2.2. MLLMs for Image Editing

  • MLLM editing shifted from one-shot generation and static prompt rewriting toward agents that decompose tasks and coordinate tools, though visual editing remains challenging.Static prompt optimization does not check whether the edited result matches the instruction; the cited agentic systems introduce sequential planning.Source: §2.2 · p. 3

3. Data Curation

MIRA-EDITING addresses the need for multi-step reasoning supervision with 150K paired samples. Its preprocessing creates and ranks varied instruction-edit trajectories, then structures them as stepwise supervision for iterative editing.

3. Data Curation

  • MIRA-EDITING contains 150K paired samples designed to provide multi-step reasoning, complex-instruction alignment, and high-fidelity editing trajectories.The paper reports additional quantitative and qualitative dataset analyses in supplementary material.Source: §3 · p. 3

3.1. Data Preprocessing Pipeline

  • Preprocessing merges atomic edits into complex instructions, creates in-order and permuted variants, rewrites instructions, and generates candidate edited trajectories.Candidates are ranked for semantic consistency, with the top-ranked result retained to form each sample alongside the image and instruction variants.Source: §3.1 · pp. 3–4

3.2. Training Data Formulation.

  • The agent’s editing workflow repeatedly analyzes the current visual state and instruction, issues an atomic edit through an external editor, and feeds the updated image back into the next step.The loop continues until the full instruction is satisfied.Source: Figure 2 · p. 4
  • Each trajectory is divided into Start, Continue, and Stop samples, supervising the first action, feedback-based refinement, and recognition of task completion.The three sample types support progressive, feedback-driven reasoning, as illustrated in Figure 3.Source: §3.2 · p. 4, Figure 3 · p. 4

4. Methodology

MIRA edits through a closed loop, using visual feedback to choose one atomic instruction at a time and decide when to stop. Its training moves from supervised imitation to step-wise reward-driven optimization.

4.1. The MIRA Framework

  • MIRA issues one atomic editing instruction per step, adapts from visual feedback, and uses a termination controller to decide whether to continue or stop.If it stops, the current image is returned; otherwise, the updated image, original image, and instruction are fed back to the policy.Source: §4.1 · pp. 4–5

4.2. Training Pipeline

  • Stage 1 fine-tunes Qwen2.5-VL-7B-Instruct on input-image, previously edited-image, and complex-instruction samples to initialize the policy.This stage uses imitation learning to align visual transformations with textual intent.Source: Figure 4 · p. 5, §4.2 · p. 5

4.1. The MIRA Framework

  • Unlike one-shot sequence planning, MIRA selects feedback-driven, goal-conditioned atomic operations and delegates them to interchangeable editing backends.The framework pairs vision-language-model reasoning with modular open-source diffusion editors.Source: §4.1 · p. 5

4.2. Training Pipeline

  • Step-wise GRPO scores editor-executed atomic edits for semantic consistency and perceptual quality, then optimizes the policy across step-level transitions.The image editor and reward model are used only for forward evaluation, not backpropagation.Source: §4.2 · p. 6

5. Experiment

The experiments compare image-editing systems on a 500-sample benchmark of complex instructions. They assess semantic consistency and perceptual quality across baseline, MIRA-enhanced, and VLM-enhanced systems.

5.1. Experiment Settings

  • The 500 test samples combine the multi-turn MagicBrush subset with CompBench examples and use rich, multisentence editing instructions.Evaluation images are standardized to 1024×1024 resolution, and instruction length is limited to 77 words.Source: §5.1 · p. 6
  • The comparison includes open-source and proprietary editors, MIRA-enhanced variants, and other VLM-enhanced systems using plug-and-play editing backbones.The MIRA variants pair with Step1X-Edit, Flux.1-Kontext-Dev, or Qwen-Image-Edit; Table 2 compares VLM-enhanced systems using Flux.1-Kontext-Dev and Qwen-Image-Edit.Source: §5.1 · p. 6, Table 2 · p. 7
  • Semantic consistency is measured with GPT-SC, Gemini-SC, Qwen3VL-SC, and EditScore-SC, while perceptual quality uses ARNIQA, TOPIQ, EditScore-PQ, and EditScore-OA.The semantic- and perceptual-quality dimensions are evaluated using official pretrained checkpoints and default repository settings.Source: §5.1 · pp. 6–7

5.2. Evaluation of MIRA Performance

MIRA improves semantic consistency and perceptual quality across open-source editors, and also generalizes to single-turn instruction refinement beyond its training domain.

5.2.1. Quantitative Results

  • Flux.1-Kontext + MIRA gains 4.98% on GPT-SC, 6.04% on Gemini-SC, 13.41% on Qwen3VL-SC, and 4.12% on EditScore-SC.Step1X-Edit and Qwen-Image-Edit also gain 6.40% and 9.41% on GPT-SC, respectively.Source: §5.2.1 · p. 7
  • MIRA 7B yields 13%–68% relative gains on GPT-SC and 15%–70% on Gemini-SC over open-source VLMs, with comparable or higher overall performance than GPT-5.Across both editing models, it reaches the best balance of semantic consistency and perceptual quality.Source: §5.2.1 · p. 7

5.2.2. Single-Turn Instruction Refinement

  • Outside its training domain, MIRA also clarifies ambiguous instructions into concise, executable edits for single-turn refinement.The paper reports consistent gains across diverse editing categories in supplementary analysis.Source: §5.2.2 · p. 7

5.3. Ablation Study

Reward-driven GRPO post-training improves MIRA across nearly all reported metrics and open-source editors, while increasing the maximum reasoning-step budget yields only small changes.

5.3.1. Effectiveness of RL

  • Step1X-Edit + MIRA-GRPO improves 9.83% on GPT-SC, 6.29% on EditScore-PQ, and 4.26% on EditScore-OA versus its SFT counterpart.The table compares SFT-only MIRA with MIRA further trained using GRPO; gains extend across backbones and both semantic and perceptual measures.Source: Table 3 · p. 8, §5.3.1 · p. 8
  • Figure 5 sequences five atomic edits—from floor and cabinet changes through stove and refrigerator recoloring—before MIRA issues a stop decision.The SFT policy imitates curated instruction–image pairs, while GRPO refines it through composite rewards from an editing model and an editing reward model.Source: §5.3.1 · p. 7, Figure 5 · p. 8

5.3.2. Reasoning Steps and Excution Quality

  • Increasing the maximum inference-step budget from 3 to 7 produces only minor metric variations, with modest gains most evident in semantic consistency.The reported changes indicate that reasoning depth is not the primary factor governing final output quality.Source: §5.3.2 · p. 8

5.4. Latency and Computational Cost Analysis

MIRA’s iterative reasoning adds latency, but its measured end-to-end time remains below GPT-Image’s and its open-source setup avoids per-edit proprietary pricing.

5.4. Latency and Computational Cost Analysis

  • MIRA’s iterative framework adds latency and computation while improving editing quality.The paper describes the framework as practical given its open-source availability and gains in semantic consistency and visual fidelity.Source: §5.4 · p. 8
  • 48.005 seconds is MIRA 7B + Flux.1-Kontext’s average end-to-end latency, versus 71.7 seconds for GPT-Image and 12.3 seconds for Nano-Banana.Measurements used one NVIDIA H100 GPU and 500 benchmark samples; MIRA averaged 4.111 iterations, with 0.746 seconds per reasoning step and 14.445 seconds per editor inference.Source: §5.4 · p. 8

5.5. Inherent Robustness and Error Mitigation

MIRA uses closed-loop visual feedback to recheck intermediate edits against the original image and instruction, then corrects deviations rather than assuming each edit succeeded.

5.5. Inherent Robustness and Error Mitigation

  • At each step, MIRA reevaluates the latest edited image against the original image and instruction, allowing it to mitigate or correct earlier execution errors.Unlike a static open-loop plan, its decisions depend on the current editing state.Source: §5.5 · pp. 8–9
  • In the example, cabinet recoloring mistakenly turns the refrigerator brown; restoring it then turns the stove white, prompting another targeted correction.MIRA issues separate atomic instructions to return the refrigerator to white and the stove to black.Source: §5.5 · p. 9
  • MIRA diagnoses editing discrepancies and dynamically adjusts its trajectory, mitigating error propagation and preserving alignment with user intent.After confirming the final output satisfies the instruction, it emits a stop signal.Source: §5.5 · p. 9

5.6. Reliability of the Termination Mechanism

MIRA’s average reasoning-step count rises with a larger budget from 3 to 5 steps, then remains nearly unchanged from 5 to 7, indicating goal-driven termination.

5.6. Reliability of the Termination Mechanism

  • From a 5-step to a 7-step maximum, average actual reasoning steps change only from 4.096 to 4.208, showing no meaningful scaling with the larger budget.MIRA averages about 4 steps on this benchmark.Source: §5.6 · p. 9
  • With maximum budgets increasing from 3 to 5, average actual steps rise from 2.976 to 4.111, consistent with instructions often requiring at least three editing actions.The passage attributes the rise to multiple subtasks in many benchmark instructions.Source: §5.6 · p. 9
  • MIRA’s termination is goal-driven rather than budget-driven: it does not use the full available budget, which the passage links to mitigating over-editing and improving reliability.The reported step count converges near four on this benchmark.Source: §5.6 · p. 9

6. Conclusion

MIRA reframes image editing as iterative perception, reasoning, and action, supported by the MIRA-EDITING dataset and GRPO-based optimization. It consistently improves semantic consistency and perceptual quality, narrowing the gap with proprietary systems.

6. Conclusion

  • MIRA reframes image editing as an iterative perception–reasoning–action process that generates atomic edit instructions and adapts through stepwise feedback.The paper presents this as a lightweight multimodal reasoning agent.Source: §6 · p. 9
  • MIRA-EDITING is a large-scale dataset for training automatic tool-use models, and MIRA’s training uses GRPO-based optimization with a customized reward function.Source: §6 · p. 9
  • MIRA consistently improves semantic consistency and perceptual quality, narrowing the gap with proprietary systems.The paper identifies iterative multimodal reasoning as an effective, scalable paradigm for controllable, high-quality image editing.Source: §6 · p. 9

Related papers

Loading 2511.21087v2…