PaperlayerView exact revision on arXiv

Source-linked AI summary

Reasoning-Informed Visual Editing

Xue Yang, Peiyuan Zhang, Yilun Zhu, Qihao Yang, Mingxin Liu, Xiangyu Zhao, Ziqian Fan, Zhaokai Wang, Yan Li, Yifan Yang, Xu Yang, Xiaosong Jia, Yue Zhou, Zhihang Zhong, Junchi Yan

arXiv:2610.12343v1cs.CV

TL;DR

Visual-editing models still struggle with complex instructions, appearance consistency, and flexible inputs, and reasoning-informed editing lacks an established benchmark. The paper introduces RISEBench++ and an evaluation framework, alongside the training-free RISE-Agent. Across evaluated approaches, instruction reasoning remains a major challenge, particularly for logical and hybrid tasks.

  • Problem

    Visual-editing methods struggle with complex instructions, preserving the original image’s appearance, and flexible input formats, while reasoning-informed editing lacks an established benchmark.

    Source: §1 · p. 2

  • Method

    RISEBench++ provides a hierarchical, bilingual benchmark, paired with human and LMM-based evaluation, while RISE-Agent combines planning, tool use, and verifier-guided refinement.

    Source: §1 · p. 3

  • Results

    Across 58 evaluated approaches, proprietary models generally perform stronger overall, but instruction reasoning remains a major challenge, particularly for logical and hybrid tasks.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The findings point to visual perception, multimodal reasoning, and precise execution as areas for advancing reasoning-informed visual editing.

    Source: §5 · p. 22

    In sequential editing examples, Qwen-Image-Edit-2509 and HunyuanImage-3.0 show cumulative visual degradation, with later outputs losing structural integrity and visual fidelity.

    Source: §4.7 · p. 21

Abstract

from arXiv · show

Large Multi-modality Models (LMMs) have made significant progress in visual understanding and generation, but still face challenges in visual editing, particularly in following complex instructions, preserving appearance consistency, and supporting flexible input formats. To study this gap, we introduce RISEBench, the first benchmark for evaluating Reasoning-Informed viSual Editing (RISE), and extend it to RISEBench++, a more comprehensive and fine-grained benchmark for this emerging task. RISEBench++ extends the taxonomy into a hierarchical scheme spanning six reasoning dimensions: Temporal, Causal, Spatial, Logical, and Counterfactual Reasoning, together with Hybrid Reasoning integrating multiple reasoning types across multi-turn edits. These dimensions are further decomposed into 12 subcategories and 65 fine-grained task types. We expand input formats to include multi-image conditioning and scale the benchmark to 1000 human-annotated test cases, released in English and Chinese. We also improve our evaluation framework, assessing Instruction Reasoning, Appearance Consistency, and Visual Plausibility with human judges and an LMM-as-a-judge approach for more reliable and calibrated judgements. Beyond benchmarking, we introduce RISE-Agent, a training-free agentic framework integrating reasoning-driven planning, tool-augmented execution, and verifier-guided refinement, outperforming most strong existing approaches across diverse RISE tasks. We evaluate 58 visual editing approaches, including 34 open-source models, 19 closed-source models, and 5 agentic methods. The results reveal substantial challenges in reasoning-based visual editing, with even the strongest evaluated approach, GPT-Image-2.5 Sunburst, achieving only 56.6% accuracy. RISEBench++ highlights the limitations of contemporary editing models, provides insights, and indicates future directions for reasoning-aware visual editing.

1 Introduction

The introduction identifies a gap in visual editing: models struggle with complex instructions, appearance consistency, and flexible inputs, while no established benchmark systematically evaluates reasoning-informed editing. It presents RISEBench++ and RISE-Agent as resources for evaluating and addressing these challenges.

1 Introduction

  • Existing open-source methods struggle to follow complex instructions, preserve image appearance, and support flexible inputs, while the lack of an established benchmark limits systematic comparison.The cited flexible-input examples include editing with either one image or multiple images alongside natural-language instructions.Source: §1 · p. 2
  • RISEBench++ evaluates reasoning-informed editing across six dimensions, 12 subcategories, and 65 task types, using 1,000 bilingual cases across single-image, multi-image, and multi-turn settings.The benchmark includes cases in English and Chinese.Source: §1 · pp. 3–4
  • The evaluation scores instruction reasoning, appearance consistency, and visual plausibility with human judges and an LMM-as-a-judge framework validated against human experts.The LMM pipeline uses dimension-specific evidence and tailored scoring rubrics for more reliable, calibrated assessments.Source: §1 · pp. 3–4
  • RISE-Agent combines instruction planning, tool-augmented execution, and verifier-guided refinement to perform complex edits without extra training.Its verifier can accept results, refine them, or trigger replanning.Source: §1 · pp. 3–4
  • Proprietary models generally outperform open-source models, but instruction reasoning remains difficult, especially for logical and hybrid tasks.GPT-Image-2.5 Sunburst achieves the best results among evaluated models, while RISE-Agent performs competitively and surpasses most proprietary models and agentic methods.Source: §1 · p. 3
  • The paper evaluates 58 visual-editing approaches and reports a progression from limited reasoning in early open-source models to stronger recent open-source and proprietary systems.The extended work expands evaluation from 8 to 58 approaches and analyzes failure modes across reasoning dimensions.Source: §1 · pp. 3–4

2 RELATED WORK

Image-editing research spans diffusion, unified multimodal, and agentic approaches, while benchmarks increasingly cover reasoning but differ in task scope and evaluation settings. RISEBench++ is positioned among these efforts as a benchmark for complex, implicit editing requirements.

2.1 Image Editing Models

  • Diffusion editors emphasize controllable, appearance-level changes, unified multimodal models combine understanding and generation, and agentic editors decompose intentions into steps and iteratively refine results.The cited examples include MagicBrush and FLUX.1 Kontext for diffusion, OmniGen and Gemini for unified models, and IntentEdit, ImAgent, and Mind-Brush for agentic methods.Source: §2.1 · p. 4

2.2 Image Editing Benchmarks

  • Existing editing benchmarks range from foundational operations to multi-image and multi-turn settings, but often use simple, explicit instructions rather than complex implicit requirements.Examples include MagicBrush, AnyEdit, I2I-Bench, and I2EBench2.0.Source: §2.2 · pp. 4–5

2.3 Reasoning-Informed Benchmarks for Visual Generation

  • Reasoning-oriented benchmarks address knowledge-intensive image generation, reasoning-informed editing, and video tasks involving temporal evolution and cross-frame consistency.The cited editing work includes KRIS-Bench, WiseEdit, UniREditBench, and GRADE; video examples include Physics-IQ and RISE-Video.Source: §2.3 · pp. 5–6

3 METHODOLOGY

RISEBench++ organizes reasoning-informed editing across six dimensions, with distinct knowledge-, perception-, imagination-, and multi-turn reasoning demands. Its construction, evaluation pipeline, and RISE-Agent address benchmark coverage, assessment, and execution of these tasks.

3.1 Benchmark Overview

  • Temporal and causal tasks are knowledge-driven, spatial and logical tasks are perception-driven, counterfactual tasks are imagination-driven, and hybrid tasks combine reasoning modes across turns.Hybrid cases require intermediate edits to guide later transformations while maintaining scene consistency.Source: §3.1 · p. 6
  • RISEBench++ contains 1,000 cases across six dimensions, 12 subcategories, and 65 task types, with 200 cases each for temporal, causal, spatial, and logical reasoning and 100 each for counterfactual and hybrid reasoning.The benchmark jointly supports reasoning, multi-turn interaction, multi-image inputs, agent support, and English and Chinese instructions.Source: §3.1 · pp. 6–7

3.2 Benchmark Construction

  • Temporal and causal cases test knowledge-grounded state changes, while spatial and logical cases test visual relation or rule inference and translation into coherent edits.The benchmark includes 200 temporal cases spanning 19 task types and 200 causal cases spanning 14 task types.Source: §3.2 · pp. 7–8
  • Counterfactual cases test anomaly correction or instructed violations of real-world knowledge, while 100 hybrid cases combine reasoning dimensions across three to five linked instructions.Hybrid tasks demand long-horizon tracking and sustained consistency across successive edits.Source: §3.2 · p. 8

3.3 Evaluation Pipeline

  • Human and LMM evaluations score instruction reasoning, appearance consistency, and visual plausibility, using text or image references for instruction checks and input-output comparisons for consistency.Visual plausibility focuses on modified regions; counterfactual edits are judged for perceptual quality without penalizing intended physical impossibility.Source: §3.3 · pp. 9–10
  • A sample counts as solved only when it scores 5 on all three metrics; accuracy is the percentage of such samples, alongside dimension scores rescaled to 100.Logical-task consistency uses a binary scheme, unlike the continuous 1-to-5 scale for temporal, causal, and spatial tasks.Source: §3.3 · p. 10

3.4 RISE-Agent

  • RISE-Agent uses a planner to infer target states, an executor to realize them with generative or programmatic tools, and a verifier to pass, refine, or replan edits.The planner may use web search or structured code tools; the executor has 12 deterministic operations, and refinement and replanning have predefined budgets.Source: §3.4 · pp. 10–11

4 EXPERIMENT

The experiments compare 58 visual-editing approaches on RISEBench++ and evaluate outputs across instruction reasoning, appearance consistency, and visual plausibility. Results show persistent reasoning challenges, especially for logical and hybrid tasks, alongside gains from RISE-Agent’s integrated components.

4.2 Main Results (LMM-as-a-Judge)

  • Under English instructions, GPT-Image-2.5 Sunburst achieved 56.6% overall accuracy on RISEBench++, the highest among evaluated closed-source models; a clear performance gap remained between closed-source and open-source models.Source: §4.2 · pp. 12, 15
  • 48.0% overall accuracy puts RISE-Agent ahead of MindBrush, IntentEdit, and ImAgent on single-image cases, though the results leave considerable room for improvement.The three methods score 21.8%, 13.2%, and 3.5%, respectively, on those cases.Source: §4.2 · p. 15
  • Strong models are generally robust across English and Chinese, but instruction reasoning trails visual-quality dimensions, and logical and hybrid tasks remain difficult.GPT-Image-2.5 Sunburst scores 27.0% on logical and 39.0% on hybrid reasoning; on Chinese instructions, it achieves 55.5% overall.Source: §4.2 · pp. 12, 15, Figure 5 · p. 12

4.3 Performance across Subcategories

  • Models handle familiar knowledge and explicit spatial structure better than image-specific evidence and implicit visual rules.The strongest closed-source result for implicit pattern induction is 31.2%; RISE-Agent reaches 47.2% on explicit rule deduction, above evaluated closed-source models.Source: §4.3 · pp. 15, 18

4.4 Fine-grained Analysis by Task Type

  • Pattern prediction reaches no more than 25.6% across seven analyzed models, while RISE-Agent leads on puzzle solving and rule transfer.Models also differ in task-specific strengths, including geometry, viewpoint generation, reading offset, and anatomical anomaly editing.Source: §4.4 · pp. 18–19

4.5 Validity of LMM-as-a-Judge

  • Human-alignment tests support Gemini-3-Flash for instruction reasoning and appearance consistency, and Gemini-3.1-Flash-Lite for visual plausibility.On instruction reasoning, Gemini-3-Flash achieves 0.48 MAE and 0.84 Spearman; the study samples 100 cases and four annotators score 200 outputs.Source: §4.1 · p. 12, Table 6 · p. 19, §4.5 · p. 19

4.6 Ablation Study of RISE-Agent

  • 48.0% is the final RISE-Agent ablation score after verifier-guided refinement, up from 12.9% for the baseline; tools raise the planner score to 42.6%.The planner alone raises the score to 40.5%; verifier refinement performs best in five of six categories, while tool use slightly reduces temporal and spatial scores.Source: Table 7 · p. 19, §4.6 · pp. 19, 21

4.7 Qualitative Examples

  • Qualitative examples show that reasoning-intensive edits require both inferring the intended state and executing it precisely, while sequential edits can accumulate visual degradation.Qwen-Image-Edit-2509 and HunyuanImage-3.0 show progressive structural and fidelity loss across turns.Source: §4.7 · p. 21

5 CONCLUSION

The conclusion presents RISEBench++ as a fine-grained bilingual benchmark and RISE-Agent as a training-free editing framework. Across 58 evaluated approaches, reasoning-informed editing remains challenging, with the strongest approach reaching 56.6% accuracy.

5 CONCLUSION

  • RISEBench++ provides 1,000 human-annotated bilingual cases across six reasoning dimensions, 12 subcategories, and 65 task types, with single-image, multi-image, and multi-turn settings.Its evaluation framework assesses instruction reasoning, appearance consistency, and visual plausibility, with LMM judgments validated against human assessments.Source: §5 · p. 21
  • 56.6% accuracy is the best result among 58 evaluated approaches, while implicit visual rules, precise reasoning-dependent edits, and consistency across successive edits remain bottlenecks.RISE-Agent integrates reasoning-driven planning, tool-augmented execution, and verifier-guided refinement; the results indicate promise for agentic approaches.Source: §5 · pp. 21–22
  • Advancing visual editing requires more than stronger generation: the conclusion also identifies robust visual perception, sophisticated multimodal reasoning, and precise execution as needed capabilities.The authors position RISEBench++ and RISE-Agent as a foundation for systematic evaluation and further research.Source: §5 · p. 22

Supplementary Material

RISEBench++ organizes its 1,000 tasks into six reasoning dimensions and a hierarchy of finer-grained categories.

A Taxonomy and Statistics of RISEBench++

  • The taxonomy groups tasks across temporal, causal, spatial, logical, counterfactual, and hybrid reasoning, spanning scenarios from common-sense knowledge to rule deduction and counterfactual imagination.The hierarchy is intended to support broad coverage and fine-grained analysis of different reasoning skills.Source: A Taxonomy and Statistics of RISEBench++ · p. 28

B Generalization Evaluation of RISE-Agent on KRIS-Bench

On KRIS-Bench, RISE-Agent is competitive with closed-source models and outperforms open-source models and existing agentic methods overall. This provides evidence of transfer to a different benchmark and evaluation setup.

B Generalization Evaluation of RISE-Agent on KRIS-Bench

  • 88.12 overall score places RISE-Agent above all open-source models and existing agentic methods on KRIS-Bench, though closed-source GPT-Image-2.5 models score higher.RISE-Agent also scores 89.14 on factual knowledge, 87.77 on conceptual knowledge, and 97.06 on instruction following.Source: B Generalization Evaluation of… · p. 28

C Detailed Performance across Three Evaluation Dimensions

Performance varies across evaluation dimensions and reasoning categories: Instruction Reasoning (IR) remains harder than Appearance Consistency (AC) and Visual Plausibility (VP), especially for logical and hybrid reasoning. These patterns are broadly consistent in English and Chinese.

C Detailed Performance across Three Evaluation Dimensions

  • IR remains considerably more challenging than AC and VP, despite many models achieving relatively strong scores on the latter dimensions.The reported dimension-level trends are broadly consistent in English and Chinese.Source: C Detailed Performance across Three… · p. 28
  • Logical and hybrid reasoning generally receive lower IR scores than temporal, causal, and spatial reasoning.The gap is especially evident among open-source models, where strong visual quality or constraint adherence does not necessarily translate into reasoning correctness.Source: C Detailed Performance across Three… · p. 28
  • The three dimensions are complementary, so visual quality or instruction compliance alone does not capture reasoning-intensive editing performance.The authors also report that logical and hybrid reasoning are especially challenging.Source: C Detailed Performance across Three… · p. 28

D Fine-grained Accuracy across Task Types

Task-level accuracy varies substantially across models and tasks, with logical and rule-intensive problems remaining difficult across model groups. Score distributions and evaluation prompts provide additional views of performance and the assessment setup.

D Fine-grained Accuracy across Task Types

E Score Distributions across Models

  • GPT-Image-2.5 Sunburst, Nano Banana Pro, and RISE-Agent show many favorable scores across IR, AC, and VP, while model differences are most pronounced in IR and AC.VP scores cluster toward the upper end across most selected models, potentially making VP less discriminative as performance improves.Source: E Score Distributions across Models · p. 29, Figure 8 · p. 40

F Human Alignment Subset

  • The human-alignment subset reports human score distributions from 1 to 5 across reasoning categories and the three evaluation dimensions.Table 10 summarizes the subset and corresponding human annotations.Source: F Human Alignment Subset · p. 29, Table 10 · p. 31

G Prompts for Automated Evaluation

  • Automated evaluation prompts are tailored to evaluation dimensions and reasoning categories to promote reliable, consistent judgments.The appendix provides prompts for general, spatial, and logical instruction-reasoning tasks using text or image references.Source: G Prompts for Automated Evaluation · p. 29, Figure 9 · p. 41, Figure 10 · p. 42
  • Reported scores are normalized from a 1–5 range to a 100-point scale, and logical tasks are not evaluated for VP.IR, AC, and VP denote Instruction Reasoning, Appearance Consistency, and Visual Plausibility.Source: G Prompts for Automated Evaluation · p. 32

10. Appearance Consistency — General tasks

The general Appearance Consistency evaluation checks whether an edited output preserves aspects of the original image that the instruction did not ask to change.

10. Appearance Consistency — General tasks

  • The evaluator compares original and modified images while ignoring instructed changes and assessing consistency in all other aspects.The corresponding prompt targets Appearance Consistency on temporal, causal, and counterfactual reasoning tasks.Source: §10 · p. 47, Figure 18 · p. 47

11. Appearance Consistency — Spatial tasks

The spatial Appearance Consistency evaluation focuses on unintended differences between the original and modified images, excluding changes specified by the instruction.

11. Appearance Consistency — Spatial tasks

  • The evaluator ignores instructed changes and assesses whether the modified image remains consistent with the original in other aspects.The corresponding prompt evaluates Appearance Consistency on spatial reasoning tasks.Source: §11 · p. 48, Figure 19 · p. 48

12. Appearance Consistency — Logical tasks: with a text reference answer

Figure 20 presents a prompt for evaluating appearance consistency on logical reasoning tasks when a text reference answer is available.

12. Appearance Consistency — Logical tasks: with a text reference answer

  • Figure 20 is a prompt for evaluating appearance consistency on logical reasoning tasks with a reference text answer.Source: Figure 20 · p. 49

13. Appearance Consistency — Logical tasks: without a text reference answer

The prompts cover appearance consistency for logical tasks without a text reference and visual plausibility across general, spatial, and counterfactual tasks. The counterfactual guidance separates perceptual quality from physical plausibility and real-world correctness.

13. Appearance Consistency — Logical tasks: without a text reference answer

  • Figure 21 presents an appearance-consistency evaluation prompt for logical reasoning tasks without a text reference answer.Source: Figure 21 · p. 50

14. Visual Plausibility — General tasks

  • Figure 22’s prompt evaluates visual plausibility on temporal and causal tasks; the general rubric scores edited-image clarity and distortion from 1 to 5.Source: Figure 22 · p. 51, §14 · p. 51

15. Visual Plausibility — Spatial tasks

  • Figure 23 presents a visual-plausibility prompt for spatial reasoning; the rubric scores edited-image clarity and distortion from 1 to 5.Source: Figure 23 · p. 52, §15 · p. 52

16. Visual Plausibility — Counterfactual tasks

  • Figure 24 presents a counterfactual visual-plausibility prompt; evaluators score perceptual quality only, not physical plausibility or real-world correctness.Source: Figure 24 · p. 53, §16 · p. 53
Loading 2610.12343v1…