Source-linked AI summary
ETCHR: Editing To Clarify and Harness Reasoning
Beichen Zhang, Yuhong Liu, Jinsong Li, Yuhang Zang, Jiaqi Wang, Dahua Lin
TL;DR
Fine-grained visual reasoning remains difficult when questions require selective focus or scene transformations. ETCHR uses a question-conditioned, reasoning-aware image editor with verification, raising average Pass@1 across five task families by 4.82 points with Qwen3-VL-8B, 5.47 with Gemini-3.1-Flash-Lite, and 4.61 with Kimi K2.5.
Problem
Existing think-with-images methods are limited by fixed toolkits or noisy intermediate images, while editors struggle to map abstract questions to useful visual transformations.
Method
ETCHR trains a decoupled, question-conditioned image editor with Reasoning Imitation and Reasoning Enhancement, then verifies edits before downstream reasoning.
Results
Average Pass@1 rises from 55.95 to 60.77 (+4.82) across five task families with Qwen3-VL-8B, with gains also reported for Gemini-3.1-Flash-Lite and Kimi K2.5.
Takeaways & Limitations
Because ETCHR is decoupled from the understanding model, it plugs into open- and closed-source MLLMs without fine-tuning them.
Takeaways & Limitations
ETCHR incurs higher time overhead than text-based reasoning and remains capped by the downstream understanding model’s reasoning ability.
Abstract
from arXiv · showhide
Multimodal Large Language Models have advanced visual reasoning, yet a purely textual chain of thought remains a bottleneck for questions that require fine-grained focus or view transformations. The ''think with images'' paradigm narrows this gap, but existing approaches are either constrained by fixed predefined toolkits or produce noisy intermediate images from unified multimodal methods. We pursue a third option: using a dedicated image editing model and decouple it with an understanding model. However, off-the-shelf image editors fail as reasoning assistants with two complementary gaps: a language-side gap, where editors trained as passive instruction-followers cannot map an abstract question to an appropriate visual transformation, and a generation-side gap, where edit correctness degrades as reasoning depth grows. Guided by this analysis, we introduce ETCHR (Editing To Clarify and Harness Reasoning), a question-conditioned, reasoning-aware image editor decoupled from the downstream understanding model and trained with a two-stage recipe targeted at the two gaps: Reasoning Imitation via supervised fine-tuning on edit trajectories, followed by Reasoning Enhancement with VLM-derived rewards for edit correctness and downstream reasoning accuracy. Since the editor is decoupled, ETCHR plugs into different open- and closed-source MLLMs in a training-free manner. Across five task families (fine-grained perception, chart understanding, logic reasoning, jigsaw restoration, and 3D understanding), ETCHR raises average Pass@1 from 55.95 to 60.77 (+4.82) with Qwen3-VL-8B, from 65.08 to 70.55 (+5.47) with Gemini-3.1-Flash-Lite, and from 76.55 to 81.16 (+4.61) with the 1T-parameter MoE model Kimi K2.5.
1. Introduction
ETCHR addresses the limitations of “think with images” by decoupling a question-conditioned, reasoning-aware image editor from the understanding model and adding edit verification. Its two-stage training improves edit usefulness and yields consistent Pass@1 gains across diverse tasks and MLLMs.
- Motivation: “Think with images” improves visual reasoning, but generating the right question-relevant intermediate image requires both suitable transformation selection and faithful rendering.Reported gains include fine-grained visual search, chart reasoning, and spatial navigation.
- Limitations: Tool-based methods restrict edits to predefined low-level operations, while unified models produce noisy intermediates and weaken specialist understanding and generation.Neither family verifies edit correctness before reasoning continues, allowing noisy edits to propagate.
- ETCHR: ETCHR decouples a question-conditioned specialist editor from the understanding MLLM, enabling broader transformations, preserved editing fidelity, and reflective rejection of unreliable edits.The editor infers the useful transformation directly from the question rather than requiring an explicit edit prompt.
- Training and inference: Reasoning Imitation uses supervised fine-tuning on question-conditioned edit trajectories, followed by Reasoning Enhancement with reasoning-aware rewards for edit correctness and downstream usefulness.Inference uses an Edit-Verify-Reason procedure that lets the understanding model reject noisy edits.
- Results: 55.95 to 60.77 (+4.82) average Pass@1 with Qwen3-VL-8B, 65.08 to 70.55 (+5.47) with Gemini-3.1-Flash-Lite, and 76.55 to 81.16 (+4.61) with Kimi K2.5.The benchmark suite spans fine-grained perception, chart understanding, logic and path reasoning, jigsaw reasoning, and 3D understanding.
2. Analysis
The analysis identifies two complementary deficiencies in current image editors: inferring useful edits from abstract questions and faithfully generating transformations requiring multi-step reasoning. These gaps motivate ETCHR’s two-stage question-conditioned imitation and reasoning-oriented enhancement.
- Two Reasoning Gaps: Reasoning-aware editing requires both language-side reasoning, which infers a helpful visual transformation from a question, and generation-side reasoning, which renders it faithfully.Generation-side challenges include non-trivial spatial or algorithmic inference, such as tracing a maze path.
- Language-Side Reasoning: Modern editors follow explicit edit instructions but struggle to recover a useful transformation from an abstract question alone.The analysis contrasts instructions like drawing a box with questions requiring spatial comparisons.
- Language-Side Reasoning: Concrete editing instructions significantly outperform abstract questions, showing that base editors lack a reliable question-to-edit mapping.Gemini-3.1-Flash-Lite generated instructions, while Qwen3-VL-8B judged correctness on 100 V*Bench and HRBench samples.
- Generation-Side Reasoning: Edit correctness is near-perfect at shortest-path length L=1 but decreases sharply as L grows, approaching zero on the longest Maze Solving and Frozen Lake Solving paths.The held-out evaluation used 100 samples with L∈{1, 3, 5, 7, 10}, with a VLM judge requiring highlighted paths to match inputs.
- Summary: ETCHR addresses the gaps through Stage I question-conditioned imitation and Stage II reasoning-oriented enhancement for multi-step generation.The two stages target the question-to-edit gap and generation robustness, respectively.
3. ETCHR: Editing To Clarify and Harness Reasoning
ETCHR is a question-conditioned image editor trained through Reasoning Imitation and Reasoning Enhancement to produce edits that expose task-relevant visual evidence. Its Edit-Verify-Reason procedure verifies candidate edits before downstream reasoning, while complementary rewards align edits with both correctness and reasoning utility.
- Training Recipe: The two-stage recipe uses supervised fine-tuning for Reasoning Imitation and reinforcement learning for Reasoning Enhancement.The reinforcement-learning stage uses VLM-derived rewards aligned with downstream reasoning utility.
- Question-Conditioned Editing: ETCHR maps an image and question to an edited image that surfaces the visual evidence needed to derive the answer.Training instances pair the input image, ground-truth edit, question, and answer.
- Training Data: Five reasoning families cover fine-grained perception, chart understanding, logic reasoning, jigsaw restoration, and 3D understanding.These families require localization, structured-plot grounding, multi-step inference, geometric reorganization, and viewpoint-related transformations.
- Task-Level Prompt Enhancement: Task-level meta-prompts are prepended to questions because question-only prompting causes severe cross-task interference.The meta-prompt evokes the editing modality appropriate to each task family and acts as a soft task-router during training.
- Reward Design: The combined reward balances downstream answer correctness with isolated edit correctness, compensating for reasoning-model ceiling effects and judge noise.The guidance reward is faithful but capability-ceiling-bound, whereas the correctness reward is broader but can accept plausible or reject correct edits.
- Edit-Verify-Reason: Edit-Verify-Reason edits the image, verifies whether the edit contains information needed to answer the question, and then reasons with the verified result.The procedure prevents misleading intermediate edits from contaminating downstream reasoning.
4. Experiments
Experiments evaluate ETCHR across five task families and multiple understanding backbones, showing advantages over prevailing visual-reasoning paradigms and clarifying the roles of staged training, complementary rewards, and selective reflection.
- Experimental Setup: Evaluations span fine-grained perception, chart understanding, logical reasoning, jigsaw restoration, and 3D understanding across Qwen3-VL-8B, Kimi K2.5, and Gemini-3.1-Flash-Lite.The benchmarks include natural-image detail recognition, charts, in-house maze and Frozen Lake tasks, shuffled MS COCO puzzles, and 3D perception.
- Overall Performance: Tool-based models have limited coverage for non-local Logic, Jigsaw, and 3D edits, while unified models avoid this limit but trail ETCHR on average.Unsupported tool-based tasks are marked as outside evaluated scope rather than failures.
- Comparison with a Closed-Source Editor: Both ETCHR and Nano Banana 2 improve over no-edit baselines on most tasks, but their 100-sample-per-benchmark comparison indicates trends rather than significance-tested estimates.The comparison uses Qwen3-VL-8B and Gemini-3.1-Flash-Lite under matched subsets to control API cost.
- Training Recipe: Stage I SFT improves all five task families on both backbones, whereas Stage II RL adds under one point on Perception and Chart and is essentially flat on Logic.The results suggest supervised imitation reactivates the encoder’s reasoning capacity for question-conditioned editing.
- Reward Mechanism: Combining Editing Correctness and Editing Guidance matches or exceeds either reward alone across all task families.Correctness filters implausible edits, while Guidance ensures edits advance the downstream reasoning chain; their relative strengths vary by task.
- Reflection Mechanism: Reflection consistently helps Perception and Chart, but gains are smaller on Logic, absent on Qwen3-VL-8B Jigsaw, and slightly negative on 3D.These results motivate selectively gating verification by backbone confidence or task family rather than applying it uniformly.
5. Related Work
Prior “think with images” research follows tool-based pipelines or unified multimodal models, while recent image editors improve instruction understanding and edit faithfulness. These lines of work motivate positioning image editing as a reasoning aid.
- Thinking with Images: Tool-based Pipelines: Tool-based pipelines couple an understanding MLLM with a deterministic renderer that executes structured actions to produce intermediate visual aids.Visual Sketchpad prompts spatial annotations and auxiliary lines for geometric and mathematical reasoning.
- Thinking with Images: Unified Multi-modal Models: Unified multimodal models interleave text and image tokens within one backbone, with Janus decoupling visual encoders for understanding and generation.Chameleon and Show-o use joint autoregressive and diffusion modeling over a shared token space.
- Image Editing Models: Modern image-to-image editors use MLLM-style encoders to parse complex instructions, while instruction-tuned diffusion editors target edit faithfulness.InstructPix2Pix trains on synthetic before/after pairs, whereas MagicBrush uses human-annotated real-image edits.
6. Conclusion
The conclusion presents ETCHR as a question-conditioned image editor addressing language-side and generation-side reasoning gaps through staged training and Edit-Verify-Reason inference, while noting limitations in exploration, downstream reasoning capacity, and time cost.
- Contributions: ETCHR is a question-conditioned image editor for think-with-images reasoning, motivated by language-side and generation-side gaps in prior work.The identified gaps concern mapping abstract questions to edits and maintaining edit quality as reasoning depth increases.
- Method: Its two-stage recipe combines Reasoning Imitation SFT and Reasoning Enhancement RL with an Edit-Verify-Reason inference procedure.The recipe targets the two identified gaps while keeping the editor decoupled from the understanding model.
- Limitations: GRPO sampling has limited semantic diversity for structural editing, constraining policy exploration and preventing uniform reinforcement-learning gains across reasoning families.The limitation is exemplified by jigsaw restoration.
- Limitations: ETCHR remains capped by the downstream understanding model, because even optimally edited intermediates cannot overcome that model’s reasoning ceiling.This is an end-to-end capability bound imposed by the downstream MLLM.
- Limitations: Image editing adds time cost to the overall reasoning process.The passage identifies additional temporal cost as a limitation of the approach.
A. Prompts
The prompts condition ETCHR’s edits and downstream reasoning on task-specific visual transformations, including marking relevant regions, tracing paths, restoring puzzles, and generating new 3D perspectives. They also ask models to verify edited-image validity before answering from the original and transformed views.
- Edit prompts: Task-specific edit prompts mark relevant regions, trace maze paths, restore jigsaws, or imagine perspectives that expose useful 3D information.Fine-grained perception and chart understanding use red boxes; logic uses a blue shortest path; jigsaw restoration reconstructs the original image.
- Verification prompts: Verification prompts judge whether boxed information is valid, paths satisfy connectivity and obstacle constraints, jigsaws are correctly restored, or objects are visible from a new perspective.Maze verification requires four-direction continuity without altered obstacles or path tiles, while frozen-lake verification checks that the path avoids holes.
- Answer prompts: Answering prompts direct models to focus on transformed views while retaining the original image for fine-grained perception, charts, mazes, frozen lakes, jigsaws, and 3D understanding.Red-box prompts prioritize corresponding key information; path prompts treat the blue path as likely correct; 3D prompts emphasize the novel perspective.
- Chart understanding: Chart prompts use precise localization of granular information to facilitate better comprehension of visual data for understanding models.The chart workflow marks question-relevant regions with a red box and asks the model to focus on the boxed area.
B. More Cases
ETCHR assists models across chart understanding, logical reasoning, jigsaw puzzles, and 3D perception by applying task-specific visual transformations. Examples include drawing maze routes, restoring scrambled images, and shifting perspective to clarify spatial positions.
- Cross-task examples: ETCHR supports reasoning across chart understanding, logical reasoning, jigsaw puzzles, and 3D perception.The paper provides examples in these four task families.
- Logical reasoning: ETCHR draws the correct path on a maze map, helping the understanding model identify the route to the exit.This visual transformation directly guides route identification.
- Jigsaw restoration: ETCHR restores scrambled jigsaw images, guiding the understanding model to solve jigsaw tasks correctly.The restoration supports determining the correct arrangement of image patches.
- 3D perception: ETCHR shifts perspective based on the question to help the model understand spatial positioning.The example asks where a woman in red is located from the perspective of a woman in green.