Source-linked AI summary
Think, Look, and Revise: Inconsistency-Aware Visual Self-Correction in MLLMs
Yu Cheng, Arushi Goel, Hakan Bilen
TL;DR
Tool-augmented MLLMs often trust external visual evidence without verifying it, limiting recovery from noisy or misaligned tool outputs. ReVISE trains verification and self-correction through reflective trajectories and targeted rewards, and consistently improves perception-intensive benchmark performance. Its scope remains bounded by dependence on synthetic trajectories, tool quality, and visual-centric settings, with a trade-off on some abstract tasks.
Problem
Existing tool-augmented MLLMs rarely verify tool outputs, limiting detection and recovery from noisy or misaligned evidence.
Method
ReVISE combines reflection-supervised multi-turn trajectories with GRPO rewards for self-correction and spatial grounding.
Results
ReVISE consistently improves performance on challenging perception-intensive benchmarks including CountBench, CVBench, and BLINK-HARD.
Takeaways & Limitations
Verification-driven reasoning improves robustness across multiple perception-intensive benchmarks by enabling models to reassess visual evidence and revise incorrect intermediate conclusions.
Takeaways & Limitations
ReVISE depends on synthetic trajectories, bounding boxes, and external tool quality, and fine-grained grounding slightly degrades performance on some abstract knowledge-intensive tasks.
Abstract
from arXiv · showhide
Tool-augmented multimodal reasoning integrates external tools (e.g., object detection, depth estimation) into multimodal large language models (MLLMs) to address perceptual bottlenecks in complex visual tasks. However, existing approaches rarely verify tool outputs, limiting their ability to detect and recover from tool failures. We propose ReVISE, a framework that equips MLLMs with verification and dynamic error recovery for tool-augmented reasoning. ReVISE introduces (1) a curated training dataset that supervises reflective behaviors, enabling models to validate tool-derived evidence, reformulate queries when visual mismatches arise, and fall back to intrinsic grounding when external tools are unreliable; and (2) a reinforcement learning based targeted rewards that encourage internal reflection and penalize spatial misalignment. Experiments on several benchmarks demonstrate consistent improvements over existing methods, highlighting the importance of error detection and correction in tool-augmented multimodal reasoning.
1 Introduction
Existing tool-augmented MLLMs add visual tools to improve intermediate grounding, but typically trust tool outputs without verification. ReVISE closes this open loop through supervised reflective trajectories and correction-oriented reinforcement learning.
- Tool-augmented MLLMs invoke visual tools such as zooming, cropping, and object detection to provide evidence for intermediate decisions.
- Existing methods propagate intermediate tool outputs without verification, limiting inconsistency detection and recovery from noisy or misaligned evidence.
- ReVISE validates tool outputs against direct image interpretation and triggers self-correction when feedback is noisy, contradictory, or missing.
- The curated dataset contains multi-turn trajectories that supervise verification and revision, while GRPO training uses correction-oriented rewards.
- ReVISE treats tool-output verification and correction as learnable behaviors and reports consistent gains on CountBench, CVBench, and BLINK-HARD.
2 Related Work
Prior work integrates external vision tools and iterative inference into multimodal reasoning, but generally relies on inference-time heuristics rather than explicitly training visual self-correction. ReVISE instead supervises revision when new evidence contradicts an initial hypothesis.
- Multimodal Large Language Models: MLLM training commonly combines supervised fine-tuning with reinforcement or preference-based learning after multimodal pretraining.
- Tool-Augmented Visual Reasoning: Early tool-augmented systems translate natural-language queries into executable programs that invoke specialized vision modules.
- Tool-Augmented Visual Reasoning: Recent methods support iterative evidence revisiting, but largely rely on inference-time heuristics and do not explicitly supervise correction-oriented reasoning during training.
- Tool-Augmented Visual Reasoning: ReVISE constructs trajectories that encourage revision after contradictory evidence and adds supervision for object-level grounding during reasoning.
3 Method
ReVISE formulates tool-integrated reasoning as a sequential decision process in which an MLLM gathers and verifies visual evidence before answering. It combines cold-start supervised trajectories with GRPO rewards for reflection, answer quality, format compliance, and spatial grounding.
- Verification Framework: ReVISE defines verification as assessing consistency among tool observations, the visual query, and the model’s own perception, then revising mismatches.
- Problem Formulation: The reasoning trajectory alternates between states, tool or answer actions, and tool observations until a final answer or maximum step count.
- Cold-Start Supervised Fine-Tuning: Supervised fine-tuning uses curated trajectories that teach tool use, verification, and self-correction, providing a cold-start capability for sequential visual reasoning.
- Policy Optimization via Reflection and Grounding Rewards: GRPO optimization adds rewards for final-answer accuracy, reasoning-format compliance, self-correction, and bounding-box grounding.
- Policy Optimization via Reflection and Grounding Rewards: The grounding objective combines mean IoU with rIoU++, which rewards positive localization improvement from initial to final box predictions.
4 Training Data Curation
ReVISE trains models to verify tool evidence, detect mismatches, and revise reasoning through diverse tool-use and tool-free trajectories. Its curated supervision covers retries, failure verification, and fallback to intrinsic visual perception.
- The reflection-rich dataset teaches ReVISE to verify tool outputs, revise inconsistent evidence, and fall back to intrinsic perception when needed.
- The training process includes four visual operations: object detection, zoomed inspection, edge detection, and depth estimation.
- Trajectory design treats tool-integrated reasoning as sequential decisions over tool invocation, feedback interpretation, and verification-driven revision.
- Five cold-start trajectory types combine positive or negative verification with tool use, retries, and tool-free fallback after failures.
- Gemini-2.5-pro generates specialized traces for each trajectory type, after which erroneous trajectories are manually filtered.
5 Experiments
Experiments evaluate ReVISE across broad multimodal benchmarks, model scales, baselines, and training analyses. ReVISE generally improves perception-intensive performance through verification, grounding, and self-correction, while gains are smaller or absent on knowledge-heavy tasks.
- 5.2 Results: ReVISE-3B gains 5.86% on CountBench, 11.28% on CVBench, and 9.94% on BLINK over Qwen2.5-VL-3B-Instruct.
- 5.2 Results: ReVISE-7B outperforms ReVPT-7B and CodeDance-7B on CountBench and several CVBench subtasks.
- 5.2 Results: ReVISE does not achieve the best results on MMMU and MathVista, whose tasks rely more on multimodal knowledge and abstract reasoning than fine-grained localization.
- 5.2 Results: The 7B model improves on most benchmarks but drops 4.04% on BLINK, 0.09% on MMVP, and 0.83% on MMSTAR.
- 5.3 Further Analysis: RL diversifies tool invocation beyond the SFT stage's frequent object-detection use, while error recovery measures successful correction of SFT-stage tool-induced failures.
- 5.4 Ablation Study: Curated reflection data raises CVBench to 77.89% and MMVP to 60.48%, while reward ablations show complementary gains from self-correction and grounding.
- 5.5 Qualitative Results: Qualitative cases show ReVISE rejecting ambiguous detections, zooming into relevant regions, and refining bounding boxes when initial crops miss context.
6 Conclusion
ReVISE improves the reliability of tool-assisted reasoning through visual self-correction, helping MLLMs identify inconsistencies and revise incorrect intermediate conclusions. Its effectiveness remains dependent on visual understanding and tool quality.
- ReVISE uses visual self-correction to reassess initial predictions and revise incorrect intermediate conclusions.The framework evaluates tool-assisted reasoning against visual evidence rather than relying solely on tools to produce answers.
- Experiments show that verification-driven reasoning improves robustness across multiple perception-intensive benchmarks.
- Revision may struggle when visual evidence is ambiguous or external tools provide inaccurate signals.
Limitations
ReVISE is constrained by its training data, tool coverage, task trade-offs, tool capabilities, and evaluation scale. These limitations may restrict robustness in open-domain settings and leave larger-model scaling unresolved.
- ReVISE relies on synthetic trajectories and bounding boxes from proprietary models, while its tools target visual-centric datasets.This reliance may inadequately capture real-world reasoning dynamics and limit robustness in open-domain scenarios.
- Fine-grained spatial grounding slightly degrades performance on abstract, knowledge-intensive tasks such as MMMU and MathVista.
- Ambiguous or catastrophic failures in predefined external vision tools can still derail the revision process.
- RL evaluations primarily use 3B and 7B models, leaving scaling to much larger MLLMs insufficiently examined.
Broader Impacts and Ethical Considerations
The work focuses on foundational algorithmic improvements for multimodal large language models using existing open-source models and public benchmarks. It does not introduce sensitive applications, deployable end-user systems, or datasets posing direct societal, ethical, or safety risks.
- The work studies visual self-correction and tool-use verification as foundational methodological improvements for MLLMs.
- The study uses existing open-source models and standard public benchmarks rather than introducing new sensitive applications or deployable end-user systems.
- The paper reports no datasets posing direct societal, ethical, or safety risks and therefore omits a dedicated risk discussion.
A Additional implementation details
The appendix describes training configurations for SFT and RL and implementation details for visual tools. The tools provide object localization, zoomed crops, and edge maps for visual reasoning.
- Experiments use NVIDIA A100 80GB GPUs, with 4 GPUs for the 3B model and 8 GPUs for the 7B model.
- SFT trains models for 2 epochs with a learning rate of 1 × 10−5 and a global batch size of 64.
- The object-detection tool returns bounding boxes and confidence scores for text object queries, then crops and upsamples detected regions.
- Edge detection returns binary edge maps that provide low-level cues for boundary-aware spatial reasoning.
C The details of our curated dataset
The supplementary materials describe prompt-driven construction of multi-turn trajectories for tool use, verification, self-correction, and grounding supervision, alongside the GRPO training setup.
- Data and prompt construction: The supplementary data section documents curated cold-start SFT data, RL-stage data construction, reflective-trace prompts, bounding-box supervision, and dataset statistics.These materials support both SFT and RL stages.
- Tool-free verification: Positive and negative tool-free verification prompts require visual re-examination, explicit grounding, and correction of hallucinations or reasoning flaws.Positive verification preserves correct answers while grounding entities; negative verification targets initial errors.
- Verification trajectories: Tool-retry and negative tool-verify prompts rewrite flawed trajectories so agents recognize mistakes, revise strategies, and reach correct answers.The generated trajectories explicitly inject a natural rethinking step after flawed reasoning or tool use.
- Task-specific grounding: Spatial verification grounds both queried objects with bounding boxes and relies on semantic or perspective cues rather than raw coordinate comparison.Counting prompts enumerate all detected instances, while attribute prompts provide the primary object’s box.
- RL preparation: Gemini-2.5-Pro generates tool trajectories and reference bounding boxes, while the RL agent follows structured think, tool_call, and answer formats using vision tools.The available tools include object detection, zooming, edge detection, and depth estimation.
- RL formulation: The GRPO setup samples grouped completions, computes task-specific rewards, and uses clipped updates with KL regularization around a reference policy.The objective encourages reward improvement within each group while stabilizing policy updates.
E.1 Robustness to Noisy Grounding Supervision
ReVISE’s grounding supervision is usually accurate and tolerates moderate box noise, while mixed-domain training recovers general reasoning performance and cold-start SFT remains important for RL.
- Grounding supervision: 98% of 1,000 inspected generated bounding boxes are correct, while 2% contain noticeable localization errors.The inspection suggests that grounding rewards generally rely on reliable spatial supervision.
- Noise sensitivity: 5% Gaussian jitter causes a 0.63-point average degradation, whereas 10% perturbation causes a 3.69-point average drop.The larger perturbation produces substantially greater sensitivity than moderate noise.
- Noise limitations: GRPO clipping limits excessively large updates from high-variance rewards but cannot determine whether a bounding box is semantically correct or remove systematic teacher bias.ReVISE additionally relies on low observed noise and answer, format, grounding, and self-correction rewards.
- Task-conditional rewards: For math and science samples, grounding rewards are disabled while answer-accuracy, format, and self-correction rewards remain active.Perception-heavy samples retain λ_ground = 1.
- Mixed-domain training: Mixed-domain training improves MMMU by 1.71 points and MathVista by 4.35 points while preserving strong CVBench performance.The reported results associate the recovery with adding general-domain reasoning data while retaining perception-heavy training.
- SFT and RL: Subsequent RL improves CountBench from 72.41 to 74.92 and CVBench from 77.89 to 83.83 after cold-start SFT.The two stages provide complementary roles: SFT initializes structured interaction, while RL improves the policy.
F Additional Qualitative Examples
The qualitative cases show that self-correction remains vulnerable to missing domain knowledge, hallucinated tools, and loss of global context after localized zooming.
- Knowledge deficit: The model reads the “Neem Oil Extract” label but answers “Organic” instead of the correct “Chemical” classification and fails to correct itself.The case attributes the failure to missing specialized agricultural knowledge and confirmation bias.
- Cumulative spatial errors: In cluttered scenes, zooming can remove global context, causing double-counting across crop boundaries and unreconciled local evidence.This occurs after noisy, overlapping object-detection outputs.
- Scope boundaries: The framework’s tools are designed for spatial, geometric, and perceptual reasoning, creating a mismatch for abstract knowledge or computation tasks.The reported limitations include toolset-task misalignment and intrinsic knowledge gaps with confirmation bias.
- Theoretical reasoning: In the statistics case, the model abandons ineffective detection, hallucinates a correlation tool, and misses the equivalence tr = tb.The failure combines tool hallucination with a theoretical reasoning limitation.
- Tool verification: A qualitative tool-use example shows the agent rejecting an unhelpful detection result and switching to direct visual examination before further reasoning.The trajectory illustrates tool verification rather than unconditional trust in a failed observation.