Source-linked AI summary
DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning
Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, Xing Yu
TL;DR
Vision-language models remain predominantly text-based in their reasoning, motivating methods that integrate visual information more deeply. DeepEyes uses end-to-end reinforcement learning and native grounding with active perception, without cold-start SFT or external specialized models. It improves multiple perception and reasoning benchmarks while developing progressively more selective active perception and diverse visual reasoning patterns.
Problem
Vision-language models primarily rely on text-based reasoning despite the importance of integrating visual information into multimodal thought.
Method
DeepEyes uses end-to-end reinforcement learning to interleave textual reasoning with native visual grounding and active perception, without cold-start SFT or external specialized models.
Results
DeepEyes significantly improves multiple visual perception and reasoning tasks, including 90.1% accuracy on V* (+18.9%) with the 7B model.
Takeaways & Limitations
DeepEyes develops selective active perception and diverse visual search, comparison, confirmation, and hallucination-mitigation reasoning patterns resembling human cognition.
Takeaways & Limitations
The approach remains constrained by the relatively weak fundamental capabilities of the small Qwen2.5-VL-7B foundation model, including shortcuts and inaccurate localization.
Abstract
from arXiv · showhide
Large Vision-Language Models excel at multimodal understanding but struggle to deeply integrate visual information into their predominantly text-based reasoning processes, a key challenge in mirroring human cognition. To address this, we introduce DeepEyes, a model that learns to "think with images", trained end-to-end with reinforcement learning without requiring pre-collected reasoning data for cold-start supervised fine-tuning (SFT). Notably, this ability emerges natively, leveraging the model's own grounding capability as an intrinsic function rather than relying on external specialized models or APIs. We enable this capability through active perception, where the model learns to strategically ground its reasoning in visual information, guided by a tailored data selection and reward strategy. DeepEyes achieves significant performance gains on general perception and reasoning benchmarks and also demonstrates improvement in grounding, hallucination, and mathematical reasoning tasks. Interestingly, we observe the distinct evolution of active perception from initial exploration to efficient and accurate exploitation, and diverse thinking patterns that closely mirror human visual reasoning processes. Code is available at https://github.com/Visual-Agent/DeepEyes.
1 INTRODUCTION
DeepEyes addresses the predominance of text-based reasoning in vision-language models by learning to interleave visual grounding with textual reasoning through end-to-end reinforcement learning. Its active perception develops from ineffective exploration into selective, efficient use, producing diverse visual reasoning behaviors and gains across perception and reasoning tasks.
- Vision-language models largely confine reasoning to text, whereas human reasoning combines cognition with sequential visual fixations.
- DeepEyes learns interleaved multimodal chain-of-thought through end-to-end reinforcement learning, without cold-start SFT or external specialized models.The model generates grounding coordinates, crops relevant regions, and incorporates them into its ongoing reasoning trajectory.
- During RL, active perception progresses from ineffective exploration to frequent effective use and finally mature, selective, efficient behavior.This evolution occurs without supervised fine-tuning for intermediate reasoning steps.
- DeepEyes develops visual search, regional comparison, visual confirmation, and detail-focused hallucination mitigation patterns resembling human cognition.
- 90.1% accuracy on V* (+18.9%) and improvements of 6.3% on HR-Bench-4K and 7.3% on HR-Bench-8K demonstrate gains on high-resolution benchmarks.The reported V* result uses the 7B model.
- Active-perception data selection and conditional rewards promote grounding-assisted problem solving and significantly advance interleaved reasoning.The reward bonus targets trajectories that successfully complete tasks through active perception.
2 RELATED WORK
Prior multimodal reasoning methods use predefined workflows or auxiliary models, while DeepEyes presents a model that decides whether to zoom into image regions or answer directly. This overview emphasizes model-controlled visual inspection as part of reasoning.
- Existing multimodal chain-of-thought methods commonly rely on predefined workflows or auxiliary models for localization, feature regeneration, or external knowledge.
- DeepEyes itself decides whether to perform a second perception by generating grounding coordinates and cropping relevant image regions.
3 METHOD
DeepEyes is a unified multimodal model that learns to think with images through iMCoT, using native visual grounding and outcome-reward reinforcement learning without cold-start supervised fine-tuning. Its agentic rollout interleaves text and image observations, supported by curated training data and perception-aware rewards.
- 3.1 DEEPEYES: DeepEyes uses end-to-end reinforcement learning to enhance native visual grounding into iMCoT, eliminating cold-start supervised fine-tuning.The approach does not rely on separate specialized models or external tools.
- 3.1 DEEPEYES: After each textual reasoning step, the model autonomously chooses between answering directly and zooming into image regions, appending returned crops to the ongoing trajectory.Active perception can repeat until an answer is generated or the maximum number of perceptions is reached.
- 3.2 AGENTIC REINFORCEMENT LEARNING: iMCoT states interleave accumulated text tokens and image observation tokens, while model-generated next-token actions drive the rollout.The rollout ends when an answer is generated or the maximum number of active perceptions is reached.
- 3.2 AGENTIC REINFORCEMENT LEARNING: The reward combines accuracy, formatting, and a conditional active-perception bonus granted only when the answer is correct and perception is triggered.Conditioning the bonus on correctness promotes perception-aware reasoning while discouraging unnecessary actions.
- 3.3 TRAINING DATA CURATION: Training data curation combines fine-grained perception, diverse chart tasks, and challenging reasoning, then filters samples by difficulty, verification, and perception utility.The curation strategy is designed to bootstrap effective active perception without an SFT cold start.
4 EXPERIMENT
DeepEyes improves perception and multimodal reasoning through active visual grounding, while RL training develops increasingly selective and diverse visual reasoning behaviors.
- Main Results: DeepEyes delivers top performance on MME-RealWorld-Lite, surpassing both 7B and 32B Qwen2.5-VL models.
- Main Results: The model achieves higher grounding accuracy and substantially reduces hallucinations by focusing on regions of interest and verifying cropped areas.
- Main Results: DeepEyes consistently improves performance across challenging multimodal reasoning benchmarks, including mathematical problem-solving.
- Training Dynamics: RL training progresses from ineffective exploration to frequent engagement and finally selective, efficient active perception with high grounding and task accuracy.The three stages are reflected in changing action frequency, response length, grounding IoU, and accuracy.
- Thinking Patterns: Emergent reasoning patterns include visual search, comparison, confirmation, and hallucination mitigation through focused visual inspection.
- Scaling Model Size: Scaling model size from 7B to 32B widens the performance gap over Qwen2.5-VL while increasing reasoning-chain length and grounding precision.
- Scaling Challenging Reasoning Data: Increasing challenging reasoning data from 23% to 42% improves both mathematical benchmarks and the V ∗ perception task.
5 CONCLUSION
DeepEyes learns to think with images through end-to-end RL, using native visual grounding and active perception without external models or cold-start SFT.
- Conclusion: DeepEyes learns “thinking with images” through end-to-end reinforcement learning and native visual grounding.
- Conclusion: The approach uses active perception, tailored data selection, and rewards to incentivize strategic visual grounding during reasoning.
- Conclusion: DeepEyes achieves competitive results across multiple benchmarks and exhibits human-like visual search and comparison patterns.
B.1 DATA DISTRIBUTION
The training corpus combines visual search, scientific diagram understanding, and multimodal reasoning data to support perception and reasoning robustness.
- Data Sources: The corpus contains three sources with distinct focuses: visual grounding, structured visual semantics, and multimodal reasoning.
- Visual Search: Visual Search contributes 47% or 22k samples from V ∗, emphasizing natural-image fine-grained perception and object-level distinctions.
- ArxivQA: ArxivQA contributes 30% or 14k samples featuring scientific plots, diagrams, and schematic charts.
- ThinkLite-VL: ThinkLite-VL contributes 23% or 11k samples for arithmetic reasoning, commonsense inference, and problem solving.
B.2 IMPACT OF TRAINING DATA
Training-data composition is decisive: fine-grained data improves high-resolution perception, reasoning data preserves mathematical ability, and chart data broadens coverage.
- Impact of Training Data: Unfiltered data offers minimal benefit, whereas curated fine-grained data substantially boosts high-resolution image handling.
- Impact of Training Data: Fine-grained specialization induces catastrophic forgetting of reasoning skills, which reasoning data preserves without sacrificing perception gains.
- Impact of Training Data: Chart data is introduced to further enhance the model’s cognitive range.
D.1 SUCCESSFUL CASES
The paper illustrates several emergent visual reasoning patterns, including search, comparison, confirmation, and hallucination mitigation. A zoom-in operation can help correct a visual mistake by focusing on an enlarged image region.
- Thinking Patterns: DeepEyes exhibits visual search, visual comparison, and visual confirmation as distinct thinking patterns.These patterns are identified in Figures 7–9.
- Hallucination Mitigation: The model uses hallucination mitigation by inspecting image details when its initial interpretation is incorrect.Figure 10 describes an initial color confusion followed by correction after zooming in.
E LIMITATIONS
The reported end-to-end reinforcement-learning approach can elicit visual reasoning abilities, but shortcuts remain in reasoning richness and target localization. The paper also relates this direction to more transparent and interpretable multimodal reasoning.
- E LIMITATIONS: End-to-end reinforcement learning still leaves shortcuts involving insufficiently rich reasoning and inaccurate target localization.These issues are attributed to limitations in the foundation model’s capabilities.
- E LIMITATIONS: The experiments use only Qwen2.5-VL-7B, whose small model size is identified as a source of relatively weak fundamental capabilities.This constrains the foundation-model setting examined in the paper.
- E LIMITATIONS: Interleaved multimodal chain-of-thought reasoning is presented as a direction for studying more transparent and interpretable AI systems.The paper connects step-by-step visual reasoning through interactive dialogues with this research direction.
G FUTURE WORK
The current visual reasoning process supports only crop operations. Future work will investigate additional tools for richer real-world visual reasoning.
- G FUTURE WORK: The current visual reasoning process includes only the crop operation.No other visual tools are included in the present process.
- G FUTURE WORK: Real-world scenarios are described as requiring a wider range of tools, including search and drawing auxiliary lines.These are examples of tools beyond cropping.
- G FUTURE WORK: Future work will explore integrating additional tool utilization into visual reasoning.The proposed expansion is motivated by the needs of real-world scenarios.