Source-linked AI summary
Thinking with Images via Self-Calling Agent
Wenxi Yang, Yuzhong Zhao, Fang Wan, Qixiang Ye
TL;DR
Scarce high-quality reasoning data make reinforcement-learning optimization of interleaved multimodal CoT challenging. The paper proposes language-only Self-Calling Chain-of-Thought, in which a main agent delegates atomic visual subtasks to virtual replicas. On V* and HR-Bench 4K, it reports better results than DeepEyes with roughly one quarter of the GPU hours.
Problem
Scarce interleaved multimodal reasoning corpora make reinforcement-learning optimization of iMCoT more challenging than language-only CoT.
Source: §1 · pp. 1–2Method
sCoT has a main agent decompose visual tasks into atomic subtasks solved by same-weight virtual replicas, then coordinate their textual responses through language-only reasoning.
Source: §1 · p. 2, §4.1 · p. 3Results
1.9% on HR-Bench 4K and 1.2% on V*, sCoT outperforms DeepEyes using only ∼25% of its GPU hours.
Source: §1 · p. 3Takeaways & Limitations
The authors conclude that structured textual decomposition and delegation can support efficient end-to-end reinforcement learning for visual reasoning.
Source: §6 · p. 8
Abstract
from arXiv · showhide
Thinking-with-images paradigms have showcased remarkable visual reasoning capability by integrating visual information as dynamic elements into the Chain-of-Thought (CoT). However, optimizing interleaved multimodal CoT (iMCoT) through reinforcement learning remains challenging, as it relies on scarce high-quality reasoning data. In this study, we propose Self-Calling Chain-of-Thought (sCoT), a novel visual reasoning paradigm that reformulates iMCoT as a language-only CoT with self-calling. Specifically, a main agent decomposes the complex visual reasoning task to atomic subtasks and invokes its virtual replicas, i.e. parameter-sharing subagents, to solve them in isolated context. sCoT enjoys substantial training effectiveness and efficiency, as it requires no explicit interleaving between modalities. sCoT employs group-relative policy optimization to reinforce effective reasoning behavior to enhance optimization. Experiments on HR-Bench 4K show that sCoT improves the overall reasoning performance by up to $1.9\%$ with $\sim 75\%$ fewer GPU hours compared to strong baseline approaches. Code is available at https://github.com/YWenxi/think-with-images-through-self-calling.
1. Introduction
Interleaved multimodal reasoning is difficult to optimize with reinforcement learning because high-quality interleaved reasoning data are scarce. sCoT replaces interleaved reasoning with language-only self-calling, and outperforms DeepEyes on the reported benchmarks with substantially fewer GPU hours.
1. Introduction
- Scarce interleaved multimodal reasoning corpora make reinforcement-learning optimization of iMCoT more challenging than language-only CoT.The authors state that this scarcity makes iMCoT agents harder to optimize and undermines reasoning and tool-calling capabilities.Source: §1 · pp. 1–2
- sCoT reformulates visual reasoning as language-only CoT: a main agent decomposes a task into atomic subtasks solved by same-weight virtual replicas, then aggregates their textual responses.The subtasks are designed to be simple enough for language-only CoT.Source: §1 · p. 2
- End-to-end reinforcement learning optimizes the main agent’s reasoning trajectory with GRPO while masking subagent outputs to avoid reward leakage.The authors say the design makes sCoT easier to optimize than iMCoT and facilitates memory savings and generalization.Source: §1 · pp. 2–3
- 1.9% on HR-Bench 4K and 1.2% on V*, sCoT outperforms DeepEyes using only ∼25% of its GPU hours.The paper reports the benchmark gains and GPU-hour comparison for sCoT versus DeepEyes.Source: §1 · p. 3
2. Related Work
Prior visual-reasoning work uses perception tools, multimodal chains of thought, or manually defined workflows. This paper instead coordinates self-invoked subtasks through language-only reasoning and applies reinforcement learning to textual trajectories.
2. Related Work
- Prior approaches enhance perception with tools such as zooming, detection, and segmentation, or use multimodal CoTs and handcrafted workflows to turn visual information into language-friendly representations.The paper presents self-invoked atomic subtasks coordinated by language-only reasoning as an alternative that does not rely on external experts or handcrafted workflows.Source: §2 · p. 3
- Prior thinking-with-images methods commonly rely on interleaved visual observations and tool invocations, whose cross-modal reasoning trajectories are difficult to incentivize directly with reinforcement learning.The paper instead applies reinforcement learning to pure language-based CoT while accessing tools through self-calling.Source: §2 · p. 3
3. Preliminary: Thinking with Images
Thinking with images can be represented as interleaved multimodal reasoning, where textual and visual modalities alternate. These long chains place demands on models, and reported performance degrades as image counts and integration difficulty increase.
3. Preliminary: Thinking with Images
- Interleaved Multimodal Chain-of-Thought alternates between textual and visual reasoning modalities.Source: §3 · p. 3
- Long interleaved chains impose a heavy reasoning burden, and state-of-the-art models show pronounced performance degradation as the number of images increases.The reported degradation is more severe when cues must be integrated across multiple key images or under noisy conditions.Source: §3 · p. 3
- Qwen2-VL-7B-Instruct suffers more than 15% accuracy drops on a two-image visual needle-in-a-stack benchmark.Source: §3 · p. 3
4. The Proposed Approach
The proposed approach decomposes visual questions into isolated, atomic subtasks handled by parameter-sharing replicas, with the main agent aggregating textual outputs. Its language-only self-calling trajectory is optimized with agentic reinforcement learning.
4.1. Self-Calling Chain-of-Thought
- The main agent breaks a complex visual task into atomic subtasks, each paired with a clear question and cropped image region; subagents return text that the main agent aggregates into an answer.The subagents are virtual replicas that share the main agent’s weights and parameters rather than separately deployed models.Source: §4.1 · p. 3
- During training, the main agent iteratively invokes parameter-sharing subagents through pure-language CoT, and GRPO incentivizes the resulting behavior using accuracy, format, and tool-use rewards.Source: Figure 3 · p. 4
- Subagents are invoked dynamically through structured calls specifying a task type, textual prompt, and image bounding box, and they operate independently without global information or communication.The calls are sequential in the pipeline; the subagents process one cropped image at a time and return concise text.Source: §4.1 · p. 4
4.2. Agentic Reinforcement Learning
- The sCoT state at step t comprises the main agent’s reasoning tokens and accumulated textual responses from subagents.The agent learns a policy to maximize long-term reward over this language-only reasoning trajectory.Source: §4.2 · p. 5
- The tool-use bonus requires a correct answer and a tool call before the answer, while GRPO optimization masks gradients on subagent observation tokens.The ordering constraint is intended to discourage post-hoc or redundant tool calls; masking focuses optimization on the model’s own reasoning and action outputs.Source: §4.2 · p. 5
5. Experiment
SubagentVL delivers strong high-resolution visual reasoning results with more constrained training resources, while its calling strategy develops over training. Ablations show that structured tool calls and visually grounded training data matter for the reported learning dynamics.
5.1. Setting
- Evaluation uses V* and HR-Bench, while Qwen2.5-VL-7B-Instruct is trained on DeepEyes data and invokes subagents sequentially during reinforcement learning.Reward components cover answer accuracy, output format, and tool use.Source: §5.1 · p. 5
5.2. Result
- Tool calls first decline as reward rises, then increase as delegation improves reward, before settling into a mature coordination pattern; the transition timing varies.The training-dynamics figure is described in terms of three stages.Source: Figure 4 · p. 6, §5.2 · p. 6
- SubagentVL surpasses baselines on V* and HR-Bench 4K and remains competitive on 8K despite more constrained training resources.The comparison covers high-resolution benchmarks.Source: §5.2 · p. 6, Table 2 · p. 7
- SubagentVL shows only marginal grounding and OCR gains over DeepEyes but a notable improvement on hallucination benchmarks; the authors attribute complex-task gains mainly to learned subagent-calling strategy.The cited benchmark tables report visual grounding, OCR, and hallucination results.Source: §5.2 · p. 6, Table 3 · p. 7, Table 4 · p. 7
5.3. Ablation Study
- Relaxed tool-call arguments converge to one call and repeated-query crops, reducing reasoning diversity; stricter requirements encourage more structured tool-calling behavior.The protocol ablation examines task type, prompt, and bounding-box arguments.Source: §5.3 · pp. 6–7, Table 6 · p. 7
- Without tool-before-answer ordering, SubagentVL may append calls after </answer> and exploit reward; the ordering constraint promotes proper sequencing.The figure caption identifies reward hacking when the ordering constraint is missing.Source: Figure 6 · p. 8, §5.3 · p. 8
- Fine + Chart stabilizes training, whereas adding Reason data lowers reward and evaluation metrics, attributed to its abstract reasoning focus mismatching sCoT's visual-grounding trajectories.The authors conclude that fine-grained visual discrimination data contributes more effectively than non-visual reasoning data for sCoT.Source: Figure 7 · p. 8, §5.3 · p. 8, Table 7 · p. 8
6. Conclusion
The conclusion presents sCoT as atomic subtasks coordinated by a self-calling agent, and reports strong results on V* and HR Bench. It also identifies protocol, reward-ordering, and training-data choices as important for stable learning and delegation.
6. Conclusion
- On V* and HR Bench, SubagentVL outperforms open-source models, prior multimodal Chain-of-Thought paradigms, and manually defined workflows.The conclusion characterizes its performance as striking.Source: §6 · p. 8
- sCoT replaces interleaved multimodal reasoning trajectories with atomic subtasks coordinated by a main self-calling agent.The conclusion says this removes the reasoning burden of managing multimodal CoTs and enables efficient end-to-end reinforcement learning.Source: §6 · p. 8
- Ablations identify strict tool-calling protocols, proper reward-ordering constraints, and carefully curated training data as essential for stable learning and effective delegation.Source: §6 · p. 8
- The conclusion characterizes structured textual decomposition with lightweight visual subtasks as a scalable, resource-efficient approach to high-resolution multimodal reasoning.Source: §6 · p. 8