Source-linked AI summary
Look-Back: Implicit Visual Re-focusing in MLLM Reasoning
Shuo Yang, Yuwei Niu, Yuyang Liu, Yang Ye, Bin Lin, Li Yuan
TL;DR
MLLMs often neglect visual input during later reasoning, and existing remedies typically inject visual information explicitly. Look-Back guides models to generate reflective <back> tokens through staged fine-tuning and reinforcement learning, enabling self-directed visual refocusing. Across multimodal benchmarks, it consistently improves reasoning and perception performance.
Problem
MLLMs often rely predominantly on text during later reasoning, while existing methods commonly use explicit visual-information reinjection.
Method
Look-Back combines reflective-data cold-start fine-tuning with reinforcement learning using a <back>-based format reward to guide self-directed visual reflection.
Results
Look-Back consistently enhances reasoning and perception performance across multiple multimodal benchmarks.
Takeaways & Limitations
MLLMs can autonomously re-focus on visual inputs during reasoning without explicit visual-information injection or structural model constraints.
Takeaways & Limitations
Cold-starting with GPT-4o-generated chain-of-thought data degraded performance, whereas refined homologous model-generated data improved it.
Abstract
from arXiv · showhide
Multimodal Large Language Models (MLLMs) have achieved remarkable progress in multimodal reasoning. However, they often excessively rely on textual information during the later stages of inference, neglecting the crucial integration of visual input. Current methods typically address this by explicitly injecting visual information to guide the reasoning process. In this work, through an analysis of MLLM attention patterns, we made an intriguing observation: with appropriate guidance, MLLMs can spontaneously re-focus their attention on visual inputs during the later stages of reasoning, even without explicit visual information injection. This spontaneous shift in focus suggests that MLLMs are intrinsically capable of performing visual fusion reasoning. Building on this insight, we introduce Look-Back, an implicit approach designed to guide MLLMs to ``look back" at visual information in a self-directed manner during reasoning. Look-Back empowers the model to autonomously determine when, where, and how to re-focus on visual inputs, eliminating the need for explicit model-structure constraints or additional input. We demonstrate that Look-Back significantly enhances the model's reasoning and perception capabilities, as evidenced by extensive empirical evaluations on multiple multimodal benchmarks.
1 Introduction
MLLMs often lose visual attention during later reasoning, while existing methods typically restore it through explicit visual reinjection. Look-Back instead trains models to self-direct visual reflection, autonomously deciding when, where, and how to refocus without additional inputs or structural constraints.
- Later reasoning stages often rely predominantly on text, causing visual attention to diminish and visual information to exert negligible influence.
- Existing approaches commonly re-input images or reinject image tokens to restore visual information during reasoning.
- A simple prompt modification can spontaneously trigger later-stage visual refocusing without explicit inputs or model-structure constraints.
- Look-Back uses reflective <back> tokens and two-stage training: cold-start fine-tuning followed by reinforcement learning with a format reward.
- The model autonomously determines when to trigger <back>, where to attend, and how to enhance visual attention during reasoning.
- Extensive benchmark evaluations report consistent gains in multimodal reasoning and perception tasks.
2 Do MLLMs Know When and How to Reflect on Visual Input?
The paper investigates whether MLLMs can spontaneously recover visual attention without external intervention. Prompt-guided <back> generation increases and targets visual attention, improving benchmark performance, although prompting alone triggers reflection inconsistently.
- MLLMs often excessively rely on text late in inference, motivating investigation of spontaneous visual-attention recovery without external intervention.
- Table 1 compares CoT and Back prompts on Math-Benchmark and reports the percentage of responses spontaneously generating <back> as the trigger rate.
- Table 2 evaluates w/o back versus w/ back on triggered instances and reports their percentage performance increase as ∆Gain.
- A modified prompt encourages <back> generation, after which the model re-examines its response against visual information.
- The <back> sequence redirects substantial attention to image tokens and grounds reasoning in corresponding visual objects rather than producing only a general image glance.
- Only 62.48% of responses triggered reflection on average, showing that carefully tuned prompts alone do not consistently elicit the behavior.
3 Method of Look-Back
Look-Back combines supervised cold-start fine-tuning with reinforcement learning to guide MLLMs toward self-directed visual backtracking during reasoning. The method uses structured <back> outputs and a modified format reward to reinforce revisiting visual information without explicit image re-injection.
- 3 Method of Look-Back: Look-Back comprises supervised fine-tuning and reinforcement learning for guiding MLLMs to spontaneously re-focus on visual inputs during inference.The two-stage framework first establishes backtracking behavior, then reinforces it during policy optimization.
- 3 Method of Look-Back: Cold-start data distinguishes semantic-level backtracking during intermediate reasoning from solution-level backtracking after a preliminary solution.Semantic-back revisits visual details and continues reasoning, whereas Solution-back prompts comprehensive reconsideration of the visual input.
- 3 Method of Look-Back: The method uses explicit output formats in which <back> verification is inserted between reasoning segments before the final answer.Semantic-back resumes reasoning after verification; Solution-back adds a rethinking stage based on the reasoning and verification contents.
- 3 Method of Look-Back: Cold-start supervised fine-tuning trains the model on image-question samples containing backtracking and answer sequences to consistently trigger the behavior.Each sample is represented as (x, q, rback, a), with the backtracking tokens concatenated with the answer sequence for training.
- 3 Method of Look-Back: GRPO reinforces visual review by modifying only the format reward, while the complete reward combines format and accuracy rewards.The format reward favors the <back> format, and the total reward balances it with response accuracy through λ.
4 Look-Back Experiments Analysis
Look-Back experiments evaluate the method across mathematical and perceptual benchmarks, compare Semantic-back and Solution-back, and test the contributions of SFT, RL, reflection rate, and visual re-focusing. Results show consistent gains, with qualitative analyses linking them to repeated visual verification and grounding.
- Mathematical Reasoning: Look-Back improves mathematical performance by 7% with Semantic-back and 7.9% with Solution-back across five benchmarks.Semantic-back rises from 48.5% to 55.5%, while Solution-back rises from 48.5% to 56.4%.
- Perceptual Reasoning: Look-Back improves perceptual performance by 6.3% with Semantic-back and 6% with Solution-back across perceptual benchmarks.Semantic-back rises from 61.3% to 67.6%, while Solution-back rises from 61.3% to 67.3%.
- Effectiveness of Look-Back: Removing either SFT or RL causes significant performance degradation, while both back mechanisms outperform standard GRPO.The ablation evaluates the contribution of each training stage and compares Semantic-back and Solution-back against GRPO without look-back.
- Ablation of Reflection Rate: The optimal SFT reflection rate is 30%–50%, with both lower and higher rates reducing performance; the study adopts 50%.The reflection-rate ablation uses Semantic-back as an example and reports 50% as the selected rate.
- Qualitative Analysis: Look-Back repeatedly increases attention to visual input during later reasoning stages for both mathematical and perceptual tasks.The qualitative analysis contrasts Look-Back with standard GRPO and identifies repeated visual verification through attention spikes.
- Qualitative Analysis: Cases across five benchmarks show Semantic-back and Solution-back correcting initial errors by grounding reasoning in visual evidence.The analyses indicate that the mechanisms can determine when, where, and how to revisit visual information.
5 Further Discussion
The discussion reports failures involving model capability, cold-start data construction, and scaling trade-offs, while comparing two backtracking strategies. These findings delimit when Look-Back training and its variants work best.
- Failed Attempts: Qwen-2-VL exhibited reward hacking by generating empty <back></back> sequences that earned format rewards without genuine reasoning.The authors hypothesize that Qwen-2-VL lacks sufficient visual-reflection capability, unlike Qwen-2.5-VL.
- Failed Attempts: Cold-starting with GPT-4o-generated chain-of-thought data degraded performance, whereas model-generated data with refined <back> insertion improved it.The authors hypothesize that homologous model outputs reduce distributional deviation during cold-start fine-tuning.
- Scaling Cold-Start Data: Increasing cold-start data from 2.5k to 10k steadily improved mathematical scores, slightly reduced perception scores, and left the overall average almost flat.The experiment used mathematical Solution-back data at four scales: 2.5k, 5k, 7.5k, and 10k samples.
- Backtracking Variants: Semantic-back performs better on perceptual tasks, while Solution-back performs better on mathematical tasks across multiple benchmarks.The authors speculate that earlier backtracking confirms visual cues, whereas later backtracking supports more comprehensive verification.
6 Related Work
Related work traces multimodal reasoning from explicit modular methods through data-driven fine-tuning and reinforcement learning toward deeper visual reasoning. Look-Back differs by learning autonomous visual refocusing rather than explicitly injecting visual information.
- Early Development of Multimodal Reasoning: Early multimodal reasoning used explicit prompts, visual sampling, visual sketching, and multi-stage module cooperation.Examples include Visual-CoT, Visual-SketchPad, and Multimodal-CoT.
- Supervised Fine-Tuning and Test-Time Scaling: Large-scale synthetic chain-of-thought data and supervised fine-tuning shifted multimodal reasoning from module-based methods toward data-driven approaches.This period followed the emergence of models such as OpenAI O1 and included test-time scaling methods.
- Reinforcement Learning Breakthroughs: Reinforcement learning expanded multimodal reasoning, while fine-grained perception methods introduced dynamic visual search and visual-textual interaction.The cited examples include DeepSeek-R1, Vstar, DyFo, and DeepEyes.
- Look-Back: Unlike methods that explicitly inject visual information, Look-Back enables models to learn when and how to refocus on visual input autonomously.The approach aims to enhance reasoning without explicit visual guidance.
7 Conclusion
The paper observes that MLLMs can autonomously refocus on visual inputs without explicit injection and introduces Look-Back to train this behavior. Across multiple benchmarks, the approach significantly enhances multimodal reasoning capabilities.
- Conclusion: MLLMs can autonomously refocus attention on visual inputs during reasoning without explicit visual information injection.Look-Back builds on this observation through a two-stage process combining supervised fine-tuning and reinforcement learning.
- Conclusion: Look-Back achieves competitive results across multiple multimodal benchmarks and significantly enhances multimodal reasoning capabilities.
Dataset
The training dataset is sampled from four established multimodal reasoning datasets selected for distinct geometric and mathematical reasoning characteristics. Geo170K contributes over 170K geometric image-text pairs generated through systematic data construction.
- Dataset: The training dataset selectively samples four established multimodal reasoning datasets with distinct geometric and mathematical reasoning characteristics.
- Dataset: Geo170K contains over 170K geometric image-text pairs produced through systematic generation from existing datasets and text-generation models.Its examples cover diverse geometric reasoning scenarios requiring visual integration.
Hyper-parameters
Look-Back uses structured prompting and backtracking behaviors to make models verify visual information during reasoning. The supplied implementation also specifies SFT/RL prompt formats and training configurations.
- Training configuration: SFT uses full-parameter fine-tuning with a learning rate of 8 × 10−7 and cosine decay, while RL uses 15k samples and 12 rollouts per sample.The supplied training details cover both supervised and reinforcement-learning stages.
- Training configuration: RL freezes the vision tower and fine-tunes only language components, using an AdamW learning rate of 1 × 10−6.This configuration is intended to maintain visual feature stability.
- Prompt design: Look-Back prompts the model to revisit and verify visual information during reasoning without explicitly re-injecting images.The approach uses structured <back> segments to trigger visual verification and subsequent reasoning.
- Prompt design: Semantic-back inserts <back> during the reasoning chain, whereas Solution-back inserts it after an initial reasoning chain for comprehensive rethinking.Both variants use visual verification, but they differ in trigger timing.
- Prompt design: The templates enforce structured outputs containing reasoning, visual verification, continued reasoning, and a boxed final answer.The formats use <think>, <back>, and \boxed{} segments.
C Case Study
The case study shows both Look-Back variants correcting flawed intermediate reasoning by checking visual evidence and revising the final prediction. Semantic-back verifies during reasoning, while Solution-back verifies after a complete initial chain.
- Case-study setup: Five examples span MathVision, MME, HalluBench, MathVerse, and TallyQA, with Solution-back used in three and Semantic-back in two.The examples cover mathematical, perceptual, geometric, and visual-counting tasks.
- Effect of visual reflection: All five examples show an initial flawed deduction examined against the image inside <back>, followed by correction to the ground-truth answer.The corrective pattern is shared despite different verification timings.
- Effect of visual reflection: Visual verification corrects errors involving shaded-area counting, landmark identification, rotation direction, arc measures, and bird counting.These examples illustrate the range of visual cues used to revise intermediate reasoning.
D Training Dynamics
The supplied training-dynamics material combines a figure description with qualitative examples showing how backtracking verifies visual content. The examples include corrections in geometry, landmark recognition, directional sequences, and counting.
- Training dynamics: Figure 6 visualizes reward, accuracy, response length, and clip ratio for GRPO, Semantic-back, and Solution-back across training steps.The passage describes both training and validation reward curves.
- Solution-back examples: In the geometry example, visual verification changes the shaded-to-unshaded area ratio from 1 : 3 to the correct 2 : 3.The correction accounts for four quarter-circle regions and one full centre circle in the shaded area.
- Solution-back examples: In the landmark example, image verification changes the answer about the Church of Saint John the Baptist in Jihlava from no to yes.The corrected interpretation is based on the church’s visible simple tower and sloped roof.
- Solution-back examples: In the sequence example, visual checking reverses the initial anticlockwise judgment and identifies clockwise motion.The direction is inferred from the dots’ movement through successive positions.
- Semantic-back examples: In the TallyQA example, visual verification identifies one small bird on the ground near the steps.The observation corrects the prior response using a specific image detail.