Source-linked AI summary
More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models
Chengzhi Liu, Zhongxing Xu, Qingyue Wei, Juncheng Wu, James Zou, Xin Eric Wang, Yuyin Zhou, Sheng Liu
TL;DR
The paper studies why multimodal reasoning models produce more hallucinations in perception tasks as reasoning chains become longer. It introduces RH-AUC and RH-Bench to evaluate the balance between reasoning and visual grounding, finding that visual attention declines with longer chains and that balance depends on model scale and training-data characteristics.
Problem
Existing studies emphasize reasoning performance, while evidence about perception-focused hallucination and the changing balance between reasoning and visual grounding remains limited.
Method
The paper introduces RH-AUC, which evaluates reasoning-hallucination balance across reasoning lengths, and RH-Bench, a diagnostic benchmark spanning reasoning and perception tasks.
Results
Longer reasoning chains reduce visual attention and increase reliance on language priors, while larger models typically achieve better reasoning-hallucination balance and training-data types and domains matter more than volume.
Takeaways & Limitations
Evaluation should jointly assess reasoning quality and perceptual fidelity across varying reasoning lengths rather than rely only on fixed-length accuracy or hallucination metrics.
Takeaways & Limitations
Fixed-length accuracy and hallucination-rate metrics fail to capture dynamic balance because reasoning and perception can peak at different reasoning lengths.
Abstract
from arXiv · showhide
Test-time compute has empowered multimodal large language models to generate extended reasoning chains, yielding strong performance on tasks such as multimodal math reasoning. However, this improved reasoning ability often comes with increased hallucination: as generations become longer, models tend to drift away from image-grounded content and rely more heavily on language priors. Attention analysis shows that longer reasoning chains lead to reduced focus on visual inputs, which contributes to hallucination. To systematically study this phenomenon, we introduce RH-AUC, a metric that quantifies how a model's perception accuracy changes with reasoning length, allowing us to evaluate whether the model preserves visual grounding during reasoning. We also release RH-Bench, a diagnostic benchmark that spans a variety of multimodal tasks, designed to assess the trade-off between reasoning ability and hallucination. Our analysis reveals that (i) larger models typically achieve a better balance between reasoning and perception, and (ii) this balance is influenced more by the types and domains of training data than by its overall volume. These findings underscore the importance of evaluation frameworks that jointly consider both reasoning quality and perceptual fidelity.
1 Introduction
Multimodal reasoning models improve reasoning through extended chains but introduce more hallucinations in perception tasks, partly because visual attention declines as reasoning length increases. The paper proposes RH-AUC and RH-Bench to evaluate the changing balance between reasoning and visual grounding.
- Extended reasoning chains improve multimodal reasoning but make perception-task answers more dependent on language priors and more prone to hallucination.
- Reasoning models allocate less attention to visual tokens and more to instruction tokens, increasing reliance on language priors and hallucination risk.
- Longer reasoning chains further weaken visual attention and increase hallucinations as models rely more on language-based reasoning than visual evidence.
- The relationship between reasoning-chain length and reasoning-hallucination performance is non-monotonic, and the optimal reasoning range varies across tasks.
- RH-AUC measures the balance between reasoning and hallucination across reasoning lengths, while RH-Bench evaluates this balance across varied reasoning and perception tasks.
- Larger models typically balance reasoning and hallucination better, while training-data types and domains matter more than training-data volume.
2 Multimodal Reasoning Can Amplify Visual Hallucination
Across perception benchmarks, multimodal reasoning models hallucinate more than non-reasoning baselines, with this deficit persisting across model scales and training paradigms. The case studies distinguish visual misrecognition from reasoning bias as two hallucination patterns.
- 2.1 Hallucination Increases Consistently Compared to Base Models: Eight reasoning-augmented models show higher hallucination rates than their non-reasoning Qwen2.5-VL backbones across five perception datasets.
- 2.1 Hallucination Increases Consistently Compared to Base Models: The hallucination deficit persists at both 3B and 7B scales, indicating that it is associated with the reasoning paradigm rather than model size.
- 2.2 Does Training Paradigm Matter? Comparison Between RL and SFT+RL: Across four perception benchmarks, the Qwen2.5-VL baseline scores highest, RL-only follows, and SFT+RL performs worst.
- 2.3 Case Study of Hallucinations in Reasoning Models: Figure 4 identifies visual misrecognition and reasoning bias as two representative hallucination patterns in multimodal reasoning models.
- 2.3 Case Study of Hallucinations in Reasoning Models: Reasoning bias can cause models to overlook early visual evidence and generate incorrect responses by increasingly relying on linguistic priors.
3 Why Reasoning Models Amplify Hallucinations?
Attention analysis links amplified hallucination to reduced visual focus in multimodal reasoning models. Increasing reasoning length further shifts attention from visual tokens toward instruction tokens and linguistic priors.
- 3.1 Hallucination Resulting from Weak Visual Attention: Reasoning models consistently assign low attention to visual tokens, with attention declining in deeper layers while instruction-token attention increases.
- 3.1 Hallucination Resulting from Weak Visual Attention: Non-reasoning models maintain relatively high and stable visual attention and progressively focus on semantically salient image regions.
- 3.1 Hallucination Resulting from Weak Visual Attention: Reduced visual attention undermines effective visual grounding and exacerbates hallucinations in reasoning models.
- 3.2 Visual Focus Declines with Longer Reasoning Chains: As reasoning chains lengthen, attention to visual tokens decreases while attention to instruction tokens intensifies, increasing reliance on linguistic cues over visual evidence.
- 3.2 Visual Focus Declines with Longer Reasoning Chains: Longer reasoning chains further degrade visual grounding and may increase hallucinations.
4 Effects of Reasoning Length on Reasoning-Hallucination Balance
The paper controls reasoning-chain length with three strategies to study its dynamic trade-off between reasoning performance and hallucination. Across tasks, moderate lengths generally perform best, but optimal ranges vary, motivating evaluation beyond fixed-length metrics.
- 4.1 Overview of Reasoning Length Control Strategies: Three strategies control reasoning length: token budget forcing, test-time scaling, and latent state steering.Token budget forcing directly limits decoding tokens; test-time scaling extends reasoning incrementally; latent state steering adjusts hidden states using long–short reasoning directions.
- 4.1 Overview of Reasoning Length Control Strategies: Latent state steering extracts long–short hidden-state differences and applies them across decoder layers to vary reasoning-chain length.The scaling factor α controls guidance magnitude, with increasing α extending the reasoning chain.
- 4.2 Dynamic Balance Between Reasoning and Hallucination: Moderate reasoning depth typically maximizes performance, while overly short and excessively long chains reduce accuracy.Across controlled-length settings, performance follows a rising-then-falling trajectory rather than improving linearly with reasoning length.
- 4.2 Dynamic Balance Between Reasoning and Hallucination: Optimal reasoning length varies by task: MathVista tends to benefit from longer chains, whereas MMHalu performs best at shorter or moderate lengths.This task dependence means a unified reasoning-length setting cannot optimize the reasoning–perception balance across benchmarks.
- 4.2 Dynamic Balance Between Reasoning and Hallucination: Zerothink consistently lowers performance on both reasoning and perception benchmarks compared with normal reasoning lengths.Retaining the reasoning structure without substantive content is associated with reduced model performance.
- 4.2 Dynamic Balance Between Reasoning and Hallucination: Fixed-length accuracy and hallucination rate fail to capture the evolving reasoning–perception balance when the two peak at different lengths.The proposed AUC-style measure summarizes balance across reasoning lengths and reflects both optimal balance and stability.
5 Evaluation on the Reasoning-Hallucination Balance
The evaluation studies reasoning–hallucination balance across reasoning lengths using RH-AUC and RH-Bench. Results examine model scale, training paradigm, and training data, finding that larger models are generally more stable and that data characteristics matter for balance.
- 5.1 Setup: RH-Bench contains 1,000 samples spanning reasoning and perception tasks, with both multiple-choice and open-ended questions.The benchmark supports integrated evaluation of reasoning ability and perceptual hallucination.
- 5.1 Setup: RH-AUC measures the area under the reasoning–hallucination balance curve across evaluated reasoning lengths, with higher values indicating better balance.Reasoning and hallucination performance are computed at each length, then normalized and integrated using the trapezoidal rule.
- Model Scaling: 7B models generally achieve higher RH-AUC and more stable performance than 3B models, especially under longer reasoning chains.The analysis attributes this pattern primarily to larger parameter size and stronger representational capacity.
- Training Paradigms: RL-only training yields a stronger reasoning–perception balance than SFT+RL, partly because it produces more concise reasoning chains.The comparison associates longer, redundant reasoning with interference in visual perception.
- Training Dataset: Training-data diversity, quality, and domain can influence balance more reliably than data volume alone.The analysis reports strong balance from domain-specific mathematical data and notes that more visual perception data does not consistently improve balance.
6 Related Work
Related work covers multimodal reasoning benchmarks and reinforcement-learning approaches for improving multimodal model reasoning. These strands establish the task and training context for studying reasoning–hallucination balance.
- Multimodal Reasoning Tasks: Multimodal reasoning benchmarks span general and domain-specific tasks that require combining visual information with knowledge or commonsense.Representative datasets include MMMU, MMVP, MMBench, MMStar, MMEval-Pro, and VMCBench.
- Reinforcement Learning in MLLMs: Recent methods use chain-of-thought supervision, preference optimization, and reinforcement learning to enhance multimodal reasoning capabilities.Examples include RLHF-V, LLaVA-Reasoner, and Insight-V, with GRPO becoming a standard paradigm following DeepSeek-R1.
7 Conclusion
The paper concludes that longer reasoning can improve complex-task performance while worsening hallucination through reduced visual attention and greater reliance on language priors. It introduces RH-AUC and RH-Bench to evaluate reasoning quality together with perceptual accuracy, while noting important limits on generalizability and causal interpretation.
- Conclusion: Longer reasoning chains enhance complex-task performance but exacerbate hallucinations by diminishing visual attention and increasing reliance on language priors.The conclusion frames reasoning-chain length and visual attention allocation as central factors in the observed balance.
- Conclusion: RH-AUC and RH-Bench provide an evaluation framework for jointly assessing reasoning quality and perceptual accuracy.The paper presents them as responses to the need for integrated reasoning–hallucination evaluation.
- Limitations: The study evaluates only Qwen2.5-VL-based models and analyzes training-data effects from public documentation rather than controlled retraining experiments.Consequently, the findings may not generalize across architectures, and training-data conclusions remain observational rather than fully causal.