Source-linked AI summary
Does Seeing More Mean Knowing More? Mono-Anchored Advantage Normalization for Multi-Source Visual Reasoning
Fanhu Zeng, Zhicong Luo, Zefan Wang, You Li, Chi Chen, Maosong Sun
TL;DR
Multi-source visual reasoning can underperform mono-source reasoning because existing RLVR methods do not explicitly distinguish information gain from modality interference. MARS uses mono-source rewards as dynamic anchors for advantage normalization, achieving average gains of 3.2% with GRPO and 4.9% with DAPO across diverse visual reasoning datasets.
Problem
Existing multi-source RLVR methods often assume complementary information, leaving modality interference and relative information gain insufficiently modeled.
Method
MARS treats mono-source rewards as dynamic anchors, incorporating multi-source information gain into advantage estimation to promote useful interactions and suppress conflicts.
Results
3.2% average gain with GRPO and 4.9% with DAPO are achieved across visual reasoning tasks with diverse vision sources.
Takeaways & Limitations
The results support mono-source-anchored advantage estimation as a consistent approach for regulating multi-source interactions across datasets and training strategies.
Abstract
from arXiv · showhide
Visual reasoning through reinforcement learning with verifiable rewards (RLVR) has achieved remarkable progress. However, when dealing with multi-source inputs, existing approaches tend to treat them as a mere accumulation of information, lacking explicit mechanisms to distinguish whether integrating additional sources yields information gain or introduces interference. Therefore, they struggle to effectively model dynamic interaction when integrating multiple sources, particularly when they differ significantly in physical properties and semantics, e.g., infrared and depth, leading to inferior performance to mono-source reasoning when a certain source holds the dominant signal. To address this issue, we propose MARS, a novel mono-anchored multi-source reasoning framework that models each visual modality as an independent information source. Specifically, by treating mono-source rewards as dynamic anchors, our method explicitly incorporates the information gain introduced by multi-source fusion into advantage normalization and adaptively emphasizes mutual promotion between sources while suppressing potential noise or conflicts during RLVR. From theoretical analysis, our method effectively quantifies information gain introduced by multi-source integration in gradient estimation, enabling consistent modality regulation. Empirical results also show impressive 3.2% and 4.9% performance gains on GRPO and DAPO across diverse datasets, confirming effectiveness of our method.
1 Introduction
Existing multi-source visual reasoning often assumes that seeing more means knowing more, overlooking interference and sometimes underperforming strong mono-source reasoning. MARS addresses this by using mono-source rewards as anchors to quantify multi-source information gain and adapt source regulation, improving GRPO and DAPO performance by 3.2% and 4.9%.
- Problem: Existing methods often overuse complementary sources without explicitly modeling interference, allowing naive fusion to underperform strong mono-source reasoning.This degradation occurs when one source contains the dominant, reliable signal.
- Motivation: RGB-only reasoning or source relationships can be inadequate under illumination variation, occlusion, and adverse weather, while infrared, depth, and multi-view provide robust complementary information.Effective reasoning therefore requires comprehensive handling of multi-source data.
- MARS: MARS treats each visual modality as an individual information source and models the information gain introduced by multi-source integration.Its advantage computation compares multi-source rewards with mono-source rewards used as anchors.
- Results: 3.2% and 4.9% improvements were achieved on GRPO and DAPO, respectively, across depth, infrared, multi-view, and text-rich understanding tasks.The experiments support the method’s effectiveness and generalizability.
- MARS: Mono-source rewards enable quantitative information-gain measurement in advantage normalization and adaptive regulation of different sources during RLVR.The theoretical analysis identifies relative information gain over mono-source reasoning as central to effective integration.
2 Related Work
Related work spans reinforcement learning with verifiable rewards, visual reasoning, and multi-source visual reasoning. Prior studies develop optimization strategies, coordinate linguistic reasoning with perceptual states, and fuse information from multiple visual sources.
- Reinforcement Learning with Verifiable Rewards: Reinforcement Learning with Verifiable Rewards guides long reasoning chains using outcome reward signals and includes regularization, stabilized policy updates, and refined optimization strategies.Pioneering systems such as DeepSeek-R1 and Kimi demonstrate that complex reasoning patterns can emerge under verifiable-reward optimization.
- Visual Reasoning: Visual reasoning coordinates linguistic reasoning with perceptual states and performs strongly on grounding and image-understanding tasks.The paper focuses on visual reasoning with multi-source data for complex multimodal understanding and deduction.
- Multi-Source Visual Reasoning: Multi-source visual reasoning jointly understands images from different sensors, times, or viewpoints because a single source may be insufficient for complete and reliable decisions.Early studies emphasize explicitly fusing features extracted from different cameras to improve robustness.
3 Methodology
MARS addresses modality interference in multi-source visual reasoning by using mono-source rewards as dynamic anchors for advantage normalization. This stabilizes policy optimization while explicitly regulating the information gain or conflict introduced by source integration.
- Motivation: Multi-source reasoning can underperform mono-source reasoning when only one modality contains task-critical information, because models fail to focus on the informative scene.This issue is attributed to modality interference and the assumption that additional images are always complementary.
- Motivation: Mono-source rollouts provide stable inductive signals that guide optimization when the key image yields more consistent semantic rewards.The mono-source signal is especially useful when modalities conflict or differ in informativeness.
- Mono-Anchored Advantage Normalization: Hybrid normalization lowers the effective mean when multi-source rewards exceed mono-source rewards and inhibits contradictory updates when a modality is decisive.This enables the method to reinforce modality mutual promotion while suppressing interference.
- Mono-Anchored Advantage Normalization: The method generates mono-source rollouts and uses their rewards with multi-source rewards to normalize advantages for multi-source rollouts.Mono-source trajectories do not directly update the multi-source policy; they adjust normalization statistics as an adaptive reference.
- Theoretical Analysis: Theoretical analysis shows unbiased gradient estimation and decomposes optimization into standard multi-source reward maximization plus multi-source information-gain regulation.The information gain is ΔIG = mean(Gmulti)−mean(Gmono), with negative values indicating modality conflict and inferior multi-source reasoning.
- Theoretical Analysis: The mono-source anchor dynamically adjusts the gradient direction according to multi-source information gain, guiding optimization toward faster convergence and better multi-source performance.The analysis characterizes the procedure as weighted multi-source reward optimization with information-gain regularization.
4 Experiments
Experiments evaluate MARS across diverse multi-source visual reasoning datasets, training strategies, model scales, and robustness conditions. Results show consistent gains from mono-source-anchored advantage normalization, including 3.2% and 4.9% average improvements under GRPO and DAPO.
- Experimental setup: Experiments use RGB, depth, infrared, multi-view, and text-rich visual modalities across VQA and grounding tasks.Qwen2.5-VL-3B serves as the base model, with accuracy and mIoU as the respective evaluation metrics.
- Overall results: Union outperforms Multi across all evaluated tasks, demonstrating exploitable multi-source information gain, while higher Union performance does not necessarily imply better MARS results.Union denotes the best performance among mono-source results, whereas Multi jointly inputs all images.
- Generalizability across different model sizes: 4.4% average gain is obtained for the 3B model and 2.8% for the 7B model, showing consistent improvements across model scales.The experiments compare models with 3B and 7B parameters.
- Ablation and analysis: MARS raises average multi-source reasoning reward from 1.49 to 1.62, approaching the mono-source reward upper bound rather than increasing the best trajectory.The analysis compares reward statistics across algorithms.
- Robustness and case study: MARS focuses on an informative modality when another source is degraded, supporting robustness to noise, motion blur, illumination changes, and occlusion.Qualitative cases show improved infrared-based grounding and better use of depth information than vanilla GRPO.
5 Conclusion
The paper frames multi-source visual reasoning as a balance between modality promotion and conflict. It proposes using mono-source rewards as dynamic anchors to emphasize informative interactions and suppress conflicting ones during RLVR training.
- 5 Conclusion: Multi-source visual modalities with different physical properties and semantics can both promote and conflict with one another.Existing methods insufficiently model these opposing interaction characteristics.
- 5 Conclusion: The proposed framework adaptively emphasizes informative cross-source interactions while suppressing conflicting ones.Its design directly targets the dual promotion-and-conflict behavior of heterogeneous visual modalities.
- 5 Conclusion: Mono-source rewards serve as dynamic anchors for multi-source reasoning under RLVR training.The anchors provide the basis for adaptive regulation of interactions between visual sources.