Source-linked AI summary
SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learning
Yong Xien Chng, Tao Hu, Wenwen Tong, Xueheng Li, Jiandong Chen, Haojia Yu, Jiefan Lu, Hewei Guo, Hanming Deng, Chengjun Xie, Gao Huang, Dahua Lin, Lewei Lu
TL;DR
VLMs remain limited in coordinating continuous reasoning with dynamic tool use for knowledge-intensive and visually complex tasks. SenseNova-MARS addresses this gap with reinforcement-learned integration of image search, text search, and image cropping, plus BN-GSPO and HR-MMSearch. It achieves state-of-the-art results across search-oriented and fine-grained visual benchmarks, including 74.3 on MMSearch and 54.4 on HR-MMSearch.
Problem
VLMs remain constrained by text-oriented reasoning and isolated tool invocation when tasks require coordinated search, cropping, and continuous visual reasoning.
Method
SenseNova-MARS uses reinforcement learning to integrate image search, text search, and image cropping in multi-turn reasoning, with BN-GSPO for agent training and HR-MMSearch for evaluation.
Results
SenseNova-MARS achieves superior performance across diverse search-oriented and fine-grained visual benchmarks, including 74.3 on MMSearch and 54.4 on HR-MMSearch.
Takeaways & Limitations
The results support SenseNova-MARS as a robust agentic VLM with effective tool invocation and multimodal reasoning across complex visual tasks.
Takeaways & Limitations
SenseNova-MARS occasionally confuses semantically similar but distinct attributes in retrieved snippets, such as “based in” versus “born in.”
Abstract
from arXiv · showhide
While Vision-Language Models (VLMs) can solve complex tasks through agentic reasoning, their capabilities remain largely constrained to text-oriented chain-of-thought or isolated tool invocation. They fail to exhibit the human-like proficiency required to seamlessly interleave dynamic tool manipulation with continuous reasoning, particularly in knowledge-intensive and visually complex scenarios that demand coordinated external tools such as search and image cropping. In this work, we introduce SenseNova-MARS, a novel Multimodal Agentic Reasoning and Search framework that empowers VLMs with interleaved visual reasoning and tool-use capabilities via reinforcement learning (RL). Specifically, SenseNova-MARS dynamically integrates the image search, text search, and image crop tools to tackle fine-grained and knowledge-intensive visual understanding challenges. In the RL stage, we propose the Batch-Normalized Group Sequence Policy Optimization (BN-GSPO) algorithm to improve the training stability and advance the model's ability to invoke tools and reason effectively. To comprehensively evaluate the agentic VLMs on complex visual tasks, we introduce the HR-MMSearch benchmark, the first search-oriented benchmark composed of high-resolution images with knowledge-intensive and search-driven questions. Experiments demonstrate that SenseNova-MARS achieves state-of-the-art performance on open-source search and fine-grained image understanding benchmarks. Specifically, on search-oriented benchmarks, SenseNova-MARS-32B scores 74.3 on MMSearch and 54.4 on HR-MMSearch, surpassing proprietary models such as Gemini-3-Pro and GPT-5.2. SenseNova-MARS represents a promising step toward agentic VLMs by providing effective and robust tool-use capabilities. To facilitate further research in this field, we will release all code, models, and datasets.
1 Introduction
SenseNova-MARS addresses the need for VLMs that coordinate planning, reasoning, and multiple visual tools in challenging search-driven and high-resolution tasks. It combines reinforcement learning, a new benchmark, and adaptive tool use to achieve strong results across search and perception evaluations.
- Motivation: The work targets VLM limitations in text-centric reasoning and isolated tool calls by enabling interleaved planning, reasoning, and multi-tool execution.The motivation is the need for cohesive, adaptive, multi-step visual understanding.
- Approach: The framework integrates image search, text search, and image cropping into a dynamic, multi-turn reasoning process.It adaptively learns when and how to invoke these tools during iterative reasoning.
- Training: SenseNova-MARS is developed at 7B, 8B, and 32B scales, with training built around cold-start data, reinforcement learning, and BN-GSPO.The cold-start stage uses approximately 3,000 samples, while BN-GSPO improves stability for multi-tool rollout trajectories.
- Main results: SenseNova-MARS-32B achieves 74.3 on MMSearch and 54.4 on HR-MMSearch, while also reaching 94.2 on V* Bench and 90.2 on HR-Bench 4K.The model attains these results across search-oriented and high-resolution perception benchmarks.
- Benchmark: HR-MMSearch is introduced as a high-resolution, search-oriented benchmark for knowledge-intensive and search-driven visual questions.The benchmark is designed to evaluate agentic reasoning and multi-tool invocation on complex visual tasks.
2 Related Works
Related work has progressed from static retrieval toward tool-augmented and agentic visual reasoning, but existing systems still struggle to coordinate visual information acquisition with external knowledge retrieval. SenseNova-MARS builds on this trajectory through broader tool integration and multi-step coordination.
- Search-augmented reasoning: Search-augmented systems evolved from static RAG corpora toward live web access, enabling models to retrieve current information during reasoning.The progression includes systems such as WebGPT, Toolformer, and SAIL.
- Visual reasoning: The “Thinking with images” paradigm supports iterative interleaving of image and text reasoning, while end-to-end RL has improved fine-grained visual-task performance.DeepEyes is cited as an open-source implementation demonstrating this direction.
- Agentic VLMs: Existing multimodal agents combine hierarchical planning, reasoning, and external tools, but effective coordination of visual analysis and knowledge retrieval remains difficult.The cited work identifies limitations in collaborative tool invocation and coherent reasoning.
3 Method
SenseNova-MARS builds an agentic VLM that interleaves reasoning with text search, image search, and image cropping for high-resolution, knowledge-intensive visual tasks. Its two-stage training combines cold-start SFT with BN-GSPO reinforcement learning, while HR-MMSearch evaluates fine-grained search reasoning.
- Task and Objective: SenseNova-MARS extends multimodal search reasoning with image cropping for high-resolution tasks where important information requires selective zooming.The action space includes text search, reverse image search, image crop, and final answering.
- Two-Stage Training: The training recipe first uses cold-start SFT to bootstrap multi-tool invocation, then applies BN-GSPO reinforcement learning to refine tool use and reasoning.The SFT dataset contains curated multi-turn interaction trajectories, with target reasoning trajectories supervised through the cold-start objective.
- BN-GSPO for RL: BN-GSPO applies two-stage advantage normalization to reduce inconsistent learning signals caused by heterogeneous prompts, trajectory structures, reward scales, and difficulty levels.The method first standardizes rewards within groups and then normalizes them across the optimizer minibatch before applying a clipped sequence-level objective.
- Data Construction: The cold-start data pipeline filters source data, synthesizes tool-use trajectories for hard samples, and applies quality verification.Its source pool merges FVQA, Pixel-Reasoner warm-start data, and expert-annotated multimodal QA pairs; hard samples are identified using eight rollouts.
- HR-MMSearch Benchmark: HR-MMSearch targets a gap in existing benchmarks by using 305 4K-resolution images and knowledge-intensive questions across eight diverse domains.The benchmark emphasizes fine-grained perception and search reasoning, including visually small or inconspicuous subjects and text.
4 Experiments
SenseNova-MARS is evaluated across search-oriented and fine-grained visual benchmarks using direct-answer and agentic workflows, with results showing strong performance and adaptive tool use. Ablations examine BN-GSPO and training-data composition.
- 4.1 Implementation Details: The evaluation covers seven search-oriented benchmarks and three visual-understanding benchmarks with GPT-4o-based or exact-match metrics.Baselines include proprietary models and open-source agentic systems under direct-answer and agentic workflows.
- 4.2 Main Results: 69.74 average across seven search benchmarks makes SenseNova-MARS-32B superior to leading proprietary and open-source models.On HR-MMSearch, it scores 54.43, exceeding Gemini-3-Pro by 5.91 points and GPT-5.2 by 6.23 points.
- 4.2 Main Results: 94.2 on V* Bench and 90.2 on HR-Bench 4k demonstrate strong fine-grained visual understanding for SenseNova-MARS-32B.SenseNova-MARS-8B also leads several visual benchmarks and gains 4.9 points on average over Qwen3-VL-8B.
- 4.3 Ablation Study: BN-GSPO achieves the best overall performance across benchmarks and more balanced improvements than GRPO and GSPO in pure RL comparisons.The authors attribute this stability to batch normalization mitigating reward-scale variance.
- 4.3 Ablation Study: Hybrid search-and-perception RL data outperforms specialized data, which overspecializes the agent and substantially harms search-oriented performance.Specialized fine-grained training raises V* Bench to 85.24 but drops search performance relative to the SFT baseline.
- 4.4 Analysis of Tool Use Behavior: SenseNova-MARS-8B uses mostly image and text search on MMSearch but balances search and cropping on the more demanding HR-MMSearch benchmark.This distribution reflects task-dependent adaptation of multimodal tool use.
5 Conclusion
The paper presents SenseNova-MARS as a multimodal agentic framework that combines tool use with multi-turn reasoning, supported by BN-GSPO and the HR-MMSearch benchmark. Experiments report superior performance, efficient tool invocation, and robust reasoning across diverse benchmarks.
- 5 Conclusion: SenseNova-MARS integrates image search, text search, and image cropping into multi-turn reasoning for search-driven and fine-grained visual tasks.The framework uses BN-GSPO to improve reasoning robustness and tool-use proficiency, while HR-MMSearch evaluates high-resolution knowledge-intensive questions.
- 5 Conclusion: Experiments report superior performance across diverse benchmarks together with efficient tool invocation and robust reasoning capabilities.
A Additional Details on Training Data
This section introduces the additional details covering training data for both cold-start supervised fine-tuning and reinforcement learning.
- A Additional Details on Training Data: The additional training-data details cover both the cold-start SFT phase and the subsequent RL phase.
A.1 Cold-Start SFT Data
The cold-start SFT dataset is built through filtering, synthesis, validation, and expert construction to support tool use and visual reasoning. It contains approximately 3,315 high-quality samples.
- A.1 Cold-Start SFT Data: Approximately 3,315 high-quality samples are retained for cold-start SFT from filtered and curated visual-reasoning sources.The sources include 1,115 FVQA trajectories, approximately 2,000 Pixel-Reasoner samples, and 200 manually constructed expert trajectories.
A.2 Reinforcement Learning Data
The RL training set combines diverse visual reasoning data, including residual FVQA, high-resolution analysis, and broad visual reasoning samples.
- A.2 Reinforcement Learning Data: RL uses a larger and more diverse dataset to generalize the model’s reasoning and tool-use policies.
- A.2 Reinforcement Learning Data: The RL training set comprises 3,695 residual FVQA samples, 4,000 DeepEyes-4K samples, and 5,729 Visual-Probe samples.These datasets support factual visual reasoning, high-resolution analysis, and broad visual reasoning, respectively.
B.1 Agentic Search
The evaluation covers agentic search and high-resolution visual reasoning through diverse benchmarks, while the text-search pipeline uses local retrieval for training and live web search for inference.
- B.1 Agentic Search: The evaluation spans agentic search benchmarks including HR-MMSearch, MMSearch, FVQA-test, InfoSeek, SimpleVQA, LiveVQA, and MAT-Search.These benchmarks cover knowledge retrieval, factual reasoning, multimodal multi-hop reasoning, and changing news.
- B.1 Agentic Search: HR-MMSearch targets fine-grained perception and search reasoning with high-resolution images, small visual subjects, and questions requiring image search, text search, or image cropping.The benchmark contains 305 image–question pairs cross-verified by three experts.
- B.1 Agentic Search: The text-search pipeline uses local Wikipedia retrieval during RL training and live web search through Serper during inference.Retrieved passages from both modes are uniformly summarized by Qwen3-32B before reaching the main model.
- B.1 Agentic Search: Visual reasoning evaluation includes V* Bench, HR-Bench, and MME-RealWorld, covering high-resolution details, ultra-high-resolution inputs, and challenging real-world scenes.MME-RealWorld contains 23,599 QA samples across 43 subtasks and five domains.
- B.1 Agentic Search: Models are compared under Direct Answer, RAG, and Agentic workflows, whose prompts control the tools available during inference.
C.4 Evaluation Metrics
Evaluation metrics are selected according to output type: semantic judging for open-ended search answers and exact matching for closed-ended visual reasoning tasks.
- C.4 Evaluation Metrics: Agentic search uses GPT-4o as an LLM judge to score Pass@1 accuracy by comparing open-ended responses with ground truth.All evaluations use a sampling temperature of 0.0.
- C.4 Evaluation Metrics: Visual reasoning benchmarks use Exact Match, with Avg@8 reported for V* Bench and HR-Bench and Pass@1 for MME-RealWorld.Avg@8 averages Exact Match across eight independent attempts per question.
D Additional Exprimental Analysis
SenseNova-MARS learns benchmark-adaptive tool use and becomes more efficient during RL training by reducing redundant tool calls.
- D Additional Exprimental Analysis: RL reduces SenseNova-MARS-7B’s average tool calls from approximately 4 to approximately 2, eliminating superfluous actions.The analysis attributes this streamlining to the RL training process.
- D Additional Exprimental Analysis: SenseNova-MARS-7B uses image cropping for V* Bench, search tools for MMSearch, and a hybrid strategy for HR-MMSearch.These patterns adapt tool selection to the demands of each benchmark.
- D Additional Exprimental Analysis: The base Qwen2.5-VL-7B relies almost exclusively on text search and ignores image cropping, limiting its fine-grained perception ability.
F Limitations
The error analysis identifies retrieval noise and ineffective tool usage as concrete limitations of SenseNova-MARS. These issues affect attribute disambiguation and the grounding of fine-grained visual entities in search queries.
- SenseNova-MARS occasionally confuses semantically similar attributes in retrieved snippets, such as “based in” and “born in.”This vulnerability is illustrated in Fig. 14.
- Retrieval noise can leave the reasoning module vulnerable to distractor information, producing hallucinated reasoning paths.
- SenseNova-MARS may fail to ground specific visual entities in search queries, replacing them with generic terms and retrieving irrelevant global statistics.The analysis attributes this issue to a gap in cross-modal alignment during tool-parameter generation.