Source-linked AI summary
Insight-V++: Towards Advanced Long-Chain Visual Reasoning with Multimodal Large Language Models
Yuhao Dong, Zuyan Liu, Shulin Tian, Yongming Rao, Ziwei Liu
TL;DR
MLLMs lack large-scale high-quality data and effective training strategies for long-chain visual reasoning, especially in video. The Insight-V series combines autonomous structured-data generation with reasoning and summary agents, while Insight-V++ adds ST-GRPO, J-GRPO, and closed-loop self-evolution. Extensive evaluations report stronger and more generalizable reasoning abilities across diverse benchmarks, including an 8.1% average gain across six visual reasoning benchmarks with LLaVA-NeXT.
Problem
MLLM long-chain visual reasoning is limited by scarce large-scale high-quality datasets and effective training strategies, with video adding spatial-temporal complexity.
Method
The Insight-V series autonomously generates and assesses structured reasoning data, separates reasoning from summarization, and Insight-V++ adds video data construction, ST-GRPO, J-GRPO, and self-evolving training.
Results
8.1% average performance improvement across six challenging visual reasoning benchmarks is reported for Insight-V integrated with LLaVA-NeXT, alongside effectiveness across diverse benchmarks.
Takeaways & Limitations
The Insight-V series provides a scalable multi-agent approach for improving MLLM visual reasoning across image and video tasks while retaining strong visual perception capabilities.
Takeaways & Limitations
Traditional DPO operates offline, so its preference data can diverge from the model’s evolving distribution and weaken training effectiveness.
Abstract
from arXiv · showhide
Large Language Models (LLMs) have achieved remarkable reliability and advanced capabilities through extended test-time reasoning. However, extending these capabilities to Multi-modal Large Language Models (MLLMs) remains a significant challenge due to a critical scarcity of high-quality, long-chain reasoning data and optimized training pipelines. To bridge this gap, we present a unified multi-agent visual reasoning framework that systematically evolves from our foundational image-centric model, Insight-V, into a generalized spatial-temporal architecture, Insight-V++. We first propose a scalable data generation pipeline equipped with multi-granularity assessment that autonomously synthesizes structured, complex reasoning trajectories across image and video domains without human intervention. Recognizing that directly supervising MLLMs with such intricate data yields sub-optimal results, we design a dual-agent architecture comprising a reasoning agent to execute extensive analytical chains, and a summary agent to critically evaluate and distill final outcomes. While our initial framework utilized Direct Preference Optimization (DPO), its off-policy nature fundamentally constrained reinforcement learning potential. To overcome these limitations, particularly for long-horizon video understanding, Insight-V++ introduces two novel algorithms, ST-GRPO and J-GRPO, which enhance spatial-temporal reasoning and improve evaluative robustness. Crucially, by leveraging reliable feedback from the summary agent, we guide an iterative reasoning path generation process, retraining the entire multi-agent system in a continuous, self-improving loop. Extensive experiments on base models like LLaVA-NeXT and Qwen2.5-VL demonstrate significant performance gains across challenging image and video reasoning benchmarks while preserving strong capabilities on traditional perception-focused tasks.
1 INTRODUCTION
MLLMs still lack scalable high-quality data and effective training for long-chain visual reasoning, especially in dynamic video settings. Insight-V and Insight-V++ address this with autonomous data generation, dual-agent reasoning and summarization, and progressively self-evolving training.
- MLLM long-chain visual reasoning remains challenging because large-scale high-quality datasets and effective training strategies are scarce.
- Video reasoning adds the need to track objects, understand action sequences, and maintain spatial-temporal coherence across many frames.
- Insight-V autonomously generates, assesses, and ranks structured long-chain reasoning paths, reducing reliance on human annotation.
- Insight-V decomposes problem solving between a reasoning agent that produces detailed chains and a summary agent that evaluates and answers from them.
- Insight-V++ extends the pipeline to video with ST-GRPO and J-GRPO, targeting spatial-temporal reasoning and summary-agent evaluation robustness.
- Insight-V++ feeds high-quality trajectories back into training to jointly re-optimize both agents without additional human-annotated data.
- 8.1% average performance improvement across six visual reasoning benchmarks is reported for Insight-V integrated with LLaVA-NeXT.
2 RELATED WORK
Related work spans vision-language reasoning, alignment optimization, and agentic visual systems. Prior approaches improve capabilities but face challenges from offline optimization, limited higher-order reasoning, and error compounding in single-agent workflows.
- Vision-Language Reasoning: Vision-language research advances fine-grained visual reasoning, localization, mathematics, and scientific problem-solving in MLLMs.
- Vision-Language Alignment: Conventional DPO is offline and can degrade as models evolve, while GRPO-based and hybrid supervised-reinforcement approaches target greater training stability and reasoning ability.
- Vision-Language Alignment: Recent vision-language reinforcement learning uses verifiable visual rewards, but purely RL-based training can struggle to induce higher-order reasoning.
- Agentic Visual Reasoning: Agentic visual systems orchestrate tools or iteratively search video segments, whereas single-agent reasoning is vulnerable to compounded intermediate errors.
3 METHOD
The method section presents Insight-V and Insight-V++ as a cohesive multi-agent system for detailed long-chain image and video reasoning while preserving visual perception capabilities.
- The proposed system is organized around three primary components for detailed long-chain image and video reasoning.
- These components form a cohesive system that addresses detailed reasoning while preserving visual perception capabilities.
3.1 Overview
Insight-V uses separate reasoning and summary MLLMs with structured training data, while Insight-V++ extends the pipeline to video and adds specialized reinforcement learning and self-evolution.
- Insight-V assigns detailed problem-solving to a reasoning model and relevance assessment to a summary model using structured, high-quality training data.
- Insight-V++ constructs high-quality video reasoning datasets with in-context scoring and introduces ST-GRPO and J-GRPO for its two agents.
- The extension develops a self-evolving system to make multi-agent reasoning more compact and adaptive.
3.2 Construction of Structured Reasoning Data
Insight-V constructs structured image and video reasoning data through progressive generation and multi-granularity quality assessment. The pipeline filters incorrect answers, scores reasoning paths, and uses in-context examples to improve video-data evaluation.
- The pipeline progressively generates structured long-chain reasoning data and applies multi-granularity assessment to improve its quality.For each query, reasoning is generated iteratively before assessment.
- Each reasoning step includes a summary, detailed response, and next action, enabling continuation or final summarization.The process samples structured responses across multiple reasoning steps.
- Incorrect final answers are filtered first, after which a scoring agent rates reasoning paths from 1 to 100 for stepwise accuracy and detail.Responses are aggregated by question and scored in a single pass.
- The resulting image dataset provides detailed reasoning for each question and supports model training.
- Video construction extends the image pipeline and uses golden video reasoning examples to improve the reliability of scored trajectories.Video reasoning is harder to assess because correctness and coherence must be verified across temporal contexts.
3.3 Model Design
Insight-V decomposes visual reasoning into collaborating reasoning and summary agents. The reasoning agent generates detailed chains, while the summary agent selectively evaluates potentially flawed chains to produce answers.
- The multi-agent framework trains a reasoning agent to generate detailed processes and a summary agent to use them selectively for answering questions.
- The reasoning agent addresses errors from combining long-chain reasoning and question answering in one process by producing specialized step-by-step reasoning.
- The summary agent is designed to evaluate reasoning paths critically, incorporating or disregarding elements when the reasoning contains errors.
- Its training data combines optimal and flawed reasoning processes to discourage copying and promote assessment of reasoning quality.
3.4 Multi-Agent Training Pipeline
The training pipeline initializes reasoning and summary agents from a base MLLM, then combines preference learning with role-specific GRPO objectives. Iterative preference-data generation and multi-agent reinforcement learning support structured reasoning, answer robustness, and stable optimization.
- Training stages: Supervised fine-tuning trains each agent for its designated role before preference optimization is applied to the reasoning model.The original Insight-V pipeline uses a two-stage strategy combining supervised fine-tuning and DPO.
- Agent initialization: A base MLLM is trained on image-text knowledge data, then used to initialize separate reasoning and summary agents.The base model is followed by role-specific fine-tuning for the two agents.
- Preference optimization: Iterative DPO refreshes preference data from successive models, reducing divergence between offline preferences and the model’s evolving distribution.Each subsequent model uses preference data generated by the preceding model.
- Reinforcement learning: GRPO estimates advantages from group-level reward statistics, avoiding a separate critic and assigning positive or negative advantages relative to group performance.The policy objective also clips importance ratios and applies KL regularization to stabilize updates.
- Role-specific GRPO: ST-GRPO and J-GRPO optimize the reasoning and summary agents with role-specific objectives, producing complementary strengths in the multi-agent system.ST-GRPO combines task, judge, answer, and format rewards, while J-GRPO trains summary evaluation and answer deduction using scored reasoning paths.
- Multi-agent interaction: The summary agent provides accurate judgments and final answers, strengthening the reasoning agent’s spatial-temporal logic through the dual-agent design.The reward design uses different objectives for spatial-temporal reasoning and summary robustness.
3.5 Multi-Agent Evolving System
Insight-V++ turns multi-agent collaboration into a self-evolving training strategy: a reasoning agent generates and refines trajectories, while a summary agent evaluates them and supplies feedback.
- Unlike earlier auxiliary-agent designs focused mainly on inference-time collaboration, Insight-V++ uses multi-agent interaction to enhance the agents’ fundamental reasoning capabilities.The distinction is between inference-time assistance and training-oriented capability improvement.
- Insight-V++ trains specialized reasoning and summary agents to generate reasoning paths, evaluate them, and provide reliable answers.The reasoning agent generates detailed trajectories; the summary agent assesses them and answers the question.
- The collaborative process iterates by conditioning each revised reasoning path on the previous output and the summary agent’s corrective feedback.The interaction continues until the summary agent accepts the path, with a maximum of three iterations for efficiency.
- The framework filters generated trajectories through its data-assessment pipeline, then uses them to fine-tune the reasoning agent and optimize the summary agent.This creates a continuous self-evolving loop from collaboratively generated data.
4 EXPERIMENTS
Experiments evaluate Insight-V and Insight-V++ across image reasoning, perception, advanced reasoning, video reasoning, and training-design ablations. The results show consistent gains across modalities, with stronger improvements on complex reasoning tasks and evidence for the multi-agent and self-evolving designs.
- Experimental setup: 10 benchmarks separately measure image reasoning and perception, using normalized group averages to assess reasoning depth and perceptual accuracy.The evaluation covers general image reasoning and perception, while advanced image reasoning adds mathematical, logical, and chart-based tasks.
- Advanced image reasoning: Insight-V++ achieves the highest average score across six advanced image reasoning benchmarks, including mathematical, logical, and chart-based assessments.The comparison is against other strong RL-based methods.
- General image reasoning and understanding: Insight-V raises LLaVA-NeXT’s average reasoning score from 46.8% to 54.9% and perception score from 65.5% to 72.3%.Reported gains include +8.0% on ChartQA, +14.3% on MMStar, and +7.8% on MMMU-Pro.
- General image reasoning and understanding: Insight-V++ raises Qwen2.5-VL’s average reasoning score from 66.3% to 71.1% while maintaining comparable perception performance.It also achieves 64.8% on MMMU and 45.6% on MMMU-Pro.
- Self-evolving design: +2.4% is the average self-evolving gain on complex reasoning benchmarks, compared with +1.6% on general image reasoning benchmarks.The reported difference indicates larger gains for tasks requiring deeper reasoning.
- Video reasoning: On video reasoning, reinforcement learning raises the average score from 47.3% to 52.9%, and self-evolving training adds 1.3%, reaching 54.2%.The results support transfer of reasoning-and-summarization decomposition from image to video.
- Further analysis: Ablations identify the multi-agent design as stronger than alternative configurations, while iterative DPO progressively improves reasoning performance.The ablations emphasize reasoning-and-summarization decomposition and iterative preference optimization.
5 CONCLUSION
Insight-V++ combines scalable long-chain reasoning data generation with multi-agent training, extends the framework to video through GRPO-based reinforcement learning, and introduces self-evolving agents. Extensive benchmark evaluations support stronger and more generalizable MLLM reasoning.
- Insight-V series integrates scalable long-chain reasoning data generation with an efficient multi-agent training framework for MLLMs.
- Insight-V++ adds GRPO-based reinforcement learning and extends the system to video reasoning tasks.
- The self-evolving architecture enables agents to cooperate and co-evolve, leading to continual performance improvements.
- Extensive evaluations across diverse benchmarks demonstrate the approach’s effectiveness and stronger, more generalizable reasoning abilities.