PaperlayerView exact revision on arXiv

Source-linked AI summary

Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning

Haoji Zhang, Xin Gu, Jiawen Li, Chixiang Ma, Sule Bai, Chubin Zhang, Bowen Zhang, Zhichao Zhou, Dongliang He, Yansong Tang

arXiv:2508.04416v2cs.CV

TL;DR

Long-video reasoning in MLLMs is limited by weak cross-modal interaction and increased hallucination in text-based CoT. VITAL uses adaptive visual tools, multimodal CoT, multi-task datasets, and DGRPO, achieving state-of-the-art results across video reasoning benchmarks, especially long-video tasks.

  • Problem

    Text-based CoT often lacks sufficient cross-modal interaction and suffers increased hallucination when MLLMs reason over long videos or long reasoning chains.

  • Method

    VITAL is an end-to-end agentic framework that adaptively samples video frames with a visual toolbox, forms multimodal CoT, trains on two multi-task datasets, and uses DGRPO.

  • Results

    VITAL-7B achieves state-of-the-art performance across 11 challenging benchmarks, including long-video question answering and temporal grounding.

  • Takeaways & Limitations

    Tool-augmented multimodal reasoning improves long-video understanding while mitigating hallucination and enhancing cross-modal interaction.

  • Takeaways & Limitations

    A reported long-video grounding case still produced an imperfect 83.2% IoU with tools, despite improving over the tool-free result.

Abstract

from arXiv · show

The video reasoning ability of multimodal large language models (MLLMs) is crucial for downstream tasks like video question answering and temporal grounding. While recent approaches have explored text-based chain-of-thought (CoT) reasoning for MLLMs, these methods often suffer from limited cross-modal interaction and increased hallucination, especially with longer videos or reasoning chains. To address these challenges, we propose Video Intelligence via Tool-Augmented Learning (VITAL), a novel end-to-end agentic video reasoning framework. With a visual toolbox, the model can densely sample new video frames on demand and generate multimodal CoT for precise long video reasoning. We observe that temporal grounding and question answering are mutually beneficial for video understanding tasks. Therefore, we construct two high-quality multi-task video reasoning datasets MTVR-CoT-72k for supervised fine-tuning and MTVR-RL-110k for reinforcement learning. Moreover, we propose a Difficulty-aware Group Relative Policy Optimization algorithm (DGRPO) to mitigate difficulty imbalance in multi-task reinforcement learning. Extensive experiments on 11 challenging video understanding benchmarks demonstrate the advanced reasoning ability of VITAL, outperforming existing methods in video question answering and temporal grounding tasks, especially in long video scenarios. Code is available at https://zhang9302002.github.io/thinkingwithvideos-page/.

1 Introduction

VITAL addresses the limited cross-modal interaction and hallucination of text-based CoT in long-video reasoning by integrating visual tools into multimodal CoT. It combines multi-task training and difficulty-aware reinforcement learning, with improved temporal grounding and video reasoning performance.

  • MLLMs’ text-based CoT often provides insufficient cross-modal interaction and increases hallucination for long videos or long reasoning chains.
  • VITAL enables models to sample relevant video content dynamically and incorporate visual tools throughout multimodal CoT reasoning.The framework is designed to connect textual reasoning with visual evidence during long-video understanding.
  • 90.7% IoU was achieved with tools versus 49.9% without tools for grounding the “Gregs Microscope book discussion” event.The tool-assisted prediction closely matched the 297.00–339.00-second ground-truth interval.
  • VITAL jointly targets temporal grounding, video question answering, and grounded question answering using multi-task reasoning data and reinforcement learning.The training resources include MTVR-CoT-72k for supervised fine-tuning and MTVR-RL-110k for reinforcement learning.
  • DGRPO is introduced to address difficulty imbalance in multi-task reinforcement learning.

2 Related Works

Prior work improves video understanding through reinforcement learning, visual tools, frame sampling, and long-context or token-compression methods. VITAL differs by combining adaptive visual-tool use with multimodal CoT for video reasoning.

  • Recent MLLM research applies GRPO-style reinforcement learning to video question answering and spatial or temporal grounding.
  • Long Video Understanding: Long-video methods use sparse sampled frames, dense tool-returned frames, token compression, context extension, or course-to-fine dynamic sampling.These approaches target the computational and temporal challenges of processing long videos.
  • VITAL’s framework adaptively invokes video tools during multi-round generation and integrates their outputs into multimodal CoT.
  • Unlike text-based CoT approaches, VITAL uses multimodal CoT to improve video understanding by incorporating visual evidence during reasoning.

3 Method

VITAL combines multi-round visual-tool interaction, multi-task video reasoning data, and difficulty-aware reinforcement learning. Its training pipeline targets efficient long-video reasoning while balancing task and sample difficulty.

  • 3.1 Tool-Augmented Learning Framework: VITAL uses an MLLM with a visual toolbox to decide whether to invoke tools during multi-round multimodal reasoning.Tool calls return new video results, forming a trajectory of reasoning steps, tool requests, and visual observations.
  • 3.1 Tool-Augmented Learning Framework: The video clipping tool receives time ranges and returns densely sampled frames from the requested segment.The method identifies video clipping as the most effective tested tool for temporal grounding and reasoning.
  • 3.2 Multi-Task Video Reasoning Training Data: VITAL constructs MTVR-CoT-72k and MTVR-RL-110k for supervised fine-tuning and reinforcement learning across temporal grounding, VQA, and grounded VQA.The datasets support basic reasoning and multi-round tool-augmented long-video reasoning through separate training subsets.
  • 3.2 Multi-Task Video Reasoning Training Data: Rollout filtering removes samples whose k rollouts all pass or all fail, retaining moderate-difficulty examples for a more balanced dataset.The pipeline then generates text-based CoT for all data and multimodal CoT for long-video data.
  • 3.3 Difficulty-aware GRPO Training: DGRPO combines accuracy, format, and tool rewards while scaling temporal-grounding accuracy and sample weights according to task and sample difficulty.Sample difficulty is averaged across G rollouts, with a soft linear weight ranging from 0.5 to 1.

4 Experiments

Experiments evaluate VITAL-7B across long-video, complex-reasoning, temporal-grounding, and grounded-VQA benchmarks, with ablations examining tools, DGRPO, and training-data composition. VITAL-7B achieves strong results, particularly for long-video understanding, while multimodal CoT and tool augmentation improve temporal localization and reduce hallucination.

  • Long Video Understanding: VITAL-7B outperforms previous best open-source models on LongVideo-Reason and VidChapters-7M, reaching 79.3% Acc and 34.7% R@0.5, respectively.The corresponding previous-best results are 67.9% Acc and 27.4% R@0.5.
  • Complex Video Reasoning: VITAL-7B consistently outperforms strong baselines on VSI-Bench, Video-MMMU, and MMVU across spatial reasoning and multi-discipline knowledge tasks.The evaluation uses complex multi-step video reasoning benchmarks.
  • Video Temporal Grounding and Grounded VQA: VITAL-7B excels on Charades-STA and ActivityNet-MR and sets new state-of-the-art results on NExT-GQA and ReXTime.These benchmarks assess short-video temporal grounding and grounded video question answering, respectively.
  • Ablation Study: Adding DGRPO raises the average score from 50.3 to 52.1 while preserving short-video perception ability and improving difficult long-video tasks.The ablation compares training stages with and without DGRPO.
  • Ablation Study: Tool-augmented reinforcement learning improves long-video perception and produces substantial gains across the evaluated benchmarks.The reported tool effects are supported by comparisons between rows 6 and 7 and delta results in Tables 2–4.
  • Ablation Study: Combining temporal grounding, reasoning VQA, and grounded VQA training data improves the average score from 37.9% to 57.1%.This combination yields the best overall performance in the training-data composition study.
  • Qualitative Analysis: Multimodal CoT uses visual evidence to determine temporal event boundaries more precisely than text-based CoT, which accumulates errors and hallucinations.A qualitative temporal-grounding comparison illustrates this difference.

5 Conclusion

The conclusion presents VITAL as a tool-augmented framework for long-video multimodal reasoning, alongside multi-task datasets and DGRPO. It emphasizes tool-augmented multimodal reasoning as important for long-video understanding.

  • Conclusion: VITAL combines tool augmentation with multimodal reasoning to improve long-video reasoning, cross-modal interaction, and hallucination mitigation.The framework is presented as an end-to-end approach for MLLMs.
  • Conclusion: The reported results highlight tool-augmented multimodal reasoning as a direction for future research in long-video understanding.This is the paper’s stated broader conclusion.

Supplementary Material

The supplementary material expands the paper’s methodology, experiments, datasets, ablations, and case analyses. It also introduces the supplementary methodology section’s focus on toolbox implementation and DGRPO.

  • Supplementary Overview: The supplement provides implementation, experiment, and dataset-construction details, followed by extended ablations, case analyses, and a limitations discussion.These sections are intended to support reproducibility and deeper evaluation.

A.1 Tool Implementation Details

The supplementary implementation describes VITAL’s visual tools, reward design, DGRPO difficulty balancing, and multi-stage training configuration. The system combines tool calls with accuracy, format, and tool rewards for multimodal video reasoning.

  • Tool Implementation Details: The visual toolbox includes video clip captioning and video clip QA tools that use timestamped segments to obtain additional evidence.The captioning tool returns a segment description, while the QA tool also receives a natural-language question.
  • Tool Implementation Details: Tool calls use Hermes-style JSON enclosed in tool-call tags, and tool responses can include JSON text or video-token sequences.Execution errors return an error message.
  • DGRPO: DGRPO addresses both task-wise and sample-wise difficulty imbalance through clamped transformations and a sample-dependent updating weight.Task settings vary by temporal-grounding difficulty, while sample difficulty is estimated from averaged rollout rewards.
  • DGRPO: DGRPO uses accuracy, format, and tool rewards, with accuracy metrics including IoU, EM, Rouge, WER, and L1.The format reward evaluates whether the response follows an allowed thinking, tool-call, and answer structure.
  • DGRPO: The format reward is 0.5 for an exact valid response structure, while the tool reward is 0.5 only after a successful tool call.For DGRPO without tools, the tool reward is removed and the format reward ranges from 0 to 1.
  • Training Configuration: Video inputs are sampled at FPS = 2, with short videos capped at 64 frames and long videos at 256 initial frames.Tool-result clips use a maximum of 64 frames and 224×224 pixels.
  • Training Configuration: Training uses two phases—post-training without tools and with tools—each containing cold-start SFT and RL stages.The tool-enabled stages train only on long-video datasets to enhance long-video understanding.

B.2 Evaluation Details

Evaluation uses VITAL-7B across eleven challenging video-understanding benchmarks under fixed decoding and response settings for reproducibility.

  • VITAL-7B is evaluated on eleven challenging video-understanding benchmarks.The evaluation details are provided in Table 11.
  • Decoding uses temperature 0.01, top p 0.001, maximum response length 1024, and at most two turns.These settings are held constant across evaluation experiments.

C.1 Dataset Statistics

The MTVR data-generation pipeline combines multimodal reasoning annotations, temporal-tool suggestions, quality filtering, and post-processing to produce datasets for supervised fine-tuning and reinforcement learning.

  • Dataset composition: The MTVR dataset contains MTVR-CoT (54k), MTVR-RL (94), MTVR-CoT-Tool (18k), and MTVR-RL-Tool (16k) subsets.The subsets span video reasoning data with varied video durations and token lengths.
  • Dataset composition: Video durations vary substantially across subsets, while question and response token lengths characterize textual complexity.Word clouds additionally visualize frequent concepts and linguistic patterns in each subset.
  • Quality filtering: Rollouts with reward range ΔR_i,j ≤ 0.05 are discarded because they provide limited learning signals for DGRPO.The filtering removes overly easy or overly hard samples whose rollouts are consistently successful or failed.
  • Annotation pipeline: Multimodal CoT annotations use three rounds: thinking, tool calling, and reflection with a concluded answer.Temporal-rounding samples receive predefined tool-parameter suggestions, while video question answering does not.
  • Quality filtering: Generated samples are excluded when reasoning is incomplete, answers mismatch ground truth, or content is irrelevant.After post-processing, MTVR-CoT-72k contains MTVR-CoT (54k) for stage-1 training and MTVR-CoT-Tool (18k) for stage-3 training.

D.1 Ablation Study on DGRPO Transformations

Ablations examine DGRPO transformation functions, tool-training initialization, resolution matching, and data scale to identify factors affecting training performance.

  • DGRPO transformations: Soft penalties outperform hard penalties and continuous transformations in the DGRPO transformation ablation.The authors attribute this to reward ranges that commonly vary from 1.0 to 2.0 after training progresses.
  • Cold start and tools: Cold-start SFT alone yields few improvements without tools, whereas adding tool training produces notable gains.Cold-start SFT benefits tool-augmented reinforcement learning, while directly applying tool-augmented DGRPO after two-stage training impedes tool-call reasoning.
  • Cold start and tools: VITAL-7B with tools significantly outperforms a resolution-matched improved baseline.The comparison controls the video-token budget by increasing the baseline’s maximum total pixels.
  • Data scale: Overall performance on long- and short-video benchmarks consistently improves as DGRPO training data size increases.Some metric fluctuations occur during training, but the aggregate trend remains positive.
  • Data quality: Representative MTVR samples provide precise temporal annotations, detailed event descriptions, and diverse real-world scenarios for short- and long-video reasoning.The analysis presents these properties as a foundation for robust and generalizable video-language reasoning.

E.2 Reasoning Quality Analysis

Reasoning-quality case studies compare VITAL-7B with and without multimodal CoT, emphasizing how visual evidence and temporal tools support long-video understanding.

  • Multimodal tool-augmented reasoning integrates visual and textual modalities with temporal-grounding and evidence-extraction tools.The case studies associate this integration with more precise event localization and deeper narrative understanding.
  • The case studies highlight advantages of multimodal tool-augmented reasoning for challenging long-video question answering and temporal reasoning.

F Limitations and Future Work

VITAL’s current toolbox is limited to temporal grounding and question answering, and the framework focuses mainly on visual features while omitting audio. Future work expands tool coverage, modalities, and adaptive tool selection for broader video understanding.

  • Limitations: The current toolbox supports temporal grounding and question answering but not spatial grounding.This limits VITAL’s ability to address tasks such as spatial grounding.
  • Limitations: VITAL mainly focuses on visual features and ignores audio information.The authors note that this may restrict the model’s overall video understanding.
  • Future Work: Future work will expand the toolbox and integrate visual and audio cues for broader video understanding tasks.Planned applications include spatio-temporal grounding, semantic segmentation, and complex-scene video analysis.
  • Future Work: The authors also plan more adaptive tool selection strategies to improve robustness and versatility in open-ended video reasoning.
Loading 2508.04416v2…