Source-linked AI summary
VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning
Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, Limin Wang
TL;DR
VideoChat-R1 addresses limited spatio-temporal reasoning in video MLLMs, especially long-range temporal associations, by integrating task-specific rewards into reinforcement fine-tuning. Joint multi-task RFT with GRPO yields state-of-the-art spatio-temporal perception while preserving general chat capabilities, and supports clue-driven answer refinement.
Problem
Video MLLMs need better reasoning for long-range temporal associations and fine-grained spatiotemporal understanding based on user queries.
Method
The paper applies GRPO-based reinforcement fine-tuning with spatio-temporal rewards across multiple video perception tasks and uses temporal clues for refined inference.
Results
+31.8 temporal grounding and +31.2 object tracking improvements are reported for VideoChat-R1 compared with Qwen2.5-VL-7B, with gains on general QA benchmarks.
Takeaways & Limitations
RFT can improve task-specific spatio-temporal perception with limited data while preserving general chat capabilities and supporting more reliable video dialogue.
Takeaways & Limitations
Existing training data is insufficient to activate truly effective video reasoning chains, and chain-of-thought shows no obvious advantage over direct output in the explored tasks.
Abstract
from arXiv · showhide
Reinforcement Learning (RL) benefits Large Language Models (LLMs) for complex reasoning. Inspired by this, we explore integrating spatio-temporal specific rewards into Multimodal Large Language Models (MLLMs) to address the unique challenges of video understanding, such as long-range temporal associations. This paper investigates how rule-based rewards, particularly temporal ones, can improve video reasoning and their generalizability. Our study proposes Reinforcement Fine-Tuning (RFT) as a data-efficient method to enhance video reasoning on specific tasks without sacrificing original capabilities. Through joint RFT on multiple spatio-temporal perception tasks, we developed VideoChat-R1, a powerful Video MLLM. VideoChat-R1 achieves state-of-the-art spatio-temporal perception, demonstrating significant improvements in tasks like temporal grounding (+31.8) and object tracking (+31.2), while also improving general QA benchmarks. The enhanced perception and preserved chat abilities contribute to a more reliable video dialogue system, leading to our ``Temporal Clue-driven Reasoning" inference schema. This work provides a foundation for developing robust, real-world video comprehension agents.
1 Introduction
The paper investigates spatio-temporal rewards for video MLLMs, targeting long-range temporal associations and fine-grained query-based understanding. It presents RFT and VideoChat-R1, which improve spatio-temporal perception while preserving general capabilities and supporting clue-driven video dialogue.
- Motivation: Video understanding requires reasoning for long-range temporal associations and fine-grained spatiotemporal understanding based on user queries.
- Research scope: The study maps how rule-based, especially temporal, rewards work in video understanding and evaluates their generalizability across video reasoning scenarios.
- Findings: RFT provides a data-efficient way to improve specific tasks with small datasets while negligibly affecting out-domain performance and original general capabilities.
- Findings: +31.8 temporal grounding and +31.2 object tracking improvements are achieved by VideoChat-R1 compared with Qwen2.5-VL-7B, alongside gains on general QA benchmarks.The reported general QA gains include VideoMME (+0.9), MVBench (+1.0), and Perception Test (+0.9).
- Applications: VideoChat-R1 combines enhanced spatio-temporal perception with preserved chat abilities to support a more reliable video dialogue system.Its Temporal Clue-driven Reasoning schema reuses reference video segments to obtain more accurate answers.
2 Related work
Related work extends reinforcement-learning reasoning techniques from LLMs to MLLMs through verifiable rewards. Video MLLM research addresses dialogue-based video tasks, while prior temporal methods face data or general-performance trade-offs.
- Reasoning in MLLMs: Recent LLM reasoning advances motivate extending verifiable reward mechanisms to multimodal large language models.
- Spatio-Temporal Understanding with MLLMs: Video MLLMs organize visual predictions through unified dialogue formats for tasks including video question answering and captioning.
- Spatio-Temporal Understanding with MLLMs: Prior temporal-modeling approaches use data augmentation or supervised preference optimization, with reported costs to general performance or substantial data requirements.
3 Methodology
The methodology applies GRPO to video MLLMs using task-specific rewards for formatting, temporal localization, classification, and captioning. These rewards are combined across tasks, then used in multi-task training and a clue-driven inference procedure.
- GRPO: GRPO generates groups of candidate responses, scores them with a predefined reward function, and prioritizes responses with higher relative scores without a critic model.A KL-divergence term constrains the optimized policy from diverging excessively from the original MLLM.
- Task coverage: The experiments cover temporal grounding, object tracking, video question answering, captioning, and quality assessment.
- Reward design: Format rewards use regular-expression matching to reward outputs that follow the required thought and answer tags.
- Reward design: IoU rewards measure the overlap between predicted and ground-truth time intervals or detection boxes for temporal grounding and object tracking.The reward characterizes the accuracy of the predicted interval.
- Reward design: Classification accuracy rewards compare the model’s predicted answer with the ground-truth answer for discriminative tasks.
- Reward design: Captioning uses an LLM judge to decompose predicted and ground-truth captions into events and reward the proportion of ground-truth events entailed by the prediction.Qwen2.5-72B is used to extract events and judge entailment.
- Training recipe: Task-specific combinations add format rewards to IoU, accuracy, grounding accuracy, or caption rewards, while training uses diverse public video datasets.
- Inference: Temporal Clue-driven Reasoning re-inputs clue-associated video segments at higher resolution and frame rate before generating a refined answer.The procedure first extracts temporal clues and selects segments from a low-resolution or low-frame-rate video.
4 Experiments
Experiments show that GRPO-based reinforcement fine-tuning improves spatio-temporal perception across tasks while largely preserving general video understanding and original capabilities. Multi-task training, task-specific reasoning behavior, and clue-based inference further support reliable video dialogue, although effective video chains of thought remain unresolved.
- Evaluation: GRPO-trained VideoChat-R1 models significantly outperform Qwen2.5-VL and single-task SFT models across spatio-temporal perception and VideoMME.The evaluation uses Qwen2.5-VL-7B as the main base model and includes general video benchmarks and task-specific tests.
- Multi-task Co-training: Mixed GRPO training across spatio-temporal tasks achieves nearly the best results across all benchmarks, indicating a synergistic multi-task effect.The co-training ablation evaluates collaborative training across multiple spatio-temporal perception tasks.
- Video Question Answering: Using just over 3,000 training samples, GRPO substantially improves NExTGQA and noticeably enhances VideoMME performance.Thinking is especially helpful for QA, while temporal clues assist relatively complex video understanding such as VideoMME.
- Clue-Perception: Clue-Perception enables significant gains on long-video benchmarks, whereas VideoChat-R1 without it shows no significant improvement over Qwen2.5-VL-7B.The training data consists entirely of short videos under one minute; Qwen2.5-VL-7B declines after clue-based processing.
- GRPO vs. SFT: GRPO outperforms SFT across in-domain tasks, out-domain tasks, and preservation of original general performance, while being less prone to overfitting in extended temporal-grounding training.Temporal grounding can continue improving with more epochs under GRPO, unlike the stronger overfitting tendency observed for SFT.
- Chain-of-thought vs. Direct Output: Chain-of-thought does not consistently improve spatio-temporal perception and can be inferior to direct output, leaving video reasoning tasks and evaluations open for further study.The authors attribute this limitation partly to insufficient training data for eliciting effective video reasoning chains.
5 Conclusions
RFT with GRPO is presented as a data-efficient approach for task-specific video improvements, enabling VideoChat-R1 to achieve state-of-the-art spatio-temporal perception while preserving general chat capabilities.
- RFT with GRPO enables task-specific improvements using limited training samples while preserving general chat capabilities.The paper characterizes RFT as data-efficient and reports preserved general chat performance.