Source-linked AI summary
VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning
Qi Wang, Yanrui Yu, Ye Yuan, Rui Mao, Tianfei Zhou
TL;DR
Video reasoning remains challenging because MLLMs often lack explicit reasoning and high-quality, visually grounded video CoT data. VideoRFT combines cognition-inspired CoT curation, cross-modal revision, two-stage RFT, and semantic-consistency rewards, and it consistently surpasses advanced MLLMs across six benchmarks. Its effectiveness may be affected by rapid motion or severe occlusion, and its CoT quality depends on the underlying reasoning model.
Problem
Video reasoning is difficult because it involves complex logic, temporal and causal structures, while existing video CoT datasets lack sufficient complexity and grounding.
Method
VideoRFT uses two-stage RFT with cognition-inspired, multi-expert CoT curation, cross-modal video revision, and a semantic-consistency reward for visual grounding.
Results
VideoRFT consistently surpasses a variety of advanced MLLMs across six video reasoning benchmarks.
Takeaways & Limitations
VideoRFT provides a foundation for future efforts in RFT-based video reasoning.
Takeaways & Limitations
Performance may be affected by rapid motion or severe visual occlusion, and generated CoT quality depends on the underlying reasoning language model.
Abstract
from arXiv · showhide
Reinforcement fine-tuning (RFT) has shown great promise in achieving humanlevel reasoning capabilities of Large Language Models (LLMs), and has recently been extended to MLLMs. Nevertheless, reasoning about videos, which is a fundamental aspect of human intelligence, remains a persistent challenge due to the complex logic, temporal and causal structures inherent in video data. To fill this gap, we propose VideoRFT, a novel approach that extends the RFT paradigm to cultivate human-like video reasoning capabilities in MLLMs. VideoRFT follows the standard two-stage scheme in RFT: supervised fine-tuning (SFT) with chain-of-thought (CoT) annotations, followed by reinforcement learning (RL) to improve generalization. A central challenge to achieve this in the video domain lies in the scarcity of large-scale, high-quality video CoT datasets. We address this by building a multi-expert-driven, cognition-inspired CoT curation pipeline. First, we devise a cognition-inspired prompting strategy to elicit a reasoning LLM to generate preliminary CoTs based solely on rich, structured, and literal representations of video content. Subsequently, these CoTs are revised by a MLLM conditioned on the actual video, ensuring visual consistency and reducing visual hallucinations. This pipeline results in two new datasets, i.e.VideoRFT-CoT-102K for SFT and VideoRFT-RL-310K for RL. To further strengthen the RL phase, we introduce a novel semantic-consistency reward that explicitly promotes the alignment between textual reasoning and visual evidence. This reward encourages the model to produce coherent, context-aware reasoning outputs grounded in visual input. Extensive experiments show that VideoRFT achieves state-of-the-art performance on six video reasoning benchmarks.
1 Introduction
Video reasoning remains difficult for MLLMs because current systems often provide answers without explicit reasoning and lack sufficiently complex, visually grounded video CoT data. VideoRFT addresses these gaps with a two-stage RFT framework, expert-curated CoT datasets, and a semantic-consistency reward.
- Video understanding requires temporal, causal, and explanatory reasoning beyond the perceptual understanding where most AI models remain limited.
- Video MLLMs are predominantly answer-driven, producing answers without explicitly revealing their reasoning processes.
- Existing video CoT datasets lack the complexity and granularity needed for advanced video reasoning, while grounding reasoning in visual evidence remains underexplored.
- VideoRFT uses a scalable pipeline in which structured video descriptions support blind LLM reasoning, followed by MLLM revision against the original video to reduce inconsistencies and hallucinations.
- A semantic-consistency reward aligns the video-description portion of reasoning traces with visual features while leaving question parsing and abstract reasoning unpenalized.
- VideoRFT favorably outperforms advanced competitors across six challenging video reasoning benchmarks.
2 VIDEORFT CoT Dataset
VideoRFT constructs large-scale video reasoning data through structured representations, cognition-inspired blind CoT generation, cross-modal revision, and factual filtering. The resulting datasets provide more elaborate, fine-grained reasoning data for training video MLLMs.
- 2.1 Data Collection: The data collection process produces 310K samples spanning diverse modalities, task types, cognitive skills, and answer formats.The answer formats include multiple-choice, numerical, free-form, OCR, and regression outputs.
- 2.1 Data Collection: The dataset covers general, mathematics, science, document, and spatiotemporal cognitive skills, including causal reasoning, symbolic logic, visual parsing, and motion prediction.
- 2.2 Cognitively Inspired CoT Generation: Each video is converted into a structured representation containing a high-level caption plus timestamped, frame-level visual metadata.The metadata includes objects, actions, scenes, spatial relations, and potential interactions.
- 2.2 Cognitively Inspired CoT Generation: A reasoning LLM generates initial step-by-step CoTs from the structured representation and question using five cognition-inspired prompting stages.The stages guide observation, task understanding, selective focus, visual reasoning, and reflective answering.
- 2.2 Cognitively Inspired CoT Generation: A cross-modal MLLM compares each initial CoT with the original video, identifies inconsistencies, and revises the reasoning while preserving its logical structure.
- 2.2 Cognitively Inspired CoT Generation: After filtering incorrect or semantically inconsistent samples, 102K high-confidence annotations remain for VideoRFT-CoT-102K.
- 2.3 Data Analysis: Compared with Video-R1, VideoRFT-CoT-102K has broader and longer CoTs dominated by dynamic, videocentric concepts, indicating more elaborate reasoning data.
3 Video Reinforcement Fine-Tuning
VideoRFT uses GRPO-based reinforcement fine-tuning with format, accuracy, and semantic-consistency rewards. The semantic-consistency reward selectively aligns the video-description portion of reasoning with sampled video frames while avoiding reinforcement of incorrect reasoning.
- 3.1 Group Relative Policy Optimization: GRPO generates K candidate responses, evaluates their rewards, and normalizes them to compute response advantages without a value model.This reduces memory requirements and training complexity compared with PPO.
- 3.2 Rule-Based Reward: Standard format and accuracy rewards are insufficient to explicitly guide visually grounded cross-modal reasoning.The semantic-consistency reward addresses this gap by targeting alignment between reasoning and visual evidence.
- 3.2.1 Semantic-Consistency Reward: The semantic-consistency reward targets the video-describing stage because it represents the model’s visual understanding underlying subsequent reasoning.A fixed-length text span is compared with uniformly sampled video frames using SigLIP representations.
- 3.2.1 Semantic-Consistency Reward: The reward is nonnegative and bounded, promoting visual semantic fidelity without penalizing abstract reasoning beyond the visual input.The formulation uses max(·, 0), min(·, 1), and scaling constant w = 2.
- 3.2 Rule-Based Reward: Rule-based rewards provide format supervision through tagged reasoning and answers, alongside task-specific accuracy supervision.The format reward checks <think>...</think> and <answer>...</answer> structure using regular-expression matching.
- 3.3 Overall Reward: The overall reward combines format and accuracy rewards, activating semantic consistency only when the accuracy reward is positive.The indicator gate prevents semantically plausible but factually incorrect reasoning from being reinforced.
4 Experiment
Experiments evaluate VIDEORFT across six video benchmarks and diagnose its training data, two-stage training paradigm, reward design, and reasoning behavior. VIDEORFT improves over its base model across all benchmarks and achieves the strongest overall performance among contemporaneous models.
- Experimental Setup: Average accuracy is measured on six benchmarks covering spatial reasoning, knowledge-intensive video QA, temporal logic, and general video understanding.The benchmarks are VSI-Bench, VideoMMMU, MMVU, MVBench, TempCompass, and VideoMME.
- Experimental Setup: The evaluation compares VIDEORFT with GPT-4o, open-source MLLMs, and contemporaneous video reasoning models.The comparison includes Qwen2.5-VL, VILA, LongVA, Video-R1, TinyLLaVA-Video-R1, and VideoChat-R1.
- Main Results: +5.0% on VSI-Bench, +7.2% on MMVU, and +7.0% on VideoMME are achieved over Qwen2.5-VL-7B.The improvements extend across all six evaluated benchmarks.
- Main Results: VIDEORFT surpasses GPT-4o by +2.8% on VSI-Bench and ranks first on four of six benchmarks against contemporaneous models.It also consistently outperforms non-RL open-source MLLMs.
- Diagnostic Experiments: Removing cross-modal CoT refinement causes -2.3% on VSI-Bench, -3.7% on MMVU, and -7.0% on VideoMME.The authors attribute these drops to errors and hallucinations in the unrevised initial CoTs.
- Diagnostic Experiments: Combining SFT and RL outperforms SFT-only and RL-only baselines across all datasets.RL-only exceeds SFT-only on four of six datasets, while the full RFT paradigm combines stable initialization with reasoning enhancement.
- Diagnostic Experiments: The gated semantic-consistency reward achieves the best overall results, including +2.2% on VSI-Bench, +3.3% on MMVU, and +3.5% on VideoMME.The reward is conditionally activated when the accuracy reward is positive.
- Diagnostic Experiments: Figure 6 illustrates reasoning traces in which VIDEORFT pauses to double-check its inference before answering.The authors describe this as an internal feedback loop for re-evaluating evidence and refining inference.
5 Related Work
Related work has advanced multimodal reasoning through chain-of-thought, staged video reasoning, symbolic memory, and textual visual representations. However, video CoT construction remains difficult because temporal complexity and visual ambiguity can produce hallucinations and weak semantic alignment.
- Multimodal Reasoning: Image and video reasoning methods commonly separate perception from reasoning or decompose reasoning into predefined stages.Examples include MMCoT, DDCoT, VoT, and STEP.
- Video Reasoning: DoraemonGPT uses symbolic memory and external tool sequences but retains a modular reasoning paradigm.
- CoT Data Construction: High-quality multimodal CoT construction is challenging because video data contain temporal complexity and visual ambiguity.
- CoT Data Construction: Converting visual inputs into textual descriptions before reasoning can cause hallucinations and weak semantic alignment.The passage cites LLaVA-CoT, Vision-R1, and R1-OneVision as examples of this strategy.
6 Conclusion
VIDEORFT extends reinforced fine-tuning to cognitive video reasoning in MLLMs through cross-modal CoT data construction and semantic-consistency-guided rewards. Across six benchmarks, it consistently surpasses advanced MLLMs and is presented as a foundation for future RFT-based video reasoning.
- Conclusion: VIDEORFT generates VideoRFT-CoT-102K and VideoRFT-RL-310K through a cross-modal pipeline designed to simulate human reasoning processes.
- Conclusion: A semantic-consistency-guided reward encourages alignment between reasoning traces and visual evidence during RL.
- Conclusion: VIDEORFT consistently surpasses advanced MLLMs across six video reasoning benchmarks.
A.1 Effect of Model Scaling on Reasoning Ability
Scaling VIDEORFT to a 3B model lowers performance relative to the 7B version but retains competitive benchmark performance. The smaller model also outperforms TinyLLaVA-Video-R1-3B, supporting competitiveness under resource constraints.
- Model Scaling: The 3B VIDEORFT variant performs lower than the original 7B version but remains competitive across benchmarks.
- Model Scaling: The 3B variant outperforms TinyLLaVA-Video-R1-3B, a lightweight contemporary model.
- Model Scaling: The authors interpret this margin as evidence of competitiveness and generalization under resource-constrained conditions.
A.2 Hyperparameter Analysis
The hyperparameter study evaluates the semantic-reward scaling factor w across multiple benchmarks. w=2 consistently performs best or near-best, while smaller and larger values slightly reduce performance.
- Table 4 evaluates how the semantic-reward scaling factor w affects performance across multiple benchmarks.
- w=2 consistently achieves the best or near-best results across all evaluated benchmarks.The study identifies w=2 as providing a favorable balance between semantic alignment and factual precision.
- Smaller w=1 and larger w=3 or w=4 values lead to slight performance declines.
A.3 Aha Moment in VIDEORFT
VIDEORFT exhibits an “Aha Moment” behavior in which the model pauses to scrutinize intermediate reasoning before making a final decision.
- Before reaching a final decision, VIDEORFT pauses to scrutinize its intermediate reasoning steps.The figure marks this reflective process in red as indicative of deliberative reasoning.
B Limitations and Future Directions
The paper identifies challenging video conditions and reliance on the underlying reasoning model as important boundaries for future work.
- Handling of Challenging Video Scenarios: Rapid motion and severe visual occlusion may reduce model effectiveness in complex video scenarios.Finer-grained visual signals and higher frame-rate sampling are suggested as possible mitigations.
- CoT Data Reliance: The quality of generated CoT annotations is closely tied to the capabilities of the underlying reasoning language model.Future work could use more advanced reasoning models to improve CoT quality and potentially VIDEORFT performance.
C Potential Social Impacts
VIDEORFT may benefit education, security, and content moderation, while also introducing risks involving misuse, misinformation, privacy, and ethical surveillance use.
- Positive Impacts: Enhanced video understanding can support intelligent tutoring systems that use video content.
- Positive Impacts: VIDEORFT can improve the efficiency and accuracy of surveillance video analysis.
- Positive Impacts: The system can aid content moderation by detecting and filtering inappropriate material.
- Negative Impacts: Misuse and over-reliance on automated video analysis may contribute to misinformation through misinterpretation.
- Negative Impacts: Surveillance deployment may raise privacy and ethical-use concerns.