Source-linked AI summary
REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understanding
Jiaze Li, Hao Yin, Wenhui Tan, Jingyang Chen, Boshen Xu, Yuxun Qu, Yijing Chen, Jianzhong Ju, Zhenbo Luo, Jian Luan
TL;DR
Text-only reflection has limitations in long-form video understanding, where dynamic visual input and missing cross-modal interaction make textual re-evaluation insufficient. REVISOR revisits relevant video segments during reflection and uses DADR in GRPO to align reasoning with selected evidence. Across four benchmarks, it improves base-model average accuracy by about 2%.
Problem
Text-only reflection has limitations in long-form video understanding because dynamic visual input requires visual re-evaluation and text-only methods lack cross-modal interaction.
Source: §1 · p. 1Method
Results
Across VideoMME, LongVideoBench, MLVU, and LVBench, REVISOR improves the base model’s average accuracy by about 2%.
Source: §1 · p. 2Takeaways & Limitations
REVISOR enhances MLLM video understanding without supplementary supervised fine-tuning or external models.
Source: §1 · p. 2With final-answer verification as the sole GRPO reward, REVISOR’s performance decreases on long-form video tasks compared with the base model, as segment choices receive insufficient feedback.
Source: §3.2 · p. 5
Abstract
from arXiv · showhide
Self-reflection mechanisms that rely on purely text-based rethinking processes perform well in most multimodal tasks. However, when directly applied to long-form video understanding scenarios, they exhibit clear limitations. The fundamental reasons for this lie in two points: (1)long-form video understanding involves richer and more dynamic visual input, meaning rethinking only the text information is insufficient and necessitates a further rethinking process specifically targeting visual information; (2) purely text-based reflection mechanisms lack cross-modal interaction capabilities, preventing them from fully integrating visual information during reflection. Motivated by these insights, we propose REVISOR (REflective VIsual Segment Oriented Reasoning), a novel framework for tool-augmented multimodal reflection. REVISOR enables MLLMs to collaboratively construct introspective reflection processes across textual and visual modalities, significantly enhancing their reasoning capability for long-form video understanding. To ensure that REVISOR can learn to accurately review video segments highly relevant to the question during reinforcement learning, we designed the Dual Attribution Decoupled Reward (DADR) mechanism. Integrated into the GRPO training strategy, this mechanism enforces causal alignment between the model's reasoning and the selected video evidence. Notably, the REVISOR framework significantly enhances long-form video understanding capability of MLLMs without requiring supplementary supervised fine-tuning or external models, achieving impressive results on four benchmarks including VideoMME, LongVideoBench, MLVU, and LVBench.
1. Introduction
Text-only reflection has clear limitations in long-form video understanding, where dynamic visual input and missing cross-modal interaction make textual re-evaluation insufficient. REVISOR adds visual review to reflection and uses DADR to guide selection of relevant video evidence; across four benchmarks, it improves base-model average accuracy by about 2%.
1. Introduction
- Text-based reflection performs well on general multimodal tasks but shows limitations in long-form video, where dynamic visual input calls for visual re-evaluation and text-only methods lack cross-modal interaction.These limitations restrict MLLMs’ reasoning improvement potential on video understanding tasks.Source: §1 · p. 1
- REVISOR first identifies segments for further examination, then integrates the initial reasoning trace with newly sampled, fine-grained visual evidence during reflective reasoning.This two-stage process is designed to yield a refined final prediction.Source: §1 · p. 2, Figure 1 · p. 2
- DADR adds a Causal Segment Sufficiency Reward to GRPO’s final-answer verification reward, aligning reasoning with reviewed segments and discouraging reliance on irrelevant or spurious visual cues.The reward encourages correct answers to be derived from the selected segments.Source: §1 · p. 2
- Across VideoMME, LongVideoBench, MLVU, and LVBench, REVISOR improves the base model’s average accuracy by about 2%.The framework does so without supplementary supervised fine-tuning or external models.Source: §1 · p. 2
2. Motivation
The motivation examines why text-only reflection struggles with long-form video and tests whether revisiting visual evidence during reflection improves accuracy. Its experiments report gains from visual rethinking, while text-only reflection yields negligible improvement.
2. Motivation
- The motivation presents text-only re-evaluation as inadequate for long-form video and preliminary experiments as evidence that visual re-evaluation significantly enhances MLLM accuracy.Long-form videos contain richer and more dynamic visual information than static images.Source: §2 · p. 3
2.1. Dilemma of Text-Based Reflection
- VL-Rethinker improves image-understanding benchmarks, but text-only reflection declines on long-form video, and a video-trained text-only control still fails to improve.The proposed explanation is that dynamic video visuals require re-evaluation and text-only reflection cannot fully integrate visual cues.Source: §2.1 · p. 3
2.2. Necessity of Integrating Visual Rethinking
- Approximately 7.3% average accuracy gain came from visual rethinking on NExT-GQA, ReXTime, and CG-Bench, while text-only reflection produced negligible improvement.The experiment supplied annotated key segments after initial reasoning for models to reconsider alongside their prior analysis.Source: §2.2 · pp. 3–4
3. Methodology
REVISOR is a two-stage framework for long-form video understanding that combines initial inference with focused visual review and reflective reasoning. DADR adds a reward for whether reviewed segments suffice to derive the correct answer.
3.1. Tool-Augmented Multimodal Reflection
- Stage 1 uses sparse, uniform video frames for initial reasoning and asks the MLLM to propose a relevant or uncertain temporal segment for review.The proposed segment is specified by start and end timestamps.Source: §3.1 · p. 4
- The visual toolbox densely resamples frames only within the proposed segment, enabling focused examination without processing the entire video at full resolution.The denser sampling uses a higher temporal resolution than the initial sparse sampling.Source: §3.1 · p. 4
- Stage 2 reflects on the question, initial reasoning, and reviewed visual evidence to refine the reasoning trace and final answer.The model can validate hypotheses, resolve ambiguities, or correct prior misinterpretations.Source: §3.1 · p. 4
3.2. Dual Attribution Decoupled Reward
- With final-answer verification as the sole GRPO reward, REVISOR training decreases performance versus the base model and gives insufficient feedback on segment selection.The imbalance means correct segments receive too little positive feedback and incorrect segment predictions too little penalty.Source: §3.2 · p. 5
- DADR combines final-answer verification with a causal segment-sufficiency reward, rewarding a reviewed segment when it alone supports the correct answer.The two rewards are weighted; the segment reward is based on an attribution answer derived from the question and densely sampled evidence from the proposed segment.Source: §3.2 · p. 5
4. Experiments
REVISOR is evaluated on four long-form video benchmarks and temporal grounding, followed by ablations of its reward mechanism and training-data composition. Results cover its performance against baselines and the effects of key training choices.
4.1. Experimental Setup
- Experiments use Qwen2.5-VL-7B, a single-stage reinforcement-learning phase following DAPO, and 25K samples from eight video datasets.Training and evaluation inputs are limited to 8192 video tokens; the optimizer is AdamW with a 1 × 10^-6 learning rate and batch size 32.Source: §4.1 · p. 6
4.2. Main Results
- REVISOR scores 65.7% on VideoMME, outperforming Qwen2.5-VL-7B by 1.4%, and gains 2.8% on VideoMME’s long subset and 2.5% on MLVU.Against Video-R1, it improves VideoMME by 4.3%; against VL-Rethinker and a video-data-trained text-only reflection model, gains are 3.6% and 2.3%, respectively.Source: §4.2 · pp. 6–7
- On Charades-STA, REVISOR reaches 51.4% mIoU, surpassing iMOVE by 4.1% mIoU and TVG-R1 by 4.7% mIoU; it exceeds TVG-R1 by 3.9% mIoU on NExT-GQA.Table 2 reports temporal grounding results for Charades-STA and NExT-GQA.Source: §4.2 · p. 7, Table 2 · p. 7
4.3. Ablation Study
- With λ2 set to 0, REVISOR’s VideoMME score falls from 65.7% to 62.2%, below the base model’s 64.3%, suggesting CSSR helps learn correct review-segment selection.Table 3 presents the DADR ablation.Source: §4.3 · p. 7, Table 3 · p. 7
- When λ2 exceeds λ1, temporal grounding improves but MLVU drops from 69.8% to 68.7%; the authors set λ1 above λ2 to balance segment selection and answer reasoning.The authors attribute the decline to over-focusing on locating the review segment rather than using it to derive the answer.Source: §4.3 · p. 7
- Adding more diverse training datasets raises MLVU performance from 66.7% to 69.8%; training only on short-video QA data raises NExT-GQA mIoU from 20.9% to 25%.Table 4 ablates training-data composition.Source: §4.3 · p. 7, Table 4 · p. 7
5. Extended Analysis
The extended analysis examines visual versus textual reflection and whether DADR improves retrieval of relevant video segments. It reports that longer textual reflection can hurt performance, while DADR-trained models retrieve more accurate segments.
5.1. Importance of Visual Rethinking
- During training, textual reflection length steadily decreases while reviewed-video length first rises and then falls, as the model learns to discard redundant segments.The authors interpret the refinement as locating minimal segments sufficient to answer the question.Source: §5.1 · p. 8
- Forcing longer textual reflection decreases video-understanding performance, supporting the authors’ conclusion that more textual deliberation is not critical for these tasks.Figure 4 contrasts the decline in reflection length during training with the performance degradation caused by prompting longer reflection.Source: §5.1 · p. 8, Figure 4 · p. 8
5.2. Accuracy of Visual Information Retrieval
- The retrieval test gives extracted segments to the same Qwen2.5-VL-7B model, so answer accuracy indirectly measures how relevant those segments are to the questions.The model architecture and parameters are held identical across conditions.Source: §5.2 · p. 8
- DADR-trained models generally revisit more accurate segments and outperform four temporal-grounding methods in relevant-segment retrieval accuracy.Figure 5 compares REVISOR variants with temporal-grounding baselines.Source: §5.2 · p. 8, Figure 5 · p. 8
6. Conclusion
REVISOR addresses the limits of text-based reflection in long-form video by revisiting essential visual content. Across four benchmarks, it improves long-form video understanding without supplementary supervised fine-tuning or external models.
6. Conclusion
- REVISOR revisits essential visual content to correct errors arising in initial reasoning.It is a tool-augmented multimodal reflection framework.Source: §6 · p. 8
- The framework incorporates DADR into GRPO to localize video segments most relevant to each query.DADR is used to train REVISOR to identify those segments precisely.Source: §6 · p. 8
- Across four long-form video benchmarks, REVISOR delivers significant performance gains without supplementary supervised fine-tuning or external models.The benchmarks named are VideoMME, LongVideoBench, MLVU, and LVBench.Source: §6 · p. 8
7. Related Work
Prior long-form video methods select question-relevant visual content through external models, agentic search, or model-internal selection. REVISOR instead integrates visual review with ongoing reasoning, extending reflection beyond text-only re-evaluation.
7. Related Work
- Long-form video methods address redundant visual content through external-model augmentation, agentic methods, or model-internalized selection.The goal is to identify and distill question-relevant information from complex visual inputs.Source: §7 · p. 12
- External-model methods use static one-shot selection, agentic approaches refine selection iteratively, and model-internalized methods embed selection in reasoning.VideoAgent uses a state–action–observation loop, VideoTree performs coarse-to-fine search, and GenS trains a sampler while keeping selection and QA decoupled.Source: §7 · p. 12
- Self-reflection work spans iterative feedback, multi-round reinforcement learning, and reflective prompt evolution [16] [1].In multimodal settings, VL-Rethinker and SRPO use self-verification or self-correction mechanisms [42] [41].Source: §7 · p. 12
- Existing multimodal reflection remains text-centric, while REVISOR adds explicit reflection over visual information.After reinforcement learning, REVISOR autonomously explores visual content needing review and uses ongoing reasoning to pinpoint critical information.Source: §7 · p. 12
8. More Experimental Details on REVISOR
The supplementary details describe REVISOR’s training setup and data, temporal-grounding results, reward ablations, and the balance between localization and answer reasoning. They also include a case in which visual review corrects an emotion judgment.
8.3. Supplementary Experimental Results
- On Charades-STA, REVISOR reaches 76.5% R@0.3 accuracy, 4.8% above iMOVE; on NExT-GQA, it reaches 25.5% R@0.5 accuracy, 4.3% above TVG-R1.The reported evaluation is temporal video grounding.Source: Table 5 · p. 13, §8.3 · p. 13
- With λ2 = 0, Video-MME performance falls from 65.7% to 62.2%, below the base model, as REVISOR struggles to identify the correct review segment.This setting applies only the Final Answer Verification Reward, without the Causal Segment Sufficiency Reward.Source: Table 6 · p. 13, §8.3 · p. 13
8.1. Detailed Experimental Setup
- The setup uses Qwen2.5-VL-7B and a single reinforcement-learning stage without additional supervised fine-tuning.Inputs were capped at 8192 video tokens; video was sampled at 1 FPS and review segments at 2 FPS, with training lasting 792 optimization steps.Source: §8.1 · pp. 12–13
8.2. Detailed Information of Training Data
- The 25K-sample training corpus covers Short Video QA, Temporal Grounding, and Grounded Video QA.It includes 20K balanced Short Video QA examples and 2K randomly selected Grounded Video QA examples; Temporal Grounding data comes from TimeRFT without further filtering.Source: §8.2 · p. 13
8.3. Supplementary Experimental Results
- Visual review identifies the caged fish’s emotion as surprise at the cartoon snake, correcting the text-only reflection model’s mistaken anger judgment.The example illustrates more accurate detail capture through review of key video segments.Source: Figure 7 · p. 14
- When λ2 exceeds λ1, temporal grounding improves but MLVU falls from 69.8% to 68.7%; λ1 = 0.6 and λ2 = 0.3 provide an effective balance.The authors report that too much emphasis on locating the review segment can reduce its use in answer derivation, while the selected balance supports localization and reasoning.Source: §8.3 · pp. 13–14
9. Training Qwen2.5-VL-7B with Textual Reflection Mechanism on Video Data
The section describes a text-only reflection comparison model trained under REVISOR’s other experimental settings and presents REVISOR case studies. These cases show precise detail capture, scene understanding, object counting, and increasingly accurate retrieval of key video segments.
9. Training Qwen2.5-VL-7B with Textual Reflection Mechanism on Video Data
- The Qwen2.5-VL-7B text-reflection comparator generates only textual output during reflection; its other experimental settings match REVISOR’s.The listed settings include a learning rate of 1 × 10−6, batch size 32, 8 rollouts, and a cap of 8192 input video tokens.Source: §9 · p. 14
10. Case Study of REVISOR Framework
- REVISOR’s case studies demonstrate more precise detail capture, more comprehensive scene understanding, and more accurate object counting.The examples are presented in Figs. 7–9.Source: §10 · p. 14
- In the 300-second video example, textual reflection becomes more concise as retrieval improves, ultimately identifying a critical 25-second segment.The segment supports the possibility that respiratory cells could be used in treatment for cardiovascular disease.Source: §10 · p. 14
11. Prompt Templates of REVISOR
REVISOR’s training prompts comprise a system prompt, an initial reasoning stage, and a reflective reasoning stage. Unlike plain-text reflection, the initial stage identifies video segments for further examination.
11. Prompt Templates of REVISOR
- The REVISOR training prompt has three components: a system prompt, initial reasoning, and reflective reasoning.Figures 12 and 13 show the reasoning-stage templates and system prompt template, respectively.Source: §11 · p. 14, Figure 12 · p. 17, Figure 13 · p. 18
- Examples show visual review correcting a scene interpretation and a team-member count that plain-text reflection got wrong.The scene example identifies a man avoiding an oncoming truck; the counting example identifies four team members wearing yellow.Source: Figure 8 · p. 15, Figure 9 · p. 15
- As training progressed, reflective text became more concise and referenced video segments became more precise.The example ultimately concludes that respiratory cells could potentially be used to treat cardiovascular disease.Source: Figure 10 · p. 16
- The initial-stage instruction asks the model to think first, then use temporal zoom to find a segment that answers the user’s question.This stage identifies critical video segments for further examination, unlike the plain-text reflection mechanism.Source: §11 · pp. 14, 17