PaperlayerView exact revision on arXiv

Source-linked AI summary

ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understanding

Yiyang Zhou, Yangfan He, Yaofeng Su, Siwei Han, Joel Jang, Gedas Bertasius, Mohit Bansal, Huaxiu Yao

arXiv:2506.01300v1cs.CV

TL;DR

Video understanding methods often rely on static, single-pass reasoning, while existing reward and annotation strategies face efficiency and scalability challenges. ReAgent-V combines entropy-calibrated frame selection, tool-augmented reasoning, real-time rewards, and multi-perspective reflection, with experiments across 12 datasets showing gains up to 9.8% across three applications. The framework also uses inference-time feedback to select training data for subsequent optimization.

  • Problem

    Static single-pass reasoning limits adaptability in complex video scenarios, while annotation-dependent and external-reward approaches face cost, efficiency, and feedback limitations.

  • Method

    ReAgent-V combines entropy-calibrated frame selection, tool-augmented reasoning, real-time reward generation, and conservative, neutral, and aggressive reflective refinement.

  • Results

    ReAgent-V achieves performance gains of 6.9%, 2.1%, and 9.8% across video understanding, video LLM reasoning, and vision-language-action alignment, respectively, across 12 datasets or tasks.

  • Takeaways & Limitations

    Inference-time rewards support answer refinement and high-quality data selection for SFT, DPO, and GRPO across diverse video understanding applications.

  • Takeaways & Limitations

    Long videos remain challenging, and heuristic reflection and evaluation rules may limit adaptive, end-to-end automatic error correction.

Abstract

from arXiv · show

Video understanding is fundamental to tasks such as action recognition, video reasoning, and robotic control. Early video understanding methods based on large vision-language models (LVLMs) typically adopt a single-pass reasoning paradigm without dynamic feedback, limiting the model's capacity to self-correct and adapt in complex scenarios. Recent efforts have attempted to address this limitation by incorporating reward models and reinforcement learning to enhance reasoning, or by employing tool-agent frameworks. However, these approaches face several challenges, including high annotation costs, reward signals that fail to capture real-time reasoning states, and low inference efficiency. To overcome these issues, we propose ReAgent-V, a novel agentic video understanding framework that integrates efficient frame selection with real-time reward generation during inference. These reward signals not only guide iterative answer refinement through a multi-perspective reflection mechanism-adjusting predictions from conservative, neutral, and aggressive viewpoints-but also enable automatic filtering of high-quality data for supervised fine-tuning (SFT), direct preference optimization (DPO), and group relative policy optimization (GRPO). ReAgent-V is lightweight, modular, and extensible, supporting flexible tool integration tailored to diverse tasks. Extensive experiments on 12 datasets across three core applications-video understanding, video reasoning enhancement, and vision-language-action model alignment-demonstrate significant gains in generalization and reasoning, with improvements of up to 6.9%, 2.1%, and 9.8%, respectively, highlighting the effectiveness and versatility of the proposed framework.

1 Introduction

ReAgent-V addresses the limitations of static, single-pass video reasoning and costly or inefficient feedback strategies with an agentic framework that combines efficient frame selection, real-time rewards, and reflective refinement. Its experiments show gains across video understanding, video LLM reasoning, and vision-language-action alignment.

  • Static single-pass LVLM reasoning limits adaptability in complex video scenarios requiring iterative reasoning or task-specific feedback.
  • Existing improvement strategies rely on expensive high-quality annotations or external reward models and templates with notable scalability limitations.
  • ReAgent-V combines efficient frame selection with real-time reward generation to refine answers during inference and filter high-quality data for SFT, DPO, and GRPO.
  • Its reflection mechanism revises predictions from conservative, neutral, and aggressive perspectives, targeting different levels of answer, entity, and reasoning changes.
  • 6.9%, 2.1%, and 9.8% performance improvements are reported across video understanding, video LLM reasoning, and vision-language-action alignment, respectively.The framework is evaluated on 12 datasets or tasks across the three applications.

2 ReAgent-V

ReAgent-V combines entropy-calibrated frame selection, tool-augmented reasoning, and multi-perspective reflection with dynamically generated feedback for efficient video understanding. Its pipeline iteratively selects relevant frames, enriches reasoning with tools, evaluates answers, and refines them using confidence-guided strategies.

  • Entropy-Calibrated Frame Selection: ReAgent-V selects task-relevant frames with ECRS, combining CLIP-based query similarity and frame information entropy.The method iteratively recalculates scores and selects frames above an increasing threshold.
  • Entropy-Calibrated Frame Selection: The frame-selection pipeline computes ECRS, repeatedly filters frames by threshold, and supplements sets below 32 frames with high-scoring non-repetitive frames.The threshold increases exponentially across iterations, favoring frames with higher ECRS.
  • Tool-Augmented Reasoning: After frame selection, the target agent dynamically chooses tools and applies them to the query and selected frames to obtain information for an initial answer.Tool use is iterative and task-dependent, supporting higher accuracy and interpretability.
  • Evaluation and Reflection: A critic evaluates the initial answer, generates sub-questions when needed, and augments the toolset to obtain updated information for correction.The updated tool outputs feed the subsequent reflection stage.
  • Evaluation and Reflection: Three reflection strategies independently revise answers: conservative changes the answer, neutral updates relevant entities, and aggressive revises reasoning and entities.When all confidence scores exceed a threshold, common components are aggregated and inconsistencies removed; otherwise, the highest-confidence answer is selected.

3 Experiments

Experiments evaluate ReAgent-V across video understanding, video LLM reasoning, and VLA alignment, with analyses of frame selection, reflection, and iterative tool use. The framework improves performance across applications while reducing reliance on extensive data or frame inputs.

  • Experimental Setup: Experiments cover video understanding, video LLM reasoning, and vision-language-action model alignment across multiple benchmarks and baselines.The evaluation includes six video-understanding benchmarks, optimization strategies for video LLM reasoning, and aligned-model comparisons under consistent settings.
  • Main Results: 6.9%, 2.1%, and 9.8% are the reported performance gains for video understanding, video LLM reasoning, and VLA alignment, respectively.The results span Tables 1 and 2 and Figure 3.
  • Video Understanding: 3.2% and 6.9% are ReAgent-V’s average gains over Qwen2.5-VL-72B and LLaVA-Video-72B without additional frame inputs.For LLaVA-Video-72B, performance is comparable to GPT-4o on part of the benchmarks while using significantly fewer frames.
  • Video LLM Reasoning: 2.1% relative average improvement over vanilla GRPO is achieved using only 45% of its training data, while ReAgent-V also exceeds full-data SFT by 4.3%.Inference-generated evaluation reports are used as rewards for selecting training data.
  • VLA Alignment: 9.8% overall improvement over GRAPE is reported for VLA alignment on SIMPLER under the same setting, including a 9.0% gain on the original task.The comparison is against GRAPE’s template reward.
  • Quantitative and Qualitative Analysis: Frame selection and reflection are analyzed as mechanisms intended to improve efficiency and performance, while reflection consistently improves video-understanding benchmarks.The case study illustrates iterative reasoning with tool use, and the reflection analysis compares versions with and without the reflection module.

4 Related Work

Related work combines LVLMs with multimodal tools and uses SFT or preference-based reinforcement learning to improve video reasoning. ReAgent-V is positioned as addressing missing real-time feedback, inefficient frame processing, and limited inference-time refinement.

  • Multimodal LLMs for Video Understanding: LVLM-based video understanding integrates OCR, ASR, object detection, and video captioning to combine visual, textual, and auditory information.These tools are applied to tasks including video question answering.
  • ReAgent-V’s Position: ReAgent-V introduces real-time inference-aware rewards, efficient frame selection, and multi-perspective reflection for dynamic prediction refinement and high-quality data selection.The framework is presented as avoiding costly annotations and static reward templates while improving adaptability and interpretability.
  • Data-Centric VLM Reasoning Strategies: SFT, DPO, and GRPO rely on large annotated datasets or human-defined reward models to improve reasoning accuracy.These strategies fine-tune base models using annotated data and rewards aligned with human preferences.

5 Conclusion

ReAgent-V combines inference-time reward generation with multi-perspective reflective refinement for video understanding. Across 12 datasets and three applications, it reports consistent gains with minimal overhead.

  • ReAgent-V introduces inference-time reward generation and multi-perspective reflective refinement in a unified agentic video-understanding framework.
  • Across 12 datasets and three applications, ReAgent-V demonstrates consistent performance gains of up to 9.8% with minimal overhead.

A.1 Dataset and Baselines

The evaluation spans diverse video-understanding, reasoning, and vision-language-action settings, using broad baseline comparisons and standardized generalization tests. ReAgent-V combines tool-augmented reasoning, reflection, and reward-based data selection across these evaluations.

  • Datasets: The video benchmarks cover long-horizon reasoning, temporal and causal understanding, egocentric video, multimodal alignment, and general video understanding.
  • Baselines: The comparison includes proprietary multimodal models and open-source video-language models with different capabilities and design philosophies.
  • Reported comparison: Across baselines, ReAgent-V is reported to improve accuracy, interpretability, and computational efficiency through entropy-guided frame selection and multi-perspective reflection.
  • Evaluation workflow: The evaluation workflow uses dynamically selected tools, clarification sub-questions, structured answer assessment, and multi-agent reflection with conservative, neutral, and aggressive evaluators.
  • VLA alignment: ReAgent-V is evaluated across in-domain, subject, physical, and semantic generalization in the SIMPLER environment using average task success rate.

B.6 Implementation Workflow of ReAgent-V

The ReAgent-V workflow begins with ECRS keyframe selection, then dynamically activates query-relevant extraction tools before merging their outputs for downstream reasoning.

  • ECRS-based keyframe selection reduces redundancy while preserving semantically relevant video content.
  • A dictionary-based mechanism dynamically activates tools such as OCR, ASR, and DET according to the input query.
  • The pipeline merges extracted textual context into the reasoning input after tool selection.

C More Visualization Results

The visualization results compare ECRS-selected frames with uniform sampling and show how selected frames preserve question-relevant interactions, actions, text, and temporal context.

  • The visualizations present the ReAgent-V inference pipeline and ECRS keyframe-selection workflow.
  • ECRS selects frames that highlight photographer-tourist interaction, whereas uniform sampling includes generic landscape shots.
  • ECRS captures the woman’s organ-removal action more clearly than uniform sampling.
  • Across examples, ECRS preserves critical moments involving text and actions while uniform sampling can miss or misalign with them.

C.2 Visualization of Evaluation Report

ReAgent-V uses visual tools, multi-agent reflection, and reward-guided fine-tuning to correct video answers and robotic actions. Its frame-selection analysis shows progressively more compact selections aligned with relevant content.

  • Visual and textual correction: Visual and textual tools correct video interpretations, including festival identification, bird counts, cat colors, and geographic regions.The corrections combine OCR, detection, CLIP-based semantics, grounding, and collaborative review.
  • Visual and textual correction: Reflective critique revises initial answers toward grounded visual and textual evidence.Examples include changing the bird count from three to two, the cat’s color to orange and white, and the terrain from tropical to polar.
  • Robotic policy refinement: Reflection-guided reward correction changes six robotic manipulation tasks from initial failure to successful execution after policy fine-tuning.The tasks include carrot, spoon, eggplant basket, cube, coke can, and sprite can placement.
  • Frame-selection behavior: ECRS selects frames near local score peaks and produces more temporally clustered choices than CLIP and entropy-based selection.The selected frames align with high-compactness regions and are intended to capture informative moments.
  • Frame-selection behavior: Across iterations, ECRS reduces selected-frame counts while self-compactness and GT-compactness decrease.The early frame reduction is sharp before stabilizing, indicating a smaller subset with tighter temporal clustering and ground-truth alignment.

D.3 Comparison of Frame Selection Methods Across Varying Input Frame Numbers

On VideoMME, ECRS is compared with alternative frame-selection strategies across increasing input-frame counts. The comparison emphasizes accuracy trends as the number of frames varies.

  • Accuracy across frame counts: ECRS consistently outperforms other frame-selection strategies across all tested frame counts on VideoMME.The reported results use LLaVA-Video-7B, while similar trends are observed across different base models.
  • Accuracy across frame counts: As input frames increase from 8 to 64, accuracy improves for all evaluated methods.The performance gap between the strategies remains consistent across the tested frame counts.
  • Evaluation setup: The section presents frame-selection strategy comparisons on VideoMME, with LLaVA-Video-7B identified as the displayed base model.The table is described as an ablation study on Visual Tools rather than as a frame-count comparison.

D.4 Ablation of Visual Tools

Visual tools improve multimodal reasoning across the evaluated base models and benchmarks, with especially large gains for smaller models. The broader framework still faces challenges on long videos and includes deployment risks such as privacy and dataset bias.

  • Visual Tools ablation: Visual Tools consistently improve performance on LongBench, EgoSchema, and VideoMME across the evaluated base models.For LLaVA-Video-7B, the reported gains include +7.0 on LongBench and +4.5 on VideoMME.
  • Visual Tools ablation: Visual-tool gains are particularly notable for LLaVA-Video-7B and Qwen2-VL-7B, while larger models show smaller but evident improvements.The reported comparison spans LLaVA-Video-72B and Qwen2.5-VL-72B as larger-model settings.
  • Limitations and deployment: Long videos remain challenging because information redundancy and complex semantic relationships hinder comprehensive and accurate understanding.The reflection and evaluation mechanisms also rely on heuristic rules or templates rather than adaptive end-to-end learning.
  • Limitations and deployment: Deployment requires consideration of privacy concerns and dataset-bias propagation despite potential benefits for education, assistive technologies, and human-computer interaction.These concerns are stated as potential risks accompanying the method’s possible applications.
Loading 2506.01300v1…