Source-linked AI summary
EVA: Efficient Reinforcement Learning for End-to-End Video Agent
Yaolun Zhang, Ruohui Wang, Jiahao Wang, Yepeng Tang, Xuanyu Zheng, Haonan Duan, Hao Lu, Hanming Deng, Lewei Lu
TL;DR
EVA addresses passive, perception-first video understanding by learning an agent that plans what, when, and how to watch through iterative summary–planning–action–reflection. Its three-stage SFT–KTO–GRPO training and dedicated datasets support adaptive video understanding, with state-of-the-art performance across multiple video benchmarks.
Problem
Existing video understanding systems process entire videos or uniformly sampled frames, while agent-based methods often rely on handcrafted, perception-first workflows rather than autonomous viewing decisions.
Method
EVA uses planning-before-perception with iterative summary–planning–action–reflection and trains through SFT cold-start, KTO correction, and GRPO optimization.
Results
EVA achieves state-of-the-art performance across multiple video benchmarks while learning an adaptive policy for multi-round perception, planning, and tool usage.
Takeaways & Limitations
EVA functions as an adaptive and self-directed video agent that balances perceptual efficiency with reasoning depth and controls redundant computation.
Takeaways & Limitations
The reasoning loop relies on predefined tool interfaces and may struggle with unseen or noisy query distributions.
Abstract
from arXiv · showhide
Video understanding with multimodal large language models (MLLMs) remains challenging due to the long token sequences of videos, which contain extensive temporal dependencies and redundant frames. Existing approaches typically treat MLLMs as passive recognizers, processing entire videos or uniformly sampled frames without adaptive reasoning. Recent agent-based methods introduce external tools, yet still depend on manually designed workflows and perception-first strategies, resulting in inefficiency on long videos. We present EVA, an Efficient Reinforcement Learning framework for End-to-End Video Agent, which enables planning-before-perception through iterative summary-plan-action-reflection reasoning. EVA autonomously decides what to watch, when to watch, and how to watch, achieving query-driven and efficient video understanding. To train such agents, we design a simple yet effective three-stage learning pipeline - comprising supervised fine-tuning (SFT), Kahneman-Tversky Optimization (KTO), and Group Relative Policy Optimization (GRPO) - that bridges supervised imitation and reinforcement learning. We further construct high-quality datasets for each stage, supporting stable and reproducible training. We evaluate EVA on six video understanding benchmarks, demonstrating its comprehensive capabilities. Compared with existing baselines, EVA achieves a substantial improvement of 6-12% over general MLLM baselines and a further 1-3% gain over prior adaptive agent methods.
1. Introduction
EVA addresses inefficient, perception-first video understanding by planning before perception and iteratively combining summaries, plans, actions, and reflections. Its three-stage training pipeline and curated datasets support adaptive video reasoning while controlling redundant computation.
- Motivation: Existing systems process entire videos or uniformly sampled frames, while many agentic methods retain fixed workflows, parameters, and sampling rates.These methods typically begin reasoning after receiving uniformly sampled frames, limiting adaptive, query-driven exploration.
- EVA Framework: EVA first reasons from the query to decide what, when, and how to watch before engaging with visual input.This planning-before-perception paradigm supports selective attention to informative moments and avoids unnecessary computation.
- EVA Framework: EVA uses iterative summary–planning–action–reflection cycles that couple visual perception, reasoning, tool use, and reflection.The agent must learn to initiate tool calls from the query, continue when visual information is insufficient, and avoid unnecessary iterations.
- Training: The three-stage pipeline combines SFT cold-start, KTO correction, and GRPO optimization to develop video-agent capabilities and improve training stability.SFT teaches tool-call formatting and frame-level reasoning; KTO uses successful and failed trajectories before online GRPO.
- Outcome: The learned adaptive policy balances multi-round perception, planning, and tool use with control of redundant computation.The paper reports state-of-the-art performance across multiple video benchmarks.
- Datasets: EVA constructs EVA-SFT with 10k samples, EVA-KTO with 11k labeled frame-selection strategies, and EVA-RL with 9.6k open-ended plus 1.1k multiple-choice questions.These datasets are designed to support stable and reproducible agentic reinforcement learning.
2. Related Works
Prior work equips video agents with external tools for active exploration, but commonly constrains them with fixed workflows and limited action freedom. EVA is positioned as a more flexible framework that dynamically adapts plans to queries and visual evidence.
- Agentic Video Understanding: Agentic video understanding extends traditional MLLMs by enabling active exploration of video content through external tools.Existing methods include tools for visual comprehension, such as external MLLM APIs and related modules.
- Tool-Integrated Reasoning Training: Tool-integrated reasoning research studies how LLM-based agents invoke external tools and optimize tool use through reinforcement learning.EVA applies this paradigm to iterative planning and selection of informative video frames.
3. Method
EVA formulates active video understanding as query-driven MDP interaction, where the agent adaptively selects visual evidence and iteratively reasons through summary, planning, action, and reflection. Its three-stage training pipeline combines supervised initialization, preference optimization, and data-enhanced reinforcement learning.
- Problem Setup: EVA models active video understanding as an MDP with belief states containing the query, text–frame history, and tool-retrieved visual evidence.The agent policy selects actions conditioned on this belief state.
- Problem Setup: The agent begins with only the query and uses a flexible tool to choose temporal windows, frame counts, and spatial resolution.Start and end times define the temporal window; nframes controls sampling density, while resize supports zoom-in and zoom-out.
- Problem Setup: Traditional fixed workflows are constrained instances of EVA, whereas EVA dynamically adapts its plan using the query and extracted visual evidence.This broader action freedom supports a more flexible agentic video-understanding paradigm.
- Data Construction: The SFT cold-start stage teaches tool-call formatting, interleaved image–text reasoning, frame-level understanding, and basic frame-selection strategies from synthetic agentic data.The data pipeline uses a teacher MLLM to generate training examples, including Summary + Planning + Action + Reflection trajectories.
- Data Construction: KTO uses successful and failed strategy trajectories to refine frame-selection preferences, addressing insufficient evidence and poorly distributed sampling.Typical failures include answering without enough visual evidence, oversampling short windows, and undersampling long ones.
- Data Construction: Data-enhanced GRPO repeatedly collects policy failures, uses them to condition a teacher MLLM on new open-ended QA generation, and retrains on the expanded dataset.The mixed-format reinforcement-learning data combines open-ended and multiple-choice questions, with task-specific accuracy rewards and a ROUGE-based reward for open-ended answers.
4. Experiments
EVA combines adaptive video-agent reasoning with a three-stage SFT–KTO–GRPO training scheme, achieving strong benchmark performance while limiting visual-token and frame usage. Ablations and workflow analyses indicate that GRPO promotes deliberate multi-round exploration and mixed GRPO data improves visually grounded reasoning.
- Main Results: EVA achieves 51.8% on LSDBench using only 6.2K visual tokens, surpassing the baseline by +2.6%.The result addresses the sampling dilemma by dynamically selecting informative frames instead of exhaustively processing video inputs.
- Main Results: EVA achieves 55.1%, 60.5%, 59.9% and 38.1% on LongVideoBench, MLVU, VideoMME and LVBench while processing about 20–30 frames per video.Frame counts are estimated using 650 visual tokens per frame, and actual usage varies with adaptive resolution.
- Main Results: In zero-shot Video-Holmes evaluation, EVA achieves 32.6%, 32.9% and 36.7% for EVAs, comparable to uniformly sampled-frame reasoning agents.The benchmark evaluates diverse reasoning abilities across long temporal contexts.
- Ablation Study: The SFT–KTO–GRPO sequence transforms tool-format imitation into strategic exploration: GRPO samples fewer frames than KTO, increases interaction rounds, and achieves the highest benchmark scores.SFT alone consumes many frames and rounds yet performs worst, while GRPO allocates visual tokens more precisely across deliberate reasoning rounds.
- Ablation Study: Mixing open-ended and multi-choice data in GRPO improves training stability and performance by discouraging plausible but visually unsupported reward-hacking guesses.The ablation compares multi-choice-only, open-ended-only, and mixed data compositions on VideoMME.
- Case Study: EVA’s query-driven workflows allocate visual tokens to needed segments, while retaining the ability to sample broadly before zooming into key frames when extensive evidence is required.Its larger action space selects both frame counts and spatial resolutions, supporting more varied and efficient workflows than prior agents.
5. Conclusion and Limitation
EVA introduces an autonomous video-understanding agent that integrates planning, tool use, and reflection through SFT, KTO, and GRPO. The resulting agent balances perceptual efficiency with reasoning depth and evolves toward adaptive, self-directed video exploration.
- EVA integrates understanding, planning, tool use, and reflection in an iterative loop for query-driven video understanding.
- Its three-stage SFT cold-start, KTO, and GRPO training paradigm learns to balance perceptual efficiency with reasoning depth.
- The trained model progressively evolves from a passive video recognizer into an adaptive and self-directed agentic watcher.
6. Appendix
EVA combines planning-before-perception with iterative tool use to adapt visual sampling, reduce unnecessary computation, and support video reasoning. Its appendix materials describe training-data generation, round-wise behavior, evaluation settings, and illustrative success cases.
- EVA Advantages: Planning-before-perception guides EVA toward question-relevant visual evidence instead of irrelevant or noisy frames.The textual plan is formed before video interaction and directs subsequent observation.
- EVA Advantages: Planning-before-perception reduces visual-token usage by identifying relevant temporal segments or using low-resolution previews when sufficient.The approach is intended to improve efficiency and accuracy on long videos.
- EVA Advantages: EVA shifts video understanding from passive observation to active perception by determining what information should be acquired.Fixed observations leave reasoning constrained by noise, sampling bias, and irrelevant content.
- Data Pipeline: The Multi-Agent Data Pipeline uses Executor and Reflective Thinker agents to evaluate actions, scrutinize tool-call parameters, and archive successful trajectories.Retrieved trajectories from an Experience Bank guide the Executor in later iterations.
- Behavior Analysis: EVA initially explores broadly, then sharply reduces frame count and temporal span while increasing resize to gather finer-grained information.This round-wise allocation contrasts with baselines that assign all visual tokens in the first round.
- Evaluation: 5% is EVA's reported SAH-ratio compared with 8.8% for Qwen2.5-VL-7B on ELV-Halluc.The benchmark evaluates hallucinations during aggregation from frame-level information to event-level semantics.
- Case Studies: EVA can use multi-turn grounding and zoom-in or directly choose dense sampling when a question requires high-resolution coverage.The illustrated cases show adaptive tool calls based on query demands and earlier visual evidence.
SYSTEM
The system directs EVA to analyze videos with a frame-selection tool and answer queries by choosing an appropriate sampling strategy. Examples include long videos requiring customized sampling and questions where dense sampling is selected directly.
- System: EVA uses a frame-selection tool to analyze the video and generate an answer to the question.
- System: For a 397-second gameplay video, EVA considers dense, sparse, keyframe, and hybrid sampling before favoring customized sampling with aggressive resizing.The query asks what triggers the “GOAAAAAL!!” animation.