Source-linked AI summary
EasyVideoR1: Easier RL for Video Understanding
Chuanyu Qin, Chenxu Yang, Qingyi Si, Naibin Gu, Dingyu Yao, Zheng Lin, Peng Fu, Nan Duan, Jiaqi Wang
TL;DR
Video RL remains difficult because diverse tasks, repeated high-dimensional visual preprocessing, and reproducible evaluation are not systematically handled by existing frameworks. EasyVideoR1 addresses these gaps with video-specific caching, hybrid data training, joint image-video support, modular rewards, and asynchronous evaluation, achieving higher throughput and accuracy-aligned benchmarking.
Problem
Existing RL frameworks lack systematic video-specific optimizations for diverse task rewards, repeated visual preprocessing, and reproducible evaluation.
Method
EasyVideoR1 combines offline video tensor caching, mixed offline-online training, joint image-video training, task-aware rewards, and asynchronous multi-benchmark evaluation.
Results
1.47× throughput improvement is achieved on 32 GPUs, while evaluation covers 22 benchmarks with reproduced accuracy closely aligned with official scores.
Takeaways & Limitations
EasyVideoR1 provides an open-source framework for efficient, extensible, and accuracy-aligned RL post-training research on video understanding.
Takeaways & Limitations
Caching increases storage requirements because video tensors are substantially less compressed than source video formats.
Abstract
from arXiv · showhide
Reinforcement learning from verifiable rewards (RLVR) has demonstrated remarkable effectiveness in improving the reasoning capabilities of large language models. As models evolve into natively multimodal architectures, extending RLVR to video understanding becomes increasingly important yet remains largely unexplored, due to the diversity of video task types, the computational overhead of repeatedly decoding and preprocessing high-dimensional visual inputs, and the difficulty of reproducible evaluation across numerous sensitive hyperparameters. Existing open-source RL training frameworks provide solid infrastructure for text and image scenarios but lack systematic optimizations tailored for video modality. In this work, we present \textbf{EasyVideoR1}, a complete and efficient reinforcement learning framework specifically designed for training large vision-language models on video understanding tasks. EasyVideoR1 makes the following contributions: (1) a full video RL training pipeline with offline preprocessing and tensor caching that eliminates redundant video decoding and yields a 1.47 $\times$ throughput improvement; (2) a comprehensive, task-aware reward system covering 11 distinct video and image problem types with unified routing and modular extension; (3) a mixed offline-online data training paradigm that combines curated high-quality trajectories with on-policy exploration, benefiting the learning of more challenging tasks; (4) joint image-video training with independently configurable pixel budgets, allowing the two modalities to mutually reinforce each other; and (5) an asynchronous multi-benchmark evaluation framework covering 22 mainstream video understanding benchmarks, with reproduced accuracy closely aligned with officially reported scores.
1 Introduction
EasyVideoR1 extends RLVR to video understanding by addressing modality-specific task diversity, visual-processing overhead, and evaluation needs. It combines a complete video RL pipeline with hybrid data training, joint image-video support, and asynchronous multi-benchmark evaluation.
- Video understanding requires reward designs spanning multiple-choice answering, OCR, temporal localization, spatial grounding, tracking, and dense segmentation.
- EasyVideoR1 adapts every stage of RL training for video, including positional encoding, mixed-modality forward passes, independent resolution budgets, and offline caching.
- 1.47× throughput improvement is achieved on 32 GPUs through offline preprocessing and caching in the complete video RL pipeline.
- Mixed offline-online training combines curated trajectories with on-policy exploration, especially benefiting challenging tasks with insufficient reward from on-policy data alone.
- Joint image-video training supports independent pixel budgets for images and videos while handling mixed-modality micro-batches under FSDP.
- The asynchronous evaluation framework supports 22 video benchmarks, with reproduced accuracy closely aligning with upstream model scores and approximately 6∼7× LVBench speedup over vanilla evaluation.
- With 32 H200 GPUs and approximately 20 hours of RL training, Qwen3-VL-8B-Instruct surpasses Qwen3-VL-8B-Thinking on multiple video understanding benchmarks.
2 Related Work
Prior open-source RL frameworks provide multimodal infrastructure, but their primary optimizations remain focused on general training efficiency, usability, or scenario flexibility rather than video-specific needs.
- Video-language pretraining has advanced through unified image-video architectures, dynamic temporal sampling, synthetic instruction data, and similarity-based approaches.
- Existing RL frameworks such as veRL, TRL, and ROLL offer multimodal support but center their optimizations on efficiency, usability, or flexible training scenarios.
3 System Design
EasyVideoR1 extends EasyR1 and veRL with systematic support for video RL training and evaluation. Its design covers video-specific pipeline adaptation, research-friendly training interfaces, and high-throughput evaluation.
- EasyVideoR1 organizes its design around video-modality RL adaptation, research-friendly algorithm-development interfaces, and a high-throughput evaluation framework.
- The training pipeline preprocesses videos offline into .pt cache files, which workers load locally during training.
3.1 Video Friendly Optimization
The video-friendly optimization layer addresses long sequences, decoding bottlenecks, mixed-modality execution, and heterogeneous task rewards through caching, coordinated pipeline changes, and modular scoring.
- Video sequences can be 10–100× longer than image sequences, while file-path-based staging causes redundant decoding and preprocessing across pipeline stages.
- Offline preprocessing decodes, resamples, and resizes videos into parameter-keyed cache files, with parallel workers and hash-based deduplication.
- Cached video tensors trade storage for GPU-hour throughput; a 10-minute video sampled at 2 fps with 256 frames requires roughly 360 MB.
- The dataset stage passes lightweight cache paths, workers load tensors locally, and uncached videos fall back to on-the-fly decoding.
- Mixed image-video training modifies dataset loading, rollout generation, and actor training across the three RL pipeline stages.
- Zero-valued dummy tensors connect inactive modality branches to the computation graph, preventing FSDP gradient synchronization failures in single-modality micro-batches.
- Independent image and video pixel and frame budgets allow modality-specific compute tuning.
- A central dispatcher routes each sample by problem_type to an independent reward module, enabling incremental extension across task types.
3.2 Research-Friendly Interfaces for Algorithm Development
EasyVideoR1 provides research-oriented interfaces for hybrid offline-online training, joint image-video learning, broad model support, and configurable RL algorithms. These interfaces target cold-start, data scarcity, and heterogeneous multimodal optimization challenges.
- Hybrid training combines offline trajectories with online rollouts to improve sample efficiency and mitigate cold-start issues.The framework exposes lightweight interfaces for both paradigms.
- The mix-policy interface inserts one offline trajectory into an otherwise on-policy rollout group without modifying the GRPO algorithm.A flag and optional quality threshold control the mechanism, which accepts trajectories from any source.
- Joint image-video training uses abundant image data alongside video to strengthen visual reasoning while learning temporal understanding.Image and video samples use separate preprocessors and independently configurable resolution budgets.
- EasyVideoR1 supports multiple Qwen vision-language model families and a broad suite of RL algorithms through unified interfaces.Switching among supported algorithms requires only a configuration change.
3.3 Fast & Comprehensive Evaluation Framework
The evaluation framework converts video benchmarking into an asynchronous, cache-backed streaming pipeline. It integrates 22 benchmarks and reports approximately 6–7× speedup on LVBench while maintaining broad task coverage.
- Video evaluation is bottlenecked by CPU-intensive preprocessing and sequential CPU–GPU execution that underutilizes hardware.The framework targets serialization between video processing and model inference.
- Precomputed frame caching removes repeated decoding, temporal sampling, and resizing from evaluation runs.Cached frames are stored on disk and loaded directly at evaluation time.
- Asynchronous queuing overlaps I/O, prefill, and decoding, while chunked prefill prevents long sequences from monopolizing GPU computation.These mechanisms maintain GPU productivity across scheduling steps.
- 6–7× speedup is achieved over vanilla inference frameworks on LVBench.The result follows from cached I/O, asynchronous queuing, and chunked prefill.
- The unified interface currently integrates 22 video understanding benchmarks spanning diverse capability categories and task types.The benchmark suite includes general, long-video, reasoning, STEM, spatial, grounding, and streaming tasks.
4 Experiments
Experiments evaluate EasyVideoR1 on representative video benchmarks and compare cached versus on-the-fly video loading. RL improves the instruct model’s accuracy, while caching substantially improves training throughput without changing token counts.
- 4 Experiments: Qwen3-VL-8B-Instruct is trained with GRPO on approximately 100K video samples using 32 GPUs and evaluated across 10 representative benchmarks.The evaluation spans general, long-video, reasoning, and STEM categories.
- 4.2 Unlocking the Potential of Instruct Models: 62.1 to 64.4 (+2.3) is the average accuracy improvement after EasyVideoR1 training.The comparison includes the base instruct model, the released thinking variant, and the RL-trained model.
- 4.2 Unlocking the Potential of Instruct Models: Video-Holmes (+6.6) and VideoMathQA (+6.7) show the largest gains, while Video-MME (+2.1), MVBench (+3.5), and LVBench (+0.7) also improve.The RL-trained model is comparable or superior to the thinking variant on most benchmarks in standard inference mode.
- 4.3 How Much Does Offline Preprocessing and Caching Help?: 1.47× overall speedup reduces average step time from 194.5s to 131.9s and raises token throughput from 797 to 1,175 tokens/s with cache-based loading.The comparison holds other configurations constant and averages measurements after excluding warmup.
- 4.3 How Much Does Offline Preprocessing and Caching Help?: Reference model forward time drops from 53.6s to 18.8s (2.85×), while actor parameter updates remain approximately 54s in both loading modes.Caching removes redundant video decoding during the reference-model stage, whereas actor updates operate on token-level gradients.
- 4.3 How Much Does Offline Preprocessing and Caching Help?: Approximately 4.93M tokens are processed per step in both modes, indicating that caching preserves training semantics while accelerating execution.
5 Conclusion
EasyVideoR1 is presented as an open-source, research-oriented platform for RL post-training of multimodal models on video understanding. It combines broad task support, flexible training interfaces, efficiency optimizations, and accuracy-aligned evaluation.
- EasyVideoR1 addresses the limited suitability of existing RL frameworks for video understanding scenarios.
- The framework supports mixed offline-online training, joint image-video training, efficient video RL, and comprehensive evaluation.It is released as an open repository intended to encourage community research and maintenance.