Source-linked AI summary
Video-MOPD: Multi-Teacher On-Policy Distillation for Video Understanding
Zhenxin Qin, Peng Shi, Cong Han, Yinlong Qian, Zequn Jie, Lin Ma
TL;DR
Video understanding requires jointly handling broad perception, temporal alignment, and complex reasoning, but these capabilities are difficult to optimize together. Video-MOPD-8B specializes a shared model with targeted reinforcement learning, unifies the experts through routed on-policy distillation and reliability-aware sampling, and achieves strong results across seven video benchmarks.
Problem
Video understanding requires complementary capabilities in general comprehension, temporal grounding, and STEM reasoning that are difficult to jointly optimize in one model.
Method
Video-MOPD-8B uses three domain experts, routed Multi-Teacher On-Policy Distillation, and Reliability-Aware Informative Sampling to consolidate specialized capabilities.
Results
Across seven benchmarks, Video-MOPD-8B improves the average score by 5.4 points over Qwen3-VL-8B-Instruct and reaches 69.12 overall.
Takeaways & Limitations
The specialize-then-unify pipeline combines complementary expert improvements into a single model for comprehensive video understanding.
Takeaways & Limitations
Future work targets long-form videos, where relevant evidence is sparse and distributed across distant temporal segments.
Abstract
from arXiv · showhide
Video understanding demands a convergence of complementary capabilities across perception, temporal understanding, and complex reasoning, which are difficult to jointly optimize within a single model. We introduce Video-MOPD-8B, an open-weight model dedicated to video understanding tasks. To fundamentally enhance its capabilities, we conduct targeted reinforcement learning (RL) optimization across three core domains: video temporal grounding (VTG), general video comprehension, and video STEM reasoning. We then unify their complementary capabilities via Multi-Teacher On-Policy Distillation (MOPD), which consolidates expert knowledge by supervising student-generated trajectories with routed teacher feedback. We further introduce Reliability-Aware Informative Sampling (RAIS), which selects examples with consistently reliable teacher supervision and large teacher-student performance gaps. Together, these components enable Video-MOPD-8B to achieve coordinated and comprehensive performance gains across diverse video understanding tasks. Extensive experiments on comprehensive benchmarks covering general video understanding, temporal grounding, video reasoning, and video STEM tasks demonstrate that Video-MOPD-8B achieves state-of-the-art performance among existing models at a comparable scale. The trained model weights are available at https://huggingface.co/LandH/Video-MOPD-8B.
1 Introduction
Video-MOPD-8B addresses the challenge of jointly optimizing broad comprehension, precise temporal grounding, and knowledge-intensive video reasoning. Its specialize-then-unify pipeline uses domain experts, routed on-policy distillation, and reliability-aware sampling, achieving strong performance across seven benchmarks.
- Motivation: Video understanding requires complementary capabilities because videos combine evolving visual content, temporal dependencies, object interactions, and causal relations.Video STEM reasoning additionally integrates visual-temporal evidence with scientific or technical knowledge and multi-step reasoning.
- Approach: Three specialized experts are consolidated through routed Multi-Teacher On-Policy Distillation using student-generated trajectories and matched teacher supervision.Each sample is routed to its domain teacher, which supplies dense token-level supervision during on-policy training.
- Results: 69.12 overall, with a 5.4-point average improvement over Qwen3-VL-8B-Instruct across seven video benchmarks.The model leads or ties six of seven benchmarks among non-expert baselines.
- Approach: Video-MOPD-8B unifies general comprehension, temporal grounding, and video STEM reasoning in one open-weight model.The pipeline targets complementary capabilities that span semantic understanding, temporal alignment, and knowledge-intensive reasoning.
- Approach: Reliability-Aware Informative Sampling retains reliable teacher supervision and prioritizes samples with larger teacher–student performance gaps.This sampling strategy is introduced to improve capability transfer during consolidation.
2 Related Work
Related work develops video multimodal models, reinforcement-learning methods for video reasoning, and approaches for integrating capabilities across domains. These lines of work motivate combining specialized optimization with capability consolidation.
- Video Multimodal Large Language Models: Video multimodal models extend image-based vision–language modeling to temporally structured visual inputs.Representative systems explore unified image–video representations and temporal video modeling.
- Reinforcement Learning for Video Understanding: Reinforcement learning has emerged as a post-training approach for improving reasoning and task-specific capabilities in multimodal video models.Video-R1 and VideoChat-R1 investigate reward-driven or reinforcement-based optimization for dynamic visual content and spatiotemporal perception.
- Capability Integration and On-Policy Distillation: Capability-integration methods include mixed or sequential reinforcement learning, offline imitation from specialized teachers, and parameter-space model merging.These paradigms make different trade-offs in how capability integration is coupled with optimization or imitation.
3 Method
Video-MOPD builds three domain-specialized experts and unifies their capabilities through routed on-policy distillation into one student. Reliability-Aware Informative Sampling selects examples with consistent teacher supervision and larger teacher–student gaps for capability transfer.
- Domain-Specific Expert Training: The pipeline first constructs three complementary specialists for broad comprehension, temporal grounding, and knowledge-intensive STEM reasoning.The experts are trained with domain-specific procedures, including reinforcement learning for general video and VTG capabilities and image-based STEM optimization for frame-level skills.
- Video-MOPD: Routed Multi-Teacher On-Policy Distillation: Domain-labeled video, image, and temporal-grounding examples are routed to the General Video, STEM, and VTG teachers, respectively, with approximately balanced domain sampling.A deterministic router selects the teacher associated with each example’s domain label.
- Reliability-Aware Informative Sampling: RAIS retains examples whose teacher is correct across all K sampled responses, then prioritizes larger teacher–student performance gaps.This combines consistent teacher supervision with informative remaining learning value.
- Video-MOPD: Routed Multi-Teacher On-Policy Distillation: The student generates on-policy trajectories, which the matched teacher evaluates under the same context for dense token-level reverse-KL supervision.Each expert guides the student only on samples matched to its specialization.
- Video-MOPD: Routed Multi-Teacher On-Policy Distillation: Video-MOPD jointly updates one student from General Video, STEM, and VTG experts, producing a unified model that performs inference with a single parameter set.The teachers and router are used only during training, rather than combining expert predictions at inference time.
4 Experiments
Across seven complementary video benchmarks, Video-MOPD-8B combines improvements in STEM reasoning, general video understanding, and temporal grounding. The experiments compare expert consolidation, parameter averaging, token-support choices, and qualitative behavior.
- 4.3 Comprehensive Model Comparison: 69.12 overall average and a 5.4-point gain over Qwen3-VL-8B-Instruct establish Video-MOPD-8B as the strongest overall model, leading or tying six of seven benchmarks among non-expert baselines.It improves the overall average from 67.28 with Param-Merge to 69.12.
- 4.3 Comprehensive Model Comparison: Policy-space consolidation outperforms direct parameter averaging, increasing the overall average from 67.28 to 69.12.The comparison isolates capability consolidation because the models operate at the same nominal scale.
- 4.3 Comprehensive Model Comparison: The General Video, STEM, and VTG experts specialize in broad comprehension, knowledge-intensive reasoning, and temporal localization, respectively, while Video-MOPD integrates these strengths.This complementary specialization motivates multi-teacher consolidation rather than relying on a single expert.
- 4.4 Distillation Support Ablation: Token-support ablations compare sampled-token feedback with Teacher Top-16, Student Top-16, and Teacher–Student overlap Top-16 under matched one-epoch training, with sampled-token supervision adopted by default.The variants emphasize different regions of teacher and student distributions.
- 4.5 Qualitative Analysis: Qualitative examples show Video-MOPD-8B reproducing expert-level cross-frame evidence integration, STEM reasoning, and temporal localization in one model, including an exact ground-truth interval recovery.Each example compares the base model, the matched expert, and Video-MOPD-8B.
5 Conclusion
Video-MOPD unifies complementary video capabilities by specializing experts and consolidating them with reliability-aware, routed on-policy distillation. Across seven benchmarks, the resulting model reaches the highest overall score and surpasses both the base model and parameter averaging.
- 5 Conclusion: Video-MOPD-8B reaches 69.12 overall, improves 5.4 points over Qwen3-VL-8B-Instruct, and exceeds direct parameter averaging by 1.84 points.The framework uses RAIS and routed Multi-Teacher On-Policy Distillation, with no teachers or routing required during inference.
- 5 Conclusion: Video-MOPD consolidates general video understanding, STEM reasoning, and precise temporal grounding into one unified post-training recipe.The framework starts from a shared base model and constructs specialized experts before unification.
- 5 Conclusion: Reliability-Aware Informative Sampling filters for reliable teacher supervision and prioritizes samples with larger teacher–student performance gaps.
6 Future Work
Future work targets long-form video understanding and stronger expert training. The planned directions address sparse, distant evidence and iterative capability expansion without forgetting existing skills.
- 6 Future Work: Future work will extend Video-MOPD to long-form videos, where relevant evidence is sparse and distributed across distant temporal segments.
- 6 Future Work: Planned improvements include efficient frame selection, hierarchical temporal representations, and memory-augmented reasoning for long-context processing.
- 6 Future Work: The authors also plan broader training data, more accurate verifiers and rewards, improved optimization, and repeated MOPD to transfer new capabilities without forgetting existing ones.
A Detailed Training Configuration
Table 3 reports the main MOPD configuration used in the study.
- A Detailed Training Configuration: Table 3 summarizes the main MOPD configuration and its corresponding values.
B Data Composition and Routing
Table 4 documents the final MOPD training mixture, while routing uses known data-domain labels to select teachers during training. The deterministic routing mechanism does not change the student architecture or operate during inference.
- Table 4 presents the composition of the final MOPD training mixture.
- Known data-domain labels deterministically select the teacher service during training.
- Routing is training-only and does not alter the student architecture or function as a learned inference router.
C Detailed Evaluation Configuration
The evaluation uses a shared decoding setup for benchmarks other than TimeLens and a dedicated temporal-localization pipeline for TimeLens. The configurations differ in sampling rate, backend, and decoding limits.
- For six non-TimeLens benchmarks, videos are sampled at 2 FPS with at most 2,048 frames using vLLM.The shared setup covers MVBench, MMVU, Video-MME, VideoMMMU, Video-Holmes, and TempCompass.
- TimeLens uses a dedicated Transformers temporal-localization pipeline with 4 FPS sampling and a 2,048-frame maximum.
- TimeLens additionally uses a 128,000 × 322 pixel visual budget, greedy decoding, and up to 4,096 generated tokens.
D Distillation-Support Variants
The sampled-token policy-gradient estimator uses teacher–student logprobability differences for student-sampled tokens, while Top-k variants broaden the supported token set.
- The sampled-token estimator computes the teacher–student logprobability difference for each token sampled by the student.
- Teacher Top-k and Student Top-k use candidate sets formed from the teacher’s or student’s highest-probability tokens.
- Teacher∩Student Top-k restricts support to tokens appearing in both candidate sets.