PaperlayerView exact revision on arXiv

Source-linked AI summary

VLMs are Good Teachers for Video Reasoning via Adaptive Test-Time Optimization

Junhao Cheng, Liang Hou, Tianxiong Zhong, Xin Tao, Pengfei Wan, Kun Gai, Jing Liao

arXiv:2606.02564v3cs.CV

TL;DR

Video generation models can produce visually plausible but logically inconsistent reasoning trajectories, while textual VLM guidance often misses fine-grained spatiotemporal details. This paper instead uses VLMs as test-time teachers that provide differentiable, task-specific rewards for online VGM optimization, achieving a 16.7-point average gain across symbolic and general-purpose benchmarks at comparable test-time cost.

  • Problem

    Video generation models often produce visually plausible but logically inconsistent trajectories, while sampling and textual VLM planning provide limited correction for rule-following failures.

  • Method

    A VLM teacher extracts task-specific process and goal rules as differentiable rewards, guiding instance-specific online optimization of a lightweight LoRA module in a VGM reasoner.

  • Results

    16.7-point average performance gain over the baseline Reasoner was achieved across VBVR-Bench and RULER-Bench, outperforming VLM-as-Solver (+0.4 points) and Best-of-N (+2.2 points) at comparable test-time cost.

  • Takeaways & Limitations

    Using VLMs as test-time teachers offers a promising way to bridge high-level logic and visual execution in generative video reasoning.

Abstract

from arXiv · show

The recent "Reasoning with Video" paradigm utilizes Video Generation Models (VGMs) to generate temporally coherent visual trajectories to complete reasoning tasks. Although state-of-the-art VGMs excel at visual quality, they often struggle to understand and follow task-specific rules, leading to logical failures across diverse reasoning scenarios. Existing efforts try to utilize Vision-Language Models (VLMs) as problem pre-solvers to produce or refine textual guidance for the VGM. However, textual descriptions fail to capture intricate spatiotemporal details, and VGMs often struggle to faithfully execute fine-grained or long-tail instructions even with a valid plan. While VLMs struggle as solvers, they possess strong perception capabilities to evaluate process-constraint satisfaction and final-goal achievement. Leveraging this strength, we introduce a paradigm shift that transitions the role of VLMs to "teachers". Specifically, a VLM teacher extracts task-specific rules to formulate differentiable rewards, guiding a VGM Reasoner via test-time online optimization of a lightweight LoRA module. This strategy enables adaptive test-time optimization and extends the reasoning capabilities beyond the VGM's intrinsic boundaries. Evaluations on symbolic (VBVR-Bench) and general-purpose (RULER-Bench) video reasoning benchmarks show that the proposed method yields a 16.7-point average performance gain, outperforming the VLM-as-Solver paradigm (+0.4 points) and Best-of-N scaling (+2.2 points) by a large margin at comparable test-time cost. These findings reveal that integrating VLMs as test-time teachers offers a promising paradigm for achieving generalizable video reasoning. Project Page: https://VLM-as-Teacher.github.io/

I. INTRODUCTION

Reasoning with Video uses VGMs to generate visual trajectories for logical tasks, but visually plausible outputs can remain logically inconsistent. The proposed VLM-as-Teacher paradigm instead derives differentiable process and goal rewards from task descriptions to optimize a VGM online at test time, yielding substantially larger gains than existing scaling and solver approaches.

  • Motivation: VGMs extend video generation to logical reasoning by synthesizing coherent visual trajectories for vision-centric challenges that language alone cannot precisely specify.
  • Limitations: Visually plausible VGM trajectories often violate task-specific rules and remain logically inconsistent with the intended goals.
  • Limitations: Best-of-N and rejection-based test-time scaling reduce stochastic errors by searching sampled videos, but provide limited gains against systematic logical and causal failures.
  • VLM-as-Teacher: VLMs serve as teachers by verifying observable process constraints and final-goal achievement even when they cannot construct executable visual trajectories.For example, a VLM can check whether a ball reaches a maze exit, preserves its identity, and avoids crossing walls.
  • Results: 16.7-point average performance gain is achieved on VBVR-Bench and RULER-Bench, compared with +0.4 points for VLM-as-Solver and +2.2 points for Best-of-N scaling at comparable test-time cost.
  • Method: The method performs online test-time optimization of a VGM using differentiable VLM rewards and task-adaptive process and goal rewards synthesized from task descriptions.These rewards are presented as sufficient conditions for successful reasoning-task completion and extend capability beyond the VGM’s intrinsic generative limits.

II. RELATED WORK

Related work reframes video generation as a substrate for visual reasoning and explores test-time scaling and VLM-guided generation. These approaches motivate using VLMs as teachers that evaluate processes and guide VGM optimization rather than acting as solvers.

  • Reasoning with Video: “Thinking with Frames” treats video generation as a computational substrate for visual reasoning rather than mere synthesis.This paradigm extends video generation toward reasoning-oriented use.
  • Reasoning with Video: Preliminary Veo-3 studies show that large-scale pre-training can enable zero-shot perceptual and manipulation behaviors on simple tasks.Recent work draws an analogy between multi-step Chain-of-Frame diagnosis and Chain-of-Thought prompting.
  • Test-Time Scaling for Video Reasoning: Test-time scaling enhances video reasoning through frame-level tree searches, evolutionary sampling, and iterative self-refinement without modifying model parameters.Video-specific methods extend test-time computation along the temporal axis.
  • Test-Time Scaling for Video Reasoning: Repeated sampling and ensemble-based scaling yield limited gains when failures are systematic, including flawed solution paths, skipped sub-goals, and physically inconsistent outcomes.These limitations reflect constraints imposed by the base models’ inherent generative capacity.
  • Integrating VLMs for Video Reasoning: The proposed teacher paradigm shifts VLMs from symbolic solvers to evaluators that formulate differentiable rewards from task-specific rules and guide VGM test-time optimization.This role bridges high-level logic and visual execution by leveraging VLMs’ strength in evaluating generative processes.

III. METHOD · A. Task Formulation

The method formulates rule-based video reasoning as generating a temporally coherent visual trajectory that obeys task-specific rules and achieves a goal. Each instance conditions a VGM on textual instructions and optional imagery, while a VLM Teacher provides case-specific supervision because generic rewards cannot capture diverse constraints.

  • A. Task Formulation: Rule-based video reasoning requires a VGM to generate a temporally coherent visual trajectory that follows task-specific rules and achieves an intended goal.The setting includes symbolic tasks such as navigation and object arrangement, plus general-purpose scenarios such as anomaly removal and hypothesis generation.
  • A. Task Formulation: A reasoning instance is defined by condition c = (p, x), where p is a textual instruction and x is an optional condition image.Given c, the VGM generates the visual reasoning trajectory.
  • A. Task Formulation: The generated trajectory is v = Gθ(c; ϵ) = {v1, v2, . . . , vT}, with θ denoting VGM parameters and ϵ denoting sampling randomness.The video consists of a sequence of visual states or frames indexed from 1 to T.
  • A. Task Formulation: Successful completion requires satisfying the final-goal predicate g(v, c) and the process-constraint predicates R(v, c) = {rm(v, c)}M.Both the achieved final state and the trajectory’s intermediate constraints determine task success.
  • A. Task Formulation: Task-specific rules vary across individual tasks and conditions, making a general reward-function set inadequate for characterizing their diverse constraints.This variability is the central formulation challenge addressed by the method.
  • A. Task Formulation: A VLM Teacher synthesizes supervision queries for each case and directly guides the VGM through test-time optimization.The formulation therefore uses case-specific supervision rather than relying solely on a fixed, general reward specification.

B. VLM-as-Teacher Framework

The framework uses a VLM Teacher to extract task-specific requirements and provide differentiable supervision for test-time LoRA optimization of a VGM Reasoner. It combines goal and process queries with efficient approximation techniques to guide valid, goal-achieving video trajectories.

  • Framework: The VLM Teacher identifies task requirements and supplies differentiable supervision to optimize a lightweight LoRA module in the VGM Reasoner.The pretrained VGM backbone and VLM Teacher remain frozen for each reasoning instance.
  • Reward Query Synthesis: The teacher synthesizes one goal-achievement query and M process-supervision queries, typically with 1 ≤ M ≤ 3.Process queries assess task-specific constraints, while the goal query assesses whether the final state satisfies the intended objective.
  • Reward Query Synthesis: Goal and process supervision are complementary: goal-only feedback permits invalid intermediate trajectories, whereas process-only feedback does not ensure task completion.All reward queries are positively phrased so that a “Yes” response indicates requirement satisfaction.
  • Online Optimization: The optimization objective combines one goal-achievement term with M process-supervision terms, and updates LoRA parameters by gradient descent through the frozen VLM Teacher.The gradients propagate through visual prediction to the VGM Reasoner’s LoRA parameters rather than updating the VLM.
  • Efficient Adaptation: Efficiency comes from a lightweight surrogate decoder, a distilled four-step generator with first-step optimization, and loss-based early stopping.The teacher evaluates uniformly sampled frames from the first-step prediction, and optimization stops when Multi-VQA ≤ τL or the maximum step count N is reached.

IV. EXPERIMENTS · A. Experimental Setup

The experiments evaluate the method on complementary symbolic and general-purpose video reasoning benchmarks, using standardized VGM, VLM-teacher, and test-time reasoning configurations. Implementation relies on a distilled Wan2.2-5B reasoner, Qwen3-VL-4B teacher, LoRA-only online optimization, and fixed sampling and stopping settings.

  • A. Experimental Setup: VBVR-Bench tests symbolic visual reasoning across abstraction, knowledge, perception, spatiality, and transformation capabilities.RULER-Bench covers general-purpose reasoning across humanity, science, hypothesis, semantics, vision, and game rule categories.
  • A. Experimental Setup: The method is compared with Sora 2, Kling 2.6, Veo 3.1, and Wan2.2 using Pass@N, PE, and VideoTPO test-time strategies.Pass@N samples N candidates with different initial noises, while PE and VideoTPO represent the VLM-as-Solver paradigm.
  • A. Experimental Setup: The default system uses a step-distilled Wan2.2-5B as VGM Reasoner and Qwen3-VL-4B as VLM Teacher.The reasoner is distilled into a four-step generator following DMD2; open-source baselines receive domain-adaptive supervised fine-tuning on VBVR-Bench’s 30K training instances.
  • A. Experimental Setup: Online optimization updates only LoRA parameters, while first-step clean-latent predictions are decoded with LightX2V’s lightweight surrogate decoder.The setup uniformly samples K = 24 frames for VLM evaluation.
  • A. Experimental Setup: The benchmarks report higher-is-better task results and average total inference generation seconds per sample as Cost.VBVR-Bench reports abstraction, knowledge, perception, spatiality, and transformation tasks; RULER-Bench reports transportation, sports, social, safety, festival, dress, food, emotion, chemistry, physics, biology, earth science, mathematics, medicine, life, subjective, and objective tasks.
  • A. Experimental Setup: Online optimization runs for at most N = 40 steps with LoRA rank 16, learning rate 5×10−5, and loss balance factor λ = 0.5.Early stopping uses τL = 0.22, approximately corresponding to overall VLM confidence 0.8 for answering “Yes” to reward queries; final videos use the optimized reasoner and standard VAE.

B. Comparison with SOTA Methods

The proposed method substantially outperforms prompt refinement and test-time scaling on VBVR-Bench and RULER-Bench at comparable inference cost. Qualitative results further show more reliable satisfaction of both final goals and process constraints during visual execution.

  • Quantitative Comparisons: 0.115 overall gain on VBVR-Bench raises the baseline from 0.666 to 0.781, with improvements of +0.111 on ID and +0.119 on OOD tasks.At comparable test-time cost, Pass@5 improves only 0.017, while VideoTPO decreases the overall score by 0.032.
  • Quantitative Comparisons: 21.8-point improvement on RULER-Bench raises the baseline Reasoner average from 46.4 to 68.2, exceeding PE, VideoTPO, and Pass@5 gains of 1.9, 3.9, and 2.7 points.The method improves all 30 evaluated task categories, whereas PE and VideoTPO decrease performance on 7 and 4 categories, respectively.
  • Quantitative Comparisons: Prompt-space methods are less reliable when tasks require precise visual execution, while direct trajectory optimization yields substantial gains on Anomaly, Color, Count, and Direction.The proposed method improves these visually demanding categories more reliably than textual specification refinement alone.
  • Quantitative Comparisons: 69s per sample on VBVR-Bench and 88 s on RULER-Bench remain comparable to Pass@5 at 70s and 90s, while below original-baseline costs of 87s and 98s.Step distillation reduces generation cost to 14 s on VBVR-Bench and 18 s on RULER-Bench, with reasoning decreases of 0.010 and 0.3 points, respectively.
  • Qualitative Comparisons: The proposed method satisfies final-goal and process-constraint queries, preserving object properties, avoiding blocked cells and duplicated instances, and completing the illustrated visual tasks.Kling 2.6, the step-distilled baseline, and VideoTPO exhibit rule-following or visual-execution failures such as misplaced objects, duplicated balls, and an uncorrected six-finger anomaly.

C. Ablation and Analysis

Ablations show that task-specific, differentiable online rewards, complementary goal and process supervision, and efficient early-stage optimization are central to performance. The method generalizes across teachers and backbones, reduces failures substantially, but can suffer from incorrect synthesized goals and visual artifacts.

  • Reward Design: Replacing online optimization with shared post-training lowers the overall score from 0.781 to 0.688, while non-differentiable Flow-GRPO rewards reduce it further.These results support instance-specific online optimization with differentiable VLM rewards.
  • Reward Design: Using fixed generic queries lowers the overall score from 0.781 to 0.712, demonstrating the importance of task-specific reward synthesis.Generic queries fail to capture each task’s specific goals and process constraints.
  • Reward Design: Removing process supervision lowers the score from 0.781 to 0.758, whereas removing final-goal supervision lowers it to 0.692, confirming complementary roles.The default balance weight is λ = 0.5, assigning equal importance to task completion and trajectory validity.
  • Efficient Optimization Designs: Step distillation and first-step optimization are essential: removing step distillation lowers the score to 0.714, while last-step optimization lowers it to 0.705.Step distillation produces task-informative predictions, and early denoising stages determine high-level motion patterns and trajectories.
  • Efficient Optimization Designs: 16 online optimization steps achieve an overall score of 0.781 on VBVR-Bench, while 20 steps add only 0.002 and 40 steps reduce it to 0.778.Loss-based early stopping averages 16 steps, indicating that optimization benefits largely saturate around this budget.
  • Failure Cases and Limitations: The method reduces failure rates from 39% to 18% on VBVR-Bench and from 67% to 29% on RULER-Bench, but incorrect goal queries and visual artifacts remain limitations.A human evaluation covered 200 generated cases; visual degradation reflects optimization for VLM-level rule satisfaction rather than pixel-level fidelity.

V. CONCLUSION

The paper introduces VLM-as-Teacher supervision for rule-based video reasoning, using task-specific differentiable rewards to adapt visual reasoning trajectories at test time. Across VBVR-Bench and RULER-Bench, it achieves a 16.7-point average gain over the baseline Reasoner and outperforms alternative strategies at comparable cost.

  • V. CONCLUSION: The VLM-as-Teacher paradigm shifts VLMs from producing textual solutions to supervising visual execution.A VLM Teacher synthesizes task-specific reward queries assessing process-constraint satisfaction and final-goal achievement.
  • V. CONCLUSION: 16.7-point average performance gain over the baseline Reasoner is achieved across symbolic VBVR-Bench and general-purpose RULER-Bench.The experiments demonstrate consistent improvements across diverse reasoning tasks.
  • V. CONCLUSION: The method substantially outperforms VLM-as-Solver and Best-of-N scaling strategies at comparable test-time cost.Efficient adaptation designs enable instance-specific refinement of visual reasoning trajectories at practical test-time cost.
Loading 2606.02564v3…