PaperlayerView exact revision on arXiv

Source-linked AI summary

Video Reasoning without Training

Deepak Sridhar, Kartikeya Bhardwaj, Jeya Pradha Jeyaraj, Nuno Vasconcelos, Ankita Nayak, Harris Teague

arXiv:2510.17045v2cs.CVcs.AIcs.LG

TL;DR

Video reasoning with large multimodal models can require costly training and produce verbose reasoning traces, raising the question of whether reasoning can be elicited without training. V-Reason uses an entropy-guided value-cache controller at inference, and the paper reports accuracy within 0.6% of an RL-trained model alongside fewer output tokens.

  • Problem

    Video reasoning methods based on chain-of-thought fine-tuning or reinforcement learning incur substantial training and inference costs, motivating a training-free way to search for reasoning traces in pretrained knowledge.

    Source: §1 · p. 1

  • Method

    V-Reason uses a trainable value-cache controller and entropy-based objective at inference to shape exploration and convergence without supervised fine-tuning or reinforcement learning.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    V-Reason narrows the accuracy gap to Video-R1-7B to within 0.6% and generates 58.6% fewer output tokens on average than Video-R1-7B.

    Source: §1 · p. 3, §5 · p. 8

  • Takeaways & Limitations

    V-Reason is reported to complement min-p and top-H decoding strategies, with gains varying across decoding conditions.

    Source: §4.1 · p. 7

    V-Reason is most effective when the needed knowledge is already represented in the pretrained model; training-based approaches remain more effective for underrepresented tasks such as VSI-MRA.

    Source: §4.1 · p. 7

Abstract

from arXiv · show

Video reasoning using Large Multimodal Models (LMMs) relies on costly reinforcement learning (RL) and verbose chain-of-thought, resulting in substantial computational overhead during both training and inference. Moreover, the mechanisms that control the thinking process in these reasoning models are very limited. In this paper, we use the entropy of the model's output distribution as a signal to study and guide reasoning behavior. We discover that high-quality models exhibit a characteristic pattern of micro-exploration and micro-exploitation cycles, followed by a later entropy peak (i.e., longer thinking) and a lower final entropy, indicating more deliberate exploration and confident convergence (i.e., avoid excessive randomness while the model is exploring or thinking through an answer). We then use these novel, theoretically-grounded insights to introduce V-Reason (Video-Reason), an inference-time optimization method that adapts the value cache of the LMM through a lightweight, trainable controller. Our proposed controller is guided by an entropy-based objective, to tune the model's behavior directly at inference, without using any RL or supervised fine-tuning. Our experiments show that V-Reason significantly outperforms the base instruction-tuned models on many video reasoning datasets, narrowing the gap with RL models to within 0.6% accuracy on average. We achieve this without any training, while offering efficiency benefits: V-Reason uses 58.6% fewer tokens than the RL model. Project Page https://deepaksridhar.github.io/vreason.github.io/

1. Introduction

Video reasoning with LMMs inherits substantial training and inference costs from chain-of-thought and reinforcement-learning approaches, motivating the question of whether reasoning traces can be surfaced without training. The paper studies entropy patterns and introduces V-Reason, reporting accuracy closer to RL-trained models with fewer output tokens.

  • Larger, more accurate models show later entropy peaks and lower final entropy, while reasoning traces include repeated micro-exploration and micro-exploitation cycles.Source: §1 · pp. 1–2
  • V-Reason uses a small trainable controller to adapt the value cache with an entropy objective, strengthening entropy cycles before a final minimization phase without supervision, data, or RL.Source: §1 · p. 2
  • V-Reason is most effective when a solution lies within pretrained knowledge; training-based approaches remain more effective for underrepresented tasks such as VSI-MRA regression.Source: §1 · p. 2
  • Across six video reasoning benchmarks, V-Reason improves the base model by 1.4% on average and narrows the gap to Video-R1-7B to within 0.6%.Source: §1 · p. 3
  • V-Reason produces 21.4% fewer tokens than the base Qwen2.5-7B-Instruct model and 58.6% fewer than Video-R1-7B on average.Its wall-clock inference time is up to 37% lower than Video-R1-7B on average.Source: §1 · p. 3

2. Related Work

Prior reasoning work elicits chain-of-thought through prompting, supervised or reward-based tuning, video-specific training, inference-time decoding, and steering. V-Reason differs by optimizing a model’s value cache at inference using its own entropy as feedback, without external models or data.

  • Prompting methods seek better chain-of-thought paths but can be task-specific, less generalizable, and dependent on manual prompt design and prompt engineering.Source: §2 · p. 3
  • Instruction-tuning and reward-based fine-tuning require supervised chain-of-thought data and expensive reinforcement-learning stages.Source: §2 · p. 3
  • Video reasoning systems such as Video-R1, VideoChat-R1, and TinyLLaVA rely on costly training; TinyLLaVA training on 50K samples takes about three days on four A100 GPUs.Source: §2 · p. 3
  • Inference-time methods include token-selection or logit-guidance approaches and sampling filters, whereas V-Reason optimizes intrinsic token distributions and can combine with min-p and top-h.Source: §2 · p. 3
  • Unlike steering methods that rely indirectly on reasoning-trained models, V-Reason modulates the value cache using the model’s own entropy without an external model or data.Source: §2 · pp. 3–4

3. Proposed Approach: V-Reason

V-Reason adjusts a pretrained LMM’s video-token value cache at inference to shape entropy and reasoning behavior, without modifying other layers. Its switching objective encourages exploration cycles before an entropy peak and entropy minimization afterward.

3.1. Inference-time Optimization

  • A small controller updates the last decoder layer’s video-token value cache at inference, while other layers and text-token inputs and outputs remain unchanged.The controller is initialized to zero and updated every kth generated token, with the first token skipped; cache normalization preserves its original magnitude while applying a directional update.Source: §3.1 · p. 4
  • The Entropy Switching Loss raises or lowers output-distribution entropy according to αk, promoting micro-exploration when αk = 1 and micro-exploitation when αk = −1.It can reinforce micro-cycles during macro-exploration and entropy minimization during macro-exploitation.Source: §3.1 · p. 4
  • The entropy EMA smooths noisy stepwise values, and the switching coefficient follows it; its slower growth yields a lower, delayed peak.The smoothing coefficient β is set to 0.98; after the EMA reaches a new global maximum, αk switches to −1.Source: §3.1 · p. 5
  • More and stronger micro-exploration and micro-exploitation cycles slow entropy growth before a delayed peak, then entropy minimization encourages faster convergence to lower final entropy.After the EMA reaches its global maximum, αk becomes −1 and macro-exploitation begins.Source: §3.1 · p. 5

3.2. Efficiency Considerations: V-Reason(Lite)

  • V-Reason reaches solutions with fewer output tokens than CoT-SFT and RL models, reducing inference time and computation despite its added optimization.The controller is limited to the last layer’s cache, reducing activation storage; V-Reason(Lite) additionally prunes 50% of video tokens from all decoder-layer KV caches and halves controller size.Source: §3.1 · p. 5, §3.2 · p. 6

3.1. Inference-time Optimization

  • Under stated assumptions, one loss-gradient step changes entropy by at most a bound, entropy remains in [0, log n], and EMA smoothing attenuates fluctuations and delays maxima.The EMA also enforces bounded oscillations by switching αk to −1 at a new global EMA maximum.Source: §3.1 · p. 6

3.2. Efficiency Considerations: V-Reason(Lite)

  • Pruning the lowest-norm 50% of video tokens can slightly improve reasoning performance on some datasets, and preserving the unpruned cache magnitude reduces pruning error.The controller is applied only to the remaining video tokens in the last decoder layer.Source: §3.2 · p. 6

4. Experiments

Across video reasoning benchmarks, V-Reason and V-Reason(Lite) improve instruction-tuned baselines and narrow the accuracy gap to Video-R1, while reducing output length. The experiments also examine task scope, model scale, decoding strategies, and entropy-loss alternatives.

4. Experiments

  • Experiments evaluate Qwen2.5-VL-Instruct across six video reasoning benchmarks, covering multiple-choice QA and regression with accuracy and Mean Relative Accuracy.The setup includes 16/32-frame settings and maximum video-pixel settings of 256/128.Source: §4 · p. 6

4.1. Video Reasoning Benchmark Results

  • V-Reason is most effective when the pretrained model already has relevant knowledge; on underrepresented VSI-MRA, it still improves over baseline by +1.4%.For a comparable compute budget, accuracy improves with more frames; reported average gains are +1.4% at 32 frames and +1.0% at 16 frames.Source: §4.1 · p. 7
  • At 7B, V-Reason/V-Reason(Lite) reach 55.9%/56.0% average accuracy at 128 px, within 0.6% of Video-R1-7B’s 56.6%.At 256 px, they reach 54.8%/54.6%; both variants average gains of [+0.9,1.3]% over Qwen2.5-VL at 3B.Source: §4.1 · p. 7, Table 1 · p. 7
  • V-Reason reduces output tokens by 58.6% versus Video-R1, with latency savings of up to 67% and 37% on average.The reported inference-time gains hold for both Full and Lite variants and stem from the proposed optimization rather than pruning.Source: §4.1 · p. 7
  • V-Reason generalizes across backbones and scales on MMVU, gaining +5.1% on Qwen3-VL, +3.0% on 32B, and +0.4% on 72B.For the 32B model, the reported scores are 72.0 vs. 69.0; for 72B, 73.0 vs. 72.6.Source: §4.1 · p. 7, Table 2 · p. 7
  • For Qwen-2.5-VL-7B, V-Reason(Lite) with best min-p decoding gains +2.0 points on MMVU, while best top-H yields +0.4; at higher temperatures it restores min-p accuracy (+6.3%).V-Reason-7B (Lite) achieves the highest score of 65.0.Source: §4.1 · pp. 7–8
  • In the Figure 3 example, V-Reason’s delayed entropy peak and lower overall entropy accompany an alternative reasoning path that reaches the correct answer.The baseline initially follows a promising path, then diverges to an incorrect answer.Source: §4.1 · p. 8, Figure 3 · p. 8
  • Always-increasing or always-decreasing entropy losses are inferior to V-Reason; even basic macro-exploration-only or macro-exploitation-only losses improve over baseline.The alternatives fail to reproduce the later-peak, lower-final-entropy dynamics associated with better reasoning.Source: §4.1 · p. 8

5. Conclusion

V-Reason uses an entropy-guided value-cache controller at inference, with a Lite variant that prunes low-norm cache entries. Across benchmarks, it narrows the gap to RL-trained models while reducing output tokens and inference time.

  • V-Reason uses a training-free value-cache controller and entropy objective to reinforce micro-exploration and micro-exploitation, delaying the entropy peak and lowering final entropy.V-Reason(Lite) prunes low l2-norm value-cache entries.Source: §5 · p. 8
  • Across benchmarks, V-Reason narrows the gap to RL-trained models such as Video-R1 to within 0.6%, while reducing output tokens by 58.6% and inference time by 37%.Source: §5 · p. 8
  • The method improves performance across 3B–72B model scales and remains robust to variations in frame sampling, pixel resolution, decoding techniques, and hyperparameters.Source: §5 · p. 8

A. Theoretical Analysis: Bounding Entropy under Switching Loss

Entropy Switching Loss bounds entropy updates and uses EMA-based switching to control exploration–exploitation oscillations; inference-time V-Reason variants are faster than Video-R1 on most evaluated benchmarks and Lite reduces memory.

A. Theoretical Analysis: Bounding Entropy under Switching Loss

  • Under the stated smoothness and boundedness assumptions, a gradient step changes entropy by a bounded amount, and entropy remains within [0, log n].The bound depends on the step size and bounded gradients; the vocabulary size n is finite.Source: A. Theoretical Analysis: Bounding… · p. 12
  • EMA smoothing suppresses fast entropy fluctuations, delays peaks, and switches the objective to entropy minimization after a global EMA maximum.This bounds oscillations by promoting increases during sustained entropy rises and decreases after a peak.Source: A. Theoretical Analysis: Bounding… · pp. 12–13

B. Implementation Details

  • The implementation uses AdamW without weight decay, EMA factor β = 0.98, gradient clipping at 1.0, and a default controller setting k = 4.The best accuracy is reported over ten learning rates from 5e-5 to 5e-4.Source: B. Implementation Details · p. 13

C. Inference Time and GPU Memory

  • V-Reason cuts inference time by approximately 20–67% versus Video-R1 across evaluated benchmarks, while narrowing the accuracy gap to within 0.6%.The exception is VideoMMMU; V-Reason adds 0.23 seconds versus the unpruned version when using Lite.Source: C. Inference Time and GPU Memory · p. 14
  • V-Reason(Lite) reduces average memory requirements by 11.6% versus full V-Reason and stays below 32GB peak GPU memory on tested datasets.Memory drops by 20% on datasets with longer output sequences such as VideoMMMU.Source: C. Inference Time and GPU Memory · p. 14

D. Analysis on Video duration

Video duration affects V-Reason and its Lite variant differently: the full model improves across the duration groups, while Lite loses performance on medium and long videos.

D. Analysis on Video duration

  • The evaluation compares full and Lite V-Reason across short, medium, and long VideoMME clips.Table 7 reports a breakdown by duration category.Source: D. Analysis on Video duration · pp. 14–15
  • The full V-Reason model improves on short videos by 1.8%, medium videos by 0.8%, and long videos by 0.9%.The duration groups are short (under two minutes), medium (4–15 minutes), and long (30–60 minutes).Source: D. Analysis on Video duration · pp. 14–15
  • V-Reason(Lite) improves short-video performance by 1.8%, comparable to the full model, but declines on medium and long videos.The authors attribute the longer-video decline to pruning that may remove important temporal or contextual details.Source: D. Analysis on Video duration · p. 15

E. Image Understanding Results

V-Reason also improves image-understanding performance over Qwen2.5-VL-7B, with modest gains on benchmarks that are non-reasoning-centric and already saturated.

E. Image Understanding Results

  • V-Reason gains an average of +0.6% over Qwen2.5-VL-7B on image-understanding benchmarks, with gains exceeding +1% on TextVQA and MMVet.The authors describe the gains as modest given the non-reasoning-centric benchmarks and strong baseline performance.Source: E. Image Understanding Results · p. 15
  • The authors frame the image-benchmark gains as modest because the benchmarks are non-reasoning-centric and the baseline is highly saturated.Source: E. Image Understanding Results · p. 15
  • The image results suggest inference-time optimization benefits may extend beyond video reasoning to general visual reasoning tasks.The authors connect these improvements to controlling exploration–exploitation behavior at inference without RL or additional training.Source: E. Image Understanding Results · p. 15

F. Results on SFT/RL trained models

V-Reason improves both CoT-SFT and RL-trained models without additional training, supervision, or parameter changes, while the authors frame it as complementary to training-based approaches.

F. Results on SFT/RL trained models

  • The authors characterize V-Reason as inference-time control that surfaces and refines existing reasoning capacity rather than fundamentally expanding it.Source: F. Results on SFT/RL trained models · p. 15
  • V-Reason improves both CoT-SFT and RL-trained models, with gains of +0.2–0.7% for RL models and up to +0.9% for CoT-SFT models.The authors report larger gains for CoT-SFT models, whose reasoning behavior is less optimized.Source: F. Results on SFT/RL trained models · p. 15
  • The reported gains require no additional training, supervision, or modification of model parameters, and the method remains complementary to SFT and RL.A +5.1% gain is reported for Qwen-3-VL-8B.Source: F. Results on SFT/RL trained models · p. 15

G. Ablation Studies

The ablations find that inference-time optimization drives the reasoning gains, remains robust to a fixed learning rate, and improves over the base model across tested step-sizes. Structured entropy regulation and alpha switching are associated with delayed entropy peaks, while smaller step-sizes trade inference efficiency for accuracy.

Pruning-Only

  • Combining V-Reason with pruning raises the average gain from −0.2 to 1.3, indicating that reasoning gains derive mostly from inference optimization.Pruning alone maintains original performance with a −0.2% average change across datasets.Source: G. Ablation Studies · pp. 15–16, Table 8 · p. 16

Learning rate

  • A fixed 3e-4 learning rate maintains the reported average performance, with similar gains on VSI-Bench, VideoMMMU, and MMVU and negligible drops on three other datasets.The reported drops are on MVBench, Tempcompass, and VideoMME.Source: G. Ablation Studies · p. 16, Table 8 · p. 16

Optimization Step-size

  • V-Reason outperforms the base model at every tested step-size on MMVU, while smaller step-sizes improve accuracy at the cost of slower inference.Smaller step-sizes entail more optimization steps; the ablation is shown in Figure 4.Source: G. Ablation Studies · p. 16

Optimization Objective

  • Alternative objectives fail to reproduce the later entropy peak and lower final entropy associated with better reasoning, supporting structured entropy regulation.The paper reports that the entropy-switching objective outperforms min-entropy and max-entropy alternatives in Table 3.Source: G. Ablation Studies · p. 16

Alpha Switching

  • Before the EMA peak, V-Reason uses fewer α = 1 micro-exploration steps and more α = −1 micro-exploitation steps, lengthening macro-exploration and delaying the entropy peak.Figure 5 plots the alpha histogram before the EMA peak.Source: G. Ablation Studies · p. 16

H. Entropy Curves with Other Models

Entropy patterns associated with stronger models recur across Llama, Phi, and Qwen analyses, although absolute entropy values are not directly comparable across differing model recipes and tokenizers. The paper also identifies limits tied to pretrained knowledge, video pruning, and absent task-specific supervision, and outlines possible future directions.

H. Entropy Curves with Other Models

  • Llama 3.1-8B-Instruct, the larger and higher-performing model, has a delayed entropy maximum relative to Llama 3.2-3B-Instruct on 100 MATH samples.Both curves show macro-exploration and macro-exploitation with micro-cycles; absolute entropy values are not directly comparable across training recipes and tokenizers.Source: H. Entropy Curves with Other Models · p. 16, Figure 6 · p. 17
  • Phi 3-mini-4k-Instruct entropy trajectories show the same macro-exploration and macro-exploitation pattern observed in Qwen and Llama analyses.The Phi analysis averages 100 MATH samples.Source: H. Entropy Curves with Other Models · p. 16
  • For Qwen2.5 LLMs, the more accurate 7B model shows a delayed peak and lower entropy overall, but the value-cache controller is not directly feasible for text-only inputs.The text-only inputs lack the distinct video-token modality embeddings used by the controller.Source: H. Entropy Curves with Other Models · p. 16, Figure 7 · p. 17

I. Limitations

  • V-Reason may provide only modest gains on tasks under-represented in the pretrained model, because its alternative-path exploration relies on pretrained knowledge.Source: §I · p. 17
  • V-Reason(Lite) improves memory efficiency but measurably loses accuracy on medium- and long-duration videos, where pruning may discard useful temporal information.Source: §I · p. 17

J. Future Work

  • The authors identify V-Reason as the first work targeting video reasoning without training and suggest integrating its inference-time entropy objective into training as a possible route to stronger gains.They also propose exploring alternative inference-time metrics and loss functions.Source: J. Future Work · p. 17
  • Without task-specific supervision, V-Reason may capture nuanced reasoning strategies less effectively than reinforcement-learning approaches; combining supervised fine-tuning with inference-time optimization is proposed.Source: J. Future Work · p. 17
  • The authors report similar entropy trends for video and language models on MATH, while describing extension to LLMs as a future direction and noting the controller's text-only input constraint.The proposed value-cache controller is not directly feasible for text-only inputs lacking distinct video-token modality embeddings.Source: H. Entropy Curves with Other Models · p. 16, J. Future Work · p. 17

K. Additional Qualitative results

Qualitative examples show V-Reason correcting several baseline errors across varied video tasks, while other examples show both systems answering correctly with differing reasoning traces. Across those latter examples, V-Reason has longer exploration and lower overall entropy, but its output-token use varies.

K. Additional Qualitative results

Loading 2510.17045v2…