Source-linked AI summary
V-Zero: Answer-Label-Free On-Policy Distillation with Contrastive Evidence Gating for Fine-Grained Visual Reasoning
Haoxiang Sun, Zhihang Yi, Langxuan Deng, Yuhao Zhou, Peiqi Jia, Jian Zhao, Li Yuan, Jiancheng Lv, Tao Wang
TL;DR
Fine-grained visual reasoning needs grounded local evidence, but existing reinforcement-learning and supervised approaches require costly exploration, verification rules, or extensive textual labels. V-Zero uses contrastive teacher-side visual evidence gating for answer-label-free on-policy distillation, improving benchmark performance while reducing training cost.
Problem
Fine-grained visual reasoning lacks an approach that avoids costly RL exploration, predefined verification rules, and large-scale textual answer labels.
Method
V-Zero contrasts teacher replays under positive target-region and negative visual views to gate dense token-level distillation of student-sampled trajectories.
Results
V-Zero improves fine-grained visual reasoning by 3.1 points on average while cutting training cost by over 5× versus SFT and over 10× versus RL.
Takeaways & Limitations
V-Zero supports answer-label-free visual reasoning with strong generalization and standard full-image inference at test time.
Takeaways & Limitations
V-Zero assumes teacher-side access to privileged information, such as localized crops, that is unavailable to the student.
Abstract
from arXiv · showhide
Fine-grained visual reasoning requires multimodal large language models (MLLMs) to identify task-relevant visual evidence and ground their reasoning in local image regions. Existing agentic methods typically rely on reinforcement learning with verifiable rewards or supervised fine-tuning on large-scale annotated reasoning traces, leading to costly exploration, hand-designed verification rules, or heavy dependence on textual supervision. A natural way to avoid such external answer labels is to learn from trajectories sampled by the student itself, which points to On-Policy Distillation (OPD). To understand what OPD can and cannot provide for visual reasoning, we revisit it as negative-free stop-gradient alignment. This perspective shows that, although OPD provides effective token-level correction, its ceiling is constrained by the absence of trajectory-level discrimination. Motivated by these observations, we propose V-Zero, an answer-label-free framework for visual reasoning with contrastive evidence gating. V-Zero uses no annotated textual answer labels; instead, during training it pairs a question-relevant regional crop with a negative visual view to evaluate student-sampled trajectories and gate dense token-level distillation. Experiments on multiple visual reasoning benchmarks show that V-Zero consistently improves fine-grained visual reasoning while preserving strong generalization. Notably, V-Zero is more than 5$\times$ faster than previous supervised fine-tuning methods and more than 10$\times$ faster than reinforcement learning baselines. Code and dataset will be released at https://github.com/eVI-group-SCU/V-Zero
Introduction
Fine-grained visual reasoning requires MLLMs to inspect local details and ground reasoning in task-relevant image regions, while existing RL and SFT approaches incur costly exploration or extensive textual supervision. V-Zero addresses these limitations by adding contrastive evidence gating to answer-label-free on-policy distillation, improving reasoning efficiency and performance.
- Motivation: Fine-grained visual reasoning requires models to inspect local details, identify task-relevant evidence, and reason over specific image regions.This distinguishes it from general visual understanding.
- Limitations of Existing Methods: Existing agentic approaches often rely on reinforcement learning with costly exploration and predefined verifiable training rules, while SFT requires large-scale annotated image-text data.These requirements motivate alternatives that reduce exploration and textual supervision.
- On-Policy Distillation: Standard OPD provides dense supervision on student-sampled trajectories but lacks explicit trajectory-level discrimination when reasoning drifts into erroneous paths.It treats all student-generated prefixes uniformly, limiting correction to token-level guidance.
- V-Zero: V-Zero contrasts paired positive and negative visual evidence views to estimate rollout reliability and gate dense answer-label-free on-policy distillation.The teacher replays each student trajectory under both evidence views while retaining the student-side rollout process.
- Results: 3.1 points is V-Zero’s average improvement over the Qwen3.5-4B base model across multiple visual reasoning benchmarks.The gains come from training-time visual evidence crops rather than ground-truth answer labels, while preserving strong generalization.
- Results: Less than half is the computational budget V-Zero uses compared with prior methods, while preserving strong generalization.V-Zero therefore combines answer-label-free training with improved efficiency.
Revisiting OPD as Negative-Free · Stop-Gradient Alignment
The paper reframes OPD as negative-free stop-gradient alignment that delivers dense token-level correction on student-induced states but lacks trajectory-level discrimination. Its privileged-information view clarifies the asymmetric student–teacher setup and motivates evaluating whole reasoning paths rather than only local continuations.
- Stop-Gradient Alignment: OPD provides dense token-level correction by matching student predictions to teacher targets on sampled prefixes, but lacks trajectory-level discriminative supervision.This establishes the paper’s initial characterization of OPD before introducing V-Zero.
- On-Policy Distillation with Teacher-Side Views: OPD trains on states generated by the student itself, treating sampled trajectories as stop-gradient data for teacher-guided correction.The teacher is queried on the same student-induced prefixes, and the student matches teacher predictions on visited states.
- On-Policy Distillation with Teacher-Side Views: Students may sample groups of G on-policy trajectories per prompt, with the standard single-rollout case recovered when G = 1.The resulting group rollout distribution is denoted πG_s(· | x).
- On-Policy Distillation with Teacher-Side Views: With privileged information, the student samples from the original prompt x while the teacher conditions on additional information z unavailable to the student.Examples include a localized crop or a reference solution.
- An Asymmetric Alignment View of OPD: OPD aligns asymmetric student and teacher views that induce predictive distributions over the same next-token decision, with the student as the online branch and teacher as a fixed target.Standard OPD shares context across views, while teacher-side information augments the teacher view.
- An Asymmetric Alignment View of OPD: OPD is therefore a negative-free stop-gradient alignment objective, but it cannot explicitly score whether a full trajectory is correct.After an erroneous reasoning path, the teacher supplies only local next-token targets conditioned on that prefix, allowing locally plausible continuations without trajectory-level assessment.
Method
V-Zero augments on-policy distillation with contrastive evidence gating: the student samples trajectories from the full image, while the teacher compares positive and negative visual views to estimate rollout reliability. It then gates positive-view token-level distillation, providing dense on-policy supervision without annotated textual answers or external rewards.
- Evidence-conditioned rollouts: The student samples sibling trajectories from the full image, while the teacher replays each with the original image plus paired positive and negative evidence views.The positive view is a task-relevant target-region crop; the negative view is an equal-size crop sampled outside that region after 2× downsampling.
- Contrastive evidence gating: The positive-minus-negative teacher support gap is aggregated into a trajectory-level evidence score and normalized among sibling rollouts under each prompt.The normalized evidence advantage measures whether a rollout is better visually grounded than its siblings.
- Contrastive evidence gating: A clipped, non-negative stop-gradient gate strengthens OPD for visually supported rollouts and suppresses rollouts whose teacher support does not improve with relevant evidence.Clipping bounds keep the OPD update stable.
- Positive-view distillation: After evidence comparison, V-Zero discards the negative view and distills only from the positive teacher distribution using a gated sampled reverse-KL objective.Evidence views determine how much to learn from each rollout, while the OPD target remains the positive teacher distribution.
- Training objective: This separation constructs dense on-policy supervision without annotated textual answer labels or external reward signals.Paired visual evidence views provide trajectory-level discrimination, while OPD supplies token-level imitation.
Experiments
V-Zero improves fine-grained visual reasoning over the Qwen3.5-4B backbone and achieves competitive cross-system performance without annotated textual answers. Ablations attribute these gains to contrastive evidence gating, while training is substantially faster than prior baselines.
- Efficiency: V-Zero takes 4.8 h, compared with ∼1 day for ZwZ and ∼2 days for DeepEyes, corresponding to > 5× and > 10× speedups.The speedups are conservative because V-Zero uses RTX PRO 6000 GPUs with weaker practical BF16 throughput than the H100 baselines.
- Main results: +4.7 on VStar, +3.4 on HR-4K, +2.0 on HR-8K, and +5.5 on ZoomBench versus Qwen3.5-4B.These gains are obtained with the same inference setting and strengthen reasoning over high-resolution, localized visual evidence.
- Main results: V-Zero achieves the best scores among visually grounded reasoning systems on HR-4K, HR-8K, ZoomBench, and MMStar.Because compared systems use different backbones, this is a cross-system result rather than a controlled backbone-matched ablation.
- Training setup: V-Zero uses paired positive and negative visual evidence views instead of annotated textual answer labels, reasoning traces, or final answers.The positive crop preserves the relevant region, while the negative crop is a 2× downsampled equal-size region sampled from an irrelevant area.
- Ablations: The 27B-to-4B configuration gives the higher perception average; a 9B teacher is stronger on VStar and HR-8K, while a 27B teacher leads on HR-4K and ZoomBench.These comparisons come from teacher–student size ablations using the same 4B student.
Discussion and Related Work
Fine-grained visual reasoning requires models to identify localized visual evidence, motivating agentic methods that interleave reasoning with visual operations. V-Zero instead studies OPD through negative-free alignment and trajectory-level contrastive evidence gates from paired visual views.
- Agentic Visual Reasoning: Fine-grained multimodal reasoning depends on identifying and using small, critical visual evidence.Standard MLLMs struggle when answers require localized visual search rather than global scene understanding.
- Agentic Visual Reasoning: Recent agentic methods train MLLMs to interleave reasoning with visual operations and gather new visual observations during inference.
- Related Work: 2026a extends OPD to temporal video grounding, where teacher token-level supervision on on-policy trajectories outperforms GRPO with faster convergence and lower computational cost.
- Novelty: V-Zero studies OPD for fine-grained visual reasoning as negative-free stop-gradient alignment with trajectory-level contrastive evidence gates.The gates convert teacher-side evidence comparisons under paired positive and negative visual evidence views into trajectory-level supervision.
Conclusion
V-Zero improves fine-grained visual reasoning without annotated textual answer labels by addressing a limitation of standard token-level OPD. It introduces trajectory-level discrimination through sibling rollouts and teacher-side positive and negative visual evidence views.
- V-Zero improves fine-grained visual reasoning without annotated textual answer labels.
- Standard token-level distillation on student-induced prefixes lacks trajectory-level discrimination.The framework is motivated by a negative-free stop-gradient alignment view of OPD.
- V-Zero samples sibling rollouts from the full image and replays them with teacher-side positive and negative visual evidence views.