Source-linked AI summary
Self-Supervised Visual On-Policy Distillation
Yijiang Li, Yijun Liang, Yunjie Tian, Bingyang Wang, Ke Zhang, Zhenfei Yin, Di Fu, Philip Torr, Nuno Vasconcelos
TL;DR
Visual on-policy distillation lacks informative teacher–student asymmetry when privileged supervision or a stronger teacher is unavailable. S2VOPD creates that asymmetry by degrading the student’s view, improving Qwen3.5-4B from 70.7% to 77.4% across six fine-grained perception benchmarks.
Problem
On-policy distillation needs an informative teacher–student asymmetry, but conventional approaches obtain it from stronger teachers or privileged supervision.
Method
S2VOPD distills predictions from an EMA teacher viewing the original image into a student operating on an augmented, degraded view without privileged supervision.
Results
Across six fine-grained perception benchmarks, S2VOPD improves Qwen3.5-4B from 70.7% to 77.4% average accuracy while improving both fine-grained perception and mathematical reasoning.
Takeaways & Limitations
S2VOPD shows that withholding task-relevant visual information from the student can provide an effective self-supervised distillation signal without privileged annotations.
Takeaways & Limitations
Augmentations must preserve question-relevant information, because larger visual gaps can become uninformative when required evidence is removed.
Abstract
from arXiv · showhide
Visual on-policy distillation relies heavily on an informative teacher-student asymmetry, through either a larger, stronger teacher or privileged supervision, such as reference answers or ground-truth regions of interest. This raises a fundamental question: where can informative asymmetry come from when nothing privileged is available? We answer this by inverting where the asymmetry comes from. Rather than adding privileged information to the teacher, we subtract information from the student. This asymmetry creates the same effective learning signal for free as a teacher with access to information unavailable to the student, without ground-truth annotations, rewards, or a separate stronger teacher model. Building on this principle, we introduce Self-Supervised Visual On-Policy Distillation (S$^2$VOPD), a simple yet effective method that constructs on-policy learning signals from asymmetric augmented views. S$^2$VOPD distills the teacher's distribution conditioned on the original image on-policy into the student distribution conditioned on a strongly augmented view of the same image. We systematically explore a broad design space of visual augmentations and uncover that (1) asymmetry matters: all four augmentation families improve performance, while symmetric self-distillation degrades it; (2) strength matters: performance peaks at a moderate strength; and (3) the gap must remain task-consistent: augmentations that completely remove the question-relevant evidence can induce large but uninformative discrepancies. Across six fine-grained perception benchmarks, S$^2$VOPD improves Qwen3.5-4B from 70.7% to 77.4%, above all open-source models compared, up to Qwen3-VL at 235B, and surpasses GPT-5.4. While holding training data the same, it recovers 96% of the improvement achieved by methods with privileged information. Website is at https://williamium3000.github.io/s2vopd
1 Introduction
S2VOPD creates informative teacher–student asymmetry without privileged supervision by giving the teacher the original image and the student an augmented view. A controlled augmentation study identifies moderate, task-consistent gaps as effective, yielding substantial benchmark gains for Qwen3.5-4B.
- Method: S2VOPD withholds visual information from the student rather than adding privileged information to the teacher, creating a predictive discrepancy without ground-truth annotations or a stronger teacher.The teacher observes the original image, while the student operates on a degraded view of the same input.
- Method: The method uses an exponentially moving average teacher conditioned on the original image and distills its distribution into student trajectories generated from an augmented view ˜x = T(x).Given image x and question q, the student generates on-policy trajectories from the transformed view.
- Augmentation study: All four asymmetric augmentation families improve performance, whereas symmetric self-distillation degrades it; performance peaks at a moderate teacher–student gap.The gap must also remain task-consistent rather than removing question-relevant evidence.
- Results: 6.7% gain: S2VOPD raises Qwen3.5-4B average accuracy from 70.7% to 77.4% across six fine-grained perception benchmarks, exceeding Qwen3-VL-Instruct-235B and GPT-5.4 and matching Qwen3.5-397B.The compared accuracies are 75.8% for Qwen3-VL-Instruct-235B and 72.8% for GPT-5.4.
2 Related Work
Prior work derives self-supervision from augmented views, while augmentation is less common in vision-language training because perturbations may alter instruction-relevant content. On-policy distillation reduces train–inference mismatch but typically requires external guidance; this work instead creates asymmetry by augmenting the student’s view.
- Self- and semi-supervised learning: Self-supervised methods align augmented views or match students to stop-gradient or momentum teachers, including SimCLR [Chen et al., 2020], MoCo [He et al., 2020], BYOL, SimSiam [Chen and He, 2021], and DINO [Caron et al., 2021].
- Visual augmentation in vision-language models: Augmentation is less common in vision-language training because geometric or photometric changes can alter instruction-relevant text, object identity, and spatial relations.
- Knowledge distillation and on-policy distillation: On-policy distillation trains students on their own trajectories with dense token-level teacher supervision, reducing the train–inference mismatch of offline distillation.
- Knowledge distillation and on-policy distillation: This work replaces privileged teacher information with a strongly augmented student view and studies which augmentation properties make the resulting teacher–student discrepancy effective.
3 Method
S^2VOPD creates teacher–student asymmetry by degrading only the student’s visual input, then distills clean-view EMA-teacher distributions onto the student’s own trajectories. Its augmentation framework systematically varies visual information, transformation strength, composition, and application probability, with downscaling plus Gaussian noise as the default recipe.
- Asymmetric on-policy distillation: S^2VOPD withholds visual information from the student while an EMA teacher retains the clean image, creating on-policy supervision without labels, rewards, privileged annotations, or a stronger teacher.The teacher is updated by exponential moving average, and the augmentation is the sole source of supervision.
- Asymmetric on-policy distillation: The student samples rollouts from an augmented view, while the teacher evaluates identical prefixes from the clean view and transfers its next-token distributions.The divergence is evaluated at prefixes sampled from the student policy, making supervision on-policy at inference-time states.
- On-policy objective: Training uses a generalized Jensen–Shannon divergence with α = 0.5, restricting both distributions to the teacher’s top-k tokens to stabilize optimization.The generalized divergence interpolates forward and reverse KL and remains bounded when distribution overlap is limited.
- Augmentation space: The augmentation space varies information-reduction, geometric, photometric, and occlusion operators through controllable strengths, probabilities, composition, and global application rates.Information reduction includes downscaling, blur, pixelation, filtering, noise, and visual-token dropping; occlusion removes localized regions.
- Best recipe: The default transformation always downsamples the student view and independently adds DDPM-style Gaussian noise with probability 0.5 at t = 200.Downscaling reduces visual tokens without restoring resolution, while the noise has standard deviation approximately 0.11 and lowers rollout and forward-pass costs.
4 Experiments
Across six perception benchmarks, S2VOPD substantially improves Qwen3.5-4B without privileged annotations and outperforms open-source and privileged-information baselines. Experiments show that effective self-supervised distillation requires asymmetric, moderately strong, task-consistent augmentations, while performance is robust to teacher-update details.
- Overall results: S2VOPD raises Qwen3.5-4B average accuracy from 70.68% to 77.44% (+6.76%), outperforming all open-source models including Qwen3-VL-Instruct-235B at 75.75% and matching Qwen3.5-397B at 77.44%.The comparison uses six fine-grained perception benchmarks and includes models trained with privileged information.
- Fair comparison: On Vision-OPD-6K, S2VOPD achieves the best average among non-privileged methods, improving over the strongest self-rewarding baseline by 2.0% at 4B and 0.7% at 9B.It exceeds all supervised baselines at 4B and matches Vision-OPD as runner-up overall at 9B without privileged annotations.
- Augmentation analysis: Every augmentation family improves the base model, whereas symmetric self-distillation falls to 65.21% from 70.58%; information reduction performs best at 75.65%.Photometric, geometric, and occlusion augmentations reach 74.40%, 74.30%, and 72.44%, respectively, under the fixed analysis protocol.
- Augmentation analysis: Performance peaks at a moderate teacher–student gap and declines when augmentation becomes too weak or too strong.The gap is measured by token-level JS divergence averaged over the first ten training steps, and useful asymmetry depends on both gap magnitude and semantics.
- Augmentation analysis: Task-inconsistent gaps hurt performance: cropping declines from 71.53% to 68.76% and 67.44% as strength increases, despite the strongest crop producing the largest predictive gap.The result shows that augmentations must preserve question-relevant information; downscaling with Gaussian noise is the most robust default.
- Ablations: Removing augmentation collapses performance to 70.52% versus the 70.68% base, while freezing the teacher costs only 0.40% and still reaches 75.95%.Performance remains within 0.8% across teacher decay values from 0.95 to 0.999, indicating that view asymmetry matters more than teacher self-improvement or the precise update rate.
5 Conclusion
S2VOPD creates informative teacher–student asymmetry without privileged annotations by subtracting task-relevant information from the student view. Its effectiveness depends on the magnitude and semantics of the induced information gap.
- 5 Conclusion: S2VOPD creates informative teacher–student asymmetry without privileged annotations by subtracting task-relevant information from the student view.The teacher observes the original image while the student learns from a degraded view.
- 5 Conclusion: The information gap turns the model’s own predictions into a perception-aligned training signal.
- 5 Conclusion: S2VOPD’s effectiveness is governed by the magnitude and semantics of the induced information gap.