PaperlayerView exact revision on arXiv

Source-linked AI summary

VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation

Kangning Zhang, Yixing Li, Shuai Shao, Qingyao Li, Zhengxi Lu, Zhiyuan Yao, Jianghao Lin, Wenxiang Jiao, Yuan Lu, Weiwen Liu, Weinan Zhang, Yong Yu

arXiv:2607.28590v1cs.CVcs.CL

TL;DR

Multimodal on-policy distillation mixes visual evidence with linguistic and teacher-specific effects, making visual support for corrections difficult to estimate. VAD uses counterfactual evidence interventions to reconstruct student-anchored targets, and consistently outperforms direct privileged-view distillation and visual-advantage weighting across six benchmarks at 4B and 9B scales.

  • Problem

    Existing multimodal on-policy distillation methods retain source-mixed teacher corrections, limiting attribution of corrections supported by visual evidence.

  • Method

    VAD contrasts evidence-present and evidence-removed teacher views, projects corrections onto the resulting signed visual proxy, and reconstructs a student-anchored target.

  • Results

    Across six fine-grained visual benchmarks and 4B and 9B students, VAD consistently improves over direct privileged-view distillation and visual-advantage weighting.

  • Takeaways & Limitations

    Attribution-based reconstruction concentrates visual, task-relevant corrections and strengthens correct-token support and wrong-token suppression over direct source-mixed distillation.

  • Takeaways & Limitations

    A single contrastive vector may bias compositional evidence, while the attributed component can retain nonvisual teacher effects and the residual remains source-mixed.

Abstract

from arXiv · show

Multimodal on-policy distillation (OPD) transfers fine-grained visual knowledge by supervising student-generated trajectories with a privileged-view teacher. Yet its next-token corrections are source-mixed, combining visual signals with linguistic priors and teacher-specific effects. The key challenge is to estimate which corrections are supported by visual evidence, not merely where or how strongly to distill. We introduce Visual Attribution Distillation (VAD), a counterfactual target-reconstruction algorithm that estimates the visually attributable part of a teacher correction. At each student-generated prefix, VAD evaluates the same fixed teacher with the relevant evidence present and removed. The corresponding change in centered log-probabilities defines ut, a signed proxy for the visual evidence direction that estimates how revealing the evidence supports or refutes candidate tokens. VAD projects the original correction onto this proxy to obtain an intervention-aligned component and a proxy-unexplained residual, then reconstructs a student-anchored target from the former. During training, this reconstructed target supplies the primary supervision signal, while the privileged teacher contributes a weak regularizer. Across six fine-grained visual benchmarks at 4B and 9B scales, VAD outperforms direct privileged-view distillation and visual-advantage weighting. Token- level and controlled-target analyses show that the proxy-aligned component is enriched in task-relevant visual corrections and yields stronger target shifts, especially when evidence refutes a mistaken answer. These results support counterfactual target reconstruction as an effective alternative to source-mixed supervision.

1 Introduction

The introduction identifies source-mixed teacher corrections as a core limitation of privileged multimodal on-policy distillation and proposes Visual Attribution Distillation (VAD) to isolate intervention-aligned visual evidence. VAD reconstructs student-anchored targets and consistently improves across six fine-grained visual benchmarks at 4B and 9B scales.

  • Motivation: Only 23.2% of strongest response-token corrections for 4B and 22.8% for 9B are strongly aligned with the teacher’s evidence-conditioned response.This diagnostic motivates attributing which teacher-correction component is supported by visual evidence.
  • Problem: Direct privileged-teacher matching and positive visual-advantage weighting do not explicitly estimate which corrections are supported by controlled visual interventions.Positive visual advantage can miss refutation when evidence lowers the probability of a mistaken student token.
  • Method: VAD queries the same fixed teacher with evidence present and removed, using the centered-log-probability shift as an intervention-derived proxy for visual evidence direction.The textual context remains unchanged between the two teacher queries.
  • Method: VAD projects the original teacher correction onto the proxy, treats the remainder as proxy-unexplained, and reconstructs a target around the full-image student distribution.The method separates visual support from refutation and applies a weak teacher-correction regularizer for stable optimization.
  • Results: Across six fine-grained visual benchmarks and 4B and 9B students, VAD consistently improves over direct privileged-view distillation and visual-advantage weighting.Semantic token analyses and controlled target studies support the intended interpretation of the proxy-aligned correction and its support–refutation behavior.

2 Related Work

Prior work advances distillation through student-induced states, privileged information, improved visual evidence acquisition, and visual interventions. VAD instead focuses on attributing privileged teacher corrections and reconstructing supervision from evidence-supported components.

  • Knowledge distillation: Recent distillation methods train on student-induced states using reverse KL, unified divergences, skew KL, adaptive generations, and related objectives.The passage names MiniLLM, GKD, DistiLLM, and DistiLLM-2 as examples.
  • Privileged distillation: Privileged distillation strengthens teachers with information unavailable to deployed students, including solution histories, visual-thought traces, partially privileged guidance, and reasoning contrasts.Examples include On-Policy Context Distillation, Visual-OPSD, Anchored Residual OPD, and Contrastive OPD.
  • Visual evidence acquisition: Visual reasoning models improve resolution and multi-scale detail, yet diagnostic and high-resolution benchmarks still expose failures on decisive local evidence.The passage cites Qwen2-VL, Qwen2.5-VL, LLaVA-UHD, and Mini-Gemini alongside these benchmark limitations.
  • Visual evidence acquisition: View-acquisition methods search, edit, or operate on images to expose fine detail, but they incur inference-time search or tool-use costs.Examples include V*, ZoomEye, ReFocus, DeepEyes, Thyme, DeepEyesV2, and SenseNova-MARS.
  • Region-to-image training: Region-to-image training internalizes privileged local evidence, whereas VAD targets construction by excluding teacher changes not explained by the controlled evidence intervention.The passage contrasts Zooming without Zooming, Vision-OPD, and ViCuR with VAD’s target-construction focus.
  • Visual attribution: Unlike visual-contrast OPD methods that determine where, how strongly, or from which policy to distill, VAD reconstructs supervision by selecting correction components warranted by controlled evidence changes.Decomposed OPD reconstructs a visual target from language-prior and visual-information signals, while VAD projects privileged corrections onto an evidence-intervention proxy.

3 Visual Attribution Distillation

VAD uses counterfactual teacher views at student-generated prefixes to isolate visual evidence responses and reconstruct student-anchored supervision targets. The reconstructed target provides primary JSD supervision, while a weak teacher regularizer preserves non-visual behavior and inference remains a standard full-image student policy.

  • Counterfactual teacher views: VAD reuses each student-generated prefix with a fixed teacher evaluated on evidence-present and evidence-removed views to measure the visual intervention response.The student samples y from the full image, while training-only views x+ and x− produce the counterfactual teacher distributions.
  • Visual attribution: The signed proxy u_t marks tokens whose relative probability rises or falls when evidence is revealed, with magnitude measuring response strength.u_t is treated as a proxy for visual information, not as an oracle grounding label or pixel gradient.
  • Target reconstruction: VAD projects the teacher correction onto the proxy, retaining only agreeing support and refutation shifts, then applies the resulting visual correction to the student distribution.The remaining correction is an unattributed residual, while the norm of the visual component defines the visual-correction budget for refinement.
  • Optimization and limitation: The reconstructed q_VAD target replaces the standard OPD target in token-level JSD supervision, while a weak privileged-teacher regularizer anchors semantics, formatting, length, and stopping.The visual-only ablation caused longer, repetitive responses with delayed commitment and unstable formatting and stopping, motivating the regularizer.
  • Inference: At inference, VAD uses only the optimized full-image student and requires no teacher call or auxiliary visual view.The teacher, crop, and degraded view are used only to construct training targets.

4 Experiments

Across six fine-grained visual benchmarks, VAD delivers the strongest matched post-training accuracy at both 4B and 9B scales, while its counterfactual decomposition concentrates visually grounded decision content. Ablations further show that branch-aware target reconstruction drives the main gain, with benefits preserved on held-out tasks and across robust training settings.

  • Overall accuracy: VAD reaches 78.32 and 79.93 Avg6 at 4B and 9B, leading the best scale-matched alternative by 2.40 and 2.80 points.Against Decomposed OPD, the gains are 2.95 and 2.88 points, respectively.
  • Overall accuracy: VAD exceeds both Vision-OPD and Decomposed OPD on every benchmark at both scales, leading all controlled methods on six 4B evaluations and five of six 9B evaluations.At 9B, VA-OPD leads ZoomBench by 0.35 points.
  • Semantic decomposition: The visual direction assigns 42.0% of relative top-5 composition to visual attributes, objects/content, and A–D decisions, versus 26.7% in the full correction and 17.7% in the residual.Token analyses likewise foreground answer symbols and visual attributes in rvis_t, while rres_t emphasizes linguistic and degradation-related content.
  • Target ablations: Branch-aware reconstruction raises Avg6 from 77.06 to 78.06 without regularization, while the weak anchor lifts the one-sided and VAD targets to 77.52 and 78.32.Full VAD is 0.80 points above regularized one-sided projection and 0.26 points above its target-only counterpart.
  • Held-out generalization: VAD achieves the highest held-out average among post-trained models at 4B and 9B, scoring 79.14 and 82.62 and improving over scale-matched Base by +0.24 and +0.23 points.It is the only post-trained method with a positive ∆ at both scales.
  • Training sensitivity: The best setting, (λ, τ+) = (0.10, 0.80), reaches 78.32, nearby configurations stay within 0.20 points, and JSD leads aggregate accuracy at both scales.JSD reaches 78.32 Avg6 at 4B and 79.93 at 9B.

5 Conclusion and Limitations · Appendix

VAD addresses source-mixed teacher corrections by contrasting evidence-present and evidence-removed views to reconstruct student-anchored targets. Its limitations are that single-view-pair contrastive vectors may bias compositional evidence, while projection does not cleanly separate visual from nonvisual teacher effects.

  • 5 Conclusion and Limitations: VAD contrasts evidence-present and evidence-removed teacher views to attribute visually aligned corrections and reconstruct student-anchored targets.The method uses separate support and refutation budgets.
  • 5 Conclusion and Limitations: Across six benchmarks, VAD leads scale-matched methods at 4B and 9B while preserving Base-level held-out performance.
  • 5 Conclusion and Limitations: Semantic and offline analyses show that the attributed direction concentrates visual evidence.
  • 5 Conclusion and Limitations: Each intervention is represented by one contrastive vector from a single view pair, which may bias compositional evidence.Multiple views or learned directional bases could provide a richer estimate.
  • 5 Conclusion and Limitations: The current projection yields semantic enrichment rather than an identifiable separation of visual and nonvisual effects.The attributed component can retain nonvisual teacher effects.
  • 5 Conclusion and Limitations: The residual remains source-mixed under the current projection.

A Visual-Alignment Diagnostic

The diagnostic evaluates frozen 4B and 9B models on deterministic responses by comparing full-image student predictions with evidence-present and evidence-removed crop teachers. Only 23.2% of top-quartile corrections align visually for 4B, and 22.8% for 9B.

  • Diagnostic setup: The diagnostic uses frozen Qwen3.5-4B and Qwen3.5-9B Base models, each generating deterministic full-image responses for 512 examples at temperature 0 and seed 42.At every response position, the model’s prefix is reused to evaluate the full-image student, evidence-present crop teacher, and evidence-removed crop teacher.
  • Diagnostic setup: The analysis retains the top-128 candidates from each view and evaluates their union on shared support.Correction strength and visual alignment are measured on this shared candidate support.
  • Results: 23.2% of top-quartile 4B teacher corrections also have top-quartile visual alignment, corresponding to 3,727/16,085 examples.The thresholds are τD = 0.217903 and τρ = 0.514454.
  • Results: 22.8% of top-quartile 9B teacher corrections also have top-quartile visual alignment, corresponding to 2,687/11,791 examples.The thresholds are τD = 0.258831 and τρ = 0.481173.

B Controlled Post-Training Algorithms

The controlled comparison holds backbone, data, rollout, and update budgets constant to isolate how six algorithms obtain and apply learning signals. VAD differs by reconstructing student-anchored targets from counterfactual visual attribution rather than directly matching or reweighting teacher supervision.

  • Controlled comparison: All controlled rows share scale-matched initialization, training data, rollout budget, and total update budget, isolating learning-signal and optimization differences.This comparison avoids confounding algorithmic results with backbone size or data volume.
  • GRPO: GRPO optimizes clipped policy ratios from group-normalized answer-correctness rewards, with a reference-policy KL term controlling drift.It increases above-average responses and decreases below-average responses within sampled response groups.
  • Vision-OPD: Vision-OPD directly distills a crop-conditioned teacher into a full-image student by minimizing token-level distributional divergence on student-generated prefixes.Both policies evaluate the same prefixes, but the crop-conditioned teacher distribution serves directly as the local target.
  • VAD: VAD compares evidence-present and evidence-removed teacher views, separates visual support from refutation, and applies the attributed direction to reconstruct a student-anchored target.A budgeted asymmetric transformation produces the reconstructed target, while direct teaching is retained only weakly.
  • Comparison boundary: The six methods use distinct signals: answer-level reward, dense teacher matching, token- and rollout-level reweighting, trajectory gating, target decomposition with gradient steering, and attribution-based reconstruction.These correspond respectively to GRPO, Vision-OPD, VA-OPD, V-Zero, Decomposed OPD, and VAD.

C Training Efficiency and Compute Cost

VAD operates in the same training-cost regime as Vision-OPD and VA-OPD while delivering substantially higher fine-grained visual accuracy. At 4B, its modest timing overhead accompanies Avg6 gains over both baselines.

  • Cost measurement: Training cost is measured over the first 65 optimization steps, or one epoch, using mean step time, standard deviation, and accumulated training time.Runs use 8 GPUs, a global batch size of 96, and 8 rollouts per prompt; benchmark evaluation is excluded from the training-core timer.
  • Cost measurement: Table 5 reports step time as the mean ± standard deviation across 65 optimization steps for matched H800 runs, where lower is better.The comparison covers training efficiency at the 4B and 9B scales.
  • Training cost and accuracy trade-off: 7.84 minutes per step, 8.49 hours per epoch, and 67.9 GPU hours are required by VAD at 4B, with Avg6 gains of 2.40 and 3.40 points over Vision-OPD and VA-OPD.These costs represent only 3.8% more step time than Vision-OPD and 3.1% more than VA-OPD.
  • Training cost and accuracy trade-off: 11.11 minutes per step, 12.03 hours per epoch, and 96.3 GPU hours are required by VAD at 9B.The passage reports these figures as part of the cross-scale training-cost comparison with Vision-OPD and VA-OPD.

D Token-Level Comparison of Distillation Mechanisms · D.1 Unified Token Notation

This section unifies four distillation mechanisms at a fixed student-generated prefix, distinguishing target distributions from loss aggregation. It defines teacher corrections and VAD’s centered counterfactual visual-response coordinate.

  • D Token-Level Comparison of Distillation Mechanisms: The comparison places Vision-OPD, VA-OPD, Decomposed OPD, and VAD in one token-level notation.It builds on the algorithm-level overview in Appendix B.
  • D.1 Unified Token Notation: All mechanisms are compared at a fixed student-generated prefix, for a response position t and shared candidate set V_t.The notation follows the same tail aggregation, numerical stabilization, and normalization used in Section 3.
  • D Token-Level Comparison of Distillation Mechanisms: The framework separates the distribution used as the target from how losses are aggregated across tokens and rollouts.These operations need not have the same effect on the final parameter gradient.
  • D.1 Unified Token Notation: At each candidate token i, p, T, and C denote the full-image student, evidence-present teacher, and evidence-removed teacher distributions.The support assumes p_i > 0, T_i > 0, and C_i > 0.
  • D.1 Unified Token Notation: a_i is the privileged teacher’s prescribed log-probability correction relative to the student, whereas b_i is the fixed teacher’s evidence-conditioned response.b_i compares the evidence-present and evidence-removed views under the same prefix.
  • D.1 Unified Token Notation: VAD centers b_i to form the visual-response coordinate used for attribution.This coordinate captures the fixed teacher’s evidence-conditioned log-probability response for each token.
  • D.1 Unified Token Notation: The common centering term changes no pairwise log-odds.Centering therefore preserves pairwise comparisons while defining VAD’s visual-response coordinate.

D.2 Target Matching, Reweighting, and Reconstruction

The section contrasts target matching, reweighting, and reconstruction strategies in multimodal on-policy distillation. VAD reconstructs a student-anchored target from signed counterfactual visual responses rather than copying the complete privileged-teacher odds.

  • Vision-OPD: Vision-OPD uses the evidence-present crop teacher distribution T directly as the per-position target for the full-image student.Every coordinate of the privileged-teacher correction can affect the local target.
  • VA-OPD: VA-OPD reweights sibling rollouts and token-level losses by visual advantage while retaining the evidence-present teacher distribution Tn,t as each local target.Its weights redirect aggregate updates but do not reconstruct a different per-position target.
  • Decomposed OPD: Decomposed OPD reconstructs a target by preserving the student’s text-only prior and injecting the teacher’s multimodal information gain.It also combines the resulting visual divergence with standard multimodal distillation through Visual Gradient Steering.
  • VAD: VAD uses every coordinate of the centered counterfactual response to separate visual support from refutation and construct a signed, budgeted target.Its primary supervision sets target odds through this response, while the direct-teacher term remains a separate weak stability regularizer.

D.3 Correct-Token Promotion and Wrong-Token Suppression

VAD’s student-anchored target promotes a correct token over an incorrect alternative when its visual-attribution shift is larger, while absolute probability changes remain governed by softmax normalization. This distinguishes VAD’s signed evidence comparison from complete-teacher and clipped visual-advantage criteria.

  • Absolute probability changes: A token’s absolute probability increases when its shift exceeds the expected exponentiated shift under the student distribution.The probability ratio is exp(δ_i) E_k∼p[exp(δ_k)].
  • Absolute probability changes: A negative shift alone does not guarantee an absolute probability decrease because softmax normalization depends on all candidate coordinates.
  • Correct-token promotion: VAD favors a correct token c over an incorrect alternative w exactly when δ_c > δ_w.This criterion compares their relative odds rather than requiring either token’s absolute probability to move in a particular direction.
  • Baseline comparison: Vision-OPD uses the complete-teacher condition a_c > a_w, whereas VA-OPD keeps that local target condition and reweights positions by rollout and token groups.
  • Baseline comparison: Positive clipping sets VAn,t = 0 for nonpositive sampled-token log-ratios, so VA-OPD does not explicitly distinguish visual refutation from other low-advantage cases.The low-advantage group still receives distillation loss.

D.4 Scope of the Comparison

The comparison separates algebraic supervision behavior at fixed prefixes from causal differences between trained checkpoints. Figure 4 examines immediate supervision directions, whereas Table 1 evaluates accumulated optimization outcomes across complete trained systems.

  • Algebraic identities at fixed prefixes do not establish causal differences between trained checkpoints.
  • Figure 4 isolates immediate correct-token support and wrong-token suppression, with VA-OPD lacking a separate direction because its local target remains p+.
  • Table 1 compares complete trained systems under a common evaluation pipeline, measuring accumulated optimization outcomes rather than frozen-state local objective behavior.
Loading 2607.28590v1…