PaperlayerView exact revision on arXiv

Source-linked AI summary

Progressive-View On-Policy Distillation for Regional-to-Global Transfer in Multimodal LLMs

Shanfeng Huang, Zhou Fang, Song Xiao, Hai Du

arXiv:2609.32333v1cs.CVcs.AI

TL;DR

Fine-grained visual details can be difficult for multimodal models to capture, while crop-based guidance must transfer to full-image inference. PVD progressively shifts the student from crop views toward the full image and reweights token supervision; PVD-full improves benchmark accuracy over reward-free and reward-matched baselines.

  • Problem

    Fine-grained visual details may occupy only a tiny fraction of high-resolution images, and regional-to-global distillation’s standard objective starts with the full-image view and uniformly weights tokens.

    Source: §1 · p. 1

  • Method

    PVD progressively shifts sampled views from privileged crops toward the full image through a padded crop, while regional-advantage weighting uses the teacher–student gap to reallocate token supervision.

    Source: §1 · p. 1, §2.2 · p. 2

  • Results

    77.51, +2.01 over reward-free Vision-OPD and +1.00 over its reward-matched counterpart: PVD-full improves on both across perception, visual mathematics, and general QA benchmarks.

    Source: §1 · p. 1, §3.2 · p. 3

  • Takeaways & Limitations

    PVD-full requires neither region annotations nor inference-time search, and reward-free PVD-distill still gains 1.16 points over V-OPD.

    Source: §3.2 · p. 3, §4 · p. 4

Abstract

from arXiv · show

Regional-to-global distillation uses crop-conditioned guidance to improve full-image understanding. The challenge is to effectively transfer the teacher's crop-based advantage to the student's full-image inference. We propose progressive-view on-policy distillation (PVD), which shifts the student's view distribution from the crop toward the full image through an intermediate aspect-preserving padded crop. The padded crop preserves regional content while matching the full image's visual-token grid. Across stages, the view mixture assigns increasing probability to the full image. A lightweight regional-advantage weighting reallocates token-level supervision using the crop-conditioned teacher-student log-probability gap. Evaluated under each sampled input, it applies mild reweighting when the gap is small and emphasizes higher-gap tokens when the gap widens. A Jensen-Shannon metric decomposition interprets this schedule as a shift from matched-input imitation toward the deployment objective. Across benchmarks spanning perception, visual mathematics and general multimodal question answering, PVD-full reaches an average accuracy of 77.51 over three seeds, improving on the reward-free distillation baseline by 2.01 points and on its reward-matched variant by 1.00 point. In the reward-free setting, PVD-distill still gains 1.16 points.

1. INTRODUCTION

Regional-to-global distillation transfers crop-based visual detail into full-image inference, but existing objectives begin with the full-image view and uniformly weight tokens. PVD progressively shifts views toward the full image and reallocates supervision toward tokens resolved by the privileged crop.

1. INTRODUCTION

  • Regional-to-global distillation transfers crop-conditioned guidance into model weights, enabling full-image inference with one student forward pass rather than per-query visual search.Prior guided search, zooming, and crop-and-reask methods recover detail at the cost of added latency and deployment machinery.Source: §1 · p. 1
  • The standard regional-to-global objective starts with the student’s full-image view and weights response tokens uniformly, despite privileged-view advantages concentrating in a minority of tokens.PVD links view sampling and token weighting through the teacher–student gap under the sampled view.Source: §1 · p. 1
  • PVD shifts the student’s view distribution in stages from privileged crops toward the deployment view, using an aspect-preserving padded crop to preserve regional content while aligning visual-token grids.Every stage keeps probability mass on more than one view, separating token-grid alignment from later surrounding context.Source: §1 · p. 1
  • 77.51, +2.01 over reward-free Vision-OPD and +1.00 over a reward-matched control: PVD-full combines progressive distillation with task reward.Neither signal alone matches the combination.Source: §1 · p. 1

2. PROPOSED METHOD

PVD replaces fixed full-image, uniform-token distillation with stagewise view sampling and gap-based token weighting. Its metric decomposition interprets the schedule as a shift from crop-matched imitation toward deployment alignment, and the method can be combined with task reward.

2.1. Regional-to-global self-distillation

  • On-policy rollouts come from the student’s sampled view, and each token’s loss is a top-K Jensen–Shannon divergence between teacher and student next-token distributions.The teacher is an exponential-moving average of the student weights and conditions on the annotated crop.Source: §2.1 · pp. 1–2

2.2. A view-conditioned transfer objective

  • Regional-advantage weighting uses the privileged-view log-probability gap to reallocate token supervision, with less selectivity at small gaps and more emphasis on higher-gap tokens.It reuses computed log-probabilities and adds no box signal, auxiliary scorer, or forward pass beyond the crop-conditioned teacher.Source: §2.2 · p. 2
  • PVD samples the student’s view from a crop, a grid-aligned padded crop, and the full image in stages, while the teacher remains crop-conditioned.The padded crop preserves regional content while matching the full image’s canvas and visual-token grid; the full image then adds surrounding context.Source: §2.2 · p. 2

2.3. A metric interpretation of PVD

  • The metric decomposition frames progression as trading crop-input matching for deployment alignment: crop views retain deployment slack, while the full-image endpoint makes the bound tight.Shifting probability toward later views reduces expected deployment slack, although larger crop-to-view distance can make the surrogate harder.Source: §2.3 · pp. 2–3

2.4. Joint training with task reward

  • PVD combines distillation with a group-relative policy-gradient loss on binary answer reward, applying both signals to every token of every rollout.The distillation signal supervises toward the teacher distribution, while task reward checks answer correctness rather than specifying which tokens to change.Source: §2.4 · p. 3

3. EXPERIMENTS

The experiments evaluate PVD’s benchmark performance, its schedule and token-weighting components, stage diagnostics, reward complementarity, and transfer beyond the training format. PVD-full outperforms both reward-free and reward-matched baselines overall.

3.1. Setup

  • Experiments use 6,240 region-grounded natural-photograph examples and evaluate nine benchmarks across perception, visual math, and general multimodal and diagram QA.The training set is 99.98% four-option multiple choice; evaluation weights the nine benchmarks equally.Source: §3.1 · p. 3

3.3. Ablation: schedule, exposure and token allocation Progressive scheduling improves over exposure-matched static

  • 76.34±0.29 for the progressive schedule exceeds the exposure-matched static mixture’s 75.67 ± 0.28; adding RA raises the average to 76.66 ± 0.30.The schedule’s 0.67-point margin over static mixing accounts for approximately 80% of its 0.84-point gain over V-OPD.Source: Progressive scheduling improves over… · pp. 3–4, Table 1 · p. 4

3.2. Main results

  • 77.51 ± 0.26 overall; PVD-full improves on V-OPD by 2.01 points and V-OPD with GRPO by 1.00 point.It improves on all three task families and has the best mean on 7 of 9 individual benchmarks.Source: §3.2 · p. 3, Table 1 · p. 4

3.4. Progressive view sampling shapes token supervision

  • Across stages, the mean privileged-view gap is 2.4, 5.2, and 4.6; the high-gap token fraction is 5.9%, 12.5%, and 12.0%.The unweighted JSD is 4.4, 12.2, and 10.2 in units of 10^-3, while RA weight ranges widen in later stages.Source: §3.4 · p. 4, Table 2 · p. 4

3.5. Complementarity with task reward

  • PVD-full combines task reward and distillation, reaching 77.51±0.26; reward-free PVD-distill reaches 76.66, below PVD-full.Adding distillation reduces the all-wrong rollout-group fraction to 12.7% for PVD-full, compared with 23.8% for reward-only GRPO.Source: §3.5 · p. 4, Table 3 · p. 4

3.6. Generalization beyond the training format

  • Against reward-matched V-OPD with GRPO, gains are +0.26 on perception, +1.48 on visual math, and +1.68 on general QA.Training uses only region-grounded perception questions over natural photographs, while gains are larger on visual math and general QA than on perception.Source: §3.6 · p. 4

4. CONCLUSION

PVD couples a stagewise regional-to-global view schedule with teacher-gap-based token reweighting. With task reward, PVD-full reaches 77.51 ± 0.26 and deployment requires neither region annotations nor inference-time search.

4. CONCLUSION

  • 77.51 ± 0.26: PVD-full exceeds reward-free and reward-matched baselines by 2.01 and 1.00 points, respectively.Source: §4 · p. 4
  • PVD couples a stagewise regional-to-global view schedule through a token-grid-aligned padded crop with teacher-gap-based token reweighting.The padded crop preserves regional content while aligning token geometry.Source: §4 · p. 4
  • The schedule accounts for most of the reward-free gain, and deployment requires neither region annotations nor inference-time search.Source: §4 · p. 4

5. COMPLIANCE WITH ETHICAL STANDARDS

The work involved no human participants or animals, so ethical approval was not required. No new data were collected, and experiments used publicly released research resources.

5. COMPLIANCE WITH ETHICAL STANDARDS

  • No human participants or animals were involved, and no ethical approval was required.Source: §5 · p. 5
  • No new data were collected.Source: §5 · p. 5
  • Experiments used publicly released model weights, a publicly released training set, and publicly released multimodal evaluation benchmarks intended for research use.Source: §5 · p. 5

Related papers

Loading 2609.32333v1…