PaperlayerView exact revision on arXiv

Source-linked AI summary

BAS-OPD: Budget-Aware Selective On-Policy Self-Distillation for Fine-Grained Multimodal Perception

Zihan Chen, Hengguang Zhou, Yuan Kang, Yiming Zhang, Wenhui Fang, Zenghui Ding, Yining Sun, Cho-Jui Hsieh

arXiv:2609.25891v1cs.CVcs.AI

TL;DR

Fine-grained visual answers can depend on local evidence that full-image MLLMs may fail to preserve, while querying a teacher for every rollout makes supervision costly. BAS-OPD selectively allocates crop-teacher supervision under a budget and reports strong benchmark performance with reduced teacher cost, while retaining single-pass full-image inference.

  • Problem

    Fine-grained answers can depend on small objects, local attributes, or spatial relations in high-resolution scenes, while OPD teacher queries for every rollout tie supervision cost to the full batch.

    Source: §1 · p. 2

  • Method

    BAS-OPD selects on-policy rollouts for crop-teacher scoring under a strict sample-level budget, using random, student-entropy, or learned utility selection.

    Source: §1 · p. 2

  • Results

    BAS-OPD 9B averaged 84.23%, 3.00 points above Qwen3.5-9B, and ranked first on five benchmarks and second on two among open-source single-forward-pass models.

    Source: §3.2 · p. 5

  • Takeaways & Limitations

    BAS-OPD improves the training accuracy–cost trade-off while preserving single-pass full-image inference.

    Source: §5 · p. 9

    Selection analyses use Qwen3.5-4B and one seed, with broader seeds, backbones, and matched 9B ablations untested.

    Source: §5 · p. 9

Abstract

from arXiv · show

Multimodal large language models (MLLMs) often struggle with fine-grained visual perception when processing complete images, as critical evidence may only appear in local regions. On-policy self-distillation (OPD) enables transferring privileged visual knowledge from informative views to full-image policies, but querying the teacher for every rollout introduces substantial supervision costs. In this work, we propose BAS-OPD, a budget-aware selective OPD framework that allocates teacher supervision under limited query budgets. Instead of querying all rollouts, BAS-OPD selects informative samples while maintaining full-batch student generation. We explore random, uncertainty-based, and learned utility-based selection strategies, where the learned selector estimates query value from detached rollout statistics and online utility signals derived from student--teacher agreement and teacher confidence without additional student forward passes. BAS-OPD only changes training-time supervision allocation and preserves single-pass full-image inference. Experiments on fine-grained multimodal perception benchmarks demonstrate that BAS-OPD achieves strong performance while substantially reducing teacher supervision costs, highlighting the effectiveness of selective OPD under constrained budgets.

a BAS-OPD 9B

BAS-OPD 9B’s reported mean accuracy is 81.60%, with a +2.81 pp result across seven visual understanding benchmarks.

a BAS-OPD 9B

1 INTRODUCTION

BAS-OPD addresses fine-grained perception by selectively directing crop-teacher supervision to full-image student rollouts under a query budget. It reports strong results across seven benchmarks while preserving single-pass full-image inference.

1 INTRODUCTION

  • Fine-grained perception remains brittle when answers depend on small objects, local attributes, or spatial relations in high-resolution scenes.Resizing or near-uniform spatial encoding can weaken the visual signals that determine an answer.Source: §1 · pp. 1–2
  • Prior high-resolution and inference-time visual-search approaches trade against larger visual-token budgets, repeated patch processing, or additional inference calls.They recover detail at inference time rather than teaching a single-pass full-image policy to internalize focused regional evidence.Source: §1 · p. 2
  • Existing OPD methods typically query the teacher for every eligible rollout, tying supervision cost to the full batch even when feedback is redundant or weakly informative.Source: §1 · p. 2
  • BAS-OPD selects on-policy rollouts before crop-teacher scoring, while the full-image policy still generates responses for the entire batch.The learned selector uses detached rollout statistics, and its online utility combines student–teacher divergence, top-k overlap, and teacher confidence without another student forward pass.Source: §1 · p. 2
  • BAS-OPD 9B averaged 84.23%, 3.00 points above Qwen3.5-9B, and ranked first on five benchmarks and second on two among open-source single-forward-pass models.Source: §1 · p. 3
  • The paper frames budget-aware OPD, online utility selection, and accuracy–cost and selection diagnostics as its contributions.The selection policies include random, student-entropy, and learned utility approaches, with unselected rollouts excluded from the crop-teacher forward pass.Source: §1 · p. 3

2 METHOD

BAS-OPD keeps full-batch student generation while selecting which rollouts receive crop-teacher scoring under a strict sample budget. It learns query utility from rollout statistics and online teacher signals, then distills only selected samples while preserving single-pass full-image inference.

2.1 BUDGET-AWARE SELECTIVE OPD

  • BAS-OPD generates responses for the full batch but selects samples before crop-teacher scoring, so unselected rollouts receive no teacher distribution or OPD target.The budget reduces the teacher input batch itself rather than masking losses after a teacher forward pass.Source: §2.1 · p. 3

2.2 LEARNED UTILITY SELECTION

  • The learned selector predicts query utility from detached rollout features, including likelihood, entropy, response and prompt lengths, and crop area, without another MLLM forward pass.A lightweight predictor uses normalized features to estimate utility.Source: §2.2 · p. 4
  • Before the predictor is updated, selection is uniform; afterward, the method reserves queries for random exploration and assigns the rest to the highest-scoring candidates.Exploration is drawn from remaining candidates to avoid a self-confirming selection loop while preserving the strict budget.Source: §2.2 · p. 4
  • Online utility targets combine student–teacher divergence, top-k overlap, and teacher confidence to identify large, support-consistent, confident corrections.Only queried samples receive observed feature–utility pairs, which train the selector through replay using weighted Huber regression and pairwise ranking.Source: §2.2 · p. 4

2.3 SELECTIVE DISTILLATION AND INFERENCE

  • The budgeted objective distills selected valid tokens, while training-time selection leaves inference as a single full-image forward pass without crops, a selector, or an external visual tool.At full budget, the objective recovers full-query OPD with the same token-mean reduction.Source: §2.3 · p. 5

3 EXPERIMENTS

BAS-OPD 9B outperforms the strongest open-source single-forward-pass model by average score, while learned selection at a 25% query budget achieves higher 4B accuracy than full querying with substantially fewer teacher-scored tokens. Experiments also examine benchmark coverage, query-budget behavior, selector components, and gradient geometry.

3.1 EXPERIMENTAL SETUP

  • The 4B experiments use seven benchmarks spanning fine-grained perception, broader visual capabilities, and object hallucination, with evaluation on original full images only.Training used 3,120 full-image–crop pairs; deterministic decoding and benchmark-specific scoring were used, with GPT-5.5 judging remaining free-form responses.Source: §3.1 · p. 5

3.2 EXPERIMENTAL RESULTS

  • BAS-OPD 9B averaged 84.23%, exceeding Qwen3.5-9B, the strongest non-BAS open-source single-forward-pass model by average score, by 3.00 points, and ranked first on five and second on two benchmarks.Source: §3.2 · p. 5
  • At 4B and 9B, BAS-OPD led on six benchmarks, tied the best MMVP score, and improved OPSD averages by 3.03 and 3.24 points, respectively.The comparison used a 25% teacher-query budget; the largest gains over the strongest alternatives were on V*Bench and ZoomBench.Source: §3.2 · p. 6, Table 2 · p. 6

3.3 ABLATION AND EFFICIENCY ANALYSIS

  • 81.60% mean accuracy for Learned-25 exceeded 79.76% for full querying while using 15.78% of full-query teacher-scored tokens.Learned-25 improved mean accuracy by 2.35, 1.36, and 1.84 points over Random-25, Entropy-25, and full querying, respectively.Source: §3.3 · p. 7, Table 3 · p. 7
  • At 10%, 25%, 50%, and 100% query budgets, seven-benchmark means were 79.13%, 81.60%, 80.55%, and 79.76%, respectively.The 25% budget led six benchmark sweeps, while 50% led on POPE by 0.05 points.Source: §3.3 · p. 7, Figure 2 · p. 7
  • At a 25% budget, learned ranking captured 58.14% of utility, versus 72.70% for oracle ranking and 25% for random ranking.The top-quartile mean utility was 1.75 times the next quartile (5.52 × 10−3 versus 3.15 × 10−3; Holm-adjusted p = 0.006).Source: §3.3 · p. 7, Figure 3 · p. 8
  • Across six checkpoints, the selected 25% produced 1.66–2.16× larger mean OPD gradient norms than the unselected 75%, with partial alignment.The analysis uses 20 batch pairs per checkpoint from a single trajectory; it does not establish that unselected rollouts are harmful or that 25% is optimal. The utility audit uses training-distribution batches and a frozen final student rather than the unserialized EMA teacher, diagnosing policy behavior rather than out-of-distribution generalization.Source: §3.3 · p. 8, Figure 4 · p. 8
  • The full selector achieved the highest seven-benchmark average, 81.60%, and removing components lowered average accuracy by 2.79–3.66 points.Teacher confidence removal caused the largest drop; MMStar ablation drops were 7.80–9.87 points.Source: §3.3 · p. 8, Table 4 · p. 9

4 RELATED WORK

Prior OPD work transfers teacher behavior along student-generated trajectories, while multimodal work has applied privileged regional supervision to full-image policies. Existing approaches to fine-grained visual evidence alter visual processing or add inference steps; BAS-OPD instead allocates regional supervision during training.

4.1 ON-POLICY DISTILLATION

  • BAS-OPD addresses teacher-query allocation under a fixed budget, complementing OPD methods focused on learning objectives, supervision quality, or privileged regional evidence.Prior methods typically query the teacher for every eligible rollout.Source: §4.1 · p. 9

4.2 FINE-GRAINED VISUAL UNDERSTANDING FOR MLLMS

  • High-resolution architectures and inference-time region search improve access to local evidence but increase visual processing or add inference steps.BAS-OPD changes neither the visual architecture nor inference procedure, retaining single-pass full-image inference while selectively allocating regional supervision during training.Source: §4.2 · p. 9

5 CONCLUSION AND LIMITATIONS

BAS-OPD allocates crop-teacher supervision under a fixed query budget using online utility estimates. Learned-25 improves the Qwen3.5-4B benchmark average while reducing teacher-scored tokens, and the paper reports limitations in selector-evaluation scope and timing portability.

5 CONCLUSION AND LIMITATIONS

  • 81.60% average accuracy versus 79.76% for Qwen3.5-4B Learned-25 uses 25% of candidate queries and 15.78% of full-query scored tokens; BAS-OPD 9B reaches 84.23%.The method improves the training accuracy–cost trade-off while preserving single-pass full-image inference.Source: §5 · p. 9
  • Selector analyses use Qwen3.5-4B and one seed, leaving broader seeds, backbones, and matched 9B ablations untested.Wall-clock results depend on implementation and hardware utilization; the frozen-selector audit measures training-distribution subset retrieval rather than out-of-distribution generalization.Source: §5 · p. 9

A IMPLEMENTATION DETAILS

The implementation includes random and entropy-based query-budget baselines, plus a learned utility predictor with detached rollout features and shared configuration across query budgets.

A.1 RANDOM AND ENTROPY SELECTION BASELINES

  • Random-25 uniformly samples eligible rollouts without replacement, and only selected rollouts are sent to the crop teacher.It has no trainable selector and requires no additional student forward pass.Source: §A.1 · p. 12
  • Entropy-25 ranks candidates by mean valid-token entropy when available, otherwise using mean sampled-token NLL from stored rollout log-probabilities.The heuristic adds no trainable parameters or additional student forward pass.Source: §A.1 · p. 12

A.2 PREDICTOR CONFIGURATION AND FEATURE HANDLING

  • The learned predictor uses detached rollout features including response-level NLL statistics, optional entropy, response and prompt lengths, and crop-to-full-image area ratio.The predictor has layer normalization, one hidden linear layer with SiLU, and a scalar output; its running feature statistics are updated during extraction and held fixed during replay updates. Shared budget settings include CPU ranking by predicted utility without response-token cost division, with unavailable features set to zero.Source: §A.2 · p. 12, Table 5 · p. 12

A.3 SELECTOR TRAINING AND UPDATE HANDLING

Selector training stores observed feature–utility pairs for replay, updates predicted utility rankings with detached data, and handles empty selected sets without crop-teacher scoring or new utility labels.

A.3 SELECTOR TRAINING AND UPDATE HANDLING

  • A bounded FIFO replay buffer stores observed feature–utility pairs with collection steps, and replay minibatches are sampled uniformly without replacement.Items receive recency weights based on their age and a specified half-life.Source: §A.3 · p. 12
  • The ranking loss trains predicted ordering for labeled pairs with utility differences above 10^-6, while detached features and targets and a separate optimizer prevent selector loss from updating the MLLM.Pair contributions are weighted by the square root of the products of their recency weights.Source: §A.3 · p. 13
  • When the selected set is empty, crop-teacher scoring is skipped, the distillation term is zero, and no replay update occurs because no new utility labels are available.The remaining updates follow standard OPD optimizer and teacher-update rules.Source: §A.3 · p. 13

B.1 TRAINING CONFIGURATION

The training configuration section identifies four retained selection-policy runs and states a maximum model length of 9,216 tokens. The optimizer and remaining training settings are specified in the main experimental setup.

B.1 TRAINING CONFIGURATION

  • The maximum model length is 9,216 tokens.The optimizer and remaining training settings are specified in the main experimental setup.Source: §B.1 · p. 13
  • Table 6 records the batch configurations and step counts for four retained policy runs.The maximum model length is 9,216 tokens.Source: Table 6 · p. 13, §B.1 · p. 13

B.2 EVALUATION PROTOCOL

Evaluation uses deterministic generation with benchmark-native parsing where available and GPT-5.5 judgments for remaining free-form answers. CV-Bench and POPE results follow specified reporting conventions.

B.2 EVALUATION PROTOCOL

  • Generation uses temperature 0 and seed 42.Benchmark-native option or exact-answer parsing is used when available; remaining free-form answers are judged with GPT-5.5 at temperature 0.Source: §B.2 · p. 13
  • CV-Bench is reported as the macro-average of its 2D and 3D accuracy categories, while POPE is reported by accuracy in the policy comparison.Source: §B.2 · p. 13
  • The POPE evaluation contains 9,000 samples.Source: §B.2 · p. 13

C TEACHER-SUPERVISION COST

Learned-25 combines the highest aggregate accuracy with the fewest teacher-scored tokens, while reducing teacher-forward time and total step time relative to full querying. The section reports teacher calls, scored tokens, and time measurements together, with time comparisons qualified by measurement limits.

C TEACHER-SUPERVISION COST

  • Training cost is recorded as selected teacher calls and response tokens scored by the teacher.Figure 5 summarizes average accuracy and teacher cost; Tables 7 and 8 provide supervision counts and observed training times.Source: C TEACHER-SUPERVISION COST · p. 13, Figure 5 · p. 14, Table 7 · p. 14
  • Learned-25 combined the highest aggregate accuracy with the fewest scored tokens, using 15.78% of full-query teacher-scored tokens versus 26.28% for Random-25 and 27.65% for Entropy-25.All three policies queried one quarter of their candidates; the token reductions reflect shorter selected responses.Source: C TEACHER-SUPERVISION COST · p. 13, Figure 5 · p. 14, Table 7 · p. 14
  • Learned-25 reduced accumulated teacher-forward time by 16.02% and total step time by 9.84% relative to full querying.Student generation, communication, data loading, and rollout scheduling remain substantial costs.Source: C TEACHER-SUPERVISION COST · pp. 13–14, Table 8 · p. 14
  • The observed time measurements are secondary because code path, device utilization, and rollout scheduling are not fully controlled across runs.Source: Table 8 · p. 14

Related papers

Loading 2609.25891v1…