Verifier-free Test-Time Sampling for Vision-Language-Action Models

Suhyeok Jang; Dongyoung Kim; Changyeon Kim; Youngsuk Kim; Jinwoo Shin

Source-linked AI summary

Verifier-free Test-Time Sampling for Vision-Language-Action Models

Suhyeok Jang, Dongyoung Kim, Changyeon Kim, Youngsuk Kim, Jinwoo Shin

arXiv:2510.05681v2Oct 7, 2025cs.ROcs.AIcs.LG

TL;DR

VLAs struggle with high-precision control, while external-verifier test-time scaling requires additional training and may not generalize to unseen conditions. MG-Select uses KL divergence from a masked-condition reference distribution to select among sampled actions, and consistently improves base models across simulation and real-world benchmarks.

Problem
VLAs remain limited on high-precision tasks, and existing test-time scaling methods rely on externally trained verifiers that require additional training and may fail to generalize to unseen conditions.
Method
MG-Select selects among multiple sampled actions using KL divergence from a reference distribution produced by the same VLA with randomly masked state and language conditions.
Results
MG-Select consistently improves base VLAs across diverse simulation and real-world benchmarks, including 28% and 35% relative gains on real-world in-distribution and out-of-distribution tasks.
Takeaways & Limitations
Condition-masking confidence enables verifier-free test-time action selection that improves policy precision and robustness using the model itself.

Abstract

from arXiv · show

Vision-Language-Action models (VLAs) have demonstrated remarkable performance in robot control. However, they remain fundamentally limited in tasks that require high precision due to their single-inference paradigm. While test-time scaling approaches using external verifiers have shown promise, they require additional training and fail to generalize to unseen conditions. We propose Masking Distribution Guided Selection (MG-Select), a novel test-time scaling framework for VLAs that leverages the model's internal properties without requiring additional training or external modules. Our approach utilizes KL divergence from a reference action token distribution as a confidence metric for selecting the optimal action from multiple candidates. We introduce a reference distribution generated by the same VLA but with randomly masked states and language conditions as inputs, providing action uncertainty while remaining aligned with the target task distribution. Additionally, we propose a joint training strategy that enables the model to learn both conditional and unconditional distributions by applying dropout to state and language conditions, thereby further improving the quality of the reference distribution. Our experiments demonstrate that MG-Select provides a reliable reference for action selection through task-relevant condition masking and consistently improves base models across diverse simulation and real-world benchmarks.

1 INTRODUCTION

MG-Select addresses precision failures in VLAs by selecting among sampled actions using an uncertainty-aware reference distribution rather than external verifiers. Across simulation and real-world benchmarks, it consistently improves base policies, including 28% and 35% relative gains on real-world in-distribution and out-of-distribution tasks.

1 INTRODUCTION: VLAs remain limited on high-precision manipulation, where greedy decoding can bottleneck performance despite extensive pre-training.Fine-grained tasks such as grasping remain challenging, and repeated sampling with external verifiers requires additional training.
1 INTRODUCTION: Likelihood-based selection can fail because fine-tuned VLAs memorize expert trajectories, concentrating action-token probabilities and causing samples to converge.MG-Select is designed to avoid this limitation by using the masking distribution as its confidence baseline.
1 INTRODUCTION: MG-Select selects among multiple action candidates using KL divergence from a reference distribution generated with randomly masked state and language conditions.The masked-input distribution supplies action uncertainty while remaining aligned with the target task distribution.
1 INTRODUCTION: A joint training strategy further improves the reference distribution by teaching the model conditional and unconditional behavior through state and language dropout.This training is intended to make confidence estimates more reliable during test-time scaling.
1 INTRODUCTION: 28% and 35% relative gains were achieved on real-world in-distribution and out-of-distribution tasks, respectively, while MG-Select improved VLAs across diverse benchmarks.The method also produced a 168% relative gain over vanilla greedy decoding on RoboCasa with 30 demonstrations.

2 PRELIMINARIES

The paper formulates robot control as imitation learning for a language-conditioned autoregressive VLA. The policy maps visual and proprioceptive state plus language instructions to action distributions, then generates discrete action tokens autoregressively.

2 PRELIMINARIES: Imitation learning trains the policy to match expert actions in demonstration trajectories within a reward-free Markov decision process.The formulation includes state and action spaces, transition dynamics, discounting, and an initial-state distribution.
2 PRELIMINARIES: A language-conditioned VLA maps state and instruction inputs to a probability distribution over actions and is trained on paired demonstrations and instructions.The state is decomposed into visual observation and proprioceptive components.
2 PRELIMINARIES: The policy represents state as visual observation and proprioceptive state, enabling action distributions conditioned on both components and the language instruction.These distributions are used for repeated sampling of candidate actions in test-time scaling.
2 PRELIMINARIES: In autoregressive decoding, the model produces a categorical distribution over discrete action tokens at each step, conditioned on the preceding token prefix.The next-token probabilities are obtained by applying softmax to the model’s logits.

3 METHOD

MG-Select scales autoregressive VLA inference by sampling multiple actions and selecting among them with confidence from condition-masked reference distributions. Joint training preserves standard fine-tuning performance while improving these reference distributions, with masking variants chosen according to task environment.

3 METHOD: MG-Select samples multiple candidate actions and selects the final action using a confidence metric based on a reference action-token distribution.The framework combines candidate generation, best-of-N selection, reference-distribution confidence, and joint training.
3.2 CONDITION-MASKING DISTRIBUTIONAL CONFIDENCE FOR TEST-TIME SAMPLING: KL divergence between predicted and reference action-token distributions provides the confidence signal for identifying the most promising candidate.The reference distribution is intended to represent uncertainty while remaining sufficiently close to the target action distribution.
3.2 CONDITION-MASKING DISTRIBUTIONAL CONFIDENCE FOR TEST-TIME SAMPLING: Condition masking removes text, state, or both inputs to approximate failure modes in which essential task-solving conditions are ignored.The resulting reference distributions are generated by the same VLA under altered conditioning.
3.2 CONDITION-MASKING DISTRIBUTIONAL CONFIDENCE FOR TEST-TIME SAMPLING: The best confidence variant depends on the benchmark: state masking works best for SIMPLER-WidowX, whereas text or text-and-state masking works better for RoboCasa.The difference reflects whether task instructions are necessary to determine the correct action.
3.3 JOINT TRAINING STRATEGY: Joint training mixes all-condition, text-masked, state-masked, and jointly masked data so the VLA learns condition-masking distributions alongside standard target-task behavior.The resulting model maintains performance comparable to standard fine-tuning and improves performance when combined with the confidence measure.

4 EXPERIMENTS

MG-Select is evaluated across simulation and real-world benchmarks, where it consistently improves VLA action precision and generalization. Ablations show that condition-masking confidence, joint training, candidate count, masking choice, and confidence aggregation materially affect performance.

4.1 Simulation Experiments: MG-Select consistently improves π0-FAST across RoboCasa tasks and demonstration scales, with a 168% relative pick-and-place gain at 30 demonstrations after joint training.The method improves the base model even without joint training, while joint training further amplifies gains, especially in the low-data regime.
4.1 Simulation Experiments: MG-Select improves SIMPLER-WidowX across all tasks, including the difficult eggplant task where the base model performs poorly.This supports robustness when the underlying policy struggles because the task background differs substantially from other tasks.
4.1 Simulation Experiments: On LIBERO, MG-Select achieves superior average performance for both π0-FAST and OpenVLA, including the challenging LIBERO-Object and LIBERO-Long suites.The result indicates compatibility with different VLA architectures and gains on suites with low base-model performance.
4.3 Ablation Studies and Analyses: MG-Select provides the strongest inference-strategy gains, while performance saturates after N = 4 candidates and text masking performs best among masking variants.Joint training further improves MG-Select, and truncating confidence aggregation to the first 5 tokens outperforms naive token-confidence summation.

5 RELATED WORK

Related work situates VLAs as strong robot-control models and frames test-time computation as a route to improve difficult outputs. It also highlights deployment efficiency as a practical concern for MG-Select.

VLAs have emerged as a prominent approach for generalist robot policies, achieving strong performance across diverse downstream tasks through large-scale robotic pre-training.
Figure 3 compares vanilla MG-Select with a single-prefill efficient deployment variant on LIBERO-Object using π0-FAST.
Test-time scaling applies additional computation to improve outputs for challenging tasks, including reasoning in language models and emerging robotics applications.

6 CONCLUSION

The paper concludes that MG-Select enables verifier-free test-time scaling by using condition-masking confidence from the model itself. It reports improved precision and policy performance across simulation and real-world benchmarks, with joint training and implementation changes further improving effectiveness and efficiency.

MG-Select uses condition-masking distributional confidence as a self-generated signal for Best-of-N action selection without external verifiers.
The framework consistently improves policy performance across diverse simulation and real-world benchmarks while addressing precision issues in single-inference paradigms.
Joint training and an optimized implementation further enhance MG-Select’s effectiveness and efficiency.

A.1 TRAINING ON SIMULATION DATA

The simulation experiments use π0-FAST and OpenVLA as base policies, with benchmark-specific training configurations and condition-dropout variants for joint imitation learning. MG-Select is tuned over sampling, candidate-count, masking, and confidence-regularization choices, with model-specific token aggregation.

The simulation study uses π0-FAST and OpenVLA as representative autoregressive VLA base models.π0-FAST uses Paligemma-3B with full fine-tuning, whereas OpenVLA uses Prismatic-7B with LoRA fine-tuning.
The simulation training setups vary by benchmark, with π0-FAST configurations documented separately across the evaluated simulation datasets.
Joint imitation learning adds condition-dropout data while otherwise following the corresponding imitation-learning configuration.The variants mask text, state, or both text and state; π0-FAST uses 10% dropout settings, with state-only dropout for SIMPLER-WidowX.
MG-Select searches over sampling temperature, candidate count, masking variant, and regularization temperature for each dataset.The reported search ranges are τ ∈ {0.1, 0.3, 0.5, 0.7, 1.0}, N ∈ {4, 8}, three masking variants, and regularization temperatures from 4.0 to 16.0.
Confidence aggregation uses the first five tokens for π0-FAST and the full fixed-length sequence for OpenVLA, which applies only text masking.

B DETAIL RESULTS ON SIMULATION EXPERIMENTS

The detailed simulation results report performance comparisons for RoboCasa and SIMPLER-WidowX under repeated-trial evaluation. They organize results by task categories and distinguish task success from grasp success where applicable.

RoboCasa results report average success rates over 50 trials across pick-and-place, open-and-close, and other task categories, with varying demonstration counts.
SIMPLER-WidowX results report both task success and grasp success rates over 24 trials on four pick-and-place tasks.Method results are averaged over three random seeds, while baseline results come from the SIMPLER paper and original papers.

C DETAIL RESULTS ON EFFICIENT DEPLOYMENT STRATEGY

MG-Select improves VLA performance across benchmarks and settings, with joint training strengthening its confidence signal and competitive performance against external verification at lower latency.

A 10%/10%/10% text/state/both-condition dropout ratio achieves the best RoboCasa performance in joint imitation learning.
Truncating action confidence to the first token performs best on SIMPLER-WidowX, although the default uses the first 5 tokens.
MG-Select significantly outperforms the base model on LIBERO across different VLA model scales.
MG-Select achieves competitive performance with RoboMonkey without an external verifier and has substantially lower latency.
Higher temperature increases reference-distribution entropy, reducing bias from over-confident masked distributions.

E.1 THEOREM

The theorem establishes that increasing temperature monotonically increases the entropy of MG-Select’s condition-masking reference distribution.

Entropy H(πmasked) is monotonically increasing for temperature τ > 0.
Using a higher temperature such as τ = 4.0 increases reference-distribution entropy and prevents confidence bias from the masked distribution’s low entropy.
The derivation expands the softmax entropy, differentiates the log-partition function and expectation terms, and shows cancellation before obtaining the temperature derivative.

F RESULTS ON CALVIN BENCHMARK

MG-Select generalizes to CALVIN’s zero-shot, long-horizon setting, consistently outperforming the base model across instruction chains while some simple tasks can incur minor stochasticity-related drops.

CALVIN evaluates zero-shot language-conditioned task execution in novel environment D after training on environments A, B, and C.
The CALVIN evaluation uses 34 tasks with a Franka Panda Arm and measures average consecutive successes across five instruction chains.
MG-Select consistently outperforms the base model across all CALVIN instruction chains, demonstrating generalizability to long-horizon multi-step planning.
On simple atomic LIBERO-Goal tasks, MG-Select can add stochasticity that slightly misaligns the gripper when the base model is already near-optimal.

H DETAIL ANALYSIS OF TRUNCATED FAST TOKENS

The truncated FAST-token analysis examines how early tokens reconstruct continuous actions and relates reconstruction error to truncated aggregation performance on RoboCasa.

The analysis compares truncated aggregation strategies with FAST-token action reconstruction error using 100 RoboCasa demonstrations.
FAST-token reconstruction error is evaluated by retaining the first K tokens, detokenizing them, and computing normalized Mean Absolute Error on RoboCasa.