DiVeR: Decision-Critical Verifier Learning for VLA Test-Time Scaling

1 Introduction

• Test-time scaling samples and ranks multiple action candidates at inference, offering an alternative to collecting new demonstrations or retraining the base policy.The paper uses the term for parallel repeated sampling and ranking, rather than sequential extended rollouts or iterative refinement.Source: §1 · p. 1
• Existing classification-based verifiers spread trajectory-level outcomes uniformly across states, although states differ in how much they help discriminate candidate actions.Routine states dominate trajectories, while sparse decision-critical states can have more consequential action differences.Source: §1 · pp. 2–3
• DiVeR estimates decision criticality from dispersion among sampled action-expert representations and reweights verifier learning toward states where candidate selection matters most.This proxy requires neither step-level supervision nor additional environment interaction.Source: §1 · pp. 2–3
• Up to +13.6% over single sample inference and +3.4% over the strongest baseline, DiVeR improves task success across policies and environments.It also surpasses the VLM-based verifier by +4.6% success rate while reducing verifier inference latency by over 700×.Source: §1 · p. 3

2 Related Works

• Geometry-aware representations, memory mechanisms, and world models strengthen VLA policies but require additional data collection, architectural modification, or retraining.The paper positions its frozen-policy, inference-time approach as complementary to these directions.Source: §2 · p. 3
• Prior language-generation work identifies sparse high-entropy forking tokens as consequential branching points, and this paper studies analogous non-uniformity in robotic control.DiVeR focuses verifier learning on sparse decision-critical states rather than treating all states uniformly.Source: §2 · p. 3
• Robotic generate-then-verify methods include costly VLM-based verifiers, verifier-free internal signals not grounded in task outcomes, and lightweight classifiers that weight visited states uniformly.DiVeR estimates decision criticality from action-expert representation variance to emphasize states with greater candidate disagreement.Source: §2 · pp. 3–4

3 Problem Setup

• At each decision step, the policy-conditioned state includes the visual observation, language instruction, and any additional policy context.The pretrained policy generates an action chunk spanning H actions, and the setup samples N candidate chunks.Source: §3 · p. 4
• A learned verifier assigns a scalar score to each candidate action chunk.The verifier maps the state and action chunk to a real-valued score.Source: §3 · p. 4
• The verifier provides an inference-time selection rule for choosing the candidate most likely to satisfy the instruction.The VLA proposes plausible robot behaviors, while the verifier ranks candidates for execution.Source: §3 · p. 4

4.1 Success-Visitation Verifier

• At the population optimum, the success-versus-failure visitation-density log ratio supplies a verifier score for ranking state–action candidates.The discriminator is trained from successful and failed trajectory outcomes.Source: §4.1 · pp. 4–5

4 Method

• DiVeR estimates candidate disagreement from frozen-policy action representations, converts it into decision weights, and uses weighted verifier training to rank sampled action chunks.The framework is summarized in Figure 2.Source: §4 · p. 4, Figure 2 · p. 5

4.2 Decision Criticality from Action-Representation Variance

• Mean per-coordinate variance across sampled action representations serves as decision criticality: lower dispersion marks routine states, while higher dispersion indicates stronger candidate disagreement.The paper reports that this score correlates with rollout-based oracle criticality.Source: §4.2 · p. 6, Figure 3 · p. 6

4.3 DiVeR: Decision-Weighted Verifier Learning

• DiVeR standardizes criticality scores within each trajectory and exponentiates them into positive weights, upweighting above-average disagreement and downweighting routine states.The dataset pairs each executed-action representation and weight with the trajectory-level outcome.Source: §4.3 · pp. 6–7
• The verifier minimizes decision-weighted binary cross-entropy, with weights derived from frozen-policy candidate disagreement rather than trajectory outcomes.The weighting increases the relative contribution of states where candidate discrimination may be more useful.Source: §4.3 · p. 7

5.1 Experimental Setup

• Experiments cover LIBERO-Long and RoboCasa simulations, plus four Franka Research 3 tasks using π0.5 fine-tuned with 40 teleoperation demonstrations per task.The simulations use 30 training and 20 evaluation episodes per task; real-robot evaluation uses 24 episodes per task.Source: §5.1 · p. 7

5.2 Main experiments

• For π0 with N = 32 on LIBERO-Long, DiVeR improves success rate by +5.8% over N = 1 and +2.8% over the best-performing baseline.Table 1 reports average success rates across varying candidate counts.Source: §5.2 · p. 7, Table 1 · p. 8
• On RoboCasa with N = 16, DiVeR improves average success rate over N = 1 by +5.9% for π0 and +11.6% for π0.5.The gains span diverse task categories rather than being biased toward one type.Source: §5.2 · p. 8, Table 2 · p. 8
• 71.9% average success on the Franka Research 3, versus 65.6% for Uniform and 58.3% for Base, across four tasks.These correspond to gains of +6.3 and +13.6 percentage points, respectively.Source: §5.2 · p. 8, Table 3 · p. 9

5.3 Quantitative Analysis

• Decision-criticality weighting adds +3.2 and +4.1 percentage points over uniform training for π0 and π0.5, respectively, achieving the best component-analysis performance.Random selection yields only marginal gains over N = 1; uniform training improves by +2.7 and +7.5 points over baseline.Source: §5.3 · p. 8, Table 4 · p. 9
• Action-expert representations outperform raw-action verifier inputs, while adding image features gives no further benefit; DiVeR also beats RoboMeter by +4.6 percentage points and is over 700× faster.For RoboMeter, verification latency falls from 0.743 s to 0.001 s.Source: §5.3 · p. 9, Table 5 · p. 9, Table 6 · p. 9
• DiVeR transfers effectively across RoboCasa task categories, and the MLP performs best across both VLA policies among the tested verifier architectures.The architectures use matched parameter budgets; the authors select the MLP for its performance and low inference cost.Source: §5.3 · pp. 9–10, Figure 4 · p. 9

5.4 Qualitative Analysis

• From identical real-world decision-critical states, DiVeR selects actions that complete drawer closing and block stacking, while the uniformly trained verifier's choices fail.The figure contrasts the resulting trajectories for both tasks.Source: §5.4 · p. 10, Figure 5 · p. 10

6 Conclusion

• DiVeR estimates decision criticality from sampled action-representation dispersion and gives greater training emphasis to sparse states where candidate selection is consequential.Across LIBERO-Long, RoboCasa, and real-world manipulation, it improves task success over verifier-free and learned baselines through verifier-guided Best-of-N selection, with negligible verifier inference overhead.Source: §6 · pp. 10–11

A Algorithm

• The algorithm constructs a decision-weighted dataset from rollout trajectories, computes criticality-based timestep weights, and updates the verifier with decision-weighted binary cross-entropy.At inference, it samples candidate action chunks, scores their action-expert representations, and executes the highest-scoring candidate.Source: A Algorithm · p. 17

B.1 Implementation

• A two-layer MLP uses final-layer action-token representations averaged across tokens for both criticality estimation and verifier training; the VLA policy stays frozen.The verifier uses 64 hidden dimensions and dropout 0.3; training samples K = 32 candidate chunks per state to estimate representation variance.Source: §B.1 · pp. 17–18

B.2 Hyperparameters

• Verifier hyperparameters are tuned on RoboCasa with π0.5, then fixed across the remaining benchmarks and VLA policies to test transfer of one configuration.This protocol avoids benchmark-specific tuning.Source: §B.2 · p. 18

B.3 Dataset statistics

• Rollout datasets are class-imbalanced across policies and environments, yet the verifier trains effectively without artificially balancing outcomes during data collection.The dataset statistics table reports successful and failed episodes and their environment steps for π0 and π0.5.Source: §B.3 · p. 18, Table 8 · p. 18

B.4 Evaluation Tasks

• The study evaluates LIBERO-Long, 18 RoboCasa Atomic-Seen tasks, and four real-world manipulation tasks on a Franka robot.The real-world tasks span object placement, stacking, and articulated-object interaction.Source: §B.4 · p. 18, Table 10 · p. 19, Table 11 · p. 19

C.1 Does Measured Criticality Reflect Actual Decision Consequence?

• Measured criticality correlates positively with rollout-based oracle criticality (Spearman ρ = 0.493, p = 8.09 × 10−23).Oracle criticality measures the potential gain from selecting the best candidate rather than choosing uniformly at random; Figure 7 compares the two quantities.Source: §C.1 · p. 20, Figure 7 · p. 20

C.2 Autoregressive Model

• With N = 4 candidates, DiVeR improves OpenVLA success rate by +3.0 percentage points over the pretrained policy, +1.0 over MGSelect, and +0.5 over SVM.This evaluation tests applicability to an autoregressive model in addition to the paper’s main flow-matching VLAs.Source: §C.2 · p. 20, Table 12 · p. 20

C.3 Inference Latency of Action Sampling

• At N = 16, candidate sampling takes approximately 360 ms for π0 and 205 ms for π0.5, while the verifier scores each chunk in less than 0.5 ms.The paper uses N = 16 as a trade-off between candidate diversity and sampling cost; Figure 8 presents chunk-level sampling latency.Source: §C.3 · p. 21, Figure 8 · p. 21

C.4 Effect of the Number of Samples K for Criticality Estimation

• On RoboCasa with π0.5, performance improves as criticality-estimation samples K increase from 4 to 32, with only marginal gains at 64.The authors use K = 32 as a trade-off between estimation quality and data-collection sampling cost; Table 13 reports the sample-count ablation.Source: §C.4 · p. 21, Table 13 · p. 21

C.5 Representation Extraction

• On RoboCasa with π0.5, averaged action-token representations and the final action-expert layer perform best among the tested extraction choices.These choices are the default representation for both decision-criticality estimation and verifier training; Table 14 reports success rates by token position and layer.Source: §C.5 · p. 21, Table 14 · p. 21

C.6 Visualization of Action-Expert Representations

• Figure 9 shows task-dependent organization and partially distinct successful and failed behavior regions in action-expert representations.The authors note that the visualizations alone do not establish that outcome-related structure is fully shared across unseen tasks.Source: §C.6 · pp. 21–22, Figure 9 · p. 22

D Derivation of the Success-Visitation Ratio

• Assuming balanced sampling, the discriminator is trained with binary cross-entropy to distinguish normalized state–action visitation densities from successful and failed trajectories.The derivation minimizes the objective pointwise over state–action pairs.Source: D Derivation of the Success-Visitation… · pp. 22–23
• Its logit recovers the log success-to-failure visitation density ratio, while decision weighting makes the verifier score a candidate-ranking score rather than an exact density-ratio estimate.The derivation states the unweighted result; DiVeR reweights learning by decision criticality.Source: D Derivation of the Success-Visitation… · pp. 22–23
• The population-optimal discriminator is the successful visitation density divided by the sum of successful and failed visitation densities.The derivation gives f⋆ = ρ+/(ρ+ + ρ−).Source: D Derivation of the Success-Visitation… · pp. 22–23

E Baselines

• All selection rules use a shared candidate set sampled from the same frozen policy, so performance differences isolate the selection rule rather than the proposal distribution.Learned baselines also share the rollout dataset, optimizer, batch size, and class-balanced sampling.Source: E Baselines · pp. 23–24
• KDPE selects candidates by estimated action-chunk density, whereas MG-Select scores candidates by a kernel-density ratio against samples from a masked-conditioning policy.KDPE is mode-seeking and uninformed by task outcomes; MG-Select adapts its conditioning-support score to flow-matching policies without token likelihoods.Source: E Baselines · p. 24
• TACO scores behavior-distribution support without trajectory outcomes, whereas SVM learns from successful and failed visitation with uniformly weighted training samples.SVM also uses image features and raw action chunks rather than the policy’s action-expert representation.Source: E Baselines · pp. 24–25

F Additional Literature Survey

• Prior work on motor variability, process rewards, failure detection, and temporal credit assignment motivates emphasizing trajectory states where action selection has greater consequences.The cited examples include structured variability in human manipulation and learning signals localized to intermediate steps or state–action pairs.Source: F Additional Literature Survey · p. 25

• Real-world experiments cover a limited set of manipulation tasks and do not fully characterize transfer across broader environments or robot embodiments.Source: G Limitations and Future Work · p. 25
• Because DiVeR selects candidates from a frozen policy, performance ultimately depends on the quality and diversity of its available action candidates.Source: G Limitations and Future Work · p. 25
• Future work could explore broader real-world evaluation and stronger candidate generation mechanisms.Source: G Limitations and Future Work · p. 25