PaperlayerView exact revision on arXiv

Source-linked AI summary

Scaling Verification Can Be More Effective than Scaling Policy Learning for Vision-Language-Action Alignment

Jacky Kwok, Xilun Zhang, Mengdi Xu, Yuejiang Liu, Azalia Mirhoseini, Chelsea Finn, Marco Pavone

arXiv:2602.12281v2cs.ROcs.AIeess.SY

TL;DR

VLA actions can misalign with language instructions, motivating test-time verification of the intention-action gap. The paper studies test-time scaling, introduces CoVer and the CoVer-VLA hierarchical pipeline, and reports gains across simulated and real-world evaluations. Its results support allocating compute to deployment-time reasoning and verification rather than policy training alone.

  • Problem

    VLA-generated actions can misalign with natural-language instructions, while policy-pretraining approaches provide limited robustness and may cause catastrophic forgetting.

  • Method

    The paper combines instruction rephrasing, repeated action sampling, a contrastive CoVer verifier, and hierarchical test-time selection of prompts and action chunks.

  • Results

    CoVer-VLA substantially improves performance across simulated and real-world settings, including 45% absolute improvement on real-world tasks, 18% on SIMPLER environments, and 9% on PolaRiS.

  • Takeaways & Limitations

    Allocating compute to reasoning and verification at deployment can be more effective than scaling policy training alone for robust policy deployment.

Abstract

from arXiv · show

The long-standing vision of general-purpose robots hinges on their ability to understand and act upon natural language instructions. Vision-Language-Action (VLA) models have made remarkable progress toward this goal, yet their generated actions can still misalign with the given instructions. In this paper, we investigate test-time verification as a means to shrink the "intention-action gap." We first characterize the test-time scaling laws for embodied instruction following and demonstrate that jointly scaling the number of rephrased instructions and generated actions greatly increases test-time sample diversity, often recovering correct actions more efficiently than scaling each dimension independently. To capitalize on these scaling laws, we present CoVer, a contrastive verifier for vision-language-action alignment, and show that our architecture scales gracefully with additional computational resources and data. We then introduce CoVer-VLA, a hierarchical test-time verification pipeline using the trained verifier. At deployment, our framework precomputes a diverse set of rephrased instructions from a Vision-Language-Model (VLM), repeatedly generates action candidates for each instruction, and then uses the verifier to select the optimal high-level prompt and low-level action chunks. Compared to scaling policy pre-training on the same data, our verification approach yields 22% gains in-distribution and 13% out-of-distribution on the SIMPLER benchmark, with a further 45% improvement in real-world experiments. On the PolaRiS benchmark, CoVer-VLA achieves 14% gains in task progress and 9% in success rate.

1. Introduction

VLA models remain vulnerable to an intention-action gap, motivating test-time verification as an alternative to scaling policy training. The paper develops CoVer-VLA, which combines diverse instruction rephrasing, action sampling, and hierarchical verification to improve alignment and downstream performance.

  • Motivation: VLA models can misalign generated actions with language instructions, causing costly errors in human-centric environments.A robot may grasp the correct object but place it in the wrong receptacle, potentially damaging the object or creating a safety hazard.
  • Motivation: Existing approaches based on policy pre-training generally provide incremental gains, degrade under perturbations, and can cause catastrophic forgetting of multimodal reasoning.These limitations motivate examining whether additional computation should be allocated at test time instead.
  • Test-time scaling: Jointly scaling rephrased instructions and generated actions produces more diverse proposals and often recovers correct actions more efficiently than scaling either dimension alone.Under an oracle verifier, action error decreases as the number of rephrased instructions increases.
  • CoVer-VLA: CoVer is a contrastive verifier for vision-language-action alignment that supports instruction optimization and action verification.Its architecture combines a text-aware visual encoder with an action encoder designed to capture long-range temporal dependencies in action chunks.
  • CoVer-VLA: CoVer-VLA precomputes rephrased instructions with boot-time VLM reasoning, generates action candidates per instruction, and selects high-level prompts and low-level action chunks through verification.The hierarchical pipeline performs offline reasoning before deployment and scores instruction-action pairs during execution.

2. Related Work

Prior VLA research has mainly addressed instruction-following failures through model scaling, more data, auxiliary objectives, and action sampling. This work instead treats instructions as distributions over phrasings and verifies language-behavior alignment at test time.

  • Vision-Language-Action Models: VLA models can suffer sharp success drops when instructions are rephrased despite semantic equivalence.This motivates methods that improve robustness to linguistic variation.
  • Vision-Language-Action Models: Related approaches scale model capacity, expand training data, or add objectives intended to preserve linguistic knowledge.The paper positions test-time verification as orthogonal to these training-based strategies.
  • Test-Time Scaling: Test-time scaling in robot learning commonly optimizes over multiple candidate action sequences to improve performance, consistency, or robustness.These methods extend a broader inference-time-compute paradigm developed in language reasoning, visual understanding, and agentic planning.
  • Action Verification: Action verification has used policy uncertainty, temporal consistency, value functions, and preference models as quality signals.These approaches differ from the paper’s formulation of verification as language-behavior alignment.
  • Action Verification: The paper formulates action verification as a contrastive alignment problem between language and behavior, explicitly targeting instruction-following quality.This distinguishes the proposed verifier from approaches focused primarily on low-level dynamics.

3. Test-Time Scaling Analysis

The scaling analysis compares repeated sampling, Gaussian perturbation, instruction rephrasing, and hybrid sampling for embodied instruction following. Instruction rephrasing broadens candidate actions, while combining rephrases with repeated sampling provides the strongest diversity and error reduction.

  • Sampling strategies: The analysis evaluates four action-candidate strategies: repeated sampling, Gaussian perturbation, instruction rephrasing, and hybrid sampling.Instruction rephrasing conditions policy samples on VLM-generated alternatives, while hybrid sampling combines rephrasing with repeated sampling.
  • Results: Instruction rephrasing consistently yields lower action error than vanilla repeated sampling and Gaussian perturbation.The comparison examines how linguistic diversity affects downstream policy performance.
  • Results: Hybrid sampling achieves greater candidate diversity by combining instruction rephrasing with repeated sampling.It explores radically different actions rather than becoming stuck in a local minimum.

4. Method

CoVer reframes VLA inference as hierarchical test-time optimization over language rephrases and action candidates, using a contrastive verifier to select aligned behaviors. The method combines multimodal alignment, offline rephrase generation, and verifier-based action selection without changing the underlying policy.

  • Hierarchical prompt-action optimization: The framework jointly optimizes language and action by selecting a rephrase whose induced behavior matches the user’s intent, then verifying individual action candidates.This treats the instruction as a test-time decision variable rather than committing to one phrasing.
  • Action-level optimization: The verifier selects among M action chunks conditioned on the selected rephrase because single-policy samples can be unreliable under bias and noise.Verification uses visual observations, instruction language, action sequences, and recent action history as temporal context.
  • Contrastive verifier: CoVer learns a shared embedding space for visual observations, language instructions, and robot actions through contrastive pre-training.A fused image-text encoder extracts task-relevant visual features, while an action encoder embeds action sequences in the same space.
  • Verifier training: Rephrase augmentation expands language coverage while keeping observations and ground-truth actions fixed, enabling semantic-equivalence learning without altering the action distribution.Each original instruction is paired with multiple linguistic realizations of the same underlying intent.
  • Test-time verification: At deployment, boot-time VLM reasoning generates and caches diverse rephrases, after which CoVer-VLA hierarchically scores prompt-action pairs and selects the prompt and action for execution.Caching moves scene reasoning and embedding computation offline to reduce runtime latency and avoid redundant rephrase generation.

5. Experiments

Experiments show that CoVer benefits consistently from scaling verifier capacity, data, training resources, and ensembling, while CoVer-VLA improves robustness across simulated, OOD, real-world, and PolaRiS evaluations. Test-time verification generally outperforms or complements training-time instruction augmentation, with gains extending to challenging rephrased instructions and cluttered scenes.

  • 5.1. Verifier Scaling Results: Scaling dataset size from 8× to 64× and model capacity from 250M to 1B parameters steadily improves CoVer’s top-1 retrieval accuracy.
  • 5.1. Verifier Scaling Results: Larger contrastive-learning batches and longer training expose CoVer to more negative examples, improving convergence and results.Batch size is scaled from 2,048 to 8,192, while the paper also studies training epochs.
  • 5.1. Verifier Scaling Results: Verifier top-1 retrieval accuracy improves as synthetic instructions, model parameters, negative samples, training compute, and ensemble size increase.The study examines model size, dataset size, batch size, training compute, and ensemble size as scaling dimensions.
  • 5.1. Verifier Scaling Results: Ensembling verifiers improves action-retrieval accuracy from ensemble sizes of 1 to 8 by reducing variance from individual model biases.
  • 5.4. Simulation Evaluation Results: Training-time instruction augmentation alone yields minimal in-distribution gains, increasing performance from 41.5 to 44, with modest OOD improvements.
  • 5.4. Simulation Evaluation Results: Random rephrasing raises in-distribution performance from 41.5 to 42.3 but lowers OOD performance from 29.7 to 28.7 and varies sharply across tasks.Success ranges from 78% on Eggplant in Basket to 1% on Redbull on Plate.
  • 5.4. Simulation Evaluation Results: On PolaRiS, pairing π0.5 with CoVer increases task progress by 13.9% and success rate by 9.3% on average across three environments.The evaluation averages 50 episodes and 3 seeds.
  • 5.4. Simulation Evaluation Results: CoVer-VLA improves robustness by 16% in-distribution and 31% OOD, while verification scaling exceeds policy-learning scaling by 15% and 12%, respectively.Combining π0 (rephrase) with CoVer reaches 65.5% on ID tasks and 62.0% on OOD tasks.

6. Conclusion

CoVer-VLA uses contrastive verification and hierarchical test-time scaling to address the intention–action gap in generalist robot policies. Its improvements span simulated and real-world settings, especially under out-of-distribution conditions.

  • CoVer-VLA combines a contrastive-based verifier with hierarchical test-time scaling for generalist robot policies.The framework targets the intention–action gap during deployment.
  • The framework improves performance across simulated and real-world settings, particularly under out-of-distribution conditions.The conclusion reports broad gains without specifying individual benchmark values in this passage.
  • The authors conclude that allocating compute to reasoning and verification at deployment can be more effective than scaling policy training alone.They also suggest extending the design beyond inference optimization and improving efficiency to reduce latency.

8. Appendix

The appendix describes the evaluated environments, task settings, baselines, and verifier experiments across SIMPLER, PolaRiS, and real-world rollouts. It also documents a major failure mode of the RoboMonkey baseline caused by policy-distribution and action-chunk differences.

  • Evaluation settings: The evaluation covers 7 SIMPLER tasks, 3 PolaRiS tasks, and 2 real-world WidowX tasks, with OOD settings containing distractors and novel objects.Representative executions are shown for the benchmark and real-world environments.
  • Evaluation settings: SIMPLER tasks test fine-grained language–vision grounding, including distinguishing the blue Redbull can from a visually confusing red Coca-Cola can.Other tasks examine object discrimination in out-of-distribution scenes.
  • Evaluation settings: PolaRiS tasks require identifying target objects and categories amid clutter, then placing or using them in sequential manipulation tasks.Examples include stacking blocks on a green tray, placing foods in a bowl, and scrubbing a frying-pan handle.
  • Baselines: The baselines include vanilla π0, instruction-augmented π0 (rephrase), and RoboMonkey with action resampling and a VLM-based verifier.The appendix explicitly defines these comparison settings and their training or test-time components.
  • Baseline behavior: RoboMonkey fails catastrophically on most tasks, which the authors attribute to verifier training on OpenVLA actions and mismatch with π0’s flow-based action chunks.The reported explanation links the failure to both action-distribution mismatch and differing action-generation structure.

8.5. Verifier Performance Analysis

The verifier separates aligned from misaligned actions, favors successful rollout segments, and reduces action error as more candidates are sampled. Its frozen encoders keep added latency and training cost comparatively low.

  • Verifier discrimination: CoVer achieves 0.765 precision, 0.780 recall, and 0.772 F1 when discriminating aligned from randomly sampled misaligned actions.These results measure binary classification performance.
  • Rollout scoring: Successful trajectories receive higher verifier scores, with peaks during object approach and task completion, whereas failed trajectories often decline steadily.The score patterns are visualized across episodes.
  • Rollout scoring: The score separation supports using CoVer as a runtime monitor for detecting and rejecting low-confidence actions during deployment.This is presented as a potential utility of the observed scoring behavior.
  • Candidate scaling: 11% lower action error results when CoVer selects among N = 16 candidates instead of using greedy decoding with N = 1.Action error is defined as RMSE between the selected and ground-truth actions on held-out trajectories.
  • Efficiency: Frozen image and text encoders reduce verifier training cost because their large backbones require no backward-pass gradients.The reported backward pass is approximately 1.0 × 10^9 FLOPs versus approximately 3.3 × 10^11 FLOPs for the forward pass.
  • Efficiency: The CoVer action encoder adds only 7–8ms, and total π0+CoVer latency exceeds the base model by less than 10ms across tested configurations.At batch size 32, combined latency is 756ms, corresponding to approximately 1.3Hz for quasi-static manipulation.

8.8. Generated Rephrases from Red-Teaming Instructions

The appendix distinguishes original, red-team, generated, and random rephrased instructions used in the evaluations. Generated rephrases from an off-the-shelf VLM serve as alternatives during verification, while random rephrases define a baseline.

  • Instruction types: Original instructions come from BridgeV2 and DROID templates and are used for task labeling rather than evaluation.
  • Instruction types: Red-team instructions are challenging ERT-generated rephrases used to test robustness to more flexible user instructions.
  • Instruction types: Generated rephrases are produced by GPT-4o and used as alternative instructions during the verification process.Their quality does not explicitly affect verifier performance because similarity is computed between generated actions and the original user instruction.
  • Instruction types: Random rephrase selects one generated rephrase and serves as the π0 + random rephrase baseline.

8.9. Boot-time Reasoning Implementation

Rephrase generation is performed once at boot time, removing its cost from per-step inference. A VLM is used because visual grounding improves semantic specificity and downstream VLA performance.

  • Boot-time latency: Boot-time rephrase generation avoids adding latency during inference.Generating 8 rephrases with an off-the-shelf VLM takes approximately 11 seconds at boot.
  • Rephrase generation: The system uses an off-the-shelf VLM to interpret the scene and generate instruction rephrases.The VLM is queried once per episode, making its boot-time inference cost negligible during execution.
  • Visual grounding: VLM rephrases preserve visual details such as object color, whereas LLM rephrases can introduce semantic drift.For “put redbull can on plate,” the VLM produces color-specific alternatives such as “blue can,” while the LLM produces category-level terms such as “beverage.”

8.10. VLM Prompts for Rephrase Generation

The rephrase prompts instruct a text-transformation assistant to generate concise, diverse, semantically equivalent alternatives while using the image to ground the instruction. Examples and red-team cases illustrate how the resulting alternatives vary terminology while retaining task objectives.

  • Prompt objective: The prompt frames rephrasing as text transformation for robot manipulation tasks.It requires understanding the original instruction and generating multiple grammatically correct alternatives that preserve intent.
  • Prompt objective: Rephrases must remain semantically equivalent, concise, grammatically correct, and easy to follow.The instructions emphasize preserving the same meaning and objective while varying vocabulary and maintaining clear structure.
  • Rephrasing guidelines: Rephrasing guidelines favor simple wording, concise descriptions, diverse vocabulary, preserved objectives, and inferred object colors when reasonably deducible.They also recommend removing adverbs and varying nouns, verbs, and adjectives across alternatives.
  • Examples: Examples replace task-specific nouns and verbs while retaining the original manipulation objective.Illustrations cover placing apples, pots, strawberries, and water bottles, with alternatives such as “red fruit,” “green basket,” and “wooden desk.”
  • Visual grounding: The VLM is instructed to consider both the image and instruction, first describing the scene before interpreting the task in context.The response format also requests candidate replacements for nouns, verbs, adjectives, and adverbs.
  • Red-team examples: Red-team examples test whether rephrases preserve objects, attributes, destinations, and actions across manipulation instructions.The cases include eggplants, tennis balls, food transfer, sponge cleaning, and block placement, alongside VLM-generated alternatives and random rephrases.

9. Notation

The notation defines the observation, instruction, policy, verifier, rephrase, action-sampling, and training variables used to describe CoVer-VLA.

  • Core variables: O, A, and L denote observation, action, and natural-language instruction spaces, while o_t and h_t denote visual observation and recent action history.The original instruction is l, and l′ denotes a rephrased instruction.
  • Rephrases and actions: L_r(l) denotes the set of K boot-time language rephrases, π is the base VLA policy, and a′_k,j is an action chunk sampled from π under rephrase l′_k.The indices k and j identify the rephrase and action sample.
  • Verification variables: V_θ is the contrastive verifier, s_k,j is an alignment score, and S_k summarizes semantic reliability for rephrase k.l* is the selected rephrase and a*_t is the final verified action chunk selected for execution.
  • Training variables: f_i and a_i are normalized vision-language and action embeddings, while B, K, M, D_aug, and N describe training and sampling sizes.K and M specify the numbers of rephrases and action samples per rephrase; D_aug is augmented with N rephrases per task.
Loading 2602.12281v2…