PaperlayerView exact revision on arXiv

Source-linked AI summary

What Should Robot Policies Learn to Predict? Rethinking Supervision Targets in Vision-Language-Action Learning

Haitao Shen, Hanyu Liu, Yihan Lin, Huiping Liu, Junbo Cui, Daoheng Li, Yang Li, Jing Zhang, Wenxin Li, Yuan Yao

arXiv:2610.21915v1cs.RO

TL;DR

The paper asks whether action prediction should dominate VLA training when auxiliary targets also shape policy learning under a shared budget. It introduces a sample-level routing framework to compare action, latent-action, and future-vision supervision, finding that suitable auxiliary targets can outperform action-only training even with no more than half the samples assigned to action prediction.

  • Problem

    Because action prediction is typically dominant despite latent-action and future-vision objectives also shaping policy learning, the paper asks whether reallocating samples to these targets improves execution under a fixed budget.

    Source: §1 · pp. 1–2

  • Method

    A two-stream VLA framework with sample-level routing assigns each training sample to action prediction or one of three latent-action or three future-vision targets.

    Source: §1 · p. 2

  • Results

    Suitable latent-action and future-vision targets outperform action-only training at the same budget with action prediction receiving no more than half the samples; future vision generally leads, while the strongest latent-action target remains competitive on CALVIN.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    The findings support treating prediction targets and their training allocation as explicit design choices rather than defaulting to action-dominant training.

    Source: §5 · p. 10

Abstract

from arXiv · show

Vision-language-action (VLA) policies are typically trained with action prediction as the primary supervision signal, since robots ultimately execute actions. Auxiliary objectives such as latent-action (LA) and future-vision (FV) prediction are often added to improve policy learning, yet they compete with action prediction for the same training budget. We ask whether, under a fixed budget, allocating a substantial share of training samples to such objectives, rather than to further action prediction, can improve policy learning. We build a unified VLA framework that couples a two-stream multimodal diffusion transformer (MMDiT) with a sample-level router, which assigns each training sample to either action prediction or an auxiliary objective and thus makes supervision allocation explicit. Within this framework, we instantiate three LA targets based on LAPA and its DINO-feature variants, and three FV targets based on DINOv3, V-JEPA 2, and Wan2.2 representations. Across LIBERO, CALVIN, and four real-robot tasks, suitable LA and FV supervision outperforms action-only training under the same budget, even when action prediction receives no more than half of the training samples. FV generally outperforms LA, although the strongest LA target remains competitive on CALVIN. Diagnostic analyses suggest that DINO-based LA targets emphasize relative motion, whereas FV targets emphasize absolute position and task identity. We therefore argue that what robot policies learn to predict, and how training is allocated among these targets, should be treated as explicit design choices rather than defaulting to action-dominant training.

1 INTRODUCTION

Although robot policies must ultimately execute actions, the paper questions whether action prediction should dominate training when latent-action and future-vision objectives also shape policy learning. It introduces a routed, fixed-budget framework and reports that suitable auxiliary supervision can outperform action-only training.

1 INTRODUCTION

  • The paper asks whether allocating as many or more training samples to latent-action or future-vision prediction than to action prediction can improve execution under a fixed budget.The motivation is that these objectives provide signals about state changes and task structure while shaping the same policy during optimization.Source: §1 · pp. 1–2
  • A two-stream MMDiT and sample-level router assign each training sample to action prediction or one of three latent-action or three future-vision targets.Future-vision prediction conditions on demonstrated actions, and source-matched controls support controlled comparisons across routing allocations.Source: §1 · p. 2
  • Suitable latent-action and future-vision supervision can outperform action-only training at the same budget, even when action prediction receives half or fewer of the samples.Across LIBERO, CALVIN, and four real-robot tasks, future vision generally outperforms latent action, while the strongest latent-action configuration remains competitive on CALVIN.Source: §1 · p. 2
  • The paper contributes a unified framework and experiments across LIBERO, CALVIN, and four real-robot tasks, alongside representation and attention analyses.The analyses distinguish the representation characteristics and policy-attention effects associated with latent-action and future-vision supervision.Source: §1 · p. 2

2 RELATED WORK

The related work situates latent-action and future-vision prediction among broader supervision strategies for robot learning. This paper focuses on comparing these targets and how training samples are allocated between auxiliary and direct action prediction.

Supervision beyond action labels

  • Robot-learning supervision extends beyond action labels to visual reconstruction, language generation, vision-language co-training, and geometric alignment.The paper narrows its focus to demonstration-derived latent-action and future-vision targets.Source: §2 · p. 3

Latent actions

  • Prior latent-action work explores transition-derived tokens, joint modeling, controllable changes, formulation choices, and representation-quality links to policy performance.This paper routes each sample to a single objective rather than reweighting losses on every sample, and probes sources with shared motion readouts.Source: §2 · p. 3

Future vision

  • Prior future-vision methods predict images, videos, visual states, or features; this paper compares them with action-only training at the same total budget.It varies the action share and uses source-matched future-vision controls, including allocations where auxiliary prediction receives most samples.Source: §2 · p. 3

3 METHOD

The method studies supervision allocation under a fixed training-sample budget, using latent-action and future-vision targets alongside direct action prediction. A two-stream policy and sample-level routing select the target and loss for each training sample.

3.1 PROBLEM SETUP AND SUPERVISION TARGETS

  • The fixed budget counts sample occurrences, including repeated demonstrations, and matches sample exposure rather than FLOPs or wall-clock time.The policy predicts an action chunk over the next H steps from the current observation, instruction, and robot state.Source: §3.1 · p. 3
  • Latent-action targets are discrete tokens derived from visual transitions, while future-vision targets use continuous future visual representations.The action chunk and latent-action target span the same H steps; future-vision uses a representation offset of δ = 8, and the current representation is an MMDiT input.Source: §3.1 · pp. 3–4

3.2 POLICY ARCHITECTURE AND ROUTING

  • A pretrained VLM supplies hidden states to separate action and auxiliary MMDiT experts, which exchange information through joint self-attention.The VLM encodes the current observation and language instruction; robot state separately conditions the action expert.Source: §3.2 · p. 4, Figure 2 · p. 4
  • A sample-level router selects either the action objective or the selected auxiliary objective, with each sample occurrence contributing to only one loss.FV-routed samples condition on clean demonstrated actions, and P25, P50, and P75 denote action-loss shares; P100 controls keep the FV source and architecture but never activate the FV loss.Source: §3.2 · pp. 4–5

3.3 TRAINING OBJECTIVES

  • The action, latent-action, and future-vision objectives use dataset-defined action loss, cross-entropy over valid latent-action tokens, and flow matching, respectively.The batch objective applies the loss associated with each sample’s selected route, with a target-specific weight for auxiliary prediction.Source: §3.3 · p. 5

4 EXPERIMENTS

The experiments compare action-only training with latent-action (LA) and future-vision (FV) supervision across simulation, real-robot tasks, and representation analyses. Results show that suitable auxiliary supervision can improve performance, but the best target and allocation depend on the benchmark and training regime.

4.1 EXPERIMENTAL SETUP

  • Experiments cover LIBERO’s four suites, CALVIN ABC→D, and four real-robot manipulation tasks; simulation full-budget runs use 60k updates, with a 30k half-budget condition.LIBERO uses task success, CALVIN reports average task-chain length and five-task success, and real-robot tasks use completion progress.Source: §4.1 · p. 6

4.2 MUST DIRECT ACTION PREDICTION RECEIVE MOST SUPERVISION?

  • With P50 allocation, all three FV configurations exceed the full-budget action-only baseline on LIBERO and CALVIN, while no LA configuration does so on LIBERO.On CALVIN, LAPA-DINOv3 also exceeds the full-budget action-only baseline; the strongest reported results are 98.15% LIBERO success with Wan2.2 and 4.451 CALVIN average task-chain length with V-JEPA 2.Source: §4.2 · p. 6, Table 1 · p. 6
  • At fixed full budget, V-JEPA 2 at P25 reaches 4.522 CALVIN average task-chain length versus 4.091 for its source-matched control, but every tested P25 configuration falls below its P100 control on LIBERO.The allocation comparison is shown in Figure 3; the preferred allocation therefore depends on both target and benchmark.Source: §4.2 · pp. 6–7, Figure 3 · p. 7
  • At P50, all FV configurations beat their source-matched controls on CALVIN across four tested budgets, but gains over the shared action-only baseline are less consistent and LIBERO gains vary.The Figure 4 panels show performance across budgets; Figure 5’s full sweep finds CALVIN gains over source-matched controls for V-JEPA 2 and Wan2.2 across tested mixed allocations and budgets.Source: §4.2 · p. 7, Figure 4 · p. 7, Figure 5 · p. 8

4.3 HOW DO LA AND FV COMPARE ACROSS TRAINING REGIMES AND TASKS?

  • FV leads LA overall on LIBERO and the real-robot tasks, while CALVIN results depend on allocation and budget, with LAPA-DINOv3 competitive with leading FV configurations.On real-robot tasks, every FV configuration beats every LA configuration and the action-only baseline; on LIBERO, FV leads overall, while LA and FV rankings vary on CALVIN.Source: §4.3 · p. 8

4.4 WHY CAN LA AND FV SUPERVISION HELP POLICY LEARNING?

  • Frozen-source probes show that FV representations have lower absolute-position errors, while LAPA-DINO variants have lower relative-motion errors on unseen AgiBotWorld-Beta tasks.Feature clustering associates LAPA-DINO more strongly with motion labels and FV sources more strongly with task IDs, suggesting distinct motion- versus position/task-related cues.Source: §4.4 · pp. 8–9, Figure 7 · p. 10
  • On two LIBERO observations, LA and FV policy attention shifts toward task-relevant objects and interaction regions compared with diffuse action-only attention in the top-row example.The examples show V-JEPA 2 highlighting the stove and robot, and LAPA-DINOv3 emphasizing the gripper and nearby containers.Source: §4.4 · p. 10, Figure 8 · p. 10

5 CONCLUSION

Suitable latent-action and future-vision targets can outperform action-only training under a fixed budget, even when action prediction receives no more than half the samples. Future-vision generally performs better, while LAPA-DINOv3 remains competitive on CALVIN.

5 CONCLUSION

  • Suitable LA and FV targets outperform action-only training under the same budget even when action prediction receives no more than half the samples.The best allocation depends on the target, benchmark, and budget.Source: §5 · p. 10
  • FV leads on LIBERO and all four real-robot tasks, while LAPA-DINOv3 remains competitive on CALVIN.Source: §5 · p. 10
  • LAPA-DINO targets emphasize relative motion, whereas FV targets emphasize absolute position and task identity.Both target families carry action-related information and can redirect policy attention toward task-relevant regions.Source: §5 · p. 10
  • The findings support jointly designing predictive targets and their supervision allocation rather than defaulting to action-only training.Source: §5 · p. 10

A TRAINING AND EVALUATION DETAILS

The appendix specifies the encoders, target representations, and temporal construction used for latent-action and future-vision supervision. LA targets are sequences of categorical codes, while FV targets use continuous encoder features.

A.1 TARGET ENCODERS AND LABEL CONSTRUCTION

  • The reported experiments use six frozen target encoders, with the same encoders and checkpoints used for frozen-source diagnostics.Table 3 records the policy VLM and target encoders.Source: §A.1 · p. 15, Table 3 · p. 15
  • Each LAPA-DINO variant pairs a DINO backbone with a separately trained LARYBench quantizer using 32-dimensional quantization and a codebook of eight entries.The DINOv3 FV encoder is a separate ViT-S/16 checkpoint.Source: §A.1 · p. 15
  • With horizon H = 8, an LA target contains eight code groups, and flattening them yields Lz = HK target positions.LAPA provides K = 4 indices per group; LAPA-DINO variants provide K = 16, and the targets are categorical indices rather than continuous codebook vectors.Source: §A.1 · p. 16
  • For CALVIN and LIBERO, each LA group is computed from primary-camera frames eight frames apart and stored at its source index.An anchor at t retrieves groups for t through t + 7, so the final group depends on frame It+15.Source: §A.1 · p. 16
  • FV supervision uses the future observation at t+8, and temporal validity masks exclude anchors lacking required observations or labels.FV encoders remain frozen while their feature projectors are trainable.Source: §A.1 · p. 16

A.2 IMPLEMENTATION, OPTIMIZATION, AND EVALUATION

The appendix documents target-specific training inputs and losses, shared optimization settings, and matched routing controls. The LIBERO and CALVIN comparisons hold the principal training and inference conditions fixed while varying routing probability.

A.2 IMPLEMENTATION, OPTIMIZATION, AND EVALUATION

  • Policies predict clean eight-step chunks of seven-dimensional actions, with dataset-specific action representations and L1 or mean-squared-error losses.LIBERO and DOS W1 use elementwise L1 error; CALVIN uses mean-squared error, and validity masks exclude invalid action timesteps.Source: §A.2 · p. 16
  • Each sample contributes only its selected loss, with the auxiliary group weighted by λq: 0.1 for LAPA, 0.01 for LAPA-DINO, and 1 for FV.Each target-specific weight remains fixed across allocations and checkpoints.Source: §A.2 · p. 17
  • LIBERO configurations share 1,693 training episodes across 40 tasks, while CALVIN uses ABC training environments and evaluates on environment D.LIBERO demonstrations comprise 273,465 frames at 20 Hz.Source: §A.2 · p. 17
  • LIBERO demonstrations are shared across configurations, and the episode-count range for each suite is summarized in Table 4.Source: Table 4 · p. 17
  • All main LIBERO and CALVIN experiments use AdamW and shared settings whose learning rates decay to 5% of their peaks.The table of shared optimization settings is identified in the appendix.Source: §A.2 · p. 17, Table 5 · p. 17
  • Matched LIBERO and CALVIN routing comparisons vary routing probability while holding initialization, data, target encoder, action horizon, inputs, batch size, updates, schedule, and inference settings fixed.At P100, all samples follow the action route; FV controls retain their source encoder and architecture, while LA targets share a query-based action-only baseline.Source: §A.2 · p. 17

A.3 POLICY EVALUATION PROTOCOLS AND METRICS

LIBERO performance is reported as pooled rollout success, while CALVIN performance summarizes completion across ordered task sequences. The appendix clarifies how the metrics relate and identifies the seed coverage for reported results.

A.3 POLICY EVALUATION PROTOCOLS AND METRICS

  • LIBERO includes ten tasks in each of Spatial, Object, Goal, and Long, with 50 rollouts per task.Equal task and rollout counts make Avg. the pooled success rate over 2,000 rollouts.Source: §A.3 · p. 18
  • LIBERO values use evaluation seed 0, while CALVIN main-text results also use seed 0 and Table 6 reports seeds 0, 1, and 2.Source: §A.3 · p. 18, Table 6 · p. 18
  • CALVIN evaluates the same 1,000 ordered five-task sequences and initial states, stopping each sequence at its first failure.The completed prefix length Li ranges from zero to five.Source: §A.3 · p. 18
  • CALVIN average task-chain length summarizes the sequence outcomes, and SR-k is cumulative rather than conditional.Avg. Len. summarizes the same sequence outcomes.Source: §A.3 · p. 18

B.1 PRIMARY BENCHMARK DETAILS

This section supplies benchmark details for the CALVIN and LIBERO comparisons, including source-matched FV controls and checkpoint results. The reported table comparisons do not establish a universal ranking across training protocols.

B.1 PRIMARY BENCHMARK DETAILS

  • The table comparisons do not establish a universal ranking across training protocols.Table 7 retains CALVIN prefix-success metrics omitted from the main comparison table.Source: §B.1 · p. 18

B.2 SOURCE-MATCHED FV CONTROLS

  • Table 7 provides CALVIN prefix-success metrics underlying the main comparison, while Table 8 compares LIBERO FV results with shared action-only and source-matched controls.Table 8 holds all settings except routing fixed for source-matched controls; its columns use the same checkpoint budget.Source: Table 8 · p. 19, Table 7 · p. 19

B.3 LIBERO P50 CHECKPOINT COMPARISONS

  • Table 9 reports LIBERO P50 success rates at four checkpoints, with half of training samples routed to action prediction in expectation.The reported average is the mean of the four suite rates.Source: Table 9 · p. 19, §B.3 · p. 19

B.4 LA LOSS-WEIGHT ABLATION

The section tests LA loss weights and reports CALVIN checkpoint and allocation comparisons. Its allocation tables distinguish LA’s shared action-only baseline from FV’s source-matched controls.

B.4 LA LOSS-WEIGHT ABLATION

  • For every LA target, λLA = 1.0 performs worst; the DINO variants perform best at 0.01, while LAPA is slightly better at 0.1 than 0.01.The experiments use 0.1 for LAPA and 0.01 for both LAPA-DINO variants, and reuse these weights on CALVIN without further tuning.Source: §B.4 · p. 20

B.5 CALVIN CHECKPOINT COMPARISONS

  • Table 11 gives CALVIN P50 average task-chain lengths at each tested budget, and Table 12 reports positive P50-minus-source-matched-P100 differences on both reported metrics.The metrics are ΔAvg. Len. in tasks per sequence and ΔSR-5 in percentage points.Source: §B.5 · p. 20, Table 12 · p. 21, Table 11 · p. 21

B.6 CALVIN ALLOCATION COMPARISONS

  • Table 13 reports CALVIN allocation outcomes at 60k updates, with LA compared against the shared action-only baseline and FV against source-matched P100 controls.P25, P50, P75, and P100 denote action-routing probabilities; lower P assigns a larger expected fraction of samples to auxiliary prediction.Source: §B.6 · p. 20, Table 13 · p. 22
  • Table 14 reports the FV allocation grid at 40k CALVIN updates, with fixed batch size and optimization settings within each source.Bold marks the best tested value for each metric within each row.Source: §B.6 · p. 20, Table 14 · p. 22

B.7 REAL-ROBOT EVALUATION DETAILS

The supplementary section describes the real-robot evaluation and diagnostic protocols. It reports task-specific robot scores and defines the scope and interpretation of offline representation probes.

B.7 REAL-ROBOT EVALUATION DETAILS

  • Real-robot outcomes cover four tasks on a DOS W1 left follower arm, using P50 auxiliary configurations trained for 30k updates and evaluated over 20 trials per task.Each task score is out of 160 accumulated points, with component-level rules allowing partial credit.Source: §B.7 · p. 22, Table 15 · p. 23

C.1 DIAGNOSTIC PROTOCOLS AND READOUT TARGETS

  • The diagnostic readouts characterize information recoverable by tested probes, not policy performance; AgiBotWorld-Beta seen-validation measures generalization within seen tasks rather than to new trajectories.Seen training and validation share episodes, though no identical 45-step action segment appears in both; unseen errors use checkpoints selected on seen validation.Source: §C.1 · p. 23
  • ABS probes predict stored absolute trajectories, while delta probes predict separately transformed coordinate-wise action differences.The diagnostic setting c is distinct from the policy target offset δ; c = 1, 9, and 45 encode different action-difference strides.Source: §C.1 · pp. 23–24
  • Delta-step5 predicts nine successive action differences, and its position error uses the ground-truth initial action as an evaluation boundary condition rather than a probe input.ABS errors are scored directly, so ABS and delta errors evaluate different readout tasks.Source: §C.1 · p. 24
  • Feature–action geometry compares distances between latent representations and action descriptors, while clustering compares feature clusters with displacement-derived action labels and task IDs.The geometry descriptor captures net change rather than the time-resolved regression target.Source: §C.1 · pp. 24–25

C.2 GEOMETRY AND TASK–ACTION CLUSTERING

The section reports associations between representation geometry, diagnostic readout error, action labels, and task identity. It also specifies how policy attention maps are extracted.

C.2 GEOMETRY AND TASK–ACTION CLUSTERING

  • Trustworthiness has a Spearman coefficient of −0.93 with delta-step5 NMSE, but these are descriptive associations with offline readout error, not policy performance.Table 17 expands Figure 7a and reports geometry and regression measurements on AgiBotWorld-Beta seen validation.Source: §C.2 · p. 25, Table 17 · p. 25
  • Table 18 reports complementary associations between feature clusters, displacement-derived action labels, and task IDs.The supplied text specifies the compared quantities but does not state their results.Source: §C.2 · p. 25, Table 18 · p. 26

C.3 POLICY ATTENTION MAPS

  • Policy attention maps use the action expert’s cross-attention to VLM conditioning tokens, averaged across the last four layers, 12 heads, and three inference noise seeds.The maps use eight action-token queries and exclude the state token.Source: §C.3 · p. 26

Related papers

Loading 2610.21915v1…