PaperlayerView exact revision on arXiv

Source-linked AI summary

Looking Back to Move Forward: Temporal Verification for Generative Robot Policies

Haoxuan Wang, Wayne Wu, Yan Yan, Bolei Zhou

arXiv:2609.39038v1cs.RO

TL;DR

Generative policies can make errors that compound into difficult out-of-distribution states, while existing action verification overlooks trajectory continuity and can require costly verifiers and supervision. TeV conditions contrastive energy verification on recent observation–action history and uses it to rank candidates and guide generation. Simulation and real-world experiments report improved task success and smoother trajectories.

  • Problem

    Existing action verification evaluates candidates from the current observation alone and often relies on large verifiers and additional supervision, leaving trajectory continuity unaccounted for.

    Source: §1 · p. 2

  • Method

    TeV uses a temporal token summarizing recent observation–action history to condition a contrastive energy verifier that ranks action chunks and guides intermediate flow samples.

    Source: §1 · p. 2

  • Results

    TeV improves task success by 6–18 percentage points and produces smoother execution trajectories in simulation and real-world experiments.

    Source: §1 · p. 2

  • Takeaways & Limitations

    These results demonstrate the benefits of temporally aware verification for generative robot control.

    Source: §1 · p. 2

Abstract

from arXiv · show

Generative policies have emerged as a promising paradigm for robot learning, combining expressive generative action modeling with scalable imitation learning from large demonstration corpora. However, heterogeneous demonstrations can induce suboptimal action chunks whose errors compound over time, eventually driving the robot into out-of-distribution states from which recovery is difficult. Action verification offers a test-time scaling strategy for mitigating this failure mode by sampling multiple candidate actions and using a verifier to select one for execution. Existing approaches, however, remain temporally myopic and costly to train, evaluating candidates from the current observation alone without accounting for trajectory continuity and often relying on large verifiers and additional expert demonstrations. In this paper, we introduce Temporal Verification (TeV), an efficient temporally aware action verification framework for flow-matching VLAs. TeV first learns a temporal token that summarizes recent observation--action history, enabling candidate chunks to be evaluated as continuations of the execution trajectory rather than as isolated predictions. Conditioned on this token, TeV constructs positive--negative pairs without additional expert demonstrations or preference annotations and trains an energy-based verifier contrastively to assign lower energy to higher-quality, trajectory-consistent action chunks. Beyond post-hoc ranking, TeV further uses the learned energy landscape to guide intermediate flow samples toward lower-energy regions, improving candidates before final selection. Extensive experiments in simulation and real-world settings demonstrate that TeV provides reliably ranks action candidates, improves task success rates, and produces smoother execution trajectories.

1. Introduction

Generative flow-matching policies can make errors that compound into difficult out-of-distribution states, while existing verification is temporally myopic and costly. TeV uses recent execution history to rank and guide action chunks; the paper reports higher task success and smoother trajectories.

1. Introduction

  • Errors in action chunks from heterogeneous demonstrations can compound, moving robots into out-of-distribution states where recovery becomes difficult.Retraining and fine-tuning can mitigate the issue but incur substantial cost and risk degrading generalist capabilities.Source: §1 · p. 1
  • Verification-based test-time scaling samples multiple action chunks and selects one using a separate verifier, without modifying the base policy.It allocates additional inference-time compute to candidate selection.Source: §1 · p. 2
  • Existing verification evaluates candidates from the current observation alone, overlooking cross-chunk coherence that matters for smooth, stable execution.The authors associate this temporal myopia with oscillation, jerky motion, and task failure.Source: §1 · p. 2
  • TeV summarizes recent observation–action history in a temporal token and uses a lightweight energy verifier to rank candidates and guide intermediate flow samples.The verifier is trained contrastively with expert chunks as positives and temporally mismatched or policy-generated chunks as negatives.Source: §1 · p. 2
  • TeV improves task success by 6–18 percentage points and produces smoother execution trajectories in simulation and real-world experiments.The authors describe it as a lightweight add-on requiring no additional expert demonstrations.Source: §1 · p. 2

2. Related Work

Prior work addresses action verification, temporal consistency, and inference-time steering through distinct strategies. TeV combines history-conditioned verification with temporal coherence and generation-time steering.

2. Related Work

  • Related verification methods include VLM-based evaluators, RL-based adaptation or value functions, and TACO's pseudo-count preference for candidates near successful training modes.Unlike these methods' static-timestep observations, TeV conditions verification on recent execution history.Source: §2 · p. 2
  • Temporal-consistency approaches include ACT's averaging of overlapping chunks, BID's selection for consistency with prior predictions, and asynchronous methods that address stale observations.TeV instead learns a history-conditioned energy verifier for expert-compatible continuation chunks.Source: §2 · pp. 2–3
  • Conventional verification ranks candidates using current observations, while temporal-consistency methods refine predictions using previous chunks; TeV combines temporal context with both guidance and selection.TeV's learned verifier is conditioned on past observation–action context.Source: Figure 1 · p. 3
  • Inference-time steering has used external objectives, learned dynamics, or human input; TeV extends contrastive energy verification from post-hoc ranking to steering generation.Figure 1 situates TeV alongside action verification and temporal-consistency approaches.Source: §2 · p. 3, Figure 1 · p. 3

3. Preliminaries

Flow-matching VLAs produce action chunks by integrating a learned velocity field from noise, while contrastive energy modeling learns relative compatibility between positive and negative samples. TeV applies that verifier within a history-conditioned generation and selection pipeline.

3. Preliminaries

  • A flow-matching VLA integrates a learned velocity field from an initial noise sample to produce an action chunk, with the update performed over normalized flow time.The integration begins at x0 and terminates at x1.Source: §3 · p. 3
  • Contrastive energy modeling assigns lower energy to positive than negative samples by a margin, matching verification's need for relative ranking rather than calibrated rewards.Under the shared context, the loss encourages positives to have lower energy than negatives by at least m.Source: §3 · p. 3
  • TeV encodes observation–action history into a temporal token that conditions its energy verifier, which guides intermediate flow samples and ranks completed chunks.Expert chunks are positives; temporally mismatched chunks and policy-generated deviations serve as negatives.Source: Figure 2 · p. 4

4. Methodology

TeV augments flow-matching VLAs with a temporal token and a lightweight energy-based verifier, using execution history to train, guide, and rank action chunks.

4.1. Temporal Token Learning

  • A causal temporal transformer combines observation summaries with preceding executed actions, while future-action prediction regularizes its temporal token to retain motion-relevant information.The auxiliary objective predicts the next F actions from the temporal token.Source: §4.1 · pp. 4–5

4.2. Contrastive Verifier Training

  • TeV’s verifier contrasts expert chunks with temporally mismatched expert chunks and coarsely integrated policy samples, learning action quality and temporal compatibility.Expert chunks come from offline imitation data; mismatched chunks are plausible in isolation but inconsistent with execution history, while fewer Euler steps produce imperfect policy-generated alternatives.Source: §4.2 · p. 5

4.3. Verification as Guidance

  • At inference, verifier-energy gradients steer intermediate flow samples, with guidance decaying to zero before the verifier ranks completed candidates for execution.Stronger early guidance influences coarse action structure while later integration recovers the base policy’s fine-grained refinement.Source: §4.3 · pp. 6–7

5. Experiments

Across simulation benchmarks, TeV improves performance over the strongest baseline; in real-world tasks it achieves stronger overall performance with steadier trajectories. Analyses and ablations associate these gains with temporally informed verification, while showing that guidance and larger candidate sets are not uniformly beneficial.

5.1. Simulation Environment

  • TeV gains 3.8% over the strongest baseline on LIBERO-Plus and 6.4% on RoboTwin 2.0 subtasks.Temporal Ensembling and BID have mixed effects, while TACO improves performance and TACO* underperforms.Source: §5.1 · p. 8, Table 2 · p. 8
  • On RoboTwin 2.0, guidance raises verification success from 72.7% to 75.1–77.3%, but guidance alone can degrade performance and LIBERO-Plus gains little.Verification provides the robust core improvement, while guidance is useful when initial candidates need refinement and final selection can mitigate unfavorable steering.Source: §5.1 · pp. 8–9, Figure 3 · p. 8
  • On RoboTwin 2.0, coarse-integration negatives raise success from 72.3% to 78.1%, temporal-mismatch negatives reach 77.4%, and combining both reaches 81.3%.The two negative families provide complementary training signals.Source: §5.1 · p. 9
  • Success does not increase monotonically with candidate count, since larger sets can give an imperfect verifier harder ranking decisions.The paper notes that larger sets may include more out-of-distribution or low-quality chunks.Source: Table 3 · p. 8, §5.1 · p. 9
  • The learned temporal token outperforms Zero, Random, and Frozen variants, with dynamic updates providing useful recent observation–action context.Frozen reuses the first episode token, while Zero and Random replace the learned token with zero or a Gaussian random vector.Source: Figure 4 · p. 8, §5.1 · p. 9

5.2. Real-world Environment

  • TeV achieves stronger overall performance than the baselines on real-world tasks, with more stable and consistent trajectories and fewer failures associated with jerky motion.Baselines sometimes oscillate, drop bottles, spill water, or terminate pouring prematurely.Source: §5.2 · p. 9, Table 4 · p. 9, Figure 6 · p. 10
  • The verifier selects one of the two nearest candidates to the expert action chunk in more than 70% of cases, supporting the contrastive objective.The analysis compares verifier ranking with latent-space distance to the corresponding expert chunk.Source: Figure 5 · p. 9, §5.2 · p. 10
  • TeV produces smoother kinematic profiles with lower jerkiness, which is associated with fewer failures such as water spilling.Jerkiness is the per-trial mean magnitude of the third time derivative of joint positions; lower values indicate fewer abrupt motion changes.Source: §5.2 · p. 10, Figure 6 · p. 10

6. Conclusion

Temporal Verification (TeV) makes action verification temporally aware by conditioning a lightweight energy-based verifier on recent observation–action history. Experiments in simulation and real-world manipulation show improved success rates and smoother robot dynamics.

6. Conclusion

  • TeV conditions a lightweight energy-based verifier on a compact temporal token summarizing recent observation–action history.This lets candidate actions be evaluated against recent history rather than a static timestep.Source: §6 · p. 10
  • A contrastive objective trains the verifier to compare candidates by relative preference rather than estimate calibrated rewards.The verifier ranks completed candidates and steers intermediate policy outputs during generation.Source: §6 · p. 10
  • TeV uses the learned verifier both to rank completed candidates and to steer intermediate policy outputs during generation.Source: §6 · p. 10
  • Experiments in simulation and real-world manipulation show that TeV improves success rates and yields smoother robot dynamics.The conclusion also characterizes TeV as sample-efficient.Source: §6 · p. 10

A. Implementation Details

Implementation uses small added modules while keeping the base model frozen, and a rolling history buffer supplies temporal context at deployment. Cache reuse and guided candidate pruning reduce inference computation.

A. Implementation Details

  • The temporal encoder has 3.0M parameters and the energy verifier 0.9M, together adding less than 0.15% relative to the base model.The base model stays frozen; only the added modules are optimized from scratch.Source: A. Implementation Details · p. 14
  • At deployment, a rolling buffer of recent observations and executed actions supplies the temporal token; missing initial history uses standard generation and later missing actions are zero-padded.Source: A. Implementation Details · p. 14
  • Inference reuses the observation’s key-value cache across candidates and applies verifier guidance during only the first half of Euler steps before retaining the lowest-energy half.The remaining integration steps run without guidance, reducing gradient computation by half with negligible accuracy degradation.Source: A. Implementation Details · p. 14

B. Experimental Details

The experimental details quantify inference-latency reductions, discrete control delays, and verifier training cost. Acceleration techniques are applied to baselines and TeV where applicable.

B. Experimental Details

  • Cache reuse shares visual-language context across candidates, while progressive pruning completes later integration with a smaller candidate set.Source: B. Experimental Details · p. 14
  • At 15 Hz (66.7 ms per action step), the base and Verification-Only configurations incur two-step control delays, while the guided configuration incurs three.Verification-Only increases wall-clock latency but remains in the base policy’s discrete delay bucket.Source: B. Experimental Details · p. 14
  • On an RTX 5080 with M = 4 candidates, latency falls approximately 20–30 ms with KV-cache reuse, 10 ms with compilation, and 30–40 ms with progressive pruning.The real-world latency analysis is identified in Table B.5; pruning is applied only where candidate scoring during generation is supported.Source: Table B.5 · p. 14, B. Experimental Details · p. 14
  • Verifier training for real-world tasks takes approximately 0.6 GPU hours, using RTX A6000 Ada GPUs.The passage says this is significantly smaller than methods relying on large-scale datasets and models.Source: B. Experimental Details · p. 14
Loading 2609.39038v1…