PaperlayerView exact revision on arXiv

Source-linked AI summary

AdaPolicy: Action-Driven Anticipatory Robot Policies via Next-Chunk Prediction

Fan Yang, Yuting Su, Chen Zhao, Fugui Fan, Peiguang Jing

arXiv:2610.34280v1cs.RO

TL;DR

Robot policies often learn anticipation from future-observation targets, while standard VLA state representations can lack incentives to advance beyond the current condition. AdaPolicy uses cascaded next-chunk prediction to shape future-aware representations from action signals, and reports effectiveness across diverse simulated and real-world settings.

  • Problem

    Future-oriented policy supervision commonly relies on future observations, and standard VLA intermediate states may lack constraints that encourage temporal progression.

    Source: §1 · p. 1

  • Method

    AdaPolicy predicts the current action chunk and an evolved latent state, then trains a cascaded head to predict the next chunk from that state.

    Source: §1 · p. 2

  • Results

    Experiments demonstrate effectiveness across MetaWorld, RoboTwin 2.0, LIBERO, and real-world tasks, while analyses find future object positions more decodable and improved support for future-goal planning.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Next-chunk prediction can shape future-aware policy representations without explicit future-observation targets, and its unroll adds zero inference-time overhead.

    Source: §5 · p. 14

Abstract

from arXiv · show

Endowing robot policies with anticipatory capabilities has proven effective in improving control performance. However, existing methods predominantly rely on explicit supervision derived from future observations, in either pixel or latent space, at the cost of additional architectural and computational complexity. Departing from explicit future supervision, we ask whether such foresight can be elicited directly from the action signal. To this end, we propose \textbf{AdaPolicy}, an \textbf{A}ction-\textbf{d}riven \textbf{a}nticipatory policy framework that shapes future-aware representations via cascaded next-chunk prediction. Specifically, an Online Action Head jointly produces the current action chunk and an evolved latent state. Building on this state, a cascaded Unrolled Action Head forecasts the next chunk, and the resulting loss flows back through the evolved state to instill implicit foresight into the online policy. Importantly, this training-only unroll requires no future-observation encoding, keeping AdaPolicy lightweight to train with zero inference-time overhead. Experiments on MetaWorld, RoboTwin 2.0, LIBERO, and real-world deployments demonstrate consistent gains across diverse settings, including up to $14$ percentage points on millimeter-precision tasks such as earphone assembly and shoelace threading. More details and visualizations are available at \href{https://teee111.github.io/AdaPolicy/}{https://teee111.github.io/AdaPolicy/}.

1 Introduction

Prior anticipatory policies rely on targets derived from future observations, while standard VLA state tokens may simply copy the current condition without future-directed constraints. AdaPolicy instead uses next-action-chunk prediction to shape future-aware representations, with reported effectiveness across varied benchmarks and supporting representation analyses.

1 Introduction

  • Existing future-supervision methods use future images, features, or structured targets, whereas AdaPolicy uses subsequent actions without explicit future-observation targets.The figure contrasts pixel reconstruction, latent alignment, and geometric targets with action-chunk supervision.Source: §1 · p. 1, Figure 1 · p. 2
  • Without future-directed constraints, standard VLA intermediate state tokens tend to become near-identity copies of the current input condition.Source: §1 · p. 1
  • AdaPolicy predicts the current action chunk and an evolved latent state, then conditions next-chunk prediction solely on that state.The EMA-updated unrolled head supplies training-time supervision, while the cascaded step is omitted at inference.Source: §1 · p. 2, Figure 1 · p. 2
  • Experiments across MetaWorld, RoboTwin 2.0, LIBERO, and real-world precision tasks demonstrate effectiveness across diverse settings.The representation analyses also find future object positions more decodable and better support for planning toward future goals.Source: §1 · p. 2
  • Next-chunk supervision makes future object positions more decodable and representations better support planning toward future goals.Source: §1 · p. 2

2 Related Work

Prior work teaches anticipation through future-observation targets in pixel, latent, or geometric forms, or through latent unrolling approaches rooted in model-based learning. AdaPolicy instead unrolls action chunks, using the subsequent action as control-aligned supervision without future-observation targets.

2.1 Explicit Future Supervision in Robot Policies

  • Pixel, latent, and geometric supervision all derive targets from future observations, with costs including appearance-focused computation, multi-stage training, or constructed annotations.AdaPolicy distinguishes itself by using the action stream rather than future-observation targets.Source: §2.1 · p. 3

2.2 Learning Anticipatory Representations via Latent Unrolling

  • Latent-unrolling methods shape internal representations without reconstructing observations, but later systems often add observation-derived guidance or reconstruction.Source: §2.2 · p. 3
  • AdaPolicy adapts decision-process supervision to reward-free chunk-based imitation by routing next-chunk error through the evolved state, not direct updates to the EMA unrolled head.Decoupled denoising timesteps and EMA parameterization prevent direct unrolled-head updates from absorbing the loss.Source: §2.2 · p. 3
  • Unlike Next Forcing’s autoregressive future-video-latent prediction, AdaPolicy unrolls action chunks and avoids future visual-stream encoding.The action chunk supplies control-aligned supervision without pixel reconstruction or feature alignment.Source: §2.2 · pp. 3–4
  • The architecture carries the evolved state from the online head into a training-time unrolled head, whose next-chunk loss backpropagates through that state.The cascaded step is omitted at inference.Source: Figure 2 · p. 4

3 Method

AdaPolicy shapes future-aware representations through cascaded next-chunk prediction, without explicit future-observation targets. Its Context Adapter forms conditioning tokens, while an online head produces the current action chunk and an evolved state used by a training-time unrolled head to predict the next chunk.

3.1 Problem Setup and Context Encoding

  • The current-chunk objective alone gives intermediate representations no explicit incentive to progress forward in time or encode temporal foresight.Source: §3.1 · p. 4
  • The Context Adapter compresses current backbone features into conditioning tokens and concatenates them with a projection of proprioceptive state.Learnable queries perform the compression; the design is compatible with pretrained visual and vision-language backbones.Source: §3.1 · p. 4

3.2 Cascaded Next-Chunk Prediction

  • The Online Action Head jointly produces the current-chunk velocity field and an evolved latent state, which conditions the Unrolled Action Head’s next-chunk prediction.The evolved state has no explicit target derived from the future observation; the next-chunk loss propagates through it to the online head and Context Adapter.Source: §3.2 · pp. 4–5

3.3 Training Objective and Inference

  • Both action heads use flow matching, with independently sampled times and a combined velocity-prediction objective that avoids future-observation reconstruction or feature alignment.The unrolled head is updated by exponential moving average rather than gradient-based updates, while the next-chunk loss gradient through the evolved state is backpropagated to the online pathway.Source: §3.3 · p. 5
  • At inference, AdaPolicy uses the pretrained encoder, Context Adapter, and Online Action Head; discarding the Unrolled Action Head adds no inference-time parameters or action-head evaluations.The current action chunk is obtained by integrating the online head’s predicted velocity field from τ0 = 0 to τ0 = 1.Source: §3.3 · p. 5

4 Experiments

The experiments evaluate AdaPolicy across simulation and real-world manipulation, then examine its design, learned representations, and optimization behavior. Across these analyses, next-chunk supervision improves task performance and shapes representations that encode future information.

4.1 Experimental Setup

  • The evaluation spans MetaWorld, RoboTwin 2.0, LIBERO, and six real-world tasks, using shared-protocol baselines and a future-feature-supervision variant for comparison.The setup includes 50-task joint training on MetaWorld and RoboTwin 2.0, four LIBERO suites, and randomized real-world initial configurations.Source: §4 · p. 5, §4.1 · pp. 5–7

4.2 Performance in Real World and Simulation

  • 86.3% average success across six real-world tasks, up from 77.0%, includes gains from 74% to 86% on Earphone Assembly and 36% to 50% on Shoelace Threading.Stacking Cups and Lid Opening remain at 100%.Source: Table 1 · p. 6, §4.2 · p. 7
  • 81.2% average success on MetaWorld and a 20.52-point gain over the chunk-only baseline on 50-task RoboTwin 2.0 accompany improvements across all four LIBERO suites.AdaPolicy reaches 95.2% average success on LIBERO versus 92.1% for the baseline, with a +7.2-point gain on Long.Source: Table 3 · p. 7, §4.2 · p. 7, Table 2 · p. 7

4.3 Analysis of Key Design Choices

  • 81.2 on MetaWorld at the same supervision span as the h=16 baseline exceeds direct longer-horizon prediction; EMA parameters, gradient flow through the evolved state, and access to that state in the second pass are important design choices.Without EMA, performance falls to 72.0; a trainable unrolled head reaches 69.4, while stop-gradient gives 67.7 versus the 67.8 baseline, and blocking condition-to-action attention gives 74.9.Source: §4.3 · p. 8, Table 5 · p. 9
  • One unroll outperforms two-step variants, independent noise-timestep sampling performs best on both benchmarks, and the default d=32 offset beats d=16 and d=48 while both alternatives still exceed baseline.Deeper supervision is noisier; the offset analysis attributes the default’s advantage to a trade-off between overlap at smaller offsets and uncertainty at larger ones.Source: §4.3 · pp. 9–10

4.4 Representation Analysis

  • Future-position probes favor AdaPolicy over both the baseline and explicit future-feature supervision, while evolved-state alignment shifts toward the next condition and latent-space planning improves reach rates.With next-chunk supervision, evolved-state similarity to the future condition is 0.9424 versus 0.6458 without it; the evolved state also aligns with the current condition at 0.9650.Source: §4.4 · pp. 11–13, Table 11 · p. 12, Table 12 · p. 12

4.5 How Does Next-Chunk Supervision Affect Optimization?

  • Next-chunk supervision lowers current-chunk training loss faster, its gradients remain positively correlated with current-chunk gradients at a moving average around 0.2, and action prediction error is lower across the chunk.The action-error advantage widens toward the middle and later timesteps, where baseline error accumulates more rapidly.Source: §4.5 · p. 13, Figure 6 · p. 13

5 Conclusion

AdaPolicy uses cascaded next-chunk prediction to shape future-aware representations from action signals, without explicit future-observation targets. Evaluations across simulation and millimeter-precision real-world tasks show gains, while the training-only unroll adds no inference-time overhead.

5 Conclusion

  • AdaPolicy converts subsequent-action errors into an implicit shaping signal for future-aware representations through cascaded next-chunk prediction.The framework requires neither future-observation reconstruction nor feature alignment.Source: §5 · p. 14
  • The latent unroll operates only during training, leaving inference-time overhead at zero compared with a standard chunk-based policy.Source: §5 · p. 14
  • Evaluations across simulation benchmarks and millimeter-precision real-world tasks demonstrate consistent and substantial gains.The paper concludes that the action stream offers a viable signal for shaping anticipatory representations through control-oriented learning.Source: §5 · p. 14

A Comparison with explicit future-supervision methods

AdaPolicy derives anticipatory supervision from the next action chunk rather than targets encoded from future observations. This avoids several explicit-supervision components, with modest training computation and no additional inference cost.

A Comparison with explicit future-supervision methods

  • AdaPolicy supervises the evolved state through next-chunk action loss, unlike pixel reconstruction or future-feature alignment that use future-observation targets.The comparison is shown in Figure 7.Source: A Comparison with explicit… · p. 19, Figure 7 · p. 19
  • Future-pixel supervision spends capacity on appearance details, while future-latent approaches require a separate encoder or anti-collapse measures and multi-stage recipes.Source: A Comparison with explicit… · p. 19
  • AdaPolicy adds 6% (hidden 512) and 11% (hidden 768) per-sample FLOPs over the chunk-32 baseline, with identical parameter count and no inference overhead.Explicit future-feature supervision requires an extra decoder, nearly doubles FLOPs (1163.8 vs. 648.2 at hidden 512), and increases parameters by 20%; Table 13 reports the training-cost comparison.Source: A Comparison with explicit… · p. 19, Table 13 · p. 19

B Real-World Experiment Details

Real-world experiments use a 6-DoF arm with chunked control, randomized task setups, asynchronous inference, and trajectory refitting. Earphone assembly and shoelace threading also include human recovery corrections, with representative successful recoveries shown.

B Real-World Experiment Details

  • The real-world platform is an AgileX PiPer 6-DoF arm with a parallel gripper, two 30 Hz cameras, 256 × 256 images, and 32-action chunks at 30 Hz.The policy uses DINOv3 visual features and absolute joint positions with continuous gripper values.Source: B Real-World Experiment Details · p. 19
  • Asynchronous inference computes a new 32-action chunk while actions execute, triggering at eight queued actions and refilling the queue with the latest 16 predictions.Figure 8 depicts inference overlapping execution.Source: B Real-World Experiment Details · p. 20, Figure 8 · p. 20
  • A cubic B-spline refits each new 16-action chunk using eight control points, adjusting the first three to match the previous trajectory and smooth chunk boundaries.The small least-squares refit adds negligible computation and no latency.Source: B Real-World Experiment Details · p. 20, Figure 8 · p. 20
  • In real-world evaluations, the wooden support block and charging-case placements are fully randomized; the case varies within approximately 5×5 cm, while the 6.5 mm earbud and 7.4 mm slot demand millimeter-level insertion precision.Random initial earbud orientation produces varying grasp poses and post-grasp configurations.Source: B Real-World Experiment Details · p. 20
  • Earphone assembly and shoelace threading add human corrections equal to 20% of each dataset, split evenly between baseline and AdaPolicy rollouts.Both policies train on the same combined data; Figures 11 and 12 show progressive realignment from misaligned attempts to successful insertion or threading.Source: B Real-World Experiment Details · pp. 20–21

C Probing Details

Linear probes compare future-position information in condition and evolved states across RoboTwin 2.0 and MetaWorld. AdaPolicy's advantage is clearest at longer prediction horizons, and its evolved state is more decodable than its condition state.

C Probing Details

  • The probes use frozen condition or evolved features, global average pooling, and a linear head to predict future object positions on one RoboTwin 2.0 task and 10 MetaWorld tasks.Stationary segments are removed; horizons are t+16, t+32, and t+48 on RoboTwin 2.0 and t+4, t+8, t+12, and t+16 on MetaWorld.Source: C Probing Details · p. 21
  • At t+48, RoboTwin 2.0 probe error is 9.44 px for AdaPolicy, 16.11 px for future-feature supervision, and 17.62 px for the baseline; at MetaWorld t+16, values are 3.24, 3.61, and 3.87 px, respectively.At short horizons, MetaWorld t+4 errors are nearly identical across variants.Source: C Probing Details · pp. 21–22
  • Across horizons on RoboTwin 2.0, AdaPolicy's evolved state is more decodable than its condition state (9.45 vs. 10.20 px), unlike the baseline's close values (15.76 vs. 16.38 px).The authors attribute the gap to next-chunk gradients organizing future information into a linearly accessible format.Source: C Probing Details · p. 22
  • Explicit future-feature targets still trail AdaPolicy on MetaWorld (2.55 vs. 2.48 px) and RoboTwin 2.0 (14.07 vs. 9.45 px).Source: C Probing Details · p. 22
  • Replacing the trainable unrolled head with its EMA-updated counterpart reduces average MetaWorld probe error from 2.61 px to 2.48 px.Source: C Probing Details · p. 22

D Evaluation Settings

The evaluation applies the same next-chunk objective across four settings that vary in embodiment, action parameterization, and camera setup. The supplied materials also describe future-object-displacement probes and a future-feature supervision baseline.

D Evaluation Settings

  • The displacement-probing tables cover RoboTwin 2.0 stack-blocks-two and 10 MetaWorld tasks, reporting test-set midpoint error in pixels.Both captions specify mean±std over three seeds; the MetaWorld protocol follows Table 14.Source: Table 14 · p. 23, Table 15 · p. 23
  • The four settings cover bimanual coordination, multi-task end-effector control, wrist-view manipulation, and single-arm precision tasks using the same objective and architecture.Only embodiment-dependent dimensions, such as action dimension, are adjusted.Source: D Evaluation Settings · pp. 22–23
  • The future-feature supervision baseline aligns decoder outputs with frozen DINOv3 intermediate-layer features from the observation at t+h.A four-layer Q-Former-style decoder uses evolved-state keys and values, with cosine loss weighted 0.5.Source: D Evaluation Settings · p. 23

E Gradient Stability Analysis of EMA Parameterization

The gradient analysis compares the EMA unrolled head with a detached copy of the online head. The reported results associate EMA with substantially lower variance in gradients from next-chunk prediction.

E Gradient Stability Analysis of EMA Parameterization

  • Without EMA, the unrolled head reuses online parameters that change at every optimization step, producing a rapidly moving target and high-variance gradients into the evolved latent state.The analysis tracks the mean, standard deviation, and variance of next-chunk gradients flowing into the online pathway.Source: E Gradient Stability Analysis of EMA… · p. 23
  • The authors interpret EMA parameterization as providing a stable, lower-variance shaping signal for anticipatory representations.The comparison is between full AdaPolicy and a variant that directly uses a detached online head for the second pass.Source: E Gradient Stability Analysis of EMA… · pp. 23–24
  • Removing EMA yields next-chunk gradient norm variance of 0.01300, while the EMA unrolled head reduces gradient variance by nearly half.Table 18 compares gradient statistics under identical training steps and data batches.Source: E Gradient Stability Analysis of EMA… · pp. 23–24, Table 18 · p. 24

F Additional Planning Probe on Condition States (ct)

A planning probe finds that AdaPolicy’s condition states perform better than the baseline feature space on average across seven MetaWorld tasks. The probe trains a separate dynamics predictor on frozen condition tokens and uses CEM to plan toward future goals.

F Additional Planning Probe on Condition States (ct)

  • The probe freezes the visual encoder and Context Adapter, then trains a separate task-specific dynamics predictor on condition tokens to plan toward a goal observation 20 environment steps ahead.The predictor is a four-layer cross-attention transformer trained with MSE; episodes 0–24 train it, 25–29 select a checkpoint, and 30–49 are held out.Source: F Additional Planning Probe on… · pp. 24–25
  • The authors conclude that next-chunk-error backpropagation into the Context Adapter helps regularize upstream conditioning tokens for downstream dynamics fitting.The probe indicates this benefit is present before the tokens enter the action head.Source: F Additional Planning Probe on… · pp. 24–25
  • 23.1% versus 17.4% reach@5 cm on average across seven MetaWorld tasks: planning over AdaPolicy condition states improves on the baseline feature space.Gains are clear on stick-push and pick-out-of-hole, while results are comparable for subtle object motions and slightly lower on basketball.Source: F Additional Planning Probe on… · p. 24, Table 19 · p. 25
  • CEM samples 300 action sequences over a four-block horizon, executes the first five-step block, and replans; success requires reaching within 5 cm of the goal.Candidates are scored by squared L2 distance to goal tokens, and the top 30 elites update the Gaussian sampling distribution over 30 iterations.Source: F Additional Planning Probe on… · p. 25

Related papers

Loading 2610.34280v1…