PaperlayerView exact revision on arXiv

Source-linked AI summary

Copper-Policy: Focus on the Representation for Robust Robot Manipulation

Zexin Feng, Yixu Feng, Lingyu Xiao, Shang Su, Kexin Zheng, Chang Xu, Mengkai Shi, Shuo Feng, Xintao Yan

arXiv:2609.32779v1cs.ROcs.LG

TL;DR

World Action Models model future scene evolution, but detailed future prediction is costly, and prior evidence raises the question of what dynamics learning must contribute to control. Copper-Policy jointly learns a compact predictive representation with its policy while preserving current-frame detail for action execution. It reports strong control performance alongside reduced training-token counts and improvements over Fast-WAM on LIBERO-Plus and real-robot scores.

  • Problem

    Generative WAMs incur training and inference costs by coupling action with explicit video or latent-video generation, while evidence that co-training can help without test-time generation raises what dynamics learning must contribute to control.

    Source: §1 · p. 2

  • Method

    Copper-Policy jointly learns a compact, task-conditioned World representation through future-embedding prediction and action decoding, while giving the Action Expert access to current-frame spatial detail.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    80.85% LIBERO-Plus total exceeds several embodied-pretrained VLA baselines; Copper-Policy also improves on Fast-WAM in LIBERO-Plus and real-robot scores while training faster with fewer parameters.

    Source: §4.2.1 · p. 6, §5 · p. 9

  • Takeaways & Limitations

    Representation diagnostics and results across simulation and real-robot tasks support the value of the World representation under common closed-loop shifts.

    Source: §5 · p. 9

    Instruction grounding remains limited, and Background and Layout evaluations reveal further scene-generalization gaps.

    Source: §4.4 · p. 6, §5 · p. 9

Abstract

from arXiv · show

World Action Models (WAMs) acquire behavioral priors by modeling future scene evolution, but predicting detailed futures in pixel or latent space incurs substantial cost. Recent evidence that co-training gains persist without test-time generation raises a question: what must a WAM learn to improve control? We introduce Copper-Policy, which learns a compact World representation with the policy rather than relying on a predefined target space. Through temporal joint-embedding prediction, it predicts future observation embeddings conditioned on task intention without reconstructing pixels. This prediction and action decoding shape the representation jointly, while the policy retains access to current-frame spatial detail for execution. Representation analyses show that the learned features better separate task-driven change from perturbations and provide complementary information for control. Compact prediction targets reduce training tokens per sample, enabling a 2B-parameter model trained in 9.67 hours on 8$\times$ RTX 5090 GPUs and 6$\times$ faster than Fast-WAM on matched A100 GPUs. Copper-Policy outperforms every compared method without embodied pretraining on RoboTwin and several embodied-pretrained VLAs on LIBERO-Plus (80.85%). On three challenging real-robot tasks, it performs comparably to $\pi_{0.5}$ and attains a higher average score. Together, these results show that Copper-Policy combines strong control performance with efficient training.

1. Introduction

Generative WAMs model future scene evolution but incur training and inference costs, prompting the question of what dynamics learning must contribute to control. Copper-Policy makes predictive representation learning part of the policy, jointly shaping a compact task-relevant representation while preserving current-frame detail for action execution.

1. Introduction

  • Generative WAMs couple action with explicit video or latent-video generation, adding substantial training burden and inference overhead.Prior evidence that co-training can help without test-time generation motivates asking what dynamics learning contributes to control.Source: §1 · p. 2
  • Copper-Policy predicts compact, task-relevant scene evolution in representation space instead of reconstructing detailed futures.The representation is intended to capture what is changing and where the task is heading, while execution-specific spatial detail remains available to the Action Expert.Source: §1 · p. 2
  • Its predictive target space is learned jointly with the policy, so predictive learning and action supervision organize the representation used for control.The Future Expert uses the compact predictive representation, while the Action Expert additionally accesses current-frame features.Source: §1 · p. 2

2. Related Work

Prior robot policies use future-derived signals in several ways, but their predictive target spaces are often fixed or staged rather than jointly shaped by the policy. Copper-Policy instead learns a compact predictive space that action decoding also uses.

2. Related Work

  • Copper-Policy’s predictive target space is the same compact World representation read by the Action Expert for action decoding.The comparison focuses on representations the policy reads for action decoding.Source: §2 · p. 2
  • Video-generation latents and direct image or video targets can preserve appearance information beyond what action prediction requires.Examples include Fast-WAM [46], LingBot-VA [25], Cosmos-Policy [22], and earlier video-prediction policies [9] [12] [23].Source: §2 · p. 2
  • Other methods learn compact, control-oriented future states, but use pretrained targets or staged training rather than jointly learning the target space with the policy.FLARE [53], WoG [39], and Being-H0.7 [33] use privileged future information without test-time rollout; Copper-Policy jointly learns the predictive target space with action decoding.Source: §2 · pp. 2–3
  • JEPA methods predict in representation space, while Copper-Policy jointly shapes that space through action decoding and predicts within it by flow matching.I-JEPA [1] and V-JEPA [2] learn visual representations without an action objective, while VLA-JEPA [40] pretrains before fine-tuning an action head.Source: §2 · p. 3
  • Copper-Policy builds on Fast-WAM’s two-stream design but learns the predictive target space jointly with action decoding.This makes prediction shape the same compact representation used for control.Source: §2 · p. 3

3. Method

Copper-Policy predicts future task-conditioned embeddings in a compact World representation and decodes actions through a separate pathway with direct access to current-frame spatial detail. The representation is jointly shaped by prediction and action objectives, with no future tokens instantiated at test time.

3.1. Overview

  • Copper-Policy uses temporal joint-embedding prediction to learn a compact task-conditioned space alongside action decoding, without predicting futures at test time.The overview describes an online–EMA compactor pair and two expert streams coupled through a structured attention mask.Source: §3.1 · p. 3

3.2. Shared Representation with Online and EMA Compactors

  • The online compactor maps observation features into compact tokens conditioned on task intention, forming the World representation used by both expert streams.The World representation is a compact, task-conditioned behavioral prior in a shared temporal embedding space.Source: §3.2 · p. 4
  • An EMA copy of the compactor supplies stop-gradient future targets and tracks the online compactor, allowing the target space to co-evolve with the online representation.The future horizon matches the action horizon.Source: §3.2 · p. 4

3.3. Temporal Joint-Embedding Prediction

  • The Future Expert predicts compact future-view targets from the current World representation, conditioned on task intention, using conditional flow matching.It predicts a velocity field matching the derivative of the interpolation from clean target to noise.Source: §3.3 · p. 4

3.4. Action Expert and Current-Frame Vision

  • The Action Expert decodes action chunks using the World representation, current-frame spatial tokens, robot state, and task intention.A spatial encoder and grid sampler produce current-frame tokens, while action decoding uses the same flow-matching formulation.Source: §3.4 · p. 4

3.5. Asymmetric Interface

  • The asymmetric interface gives future prediction only compact World tokens, while action decoding also reads current-frame detail and robot state.Current-frame detail reaches future queries only through the compact tokens, and the action loss also shapes the World representation toward information useful for control.Source: §3.5 · pp. 4–5

3.6. Compact Token Interface

  • 244 tokens per sample versus 392 for Fast-WAM, with the Future Expert attending over 180 versus 360 tokens, reduces the sequence length used during training.The passage notes that self-attention is quadratic in sequence length.Source: §3.6 · p. 5

3.7. Joint Objective and Inference

  • The action and prediction objectives are optimized jointly, while the target compactor receives no gradients; at inference, the Future Expert runs once as context and no future tokens are instantiated.The Action Expert decodes over 10 flow steps from the current observation, robot state, and instruction.Source: §3.7 · p. 5

4. Experiments

Copper-Policy is evaluated on LIBERO, LIBERO-Plus, RoboTwin, and three real-robot tasks under benchmark-specific rollout protocols. Results cover control performance, training cost, robustness, and representation analyses of temporal structure, perturbation response, and action information.

4.2.1. LIBERO and LIBERO-Plus

  • 80.85% on LIBERO-Plus exceeds several embodied-pretrained VLA baselines, while Copper-Policy averages 97.25% on standard LIBERO.Cosmos-Policy (82.2%) and π0.5 (85.7%) score higher on LIBERO-Plus.Source: §4.2.1 · p. 6

4.2.2. RoboTwin

  • 41.91% is the highest average RoboTwin success rate among methods without embodied pretraining, ahead of EventVLA at 40.65% and Fast-WAM at 39.85%.Copper-Policy scores 70.84% versus π0.5’s 70.70% on clean scenes, but 12.98% versus 46.00% under randomization.Source: §4.2.2 · p. 6

4.2.3. Real-Robot Evaluation

  • 96.3 is Copper-Policy’s real-robot average, versus 94.3 for π0.5 and 84.0 for Fast-WAM; it is best on both rigid-object tasks.Removing the World input and Future Expert lowers the average to 84.7; Figure 3 reports scores of 1, 0.5, and 0 for full, half, and no success.Source: §4.2.3 · p. 6, Figure 3 · p. 7

4.3. Efficiency

  • 2B trainable parameters and 9.67 hours on eight RTX 5090 GPUs accompany training about 6× faster than Fast-WAM on matched A100 GPUs.The comparison uses global batch 128 on eight A100 GPUs; end-to-end latency is about 85 ms, versus 75 ms for π0.5 and 80 ms for Fast-WAM, partly reflecting the frozen 300M V-JEPA encoder.Source: §4.3 · p. 6

4.4. Controlled Comparisons

  • Without joint-embedding prediction, performance stays close to the full model on LIBERO-Plus, but RoboTwin falls from 41.91 to 28.77 and the real-robot average from 96.3 to 43.3.Removing current-frame features lowers LIBERO, LIBERO-Plus, and RoboTwin aggregate scores by at most 6.74 points, yet drops the real-robot average to 20.7. Language is the clearest generalization gap at 67.34%.Source: §4.4 · pp. 6–7

4.5.1. Temporal Structure and Channel Complementarity

  • Removing joint-embedding prediction lowers 16-frame cosine similarity from 0.794 to 0.684 on LIBERO, 0.902 to 0.756 on the real robot, and 0.913 to 0.729 on RoboTwin.The attention maps more often retain maxima on background surfaces without the objective; these temporal-smoothness measurements are correlates, not proof of mechanism.Source: §4.5.1 · pp. 7–8, Figure 4 · p. 8, Table 6 · p. 8

4.5.2. Perturbation Geometry and Channel Probes

  • At rank 8, perturbation–temporal subspace overlap is 0.187 with joint-embedding prediction, versus 0.798 without it and 0.021 at random.Perturbation identity remains linearly decodable at 0.998 balanced accuracy, while paired clean/perturbed views change by 0.134 versus 0.344 mean normalized L2 distance.Source: §4.5.2 · p. 8, Figure 5 · p. 9, Table 7 · p. 9
  • The action probe reaches R2 of 0.927 using both World and current-frame features, compared with 0.802 from World features and 0.910 from current-frame features alone.On real-robot tasks, removing current-frame features drops the average from 96.3 to 20.7, showing that the temporally shaped World channel does not replace that pathway for precise control.Source: §4.5.2 · p. 9

5. Conclusion

Copper-Policy combines a jointly learned compact World representation with current-frame detail for action, and the conclusion reports strong simulation and real-robot performance. Its remaining gaps are instruction grounding and scene generalization across Background and Layout.

5. Conclusion

  • Copper-Policy learns a compact World representation jointly with its policy through joint-embedding prediction and action decoding.The asymmetric interface preserves current-frame detail for precise control.Source: §5 · p. 9
  • Diagnostics show reduced overlap between perturbation-induced and task-driven variation in zt and sustained attention to manipulated objects.The conclusion also reports value under Camera, Robot, Light, and Noise shifts.Source: §5 · p. 9
  • Instruction grounding remains limited, while Background and Layout reveal further scene-generalization gaps.Future work will learn representations that better capture task intent and scene structure across these settings.Source: §5 · p. 9

A.1. Transformer Sequence Composition

The sequence composition table specifies current, future-target, and action-token counts for RoboTwin, real-robot, and LIBERO runs, and distinguishes which tokens each expert sees.

A.1. Transformer Sequence Composition

  • RoboTwin and real-robot runs use 32 current-frame tokens and 36 World tokens, with 144 future targets and 32 action tokens per sample.The Future Expert sees only the 36 World tokens, because its mask excludes current-frame tokens.Source: Table A1 · p. 13
  • LIBERO’s multi-view layout yields 28 World tokens, while each method uses 32 action tokens.The table gives 4 future target nodes for RoboTwin and real-robot runs.Source: Table A1 · p. 13
  • For RoboTwin and real-robot runs, 68 current tokens comprise the 32 current-frame tokens and 36 World tokens.The Future Expert is masked from the current-frame tokens.Source: Table A1 · p. 13

B. Representation Analysis

The representation analysis uses frozen checkpoints and episode-level protocols to measure temporal stability, perturbation geometry, and information in the World and current-frame pathways.

B. Representation Analysis

  • Diagnostics are read-only evaluations of frozen checkpoints, with synchronized observations and language conditioning and episodes treated as the statistical unit.World tokens and channel dimensions are flattened; current-frame tokens are mean-pooled, and episode-level averages weight episodes equally.Source: §B.1 · p. 13
  • Temporal stability is measured by cosine similarity and Euclidean displacement across horizons of 1, 2, 4, 8, and 16 frames.The analysis uses 40 LIBERO, 20 RoboTwin, and 18 real-robot episodes; RoboTwin’s every-fourth-frame sampling means its absolute values are not directly comparable to the other domains.Source: §B.1 · p. 13
  • Perturbation geometry and linear nuisance and action probes are evaluated on held-out episodes, with perturbation identity and action chunk as prediction targets.The RoboTwin analysis uses camera translation, illumination change, color cast, Gaussian noise, and blur; it also reports paired clean/perturbed L2 distance and cosine similarity.Source: §B.1 · p. 13
  • Qualitative World-representation visualizations compare the full model with the version without Joint-Embedding Prediction using identical inputs and rendering schedules.Six evenly spaced steps are rendered per selected episode in the native multi-view layout.Source: §B.1 · p. 13

B.2. Attention-Map Generation

The attention-map procedures derive World and current-frame maps from model attention or source-patch responses, then reshape and overlay them on input views.

B.2. Attention-Map Generation

  • World-representation attention is captured from the final task-conditioned compactor read block and averaged over heads and compact queries.The patch sequence is split according to the original camera grids.Source: §B.2 · p. 14
  • The current-frame map measures which source patches are most similar to information carried by sampled current-frame tokens.Per-view maps are resized and overlaid on normalized RGB frames, using VIRIDIS for current-frame maps and MAGMA for World maps.Source: §B.2 · p. 14
  • The real-robot platform supplies one top and two wrist camera views for the multi-view observations.The platform uses an AgileX Cobot with two Piper arms and standard grippers.Source: Figure A1 · p. 14

C.1. Experimental Platform

The experimental setup uses a dual-arm AgileX platform with RGB observations and joint-position actions. Real-robot execution uses asynchronous real-time chunking to replan around committed actions while maintaining continuous execution.

C.1. Experimental Platform

  • The platform uses two 6-degree-of-freedom Piper arms, multi-view RGB streams, and 14-D absolute joint-position actions.Teleoperated demonstrations are stored in LeRobot format.Source: §C.1 · pp. 14–15

C.2. Real-Time Chunking

  • Evaluation scenes are sampled with jitter from fitted training-set object-position and orientation distributions, using shared fixed seeds.Each policy faces the same 50 initial scenes per task.Source: §C.2 · p. 15
  • Asynchronous RTC preserves a committed action prefix while the Action Expert regenerates the remaining chunk from the latest observation, replanning at 30 Hz.Training simulates RTC delay with committed prefixes sampled from 0 to 6 steps.Source: §C.2 · p. 15

C.3. Protocol and Outcome Counts

Policies share the real-robot evaluation setup, with manually annotated full-success, half-success, and failure outcomes. Additional observations describe differing rollout trajectories and a grasp-related orientation check outside the reported trials.

C.3. Protocol and Outcome Counts

  • All policies use the same platform, demonstrations, cameras, action space, and 50 shared-seed trials per task.Outcomes are manually annotated; the average covers 150 trials per policy variant.Source: §C.3 · p. 15
  • Table A2 reports per-task full-success, half-success, and failure counts underlying the aggregate real-robot scores.Full, half, and failed trials score 1, 0.5, and 0, respectively; partial credit applies only to towel folding.Source: Table A2 · p. 15, §C.3 · p. 15
  • Copper-Policy sometimes reaches goals along trajectories visibly different from teleoperated demonstrations, while Fast-WAM often skips banana grasp attempts.An unscored check found that rotating bananas toward demonstrated orientations recovered some grasp attempts.Source: §C.3 · p. 15

Related papers

Loading 2609.32779v1…