PaperlayerView exact revision on arXiv

Source-linked AI summary

PoLAR: Factorizing Extent and Mode in Latent Actions for Robot Policy Learning

Youngjoon Jeong, Jihwan Yu, Minsoo Jo, Junha Chun, Taesup Kim

arXiv:2606.21139v1cs.ROcs.AIcs.LG

TL;DR

Existing latent-action methods entangle transition extent with transition mode, limiting the structure available for robot policy learning. PoLAR imposes hyperbolic radial geometry so radius represents extent and direction represents mode, and it consistently improves downstream policies across simulation and real-world experiments.

  • Problem

    Existing latent-action methods entangle transition extent and mode in a single representation, leaving related transitions at different horizons insufficiently structured for policy learning.

  • Method

    PoLAR uses temporal ordering to impose hyperbolic radial structure, assigning transition extent to radius and transition mode to direction.

  • Results

    Across simulation benchmarks and real-world robot experiments, PoLAR consistently improves downstream policy performance over latent-action baselines and strong pretrained VLAs.

  • Takeaways & Limitations

    The results suggest that latent-action geometry is a useful design choice for transferring visual pretraining to robot policy learning.

  • Takeaways & Limitations

    PoLAR’s temporal-ordering assumption may break down under cyclic behavior, pauses, recovery motions, or repeated back-and-forth movements.

Abstract

from arXiv · show

Latent action pretraining learns representations of visual change from pairs of observations, but existing methods typically encode each transition as a single unstructured representation that entangles transition extent and transition mode. We introduce Polar Latent Actions with Radial structure (PoLAR), which imposes a radial-direction structure on latent actions, encouraging radius to encode transition extent and direction to retain transition mode. PoLAR uses temporal offset between two observations as a weak proxy for transition extent, encouraging latent action from observation pairs separated by larger temporal gaps to occupy larger radii. We instantiate this structure in hyperbolic space, whose expanding volume with radius offers a natural fit for more diverse transition modes at larger extents. Across in-task and large-scale pretraining settings, PoLAR improves downstream policy performance in simulation and real-world robot experiments, outperforming latent action baselines and strong pretrained VLAs. These results suggest that the geometry of the latent action space is an important design choice for transferring visual pretraining to downstream robot policy learning.

1 Introduction

Prior latent action methods entangle transition extent and mode in a single representation, obscuring relationships between similar transitions across horizons. PoLAR addresses this with polar latent geometry, using radius for extent and direction for mode, with temporal offset as a weak extent signal.

  • Motivation: Latent actions compress visual change between two observations, encouraging representations to capture transitions rather than static appearance.An inverse dynamics model infers the representation from an observation pair in the same trajectory.
  • Problem: Prior methods encode each transition as one continuous vector or discrete token sequence, entangling transition extent and transition mode.This leaves short and long versions of similar transitions without an explicit relationship in latent space.
  • PoLAR: PoLAR imposes polar geometry, using radius to represent transition extent and direction to distinguish transition mode.Temporal offset between observation pairs serves as an ordinal proxy for extent, encouraging larger gaps to occupy larger radii.

2 Related Work

Prior latent-action methods represent observation transitions with continuous or discrete bottlenecked codes that often entangle transition extent and mode. PoLAR instead uses temporal structure and radial representation geometry to encode extent by radius and transition mode by direction, with hyperbolic space providing greater angular capacity at larger radii.

  • Latent actions: Latent-action models encode observation transitions as continuous latents or vector-quantized tokens for world modeling and action-free or cross-embodiment policy learning.These representations are used in world models and policies learned from action-free videos or cross-embodiment data.
  • Latent actions: Most prior methods use a single transition code that can entangle transition extent and transition mode.PoLAR addresses this entanglement by imposing structure on the latent action.
  • Temporal structure as weak supervision: Temporal order provides weak supervision without low-level action labels or simulator states and has supported alignment, phase, reward, progress, and offline policy representations.PoLAR instead uses temporal order to structure latent actions.
  • Representation geometry: Representation geometry often assigns discriminative semantics to direction and non-semantic quantities to feature norm.The related work motivates separating transition mode and extent through direction and radius.
  • Representation geometry: Hyperbolic volume grows exponentially with radius, allowing larger-radius shells to support more angular distinctions than Euclidean space.This radial capacity motivates hyperbolic geometry for latent actions involving increasingly diverse transitions.
  • Representation geometry: PoLAR encourages transition extent to map to radius while direction distinguishes transition modes, because longer-horizon transitions can involve more diverse motions, contacts, and task-state changes.The design leaves direction to distinguish modes with greater capacity at larger radii.

3 Methods

PoLAR extends the standard latent-action pipeline with radial structure, using temporal offsets and a self-transition anchor to organize latent actions by transition extent while preserving direction for transition mode. It implements this structure with hyperbolic radial losses and factorized radial-direction quantization, then relabels demonstrations for downstream latent-policy control.

  • Latent action pipeline: The standard pipeline encodes observation pairs with an encoder and inverse dynamics model, then reconstructs future features with a forward dynamics model.Continuous latents are optionally quantized in the discrete setting, which also uses codebook and commitment losses.
  • Radial structure: PoLAR encourages radius to encode transition extent and direction to encode transition mode, using temporally ordered observation pairs as weak radial supervision.For each start observation, offsets satisfy 0 < j < k ≤ Kmax, with dataset-specific Kmax.
  • Radial losses: A self-transition latent provides a local no-change anchor, while radial losses enforce that farther temporal transitions have larger anchor-centered distances and radii.The ordering loss uses softplus penalties, with λord = 1, λrad = 0.3, and α = 0.05 in all experiments.
  • Hyperbolic radial geometry: PoLAR lifts Euclidean IDM outputs into a Poincaré ball for radial losses, while quantization and forward decoding retain tangent-coordinate latents.The curvature is c = 1 in all experiments; the Euclidean ablation substitutes Euclidean norm and ℓ2 distance.
  • Factorized radial and direction tokens: For discrete latents, a factorized codebook selects one shared radial index from aggregate norm and one direction index per latent slot from normalized slot directions.The resulting radial-direction vectors replace flat VQ embeddings in the base latent-action objective.
  • Downstream control: After pretraining, the IDM relabels demonstrations at a fixed policy horizon, and downstream control uses a latent policy with a low-level action decoder.The fixed horizon h is separate from the randomly sampled pretraining offsets (j, k).

4 Experimental Results

PoLAR is evaluated across latent-action settings, policy types, pretraining scales, and simulated and real-world control tasks. It consistently outperforms flat, Euclidean, generic visual-encoder, latent-action, and pretrained-VLA alternatives.

  • Experimental scope: The experiments span in-task and large-scale pretraining, continuous and discrete latent actions, diffusion policies and VLAs, and simulated and real-world control.In-task experiments use five RoboMimic and MimicGen tasks, while VLA experiments use a DINOv2-based tokenizer, Prismatic-7B, and BridgeData V2.
  • Radial supervision: Temporal offset provides radial supervision: PoLAR radii increase with temporal offset, whereas flat baselines remain nearly constant.The figure describes temporal offset as an effective proxy for object and robot state change.
  • RoboMimic & MimicGen: PoLAR outperforms Flat and Euclidean latent-action variants across RoboMimic and MimicGen diffusion-policy tasks under decoder-only and joint fine-tuning.It also outperforms a pretrained ResNet18 encoder jointly fine-tuned with the policy.
  • SimplerEnv: PoLAR achieves the highest average success in SimplerEnv-WidowX among PoLAR ablations, latent-action baselines, and pretrained VLA references.In the matched pipeline, PoLAR uses only BridgeData V2 for latent-action and VLA pretraining yet outperforms baselines such as π0.5.
  • Real robot: PoLAR achieves the highest final success rate on all three real-world manipulation tasks and the best overall average, outperforming π0.5, UniVLA, SmolVLA, and Villa-X.The subtask breakdown indicates gains beyond early-stage grasping or reaching.

5 Analysis

The analysis validates PoLAR’s radial organization: temporal offset tracks transition extent, radius increases with offset, and sweeping radius enlarges decoded transitions while preserving mode. Ablations and policy-focused tests show complementary radial losses, improved robustness to token errors, and benefits from multi-horizon training.

  • Radial organization: Temporal offset has the strongest Spearman correlation with object and robot state distances across RoboMimic and BridgeData V2, outperforming DINOv2, ResNet18, and pixel distances.This supports temporal ordering as a weak proxy for transition extent.
  • Radial organization: PoLAR produces gradual radial progression as temporal offset increases, whereas flat latent-action baselines remain nearly constant.The result indicates that radial supervision is needed for transition extent to be organized along radius.
  • Radial organization: With direction fixed, increasing radius makes decoded visual transitions larger while preserving their transition mode.The intervention decodes swept latents through the pretrained FDM and separately trained VQ-VAE pixel decoder.
  • Ablations: Removing either Lord or Lrad weakens downstream policy performance, while λrad = 0.3 and α = 0.05 perform best among tested radial-margin settings.The ablation is conducted on RoboMimic’s Can task, showing that the two radial losses are complementary.
  • Policy-relevant benefits: Hyperbolic PoLAR has lower wrong-token latent error than Flat and UniVLA (0.311 vs. 0.447 and 0.689), with the same trend after decoding to actions (0.143 vs. 0.238 and 0.191).These results measure normalized latent error and decoded action error under mispredicted latent-action tokens.
  • Policy-relevant benefits: PoLAR improves multi-horizon diffusion-policy performance on Coffee and Stack Three (+36.0 and +60.0 points) and Simpl­erEnv VLA performance (+4.2 points), while Flat shows no gain.On SimplerEnv, PoLAR also has higher cross-horizon gradient cosine similarity than Flat (0.486 vs. 0.305).

6 Limitations

PoLAR is evaluated with a fixed third-person, top-view camera and uses temporal ordering as weak supervision for transition extent. Extending it to multi-view settings and assessing the assumption beyond goal-directed demonstrations are identified as future directions.

  • Camera settings: PoLAR experiments use a fixed third-person, top-view camera, leaving multi-view settings as an open direction.The proposed extension includes both third-person and wrist cameras.
  • Camera settings: Extending PoLAR to multi-view settings, including third-person and wrist cameras, is identified as a natural direction for future work.This direction is consistent with recent multi-view VLA architectures.
  • Temporal supervision: PoLAR uses temporal ordering as weak supervision for transition extent, an assumption suited to goal-directed demonstrations.In such demonstrations, larger temporal offsets often correspond to larger transition extents.

7 Conclusion … A.3 Downstream Policy Fine-tuning

PoLAR factorizes latent actions into hyperbolic radius and direction, improving downstream policy performance across simulation and real-world experiments. The appendix details its pretraining, latent-policy interfaces, and downstream decoder fine-tuning setups.

  • 7 Conclusion: PoLAR uses temporal ordering to impose hyperbolic radial structure, with radius representing transition extent and direction retaining transition mode.The paper links its gains to better organized and more action-informative latent actions.
  • A.1 Latent Action Pretraining: Continuous PoLAR pretraining jointly trains a latent action model on RoboMimic and MimicGen demonstrations using a residual CNN visual feature space.BridgeData V2 instead uses a frozen DINOv2 ViT-B/14 and trains the tokenizer in DINO patch-feature space.
  • A.1 Latent Action Pretraining: Discrete PoLAR represents each latent action with one shared radial token and four direction tokens selected from continuous latent codes.The radial code quantizes the average norm, while direction codes match normalized directions.
  • A.1 Latent Action Pretraining: The tokenizer combines DINO feature reconstruction, VQ, and PoLAR radial losses, while ablations compare radial supervision and matched-capacity unfactorized codebooks.Fact. removes radial supervision, whereas Flat replaces the radial-direction quantizer with unfactorized VQ codebooks.
  • A.2 Latent Policies: After latent action pretraining, the frozen IDM relabels action demonstrations, and latent policies learn to predict those actions from execution-time observations.RoboMimic and MimicGen use continuous latent policies with fixed horizon h = 20; BridgeData V2 uses a discrete latent VLA.
  • A.3 Downstream Policy Fine-tuning: For RoboMimic and MimicGen, a conditional 1D diffusion policy predicts 20-step 7-DoF action chunks from latent actions and proprioception.Decoder-only fine-tuning freezes the latent policy, while joint fine-tuning updates both latent and diffusion policies.
  • A.3 Downstream Policy Fine-tuning: For BridgeData V2, a lightweight action decoder maps pooled VLA visual and latent-action states to normalized 7-DoF action chunks during downstream fine-tuning.The objective combines action L1 loss with autoregressive latent-token cross-entropy while keeping downstream data and optimization budgets consistent across VLA methods.

A.4 Compute Cost … B.3 Action Informativeness

The appendix reports compute requirements and specifies audits and diagnostics for temporal extent, radius-controlled predictions, and action informativeness. These procedures include state-distance comparisons, qualitative feature decoding, and held-out mutual-information and probing evaluations.

  • A.4 Compute Cost: Continuous Square training required approximately 1.7 GPU-hours for PoLAR pretraining, 2.7 GPU-hours for latent policy training, and 3.0 GPU-hours for diffusion fine-tuning.The representative run used a single NVIDIA GeForce RTX 3090.
  • B.1 Temporal Offset and State Distance: The temporal-offset audit compares offsets with z-scored low-dimensional state L2 distances across sampled RoboMimic and BridgeData V2 demonstrations.RoboMimic uses Can and Square with 50 demonstrations and offsets {1, 2, 4, 8, 12, 16, 20}; BridgeData V2 uses 49 episodes and offsets {1, 2, 4, 8}.
  • B.1 Temporal Offset and State Distance: For image-based comparisons, endpoint distances use agent-view RGB images for RoboMimic and top-view RGB observations for BridgeData V2.BridgeData V2 uses a 7D robot/end-effector proprioceptive state as its low-dimensional target.
  • B.2 Radius Sweep Visualization Protocol: Radius-sweep visualization fixes four direction tokens, varies only the shared radial token, and decodes each modified action into a predicted future DINOv2 patch-feature map.The protocol uses a pretrained BridgeData V2 PoLAR tokenizer and FDM together with current DINOv2 patch features.
  • B.2 Radius Sweep Visualization Protocol: A separately trained pixel decoder visualizes predicted DINOv2 features in pixel space and is used only for qualitative visualization, not latent-action pretraining.It is trained on BridgeData V2 frames for 50k steps with AdamW and a weighted pixel L1, pixel MSE, and DINO feature-cycle loss.
  • B.3 Action Informativeness: Action informativeness is evaluated on 4,096 BridgeData V2 samples at horizon h = 9 using flattened 70-D full action-sequence targets.Each sample pairs a pretrained-tokenizer latent action with its corresponding ground-truth 10-step action chunk.
  • B.3 Action Informativeness: Held-out mutual-information diagnostics estimate Barber–Agakov and InfoNCE bounds using 2,560 training, 512 validation, and 1,024 test samples.BA is converted from nats to bits for reporting, whereas the held-out InfoNCE bound remains in natural-log units.
  • B.3 Action Informativeness: Probe diagnostics train supervised predictors from latent actions to the same action targets and report test R2, using attentive pooling for tokenized latent actions.The attentive probe enables evaluation across representations with different token counts and uses a shared two-layer MLP with hidden dimension 128.

B.4 Wrong-Token Prediction Error · B.5 Multi-Horizon Latent Policy · B.6 Hyperbolic versus Euclidean Diagnostics

The appendix defines diagnostics for token-prediction and decoded-action errors, specifies multi-horizon latent-policy targets and training procedures, and compares hyperbolic with Euclidean PoLAR using matched setups and two geometry-focused diagnostics.

  • B.4 Wrong-Token Prediction Error: B.4 evaluates wrong-token prediction by comparing each VLA top-1 token with the corresponding target token across valid action-token slots.The tokenizer supplies target latent-action tokens, while the fine-tuned VLA supplies predicted distributions and top-1 predictions.
  • B.4 Wrong-Token Prediction Error: B.4 measures normalized wrong-token latent error as the average L2 distance between predicted and target latent code vectors over incorrect predictions.Each distance is normalized by the average norm of the two code vectors, and only incorrect top-1 token slots are included.
  • B.4 Wrong-Token Prediction Error: B.4 measures decoded action error by comparing action-decoder outputs from target tokens with outputs from VLA top-1 replacement tokens.The comparison uses the actual fine-tuned VLA and decoder and reports mean per-step L2 distance over normalized 10-step action chunks.
  • B.5 Multi-Horizon Latent Policy: B.5 forms the SimplerEnv-WidowX multi-horizon target from horizons h ∈{3, 6, 9}, yielding 15 tokens: one radial and four direction tokens per horizon.Coffee and Stack Three instead concatenate continuous latent actions for horizons h ∈{5, 10, 20}, with matched downstream demonstration settings across methods.
  • B.5 Multi-Horizon Latent Policy: B.5 trains continuous latent-action pretraining and latent-policy stages for 15k steps each, followed by joint diffusion-policy fine-tuning on Coffee and Stack Three.The provided training schedule specifies 5k joint fine-tuning steps on Coffee.
  • B.5 Multi-Horizon Latent Policy: B.5 evaluates SimplerEnv-WidowX multi-horizon variants using matched 5k-step downstream fine-tuning checkpoints.This checkpoint matching complements the continuous-experiment training schedule for Coffee and Stack Three.
  • B.5 Multi-Horizon Latent Policy: B.5 diagnoses cross-horizon interactions using pairwise cosine similarity between gradients from per-horizon branch action L1 losses on SimplerEnv-WidowX.Table A.8 reports every horizon-pair cosine value and their mean for h ∈{3, 6, 9}.
  • B.6 Hyperbolic versus Euclidean Diagnostics: B.6 compares hyperbolic and Euclidean PoLAR with matched tokenizers and downstream setups, changing only geometry in the radial losses.The diagnostics measure adjacent-horizon direction cosine similarity and probe action informativeness after removing radial scale at h = 9.

C Dataset and Evaluation Details … D Additional Results

The paper evaluates robot policies across RoboMimic, MimicGen, BridgeData V2, SimplerEnv-WidowX, and real-world WidowX tasks, using demonstrations, held-out episodes, and rollout-based assessment. The supplied passages detail task coverage, observation and action formats, and evaluation protocols, but provide no specific findings for D Additional Results.

  • C Dataset and Evaluation Details: Table A.9 summarizes demonstration counts for pretraining, downstream fine-tuning, and analysis.The supplied passage does not provide the counts themselves.
  • C.1 RoboMimic and MimicGen: RoboMimic and MimicGen cover pick-and-place, stacking, cleanup, insertion, and threading behaviors with RGB observations and low-level 7-DoF actions.Main diffusion-policy experiments use Can, Square, Stack, Mug Cleanup, and Threading; Coffee and Stack Three support multi-horizon analysis.
  • C.1 RoboMimic and MimicGen: Policies execute the first 10 actions from each 20-step sequence before replanning, with success reported over 100 rollout episodes per task.This protocol applies to the RoboMimic and MimicGen evaluation described in the supplied passages.
  • C.2 BridgeData V2: BridgeData V2 provides top-view RGB observations, language instructions, and robot actions from WidowX trajectories for discrete latent-action and VLA pretraining.Its standard actions comprise world-frame translation and rotation deltas plus a gripper command, and it is used only for pretraining.
  • C.3 SimplerEnv-WidowX: SimplerEnv-WidowX evaluation covers four tasks, fine-tuning each policy on 50 successful demonstrations and testing on 24 non-overlapping held-out episodes.The environment uses 7-DoF Cartesian end-effector delta actions and evaluates policies that predict 10-step action chunks.
  • C.4 Real-World Robot Tasks: Real-world evaluation uses three WidowX SoloAI tasks: Pick & Place Banana, Cup Stack, and Open Pot & Banana.The tasks involve banana placement, cup insertion, and opening a pot before placing a banana inside.

D.1 Detailed Simulation Results · D.2 Real-World Rollouts · D.3 Additional Radius Sweep Visualizations

The appendix details task-level simulation evaluation, representative real-world PoLAR rollouts and failure modes, and radius sweeps showing that larger radii generally yield larger visual transitions while preserving mode.

  • D.1 Detailed Simulation Results: Tables A.10 and A.11 report per-task and average success rates for RoboMimic, MimicGen, and SimplerEnv-WidowX manipulation tasks.RoboMimic and MimicGen rates average 100 rollout episodes per task, while SimplerEnv-WidowX rates use 24 held-out evaluation episodes per task.
  • D.1 Detailed Simulation Results: The simulation appendix evaluates success rates under decoder-only and joint fine-tuning settings.Table A.10 covers RoboMimic and MimicGen; Table A.11 reports four SimplerEnv-WidowX manipulation tasks and their average.
  • D.2 Real-World Rollouts: Representative successful PoLAR rollouts complete Pick & Place Banana, Cup Stack, and Open Pot & Banana tasks.The snapshots show temporally ordered task progression under the same camera and robot setup used for evaluation.
  • D.2 Real-World Rollouts: PoLAR’s observed real-world failures include unsuccessful banana grasps, failed cup stacking or picking, and incomplete banana pick-and-place after opening the pot.These cases summarize observed failure modes from the corresponding real-world rollouts.
  • D.2 Real-World Rollouts: Baseline real-world failures additionally include failing to open the pot and selecting the wrong object.The baseline figures cover a broader error set alongside grasp failures, unsuccessful cup stacking, and incomplete banana-pick stages.
  • D.3 Additional Radius Sweep Visualizations: With direction tokens fixed, increasing only the radial token generally produces larger visual transitions while preserving transition mode.Figure A.4 decodes FDM-predicted DINOv2 features for additional radius-sweep examples.
  • D.3 Additional Radius Sweep Visualizations: Table A.12 reports Square ablations for radial-loss components and radial-margin hyperparameters under decoder-only and joint fine-tuning.The highlighted row indicates the final setting.

D.4 Additional Ablation Studies

Additional ablations on Square confirm that combining Lord and Lrad yields the strongest decoder-only and joint fine-tuning performance. Radius sweeps further show that larger radii produce larger decoded visual transitions while preserving transition mode.

  • Square ablation: On Square, using both Lord and Lrad achieves the best decoder-only and joint fine-tuning performance.Table A.12 reports this same pattern in the additional ablation.
  • Radius sweep: Larger radii generally produce larger decoded visual transitions while preserving the transition mode when direction tokens are fixed.The sweep varies only the radial token.
Loading 2606.21139v1…