PaperlayerView exact revision on arXiv

Source-linked AI summary

Act2Goal: From World Model To General Goal-conditioned Policy

Pengfei Zhou, Liliang Chen, Shengcong Chen, Di Chen, Wenzhi Zhao, Rongjun Jin, Guanghui Ren, Jianlan Luo

arXiv:2512.23541v1cs.ROcs.AI

TL;DR

Long-horizon goal-conditioned manipulation remains difficult because direct action prediction lacks explicit modeling of task progress, while task specifications must remain expressive and precise. Act2Goal uses a goal-conditioned visual world model with multi-scale temporal control, coupled to motor control through cross-attention, and supports reward-free HER/LoRA adaptation. In real-robot out-of-distribution tasks, success increased from 0.30 to 0.90 within minutes of autonomous interaction.

  • Problem

    Existing goal-conditioned policies struggle with long-horizon manipulation because direct action prediction lacks explicit task progress and structured intermediate guidance.

  • Method

    Act2Goal combines a goal-conditioned visual world model, Multi-Scale Temporal Hashing, end-to-end cross-attention, and HER-based LoRA finetuning for reward-free online adaptation.

  • Results

    Real-robot success rates increased from 0.30 to 0.90 on challenging out-of-distribution tasks within minutes of autonomous interaction.

  • Takeaways & Limitations

    Goal-conditioned world models combined with multi-scale temporal reasoning provide structured intermediate guidance for robust generalization and closed-loop long-horizon execution.

Abstract

from arXiv · show

Specifying robotic manipulation tasks in a manner that is both expressive and precise remains a central challenge. While visual goals provide a compact and unambiguous task specification, existing goal-conditioned policies often struggle with long-horizon manipulation due to their reliance on single-step action prediction without explicit modeling of task progress. We propose Act2Goal, a general goal-conditioned manipulation policy that integrates a goal-conditioned visual world model with multi-scale temporal control. Given a current observation and a target visual goal, the world model generates a plausible sequence of intermediate visual states that captures long-horizon structure. To translate this visual plan into robust execution, we introduce Multi-Scale Temporal Hashing (MSTH), which decomposes the imagined trajectory into dense proximal frames for fine-grained closed-loop control and sparse distal frames that anchor global task consistency. The policy couples these representations with motor control through end-to-end cross-attention, enabling coherent long-horizon behavior while remaining reactive to local disturbances. Act2Goal achieves strong zero-shot generalization to novel objects, spatial layouts, and environments. We further enable reward-free online adaptation through hindsight goal relabeling with LoRA-based finetuning, allowing rapid autonomous improvement without external supervision. Real-robot experiments demonstrate that Act2Goal improves success rates from 30% to 90% on challenging out-of-distribution tasks within minutes of autonomous interaction, validating that goal-conditioned world models with multi-scale temporal control provide structured guidance necessary for robust long-horizon manipulation. Project page: https://act2goal.github.io/

I. INTRODUCTION

Act2Goal addresses long-horizon goal-conditioned manipulation by combining a goal-conditioned visual world model with multi-scale temporal control and end-to-end motor coupling. It also supports reward-free autonomous adaptation, improving challenging out-of-distribution success from 0.30 to 0.90 within minutes.

  • Motivation: Visual goals provide precise task specifications by encoding object configurations, spatial relations, and terminal constraints without linguistic ambiguity or explicit reward engineering.This precision is important for fine-grained, complex, multi-stage manipulation.
  • Motivation: Standard goal-conditioned policies degrade on long-horizon tasks because direct action prediction lacks explicit task progress, intermediate feasibility, and long-horizon consistency.Narrowly scoped demonstrations further encourage overfitting to state–action mappings and reliance on dense supervision.
  • Method: Act2Goal combines a goal-conditioned visual world model with Multi-Scale Temporal Hashing and end-to-end cross-attention to guide motor control.The world model generates intermediate visual states, while the policy couples those representations to an action expert.
  • Method: Multi-Scale Temporal Hashing decomposes imagined trajectories into dense proximal frames for local control and sparse distal frames for global task consistency.This structure balances fine-grained closed-loop reactivity with long-horizon planning.
  • Results: Success rates increased from 0.30 to 0.90 on challenging long-horizon out-of-distribution tasks within minutes of autonomous interaction.The result is attributed to goal-conditioned world models combined with multi-scale temporal reasoning for structured intermediate guidance.
  • Contributions: Act2Goal demonstrates zero-shot generalization across unseen objects, rearrangements, environments, and goals, while enabling reward-free online improvement through HER-style relabeling and LoRA finetuning.The adaptation mechanism relabels the policy’s own rollouts as additional goal-achieving trajectories.

II. RELATED WORKS

Prior work studies goal-conditioned policies, world models, and online improvement, but existing methods can struggle to align observations with distant goals or maintain long-horizon consistency. Act2Goal combines a purely vision-based goal-conditioned world model with MSTH to provide structured visual trajectories for unseen-task generalization.

  • Goal-conditioned Policy Learning: Earlier goal-conditioned policy work includes goal relabeling, structured reinforcement-learning goals, long-horizon reasoning, keyframe planning, and program-synthesized goals.Examples include GoalGAIL’s HER-based learning and CoA’s reverse action-sequence generation from goal keyframes.
  • Goal-conditioned Policy Learning: Act2Goal addresses prior alignment and long-horizon planning limitations by simulating structured visual trajectories with a goal-conditioned world model and MSTH.The approach is presented as enabling better generalization in unseen tasks.
  • Goal-conditioned Policy Learning: Prior methods typically rely on explicit goal supervision or struggle to align current observations with distant goals.Act2Goal targets these limitations through structured visual trajectories and multi-scale temporal planning.
  • World Models for Robotic Control: World models in robotic control support environmental simulation, synthetic-data generation, learned simulation, and policy planning with action experts.In action-expert systems, the world model provides future-state features and the action expert predicts actions accordingly.
  • World Models for Robotic Control: Act2Goal uses a purely vision-based goal-conditioned world model, differing from prior systems that condition future visual prediction on language instructions.The paper describes this as the first integration of a world model into goal-conditioned policy learning.

C. Online Autonomous Improvement

Act2Goal combines a goal-conditioned visual world model with action generation and supports reward-free online improvement through HER-style relabeling and LoRA finetuning.

  • C. Online Autonomous Improvement: HER-style relabeling and LoRA-based finetuning adapt Act2Goal directly from self-collected rollouts without task rewards or human annotations.The mechanism is described as lightweight and fully self-supervised for real-world deployment.
  • C. Online Autonomous Improvement: The policy integrates a goal-conditioned world model with an action expert through multi-scale visual features and proprioceptive state conditioning.The action expert uses an architecture isomorphic to the world model and predicts actions with flow matching.
  • C. Online Autonomous Improvement: Act2Goal’s staged learning jointly trains the world model and action expert, then supports action adaptation and optional deployment-time self-improvement.The three stages align representations, improve action performance, and enable autonomous adaptation in novel scenarios.
  • C. Online Autonomous Improvement: The architecture encodes current and goal multi-view frames, refines noisy latents into MSTH frames, and cross-attends world-model features into Action DiT blocks.This connects imagined visual states to MSTH-structured action generation.
  • C. Online Autonomous Improvement: The world model predicts structured visual sequences from current-observation and goal latents by transforming random noise through continuous flow matching.The generated latent frames can be decoded into visual states, while iterative refinement uses a learned vector field.

B. Multi-Scale Temporal Hashing for Visual State and Action

MSTH decomposes imagined trajectories into dense proximal states and sparse distal states, combining local reactive control with long-horizon goal guidance.

  • B. Multi-Scale Temporal Hashing for Visual State and Action: MSTH partitions an imagined trajectory into a proximal segment for fine-grained local dynamics and a distal segment for sparse long-horizon structure.The partition uses total trajectory length K, proximal horizon P, and vision sampling stride r.
  • B. Multi-Scale Temporal Hashing for Visual State and Action: Logarithmic distal sampling increases temporal intervals with horizon length, providing coarse but goal-aligned long-term guidance.The distal indices are determined by logarithmic spacing.
  • B. Multi-Scale Temporal Hashing for Visual State and Action: Proximal actions are predicted at every timestep, while distal actions align with distal visual states and guide long-horizon adherence.Only proximal actions are executed during deployment; distal predictions remain latent.

C. Two-Stage Offline Training

Act2Goal uses two-stage offline training to align visual trajectory prediction with action planning and improve transfer to unseen tasks and environments.

  • C. Two-Stage Offline Training: Stage one jointly trains transition trajectory prediction and action generation with flow matching after adapting a pretrained world model to MSTH trajectories.The objective aligns visual transition prediction with action planning between an initial observation and goal condition.
  • C. Two-Stage Offline Training: Optional online improvement executes episodes, relabels achieved visual states as goals, and updates LoRA parameters using action-prediction loss.The procedure repeats while performance has not converged.
  • C. Two-Stage Offline Training: The joint objective combines visual and action losses with a balancing coefficient λ set to 0.1 in the experiment.The loss is written as Lstage1 = Lv + λ · La.
  • C. Two-Stage Offline Training: Stage two fine-tunes the entire pipeline end-to-end with behavioral cloning using only the action flow matching loss.Gradients from action loss update both action-generation components and the goal-conditioned world model.
  • C. Two-Stage Offline Training: Two-stage offline training enables robust world understanding and action generation that transfer to unseen environments and tasks.This is the stated outcome of the offline training approach.

D. Online Autonomous Improvement

Act2Goal addresses the deployment limitations of imitation-learning policies through reward-free online autonomous improvement with HER.

  • HER enables autonomous performance enhancement during physical-robot deployment, addressing a common limitation of imitation-learning policies.
  • Each inference transition is collected in an edge-device replay buffer and relabeled toward the achieved goal regardless of success.

IV. EXPERIMENTS

The experiments evaluate Act2Goal’s offline generalization and its ability to improve autonomously through deployment interactions.

  • Experiments assess offline imitation-learning generalization under in-domain and out-of-domain scenarios, alongside autonomous improvement during deployment.

A. Generalization Capability Evaluation

Act2Goal is evaluated for generalization across simulated and real-world manipulation tasks, including challenging unseen configurations and objects.

  • Robotwin 2.0 evaluation uses four tasks with fixed-seed goal images and Easy and Hard modes to test seen and challenging unseen scenarios.
  • Real-world evaluation covers Whiteboard Word Writing, Dessert Plating, and Plug-In Operation, comparing head-view goals with model rollouts using success rate.
  • Act2Goal outperforms baselines in all Easy-mode tasks and 3 Hard-mode Robotwin 2.0 tasks, demonstrating superior simulation generalization.
  • Act2Goal significantly outperforms all baselines across real-world manipulation tasks, including out-of-distribution settings, using only offline imitation learning.
  • OOD tests introduce unseen words, visual variations, or an unseen bottle-insertion task to probe compositional generalization, goal following, and skill transfer.

B. Analysis of Online Autonomous Improvement

Online autonomous improvement raises performance across simulated hard-mode scenarios and unfamiliar real-world drawing and insertion tasks.

  • Success rates improve by up to 8× over the pretrained baseline across approximately three online-training rounds before convergence.
  • Using all rollouts performs best, while failed-only rollouts still produce clear improvement through HER-based experience extraction.
  • In real-world drawing, online fine-tuning steadily improves quality within 15 minutes from initially limited performance.
  • On the OOD Plug-In task, online training increases success rate from 0.30 to 0.90.

C. Effectiveness of MSTH

MSTH combines dense proximal and sparse distal visual frames to support precise local control while preserving long-horizon goal consistency. In whiteboard writing, it remains robust as word length and distribution shift increase, unlike fixed-horizon action chunking.

  • C. Effectiveness of MSTH: Generated trajectories contain fine-grained proximal frames for executable short-term guidance and sparse distal frames that preserve long-horizon task structure.Figure 8 samples three proximal and three distal frames from each generated sequence.
  • C. Effectiveness of MSTH: Fixed-horizon action chunking performs comparably on short words but drops drastically for long and out-of-domain scenarios.The degradation is attributed to compounding goal misalignment over extended sequences.
  • C. Effectiveness of MSTH: MSTH maintains high success rates across short, medium, and long words, including out-of-domain words, while improving long-horizon robustness.The method dynamically adjusts temporal abstraction and anchors execution to the final goal.
  • C. Effectiveness of MSTH: Act2Goal combines a visual world model with multi-scale temporal control to generate short-term control signals and coarse long-term trajectories.The policy is trained offline and additionally supports reward-free autonomous improvement through HER and LoRA-based finetuning.

APPENDIX

The appendix specifies Act2Goal’s model, training, deployment, task protocols, and evaluation procedure. These details include multi-scale latent and decoded frames, LoRA-only online updates, task-specific physical setups, and manually labeled real-world success rates.

  • Model: The world model predicts two proximal and two distal latent frames, decoded into nine frames at each temporal scale, while distal actions guide but are not executed.The action expert outputs 54 proximal actions, of which 50 are executed, and 9 distal actions for guidance.
  • Training: Training uses the AgiBot World dataset and a proprietary dataset, with separate world-model fine-tuning and end-to-end behavioral-cloning stages.Stage 1 fine-tunes a pretrained 1.6B-parameter Genie Envisioner before Stage 2 behavioral cloning.
  • Deployment and Online Learning: Online improvement fine-tunes only rank-64 LoRA layers using a replay buffer of 20, with each training round taking five minutes and inference latency of 200 ms.Deployment uses an AgiBot Genie-01 robot with an NVIDIA RTX 4090.
  • Task Protocols: Whiteboard writing used manual marker grasp positioning and tape to reduce marker slippage during extended trials.The bearing task used bearings heavier than 2 kg, while dessert plating used silicone toy desserts for repeatability.
  • Evaluation: Real-world success rates are manually labeled over 40 rollouts per experiment, while simulation success rates are automatically computed from 90 rollouts.Autonomous-improvement checkpoints are saved after each training round and evaluated individually.
Loading 2512.23541v1…