PaperlayerView exact revision on arXiv

Source-linked AI summary

LocoFormer: Generalist Locomotion via Long-context Adaptation

Min Liu, Deepak Pathak, Ananye Agarwal

arXiv:2509.23745v1cs.ROcs.AI

TL;DR

Locomotion policies are commonly tuned to particular embodiments, while short histories limit adaptation across diverse tasks. LocoFormer trains on randomized procedural robots and extends memory across segments and trials. It controls unseen robots and adapts to disturbances, including through improved performance on later trials.

  • Problem

    Existing locomotion policies use only a few hundred milliseconds of history, which is insufficient for effective adaptation across a large task space.

    Source: §1 · p. 2, §2.1 · p. 3

  • Method

    LocoFormer trains with large-scale reinforcement learning on procedurally generated, randomized robots and uses a Transformer-XL policy with cached long-context history across segments and trials.

    Source: §1 · p. 2, §2.1 · p. 3, §2.2 · p. 5

  • Results

    On unseen real-world robots, LocoFormer robustly controls varied embodiments and adapts across seconds or trials, including after locked-limb disturbances and failed initial attempts.

    Source: §1 · p. 2, §3.2 · p. 7

  • Takeaways & Limitations

    The authors see this recipe as potentially useful for generalist controllers for other skills, especially where large-scale data can be generated in simulation.

    Source: §5 · p. 8

    Training is resource-intensive compared with a specialist policy, and hand-crafted procedural task spaces may be difficult to design generally.

    Source: §6 · p. 9

Abstract

from arXiv · show

Modern locomotion controllers are manually tuned for specific embodiments. We present LocoFormer, a generalist omni-bodied locomotion model that can control previously unseen legged and wheeled robots, even without precise knowledge of their kinematics. LocoFormer is able to adapt to changes in morphology and dynamics at test time. We find that two key choices enable adaptation. First, we train massive scale RL on procedurally generated robots with aggressive domain randomization. Second, in contrast to previous policies that are myopic with short context lengths, we extend context by orders of magnitude to span episode boundaries. We deploy the same LocoFormer to varied robots and show robust control even with large disturbances such as weight change and motor failures. In extreme scenarios, we see emergent adaptation across episodes, LocoFormer learns from falls in early episodes to improve control strategies in later ones. We believe that this simple, yet general recipe can be used to train foundation models for other robotic skills in the future. Videos at generalist-locomotion.github.io.

1 Introduction

Short-context locomotion policies struggle to adapt across a broad task space. LocoFormer combines large-scale training on randomized procedural robots with long context, and shows adaptation on unseen robots under out-of-distribution conditions.

1 Introduction

  • LocoFormer trains on procedurally generated legged and wheeled robots with aggressively randomized physical parameters to support control across unfamiliar bodies.The randomized parameters include inertia, control gains, and joint limits.Source: §1 · p. 2
  • Existing locomotion policies typically use only a few hundred milliseconds of history, which the paper says is insufficient for adaptation across a large task space.The approach extends context by several orders of magnitude, sometimes spanning trial boundaries.Source: §1 · p. 2, Figure 3 · p. 3
  • Under doubled domain-randomization ranges, LocoFormer improves across multiple seconds of history, while prior policies stop improving beyond a few hundred milliseconds.Tests include unseen real-world robots and OOD scenarios.Source: §1 · p. 2, Figure 2 · p. 2
  • The authors suggest this approach may help bridge the sim-to-real gap and maintain function after adverse morphology changes, and envision applying it to other low-level robotic skills.The paper presents those implications as possibilities, not demonstrated outcomes beyond locomotion.Source: §1 · p. 2

2 Method

LocoFormer trains across procedurally generated robot tasks and uses long-context memory to adapt in context. The method combines multi-trial training, a shared robot representation, and a Transformer-XL policy with cached history.

2.1 Pre-training to Learn in Context

  • Training runs multiple trials within an episode, retains memory across trials, and maximizes cumulative reward over the episode.PPO training uses a sampled adaptation-time budget and proceeds from shorter trials and budgets to longer ones for deployment preparation.Source: §2.1 · pp. 3–4
  • Rather than relying on detailed kinematics, LocoFormer trains over procedurally generated bipeds, quadrupeds, and wheeled counterparts with randomized kinematic and dynamic parameters.The task space does not incorporate exact parameters from market-available robots.Source: §2.1 · pp. 3–4
  • A unified joint space provides shared observations and actions across robots, while rewards combine command tracking with penalties for torque, action smoothness, and hardware damage.The policy outputs target positions in the joint superset, from which each robot's relevant joint states are extracted.Source: §2.1 · p. 4
  • At deployment, the policy can adapt to an unseen embodiment within seconds, and may improve on later trials after an initial failure.The paper describes this as zero-shot adaptation when no adaptation time is allocated, and few-shot learning when prior-trial experience is retained.Source: §2.1 · p. 4

2.2 Policy Architecture with Long-Term Memory

  • Transformer-XL processes observation embeddings in segments, using cached hidden states from earlier segments to form current attention keys and values without backpropagating through the cache.The segment recurrence gives an effective context length of O(NL); six layers with segment length 128 can retain up to 896 timesteps, about 18 seconds at 50 Hz.Source: §2.2 · pp. 4–5
  • 10% improvement on TRON1 follows a 5-second adaptation window; overall, LocoFormer approaches expert performance and outperforms the GRU and Conditioning baselines.The comparison covers 10 unseen robots in 1,000 simulation environments per robot, with rough terrain and extensive domain randomization.Source: Table 1 · p. 5
  • For real-time inference, a KV-cache reduces dependence on timestep t from quadratic to linear and retains the most recent 2 × L − 1 timesteps.The retained history matches the maximum directly conditioned history length encountered during training.Source: §2.2 · p. 5

3 Experimental Results

In simulation, LocoFormer transfers to unseen robots and benefits from adaptation time, while its internal representations become embodiment-specific. Real-world trials show the same policy adapting to mechanical changes and failures, sometimes across episodes.

3.1 Simulation Evaluation

  • On TRON1, 5 seconds of pre-evaluation adaptation yields a 10% performance gain; the GRU baseline averages 0.37.Across ten unseen robots, LocoFormer performs close to expert policies in zero-shot and few-shot settings.Source: §3.1 · p. 6
  • Across morphologies, survival improves with longer adaptation, including under doubled domain-randomization intensity on rough terrain.The evaluation averages time to fall over 6-second rollouts, with 50K episodes per setting.Source: §3.1 · p. 6, Figure 4 · p. 6
  • By 5 seconds, initially similar representations form distinct clusters across four humanoid variants, indicating embodiment-specific internal models without robot descriptors.The analysis tracks second-layer outputs over 4,096 zero-shot rollouts.Source: §3.1 · p. 6, Figure 4 · p. 6

3.2 Real-World Evaluation

  • On four Unitree robots, including Go2 and Go2-W in rear-leg biped modes, the policy deploys zero-shot and adapts within seconds without tuning or retraining.Source: §3.2 · p. 6
  • After a knee lock turns the Go2 into a three-legged robot, LocoFormer shifts weight backward and walks after 2–3 seconds; it also redistributes load when wheeled-quadruped legs are locked.Three-legged locomotion is outside the design space described in the passage.Source: §3.2 · p. 7
  • On an unstable ankle-less biped, retained failed-trajectory information supports stable walking by the third trial, including under pushes and added weights.The passage identifies the TransformerXL cache as the mechanism for retaining and attending to prior-trial information.Source: §3.2 · p. 7
  • When Go2-W wheels are locked, LocoFormer switches from rolling to walking, then returns to rolling when they unlock; added mass prompts gait adjustment for balance.Source: §3.2 · p. 7
  • With stilts, the policy adjusts step timing and foot placement for stable walking; after lower legs are cut, it locomotes effectively after 7–8 seconds using large thigh swings.Source: §3.2 · p. 8

4 Related Work

Prior locomotion work often targets individual robots or uses short context, while LocoFormer combines diverse embodiments with long-context adaptation. The related work also situates this approach alongside meta-learning and in-context adaptation.

4.1 Legged Locomotion

  • Sim2real locomotion controllers are typically tuned for one robot with narrow dynamics randomization and hundred-millisecond context, whereas this work uses one policy across embodiments and dynamics.Source: §4.1 · p. 8

4.2 Cross-embodiment Learning

  • Cross-embodiment methods use morphology-aware architectures, parameterized spaces, imitation learning, or next-token prediction; LocoFormer instead trains at massive scale with long contexts across wheeled and legged quadrupeds and humanoids.Source: §4.2 · p. 8

4.3 Adaptation

  • Prior meta-RL and fine-tuning adapt specialist policies, while LocoFormer studies high-frequency closed-loop control with one policy across embodiments and dynamics.Source: §4.3 · p. 8

5 Conclusion

LocoFormer uses long contexts to adapt across legged and wheeled robots in the real world. The paper presents this as a potential recipe for generalist controllers beyond locomotion, particularly where simulation enables large-scale reinforcement learning.

  • LocoFormer robustly controls varied legged and wheeled robots in the real world.Source: §5 · p. 8
  • Long contexts provide information the policy uses to improve performance through emergent adaptive behavior.Source: §5 · p. 8
  • The authors propose that this recipe could train generalist controllers for other skills, especially where large-scale reinforcement learning is possible in simulation.Source: §5 · p. 8

6 Limitations

The authors identify high training cost and reliance on a hand-crafted procedural task space as limitations of LocoFormer.

  • Training LocoFormer is highly resource-intensive compared with training a specialist policy.Source: §6 · p. 9
  • The procedurally generated task space is hand-crafted, which may be difficult to design in general.Source: §6 · p. 9
  • Automated task generation from LLMs or other web-scale sources is proposed as a potentially more scalable future approach.Source: §6 · p. 9

A Implementation Details

The implementation pipeline covers robot morphology generation, environment-dynamics randomization, reward design, and policy training.

A Implementation Details

A.1 Procedural Robot Generation

The paper describes procedural generation across four robot morphology categories and a reward structure combining morphology-agnostic and morphology-specific terms.

A.1 Procedural Robot Generation

  • Procedural generation covers quadrupeds, bipeds, and wheeled counterparts, but the authors note that this morphological space is not exhaustive.The parameterization misses some features of unseen robots, including anhedral angles in both hip roll and yaw joints and joint axes offset along the x-axis.Source: §A.1 · p. 13

A.2 Reward Design

  • The unified reward structure combines morphology-agnostic and morphology-specific terms.The authors omit the time subscript t in some expressions for clarity.Source: §A.2 · p. 13

B Experiment Details

Simulation experiments include ten robots and show cross-trial adaptation on TRON1, while real-world evaluation videos are provided online.

B Experiment Details

  • TRON1 improves performance by 10% through five seconds of pre-evaluation adaptation, progressing from early failures to effective stabilization and control.The comparison set comprises ten simulated robots.Source: B Experiment Details · p. 13
  • The paper provides videos of its real-world evaluation online.Source: B Experiment Details · p. 13

C Computation Cost

LocoFormer’s training compute is amortized across 100,000 robots, and lower-compute variants show a favorable performance-compute trade-off.

C Computation Cost

  • Compared with a specialist policy, LocoFormer uses 500 times more compute for 100,000 more robots, reducing amortized compute per robot from 1 to 0.005 day.Source: C Computation Cost · p. 13
  • A lower-compute LocoFormer variant still shows clear signs of adaptation and cross-embodiment generalization, despite the trade-off with performance.The authors suggest mixed precision and ZeRO memory optimization as ways to further reduce compute requirements.Source: C Computation Cost · p. 15
  • Figure 9 compares training-set reward and average displacement on ten evaluation robots across GPU and Transformer-layer configurations; more GPUs yield the best model, with a favorable performance-compute trade-off.Source: Figure 9 · p. 15
Loading 2509.23745v1…