PaperlayerView exact revision on arXiv

Source-linked AI summary

End-to-End Reinforcement Learning for Quadrupedal Jumping in Lunar Gravity

Levin Müller, Oliver Fischer, Marco Trentini, Yusuke Tanaka, Marco Hutter

arXiv:2610.29586v1cs.RO

TL;DR

Legged robots can access rugged planetary terrain, but commanded quadrupedal point-to-point jumping under lunar gravity remains insufficiently investigated. This paper uses ballistic jump formulation and multistage reinforcement learning to train a single end-to-end policy. Simulations show accurate jumps and resilience to control freezes, while gravity-offloaded hardware demonstrates vertical jumping; horizontal hardware validation remains outstanding.

  • Problem

    Commanded point-to-point jumping by quadrupeds under lunar gravity has not yet been sufficiently investigated, despite its potential use on terrain conventional mobility cannot traverse.

    Source: §I · p. 1

  • Method

    The method combines ballistic target-to-launch-velocity formulation with multistage reinforcement learning to train a single end-to-end policy without dense motion references.

    Source: §I · p. 1

  • Results

    Landing error generally remains below 0.3 m for launch trajectories with elevation angles ≥60◦; nominal collision-free landings exceed 88% on flat and sloped terrain, and 79.7% of sloped-terrain rollouts survived a control freeze up to 1 s.

    Source: §V · p. 6

  • Takeaways & Limitations

    The results support jumping as a credible contingency behavior for escaping otherwise untraversable terrain, subject to mission risk assessment.

    Source: §V · p. 7, §VI · p. 7

    Gravity-offloaded hardware validated vertical jumps, but limitations of the offload prevented validation of horizontal jumps.

    Source: §V · p. 7

Abstract

from arXiv · show

Legged robots can extend planetary exploration to rugged and discontinuous terrain that is inaccessible to conventional rovers. In lunar gravity, jumping further expands this mobility by enabling robots to cross obstacles and gaps that cannot be traversed by walking alone. We present a multistage reinforcement-learning approach that produces a single end-to-end control policy for commanded point-to-point jumping under lunar gravity. The resulting policy has a simple architecture and relies exclusively on mission-compatible onboard sensing, making its deployment on a real lunar mission plausible. In simulation, the learned policy performs accurate point-to-point jumps with landing errors smaller than the robot's body size. We analyze the torque and power demands of lunar jumping and characterize performance degradation under extreme outlier events, including control-update freezes of up to \SI{1}{s} during flight. Hardware transfer demonstrates that the policy can successfully execute a vertical jump under lunar-gravity offloading. Furthermore, the measured torque and power profiles are consistent with the simulation results and remain within the robot's hardware limits.

I. INTRODUCTION

The paper addresses the limits of conventional mobility on rugged and discontinuous lunar terrain, where jumping can help quadrupeds cross otherwise untraversable obstacles and gaps. It develops a single end-to-end reinforcement-learning policy for commanded point-to-point jumps and evaluates the controller in simulation and gravity-offloaded hardware.

I. INTRODUCTION

  • The framework uses lunar ballistic flight to relate the target position and trajectory-apex height to the desired launch velocity.This provides the commanded point-to-point jump formulation.Source: §I · p. 1
  • A single end-to-end policy learns the full maneuver without dense motion references, online trajectory generation, or policy switching.The policy maps onboard-feasible observations to joint-position targets.Source: §I · pp. 1–2
  • The controller is evaluated on the Magnecko quadruped for jump accuracy, recovery, and robustness to random failures.Source: §I · p. 2
  • Gravity-offloading hardware tests assess transfer of the controller and the torque and power profiles.Source: §I · p. 2

II. RELATED WORK

Prior jumping approaches include model-based trajectory tracking, reference-guided reinforcement learning, and goal-conditioned policies, with trade-offs in computational complexity, reference dependence, or operating conditions. The paper’s method uses Magnecko and avoids secondary estimation modules and recurrent controller structures to support deployment.

II. RELATED WORK

  • Trajectory optimization with MPC tracking and fast online trajectory generation achieve quadrupedal jumps, but their complexity and computation demands challenge space deployment.Source: §II · p. 2
  • Reference-guided reinforcement learning can improve robustness, but depends on motion references and had not, to the authors’ knowledge, transferred to low-gravity or offloaded hardware.A goal-conditioned reference-free policy had been shown only under Earth gravity with dense velocity guidance.Source: §II · p. 2
  • Hierarchical or switching controllers add deployment modules, while prior lunar hardware-in-the-loop work focused on continuous pronking rather than a single mission jump.Source: §II · p. 2

III. METHOD

  • The controller is trained and tested on Magnecko, and omits secondary estimation modules and recurrent structures that could become failure points during temporary compute loss.Magnecko has three actuated joints per leg and a 0.62 m total extent in its default extended-leg configuration.Source: §III · p. 2

A. Jumping Physics and Task Formulation

The task uses lunar ballistic flight to define a commanded jump by touchdown position and apex height. The policy observes onboard robot state and tracks an analytically reconstructed launch velocity.

A. Jumping Physics and Task Formulation

  • Ballistic flight connects surface points with parabolic trajectories, and touchdown position plus apex height specify the commanded trajectory.The apex-height command lets the user select the parabola’s shape for the intended use case.Source: A. Jumping Physics and Task Formulation · pp. 2–3
  • The actor observes joint positions and velocities, previous actions, projected gravity, elapsed maneuver time, and recent IMU angular velocities.The critic also receives privileged state information, including contact, base, and center-of-mass quantities.Source: A. Jumping Physics and Task Formulation · p. 3
  • The 12 actions control joint PD controllers updated at 50 Hz, with targets formed using action scaling and default joint positions.Source: A. Jumping Physics and Task Formulation · p. 3
  • The commanded parabola is analytically converted into an optimal launch velocity, reformulating point-to-point jumping as launch-velocity tracking.Jump commands are sampled during training, and the corresponding launch velocity is reconstructed analytically.Source: A. Jumping Physics and Task Formulation · p. 3
  • Training uses flat terrain and planar slopes, with symmetry augmentation reported to improve convergence and base-attitude stability without additional simulation.Source: A. Jumping Physics and Task Formulation · p. 3

B. Multistage Scheme

The multistage scheme assigns distinct control objectives across a jump while retaining a single policy. Launch tracking, stage transitions, and limited sensing support coordinated behavior from preparation through recovery.

B. Multistage Scheme

  • The maneuver has five stages—STANCE, LAUNCH, FLIGHT, LANDING, and RECOVERY—with shared regularizations and stage-specific rewards.The active stage follows transition logic, although the stage itself is not observed by the policy.Source: B. Multistage Scheme · p. 3
  • STANCE permits preparation without a prescribed wind-up reference, yet instantaneous launch-velocity tracking leads the policy to learn a wind-up motion.Tracking the required velocity instantaneously discourages multiple small hops.Source: B. Multistage Scheme · pp. 3–4
  • LAUNCH tracks speed and direction with broad and narrow kernels, while a temporal exponential kernel emphasizes a single strong jump.The wide kernels preserve learning signal early; the narrow kernels enforce precision after basic behavior is discovered.Source: B. Multistage Scheme · pp. 3–4
  • Bad launches terminate if takeoff fails within 2 s or launch parameters are unacceptable, with penalties intended to accelerate and stabilize learning.Source: B. Multistage Scheme · p. 4
  • Precisely timed launch execution and a timer observation let the policy infer flight progress without dense motion references or exteroceptive sensing.Together with the asymmetric actor-critic and stage-transition heuristics, this supports distinct stage behaviors in one policy.Source: B. Multistage Scheme · p. 4
  • During FLIGHT, coordinated limb motion controls base tilt; LANDING rewards more feet contacting the ground, and RECOVERY seeks a stable stance.The recovery stance supports transition back to standard locomotion.Source: B. Multistage Scheme · p. 4

C. Simulation and Hardware Setup

The setup combines PPO-based policy training in simulation with a lunar-gravity hardware offloading testbed. The accompanying figures cover landing accuracy across the command workspace and mid-air recovery behavior.

C. Simulation and Hardware Setup

  • The simulation uses PPO with actor and critic multilayer perceptrons, each with hidden sizes {512, 256, 128}, and robot joint parameters identified using PACE.Both networks use ELU activations.Source: C. Simulation and Hardware Setup · p. 4
  • A constant-force spring reel offsets approximately 5/6 of Magnecko’s weight, while feedforward torques give the legs simulated lunar gravity.The setup supports vertical-jump experiments and simulation-to-hardware comparisons, but does not reproduce unconstrained lunar flight.Source: C. Simulation and Hardware Setup · p. 4
  • The figures cover landing accuracies for 10,000 commands on flat and sloped terrain and illustrate the controller’s mid-air recovery behavior.Source: Figure 3 · p. 5, Figure 4 · p. 5

A. Jumping Accuracy

Landing accuracy was evaluated over the trained command workspace on flat and sloped terrain. Tracking worsened with longer commanded jumps, especially on slopes.

A. Jumping Accuracy

  • Landing accuracy is shown for 10,000 commands across the full workspace, separately for flat and sloped terrain.Source: A. Jumping Accuracy · p. 5
  • Tracking accuracy appeared better at larger commanded apex heights, while landing accuracy degraded as commanded jumping distance increased.Source: A. Jumping Accuracy · p. 5
  • The degradation with jumping distance was more pronounced on sloped terrain than on flat terrain.Source: A. Jumping Accuracy · p. 5
  • The evaluation sampled 10,000 landing-position and apex-height command triplets, testing each once on flat terrain and once on slopes of 0°–5°.Slope direction was sampled from 0°–360°.Source: A. Jumping Accuracy · p. 5

B. Robustness

The controller recovers from randomized mid-flight orientations and usually completes jumps without base-ground contact. Experiments also tested temporary control freezes, including freezes lasting up to 1 second.

B. Robustness

  • After randomized mid-flight orientation initialization, median reorientation time was 0.7 s, and 99% of trials converged within 1.5 s.Convergence meant reaching within 5° of nominal base tilt.Source: B. Robustness · p. 5
  • 92.5% of flat-terrain jumps and 88.1% of sloped-terrain jumps completed without robot-base contact.Source: B. Robustness · p. 5
  • Freeze tests disabled policy target inputs for 0.2, 0.5, or 1.0 seconds, then flushed the IMU angular-velocity history buffer.Joint drivers held the last sent PD target during the interruption.Source: B. Robustness · p. 5

C. Torque and Power Demands

The torque and power analysis identifies takeoff as the dominant mechanical and electrical demand event, with pronounced peaks immediately before takeoff.

C. Torque and Power Demands

D. Hardware Feasibility

Hardware transfer was investigated using gravity offloading, but validation was limited to vertical, in-place jumps. Torque and power around takeoff were compared with a matched simulation rollout.

D. Hardware Feasibility

  • The offload could not provide translational movement, so hardware testing used vertical, in-place jumps with a commanded apex height of 0.8 m.The height was chosen because of the offload's limits.Source: D. Hardware Feasibility · p. 6
  • Torque distributions and total actuation power around takeoff were compared between hardware experiments and a matched simulation rollout.The takeoff focus reflects that, without an atmosphere, the trajectory is fully determined at takeoff.Source: D. Hardware Feasibility · p. 6
  • The hardware jumping sequence is shown in Fig. 7.The section investigates sim-to-real transfer behavior.Source: D. Hardware Feasibility · p. 6

V. DISCUSSION

The controller generally lands within a fraction of the robot's extent and remains resilient to severe control interruptions. Torque and power demands are concentrated at takeoff and remain within hardware constraints, while hardware validation was limited to vertical jumps.

V. DISCUSSION

  • For launch angles ≥60°, landing error generally stays below 0.3 m, less than half the robot’s 0.62 m total extent.The authors describe this accuracy as sufficient for contingency jumps where safely reaching a target region matters more than exact touchdown placement.Source: §V · p. 6
  • Nominal collision-free landings exceed 88% on both flat and sloped terrain, and survival on sloped terrain is 79.7% under control freezes up to 1 s.The controller also reorients the robot in mid-air during robustness experiments.Source: §V · p. 6
  • Actuation demands are mostly determined by takeoff, with demanding short-time torques and powers that remain within Magnecko’s hardware constraints.The evaluation provides a broad reference for system design.Source: §V · p. 7
  • Gravity-offloading hardware demonstrated vertical real-world jumping, but horizontal jumps could not be validated because of offload design limitations.Mission use would still require appropriate risk assessment.Source: §V · p. 7

VI. CONCLUSION AND FUTURE WORK

  • Future work calls for improved horizontal offloading validation, greater robustness and success beyond last-resort use, and soft-contact modeling for lunar regolith.The current ground-contact model is rigid.Source: §VI · p. 7
Loading 2610.29586v1…