Learning to Jump from Pixels

1 Introduction

• Blind walkers cannot anticipate the terrain ahead, limiting their ability to plan jumps across gaps and encouraging conservative behavior.A jump requires knowing where a gap is and how wide it is.Source: §1 · p. 1
• Prior jumping systems can be highly engineered around a single obstacle, with fixed gait, straight-line motion, specialized vision, and manually segmented jump phases.These constraints make extension to diverse terrain require substantial engineering effort.Source: §1 · p. 2
• The gap-world test environment uses flat regions and randomly placed variable-width gaps, simplifying the wild while retaining core vision-guided agility challenges.The authors use it to study how a system can generate diverse agile behaviors from visual observations.Source: §1 · p. 2
• DIC uses a hierarchical controller: a depth-image policy predicts body trajectories, while a blind low-level controller tracks them with whole-body impulse control.The high-level policy is trained with model-free deep reinforcement learning, and the impulse controller is suited to dynamic motion such as jumps.Source: §1 · p. 2
• DIC crosses sequences of wide gaps in real time using body-mounted depth, needs no dynamics randomization for sim-to-real transfer, and outperforms prior work [23].The system also produces different gaits as velocity and task complexity change, achieving wider jumps with variable than fixed gaits.Source: §1 · p. 2

2 Method

• Adaptive gait schedules and commanded body velocities let DIC vary contact timing and broaden feasible contact locations and forces.Forward velocity supports running, while vertical-to-forward velocity ratios control jump height and span.Source: §2 · p. 2

2.2 Low-Level Controller

• MPC and WBIC track whole-body trajectories by regulating ground reaction forces and foot contact timing, avoiding CoM tracking during under-actuated flight.MPC computes target ground forces from the desired trajectory and current state; WBIC uses them to compute joint targets and feedforward torques.Source: §2 · pp. 2–3, §2.2 · p. 4

2.1 High-Level Policy

• The high-level policy uses proprioception and an elevation map or body-mounted depth image to select body-velocity commands and gait patterns.Its action includes target linear and yaw velocity; gait may be fixed, variable, or unconstrained.Source: §2.1 · pp. 3–4

2.3 Neural Network Training

• A recurrent policy network combines CNN terrain features with proprioception, the previous action, and cyclic timing to predict actions.The recurrent architecture addresses partial terrain observations and pose-dependent variance in depth images.Source: §2.3 · pp. 4–5
• A two-stage training procedure first learns an expert with privileged ground-truth heightmaps, then trains a depth-input student with DAgger to match the expert’s action distribution.Depth-image learning is less sample- and wall-clock-efficient than heightmap learning.Source: §2.3 · p. 5

Hardware

• The Mini Cheetah uses a front-mounted RealSense D435 for real-time stereo depth, while depth processing runs offboard and trajectory tracking runs onboard.The robot is a 9 kg electrically actuated quadruped; the high-level policy communicates over Ethernet.Source: §3 · p. 5

Simulator

• The PyBullet simulator uses a robot CAD model and known camera intrinsics to generate mounted-camera observations.Source: §3 · p. 5

Gap World Environment

• The gap-world benchmark varies gap widths and flat-segment distances; test terrains are novel samples from the training distribution.Training gaps span 4 cm to maximum widths of 10, 20, or 30 cm, with flat segments randomized from 0.5 to 2.0 m.Source: §3 · pp. 5–6

Figures

• Figure 4 compares visually guided fixed-gait policies with blind policies and an ideal limit, and contrasts fixed with unconstrained gaits.Figure 5 depicts terrain observations alongside policy-modulated body velocity and contact duration for crossing 30 cm gaps.Source: Figure 4 · p. 6, Figure 5 · p. 6

Baselines

• The comparisons include a model-free Policies Modulating Trajectory Generators baseline and a model-based Local Foothold Adaptation baseline.Source: §3 · p. 6

4.1 Simulation Performance

• Above 90% of fixed-gait gap-crossing attempts succeed up to theoretical limits, and unconstrained gait policies cross wider gaps, reaching 66 cm in simulation.Fixed-gait DIC also outperforms blind locomotion and Local Foothold Adaptation, particularly on large gaps.Source: §4.1 · p. 6, Figure 6 · p. 7
• DIC handles different gap widths and gaits with one reward function and trajectory-generator structure, whereas PMTG required gait- and environment-specific tuning.Flat-ground-tuned PMTG parameters were too conservative to learn gap crossing; retuning produced irregular behaviors that overrode the trajectory generator.Source: §4.1 · p. 7

4.2 Real World Performance

• Real-time depth-based deployment on the Mini Cheetah crossed gaps up to 16 cm; using motion capture and terrain heightmaps, it crossed up to 26 cm.Across trials, foot placements varied, but DIC adapted to avoid stepping in gaps.Source: §4.2 · p. 7
• Wider-gap transfer is constrained by state-estimation drift from sensor noise and imprecise contact timing, plus foot-slip violations during aggressive motion.Source: §4.2 · p. 7

4.3 Vision and Behavioral Cloning

• Behavioral cloning from heightmaps to depth images improves performance over direct depth-image learning in most cases, and cloned Variable Pronk performs best for wide gaps.Cropping heightmaps is faster than rendering depth images, adding a wall-clock-time benefit to cloning.Source: §4.3 · p. 7
• Recurrent student policies achieve higher final performance than nonrecurrent policies, particularly in larger-gap environments requiring more dynamic motion.The authors suggest hidden state helps represent unobserved terrain from observation history.Source: §4.3 · p. 7

Model-free locomotion

• Blind locomotion methods use proprioception and robust control for challenging terrain, while simulation RL has studied discontinuous-terrain traversal without realistic perception or sim-to-real measures.Stepping-stone traversal with simulated characters is one example of the latter.Source: §5 · pp. 7–8

Figure 7

• Figure 7 presents motion-capture deployments in which contact sequences vary across trials while adapting to avoid the illustrated gaps.Source: Figure 7 · p. 8

Table 1

• Table 1 covers gap-crossing success rates across gait types, training widths, height-map or depth inputs, and behavioral-cloning policies with or without recurrence.For a model trained at maximum gap width Wmax, evaluation uses gap width Wmax − 5.Source: Table 1 · p. 8

Model-based control

• Model-based control has demonstrated dynamic blind walking, running, and obstacle jumping, as well as terrain-aware navigation with complete terrain information.The passage contrasts model-based strengths when state and model are sufficiently accurate with model-free use of unstructured or partially observed state.Source: §5 · p. 8

Interfacing Model-based and Model-Free Methods

• Hybrid approaches combine learned perception or planning with control, but cited systems are limited to conservative fixed walking or trotting and do not demonstrate flight phases planned from realistic terrain observations.RLOC uses a complete terrain heightmap and targets footholds; another framework uses a model-free high-level controller with centroidal dynamics.Source: §5 · p. 8

6 Conclusion and Discussion

• Model-free high-level trajectory prediction combined with model-based low-level trajectory tracking achieves high performance and robustness.Source: §6 · pp. 8–9
• Policies jump gaps up to 66 centimeters in simulation, but transfer reaches only 26 centimeters in the real world.Source: §6 · p. 9
• The contact-force optimizer omits robot kinematic configuration, sometimes producing infeasible or overly conservative target impulses.Source: §6 · p. 9
• The Mini Cheetah’s onboard computer cannot run the neural network alongside the low-level controller.Source: §6 · p. 9

A Details of Low-Level Controller

• Algorithm 2 repeatedly computes MPC force targets, uses WBIC and PD torques, and advances the system.Source: A Details of Low-Level Controller · p. 13

• The Raibert heuristic selects foot placements so each leg’s ground incidence angle matches its departure angle.Source: §B.1 · p. 13
• Foot placement depends on the base velocity and the duration of the next foot contact.Source: §B.1 · p. 13
• Here, d is the next placement duration, v is estimated body velocity, vcmd is commanded velocity, and k is a tunable gain.Source: §B.1 · p. 13

• A quadruped at velocity v and cycle frequency f travels v/f per gait cycle, advancing nominal foot placements by that distance.Source: §B.2 · p. 13
• The theoretical gap-crossing limit is v/f for pronking and v/(2f) for trotting.Source: §B.2 · p. 13
• At 1 m/s and 3 Hz, the Mini Cheetah’s theoretical maximum gap crossings are 33 cm for pronking and 17 cm for trotting.Source: §B.2 · p. 13
• Table B.1 lists theoretical longitudinal foot-placement limits for trotting, pronking, and bounding gaits.Source: Table B.1 · p. 14

B.3 Upper Bound on Gap Crossing Probability for Fixed Gait

• At constant velocity, foot-placement spacing d is v/(2f) for trotting and v/f for pronking.Source: §B.3 · p. 14
• For a randomly positioned gap of width h, the blind pronking gap-avoidance upper bound decreases linearly from 1 at zero width to 0 at d.Source: §B.3 · p. 14

B.4 Upper Bound on Gap Crossing Probability with Local Foothold Adaptation

• With foothold adaptation capped at Δ, a foot can fail when it lands at least Δ inside the gap edge.Source: §B.4 · p. 14
• With maximum displacement Δ, the gap-avoidance upper bound declines from 1 at width 2Δ to 0 at d + 2Δ.Source: §B.4 · p. 14

C Local Foothold Adaptation Baseline

• The FPA baseline commands constant velocity and contact patterns, then locally adjusts footholds using a heightmap, a safety heuristic, and nearest-safe-location search.The evaluation assumes FPA has privileged access to the true terrain heightmap.Source: C Local Foothold Adaptation Baseline · p. 14
• DIC outperforms theoretical FPA limits across tested maximum foot displacements of 2, 4, and 6 cm, with larger improvement on wide gaps.The authors note that FPA is complementary to DIC's velocity and contact-schedule adaptation and expect combining them to combine their performance improvements.Source: C Local Foothold Adaptation Baseline · p. 14

D PMTG Baseline

• PMTG augments model-free RL with cyclic leg trajectories and residual commands, with target joint positions computed from target foot positions by inverse kinematics.The policy regulates trajectory-generator frequency and predicts foot-position residuals at each 0.025-second step.Source: D PMTG Baseline · pp. 14–15
• The baseline reward combines forward-distance and velocity-threshold terms with orientation stability, gap-crossing reward, and a gap-avoidance penalty.The gap-crossing reward is binary (0/500), while the avoidance penalty terminates the environment if no gaps are crossed after 180 time steps.Source: D PMTG Baseline · pp. 15–16
• Without shaped rewards, PMTG trots in place instead of crossing 10 cm gaps, whereas whole-body trajectory modulation learns crossing faster and achieves higher performance.Gap-crossing reward and gap-avoidance penalty are critical for the baseline but not for the proposed controller.Source: D PMTG Baseline · p. 16, Figure D.3 · p. 16
• A conservative trajectory generator inhibits gap crossing, especially for pronking; relaxed ranges improve success but tend to exploit the simulator.The relaxed-range behavior is associated with large residual commands and motions unsuitable for sim-to-real transfer.Source: Figure D.4 · p. 16

E PMTG Sim-to-Real Results

• A forward-walking policy using the PMTG architecture was trained and deployed on flat ground to verify the baseline implementation.Figure E.6 illustrates the deployment.Source: E PMTG Sim-to-Real Results · p. 16, Figure E.6 · p. 18
• The PMTG gap-crossing experiments train trotting and pronking policies with different ranges for trajectory-generator frequency and foot-position residuals.The tested relaxed, conservative, and flat-trained conservative configurations use different parameter ranges.Source: E PMTG Sim-to-Real Results · pp. 16–17
• Relaxed PMTG parameter ranges improve gap crossing, particularly for pronking, but produce foot dragging, high foot swings, unrealistic velocity changes, and large residual commands unsuitable for sim-to-real.The comparison uses relaxed and conservative trajectory-generator parameter ranges.Source: E PMTG Sim-to-Real Results · p. 17, Figure D.5 · p. 17

F Depth Image Preprocessing

• Depth preprocessing downsamples images from 480 × 360 to 160 × 120, crops 20 pixels from the left, clips depth to 0.1–1.0 m, and applies hole filling.The crop removes the variable invalid band from the stereo sensor while retaining sufficient terrain information; Figure F.7 shows before-and-after examples.Source: F Depth Image Preprocessing · pp. 17–18, Figure F.7 · p. 19