PaperlayerView exact revision on arXiv

Source-linked AI summary

Test-Time Motion Steering for Perceptive Humanoid Fall Recovery

Osher Azulay, Cheng-Lin Hsieh, Stella X. Yu

arXiv:2610.30061v1cs.RO

TL;DR

A fallen humanoid may have several ways to stand, but reliable recovery alone does not let a task choose among them, and steering must anticipate physical consequences. RISE composes diverse recoveries and uses a visual diffusion planner to steer predicted body states before generating motion latents for execution. Simulation shows diverse, terrain-adaptive recovery and test-time steering, with real-world demonstrations of perceptive and steerable recovery.

  • Problem

    Reliable fall recovery does not by itself let task objectives choose how the humanoid stands, and steering must account for scene-dependent routes and changing support contacts.

    Source: §I · p. 1

  • Method

    RISE composes recovery movements into alternatives, learns a feedback vocabulary, and uses task costs to steer predicted future body states before generating conditioned motion latents.

    Source: §I · p. 2

  • Results

    In simulation, RISE provides diverse recoveries, terrain adaptation, and heading, speed, and path steering without retraining; real-world demonstrations show perceptive and steerable recovery.

    Source: §I · p. 2, B. Recovery Quality and Diversity · p. 5, D. Test-Time Steering of Recovery · p. 6

  • Takeaways & Limitations

    The same trained model can use task objectives to shape recovery across tested terrains, while real-world demonstrations illustrate perceptive and steerable recovery.

    Source: §I · p. 2

    On uneven terrain, accurate path tracking does not always lead to successful recovery.

    Source: §V · p. 8

Abstract

from arXiv · show

Humanoid fall recovery requires selecting a motion that suits both the surrounding scene and the task. The same fallen pose can admit several ways to stand, with different routes, support transitions, and final headings. Yet reliable recovery alone does not provide control over which of these alternatives the robot executes. Steering this choice requires anticipating the physical consequences of a requested movement. We present RISE, an approach to perceptive humanoid recovery that lets task objectives steer how the robot returns to standing at inference time. It builds on two insights: diverse recoveries can be composed from shared movements, and anticipating how these movements unfold allows recovery to be steered toward a task objective. We learn a recovery vocabulary that connects intended movements to motor actions through sensory feedback. A visual diffusion planner learns from executions of this vocabulary to predict motion intent as future body states. Task costs guide this prediction toward a desired heading, rise speed, or path. The planner then generates motion latents conditioned on the steered intent, allowing the requested movement to shape its execution. Simulation experiments demonstrate diverse recoveries, terrain adaptation, and test-time steering without retraining, complemented by real-world demonstrations on a Unitree G1. Project page: https://osheraz.github.io/RISE/

I. INTRODUCTION

Humanoid recovery must choose not only whether to stand but how, because the scene and task affect which route, support transition, and final heading fit. RISE combines perception and test-time task steering by composing diverse recoveries and steering predicted body states before generating motion latents for execution.

I. INTRODUCTION

  • Steering recovery must account for varied fallen configurations, changing support contacts, and terrain-dependent routes, while each recorded recovery supplies only one way to stand.Walls, steps, and requested headings can change which recovery fits; the robot must anticipate how a requested movement unfolds under changing contact.Source: §I · pp. 1–2
  • RISE composes shared movement patterns into alternatives, learns a feedback recovery vocabulary, and uses a visual diffusion planner to steer future body states before generating conditioned motion latents.The decoder uses proprioception and egocentric depth, and the planner learns from the resulting physical executions.Source: §I · p. 2
  • In simulation, RISE offers diverse recoveries with greater reliability and lower contact forces than prior work, and one model follows heading, speed, and path objectives across terrains without retraining.Real-world demonstrations on a Unitree G1 show perceptive and steerable recovery.Source: §I · p. 2
  • RISE combines onboard perception with test-time task steering, allowing task objectives to select a recovery rather than leaving the choice to the learned policy.The introduction describes this as the first framework for humanoid fall recovery to combine these capabilities.Source: §I · pp. 1–2

II. RELATED WORK

Prior work addresses fall recovery, perceptive motion control, or test-time steering, but the cited methods do not combine onboard scene perception with task-directed recovery. RISE draws on composed recovery data and state-first planning to steer recovery at test time.

II. RELATED WORK

  • RISE uses egocentric depth and task objectives to steer recovery heading, speed, and path, addressing the gap between perceptive recovery and test-time steering.The related-work discussion contrasts this with recovery methods whose learned policy chooses how to stand and command-driven controllers whose commands are defined before training.Source: §II · p. 2
  • Recorded recovery data is sparse and usually provides one way to stand per starting pose; RISE composes compatible recordings to create several executable alternatives.Motion graphs, physics-based augmentation, and motion matching broaden data coverage, but RISE uses motion matching specifically to provide alternatives for steering.Source: §II · p. 3
  • Joint state-action diffusion generates actions alongside states, whereas RISE generates motion latents conditioned on the completed steered state sequence for feedback execution.The authors hypothesize this ordering helps the robot follow objectives while completing recovery.Source: §II · p. 3

III. METHOD

  • RISE curates a multimodal recovery repertoire, learns a recovery vocabulary and feedback decoder, then trains a planner on the resulting motions to steer predicted intent.Task costs guide the predicted intent, which shapes execution.Source: §III · p. 3

A. Curating a Multimodal Recovery Repertoire

RISE builds a multimodal, terrain-adapted recovery repertoire and learns a feedback-based vocabulary that maps intended movements to motor actions. The vocabulary is trained on physically realized recoveries and supports adaptation to posture and nearby terrain.

A. Curating a Multimodal Recovery Repertoire

  • Compatible segments from recorded recoveries are composed into alternative ways to stand, with frame-level matching selecting junctions and segment lengths.Sources include LAFAN1, BONES-SEED, AMASS, and recordings in G1 coordinates.Source: A. Curating a Multimodal Recovery… · p. 3
  • Terrain alignment adjusts support and clearance, while geometric checks and expert-policy execution filter and realize feasible references.The expert uses coarse references and reinforcement learning to adapt execution to terrain and contact.Source: A. Curating a Multimodal Recovery… · p. 3

B. Learning a Recovery Vocabulary

  • Motion intent is represented as future body states containing joint configuration and velocity, pelvis displacement, and pelvis orientation.Recorded poses are anchored to the robot’s current pelvis position and heading.Source: B. Learning a Recovery Vocabulary · p. 4
  • An encoder compresses a desired state window into a motion latent, and a feedback decoder combines it with proprioception and egocentric depth to predict motor actions.Proprioception supports correction, while depth adapts actions to nearby terrain.Source: B. Learning a Recovery Vocabulary · p. 4
  • Online DAgger trains the decoder on corrective expert actions at states visited during execution, exposing it to varied intent–state pairings.The original coarse reference is used at the corresponding reference time.Source: B. Learning a Recovery Vocabulary · p. 4
  • A training-only motion decoder reconstructs desired state windows from motion latents, with a weighted reconstruction loss emphasizing near-future samples.The loss weights normalized joint and pelvis errors separately.Source: B. Learning a Recovery Vocabulary · p. 4

C. Learning to Plan Executable Recovery

RISE trains a planner on paired motion latents and the resulting realized states from the robot’s rollouts. It predicts steered future states first, then generates latents conditioned on those states to realize them.

C. Learning to Plan Executable Recovery

  • Offline planner sequences pair motion latents used for control with the robot’s resulting next states across varied physical and terrain conditions.The encoder and decoder are frozen while these rollouts are collected.Source: C. Learning to Plan Executable Recovery · p. 4
  • At inference, future body states are denoised first, then held fixed while motion latents are generated to realize the steered plan.Latent tokens can attend to the complete state sequence, and the factorization is p(S, Z | c_t) = p(S | c_t) p(Z | S, c_t).Source: C. Learning to Plan Executable Recovery · p. 4
  • Training alternates state prediction and conditional latent prediction with equal probability, using paired realized sequences and no task-specific labels.Small temporally correlated perturbations of conditioning states encourage tolerance to prediction errors at deployment.Source: C. Learning to Plan Executable Recovery · p. 4

D. Steering Recovery with Motion Objectives

RISE steers recovery by guiding predicted motion intent toward task costs during diffusion sampling, then generating motion latents conditioned on the steered intent. The objectives target heading, rise speed, or path.

D. Steering Recovery with Motion Objectives

IV. EXPERIMENTS

The experiments assess recovery reliability and diversity, motion prediction, task-directed recovery without retraining, and real-world demonstrations.

IV. EXPERIMENTS

  • The evaluation covers recovery quality and diversity, motion-prediction accuracy, task-directed recovery without retraining, visual feedback, prediction order, replanning, and real-world demonstrations.Source: §IV · p. 5

A. Experimental Setup

The simulation evaluates a 29-DoF Unitree G1 with a visual diffusion planner and frozen depth-conditioned decoder across held-out pose–terrain configurations. Comparisons use VIGOR and FIRM under shared trial conditions.

A. Experimental Setup

  • The evaluation uses a 29-DoF Unitree G1 in Isaac Sim with a visual diffusion planner and frozen depth-conditioned decoder.Unless otherwise noted, the planner samples one-second motion intent and latents at 50 Hz, executes 0.2 seconds, and replans; attempts last 15 seconds.Source: A. Experimental Setup · p. 5
  • Held-out initial pose–terrain configurations span flat ground, stepped pyramids, stairs, rubble, drop edges, and slopes with risers.Source: A. Experimental Setup · p. 5
  • RISE is compared with author-provided VIGOR and FIRM checkpoints using shared initial configurations, recovery criteria, and time limits.VIGOR uses visual feedback and acts deterministically; FIRM samples its blind diffusion policy without added exploration noise.Source: A. Experimental Setup · p. 5

B. Recovery Quality and Diversity

RISE improves recovery success while reducing contact forces, torques, and joint-limit violations, and its learned prior supports distinct recovery strategies. Predicted pelvis motion agrees closely with execution, with larger deviations at the hands and soles, particularly on uneven terrain.

B. Recovery Quality and Diversity

  • RISE improves success over both baselines while reducing contact forces, torques, and joint-limit violations.Source: B. Recovery Quality and Diversity · p. 5
  • Repeated recoveries from one initial configuration produce distinct trajectories, turning directions, and rise profiles, whereas VIGOR repeats one route and FIRM rarely completes stable recovery in these tests.The learned prior captures diverse and reliable recovery strategies.Source: B. Recovery Quality and Diversity · p. 5

C. Motion Prediction Accuracy

  • The prediction–execution test repeats 32 initial configurations on flat and uneven terrain, comparing pelvis, hand, and sole motion at matched times within replanning intervals.Source: C. Motion Prediction Accuracy · p. 5
  • Predicted and executed pelvis heading, height, and position agree closely, while hand and sole deviations are larger, especially on uneven terrain.This supports steering body motion through the predicted trajectory while the decoder adjusts limb motion to local support.Source: C. Motion Prediction Accuracy · p. 5

D. Test-Time Steering of Recovery

Task costs steer recovery toward desired headings, rise speeds, or paths without retraining. Steering remains more difficult on uneven terrain, where accurate path tracking may not produce a successful recovery.

D. Test-Time Steering of Recovery

  • The evaluation tests heading, speed, and path objectives on flat ground, stairs, and a slope with a riser, without retraining.Each objective and terrain uses two targets from a shared initial configuration, with 100 trials per target; paths A and B are terrain-specific references.Source: D. Test-Time Steering of Recovery · p. 6
  • Task guidance produces different headings, rise speeds, and paths from the same initial configuration while maintaining high success across tested terrains.Slower rise targets have lower errors and higher success; faster rises may be harder to control as momentum and support contacts change rapidly.Source: D. Test-Time Steering of Recovery · p. 6, Table IV · p. 6
  • On uneven terrain, heading errors increase on stairs and the slope, and rise-time errors are largest on the slope.Path errors remain close to flat-ground values, but recovery succeeds less often, so accurate path tracking alone does not ensure stable recovery.Source: D. Test-Time Steering of Recovery · p. 6

E. Role of Visual Feedback

Visual feedback during planning and execution supports recovery on uneven terrain. In comparisons with joint generation, state-first generation lowers tracking errors for all tested objectives, while longer replanning intervals increase errors.

E. Role of Visual Feedback

  • Removing depth from the planner reduces recovery success on uneven terrain even when the decoder still observes the scene.Removing decoder depth further reduces success; both blind variants show greater displacement, peak torque, and joint-limit excess.Source: E. Role of Visual Feedback · p. 6

F. State-First Generation and Replanning

  • State-first generation achieves lower tracking errors for heading, speed, and path objectives, and higher recovery success at the default replanning interval.At the longest interval, heading errors are similar, yet state-first completes recovery more often.Source: F. State-First Generation and Replanning · p. 7
  • Longer replanning intervals increase tracking errors for both models, while state-first heading success increases.Frequent replanning improves tracking without consistently improving recovery success.Source: F. State-First Generation and Replanning · p. 7
  • Beside a wall, depth observations change the recovery route away from the obstacle, while guidance can direct the robot toward the open side from the start.Blind recovery collides with the wall.Source: Figure 11 · p. 7

G. Real-World Experiments

Real-world demonstrations on a Unitree G1 show diverse, steered, and terrain-aware recoveries. The conclusion notes that on uneven terrain accurate path tracking does not always yield successful recovery.

G. Real-World Experiments

  • The real-world setup uses a 29-DoF Unitree G1 with proprioception and egocentric depth, offboard inference, and a 50 Hz decoder.DLIO estimates pelvis motion from LiDAR and IMU measurements.Source: G. Real-World Experiments · p. 7
  • From a shared supine posture, resampling diffusion noise produces left-turning, in-place, or right-turning recoveries; unguided flat-ground trials from prone and supine starts all succeed.Repeated trials reach different paths and final headings.Source: G. Real-World Experiments · p. 7
  • Guidance shapes recovery direction and route from prone and supine starts, with most flat-ground steering trials and all near-wall trials recovering successfully.Demonstrations include route steering beside a wall and across raised platforms.Source: G. Real-World Experiments · p. 8
  • On staggered steps, the robot adjusts hand and foot support while rising, and unguided recovery succeeds in most tested trials.Steering adds a challenge when requested directions or paths compete with the contacts needed to rise.Source: G. Real-World Experiments · p. 8

V. CONCLUSION AND FUTURE WORK

  • RISE steers predicted motion intent before generating motion latents for feedback execution, and simulation supports heading, speed, and path steering without retraining.The conclusion identifies a remaining gap: accurate path tracking on uneven terrain does not always lead to successful recovery.Source: §V · p. 8

Related papers

Loading 2610.30061v1…