Source-linked AI summary
Predictive Safety Curricula for Robust Legged Locomotion
Ivan Ovinnikov, Pascal Sutter, Christian Gehring, Jordis Herrmann
TL;DR
Rare safety failures can persist in legged-robot policies even when average task performance is high, and standard curricula do not explicitly target safety risk. PSC allocates experience using predicted future safety cost, and achieves the highest mean reliability across evaluated ANYmal-D conditions while improving production stair-climbing success over comparison curricula.
Problem
Standard locomotion curricula prioritize task difficulty, average performance, or learning progress rather than explicitly targeting elevated safety risk, while sparse failures make empirical estimates across many configurations data-inefficient.
Source: §1 · p. 1Method
PSC trains a shared distributional safety critic from policy rollouts and uses predicted future safety cost to prioritize terrain contexts and randomized events without changing the task reward or loss.
Source: §1 · p. 1, §2.2 · p. 3Results
PSC achieves the highest mean reliability across evaluated ANYmal-D terrain and observation conditions and improves average stair-climbing success over the baseline, PLR, and LP curricula.
Source: §3.3 · p. 7, §5 · p. 9Takeaways & Limitations
Predictions of future safety exposure provide a shared signal for allocating training experience across terrain contexts and randomized events.
Source: §5 · p. 9PSC relies on a hand-specified safety cost and can adapt experience only through environment variables exposed to the curriculum.
Source: §5 · p. 9
Abstract
from arXiv · showhide
Rare but consequential failures can persist in learned locomotion policies for legged robots even when average task performance is high, in part because standard curricula primarily adapt task difficulty rather than the distribution of safety-critical experience. We introduce Predictive Safety Curricula (PSC), a framework for allocating locomotion training experience using learned predictions of future safety cost. PSC trains a distributional safety critic from policy rollouts and uses its predictions to prioritize both terrain contexts and previously encountered randomized events. The resulting curriculum modifies the training distribution while leaving the task reward and policy-optimization loss unchanged. We evaluate PSC in controlled rough-terrain locomotion and in production locomotion systems. PSC improves reliability relative to standard terrain progression, advantage-based replay, and learning-progress curricula, with the largest gains on difficult terrain and under degraded observations. The same allocation principle transfers to two production locomotion stacks. On ANYmal-D hardware, PSC reduces shank-collision incidence by $63\%$ relative to the learning-progress curriculum across three matched training seeds, with a reduction in every seed. On a production stair-climbing platform, PSC eliminates observed shank collisions in the evaluated hardware trials. These results show that learned predictions of future safety cost can provide an effective signal for allocating training experience toward rare failure modes and improving locomotion reliability.
1 Introduction
Learned locomotion policies can remain unsafe in rare, consequential conditions despite strong average performance, while standard curricula emphasize task difficulty or learning progress. Predictive Safety Curricula (PSC) use predicted future safety cost to allocate training experience without changing the task reward or policy-optimization loss.
1 Introduction
- Rare lower-leg collisions and loss of balance can persist despite strong average performance, while standard curricula do not explicitly target elevated safety risk.Sparse failures and growing terrain and randomization spaces make reliable empirical safety estimates data-inefficient.Source: §1 · p. 1
- PSC learns future safety cost from policy rollouts and uses predictions to prioritize subsequent training experience while leaving the task reward and loss unchanged.Its learned model supports expected-cost and upper-tail prioritization across terrain and randomized environment variation.Source: §1 · p. 1
- PSC is evaluated in controlled rough-terrain locomotion and production settings, with reported reliability gains and reduced lower-leg contacts in stair climbing.The evaluation includes controlled curriculum ablations and critic diagnostics, plus repeated hardware trials with contact measurements.Source: §1 · p. 2
2 Method
PSC uses predicted future safety exposure to reallocate training across structured terrain contexts and randomized events. It trains a shared distributional safety critic while leaving the task reward and policy-optimization objective unchanged.
2 Method
- PSC adapts both structured-context allocation and randomized-event replay using learned predictions of future safety exposure.The two allocation pathways modify the training-condition distribution while allowing an existing capability curriculum to determine which structured contexts are available.Source: §2 · p. 2, §2.5 · p. 4
2.1 Training Conditions and Safety Returns
- Training conditions factor into terrain, difficulty, and optional command contexts plus randomized events, while the auxiliary safety cost trains a critic rather than changing the task reward or PPO objective.The safety return represents cumulative future safety exposure and supplies the critic's prediction target.Source: §2.1 · pp. 2–3
2.2 Shared Distributional Safety Critic
- A shared critic predicts conditional safety-return quantiles from rollout features across the full training distribution, with temporal bootstrapping propagating outcomes to preceding states.The quantile representation supports both mean and upper-tail statistics from the same critic.Source: §2.2 · p. 3
2.3 From Safety Prediction to Curriculum Priority
- PSC averages a selected state-level safety readout along each episode, then uses the resulting episode score for context priorities and event replay.The upper-tail readout is used for PSC; the mean readout and direct empirical CVaR are comparison alternatives.Source: §2.3 · p. 4
2.4 Safety-Prioritized Context Allocation
- Context allocation combines safety priorities with uniform exploration, while the underlying capability curriculum limits sampling to admitted level–command contexts.Terrain-level staleness and coverage also contribute to context priority; uninformative priorities yield uniform sampling over the admissible support.Source: §2.4 · p. 4
2.5 Safety-Prioritized Event Replay
- Event replay retains previously sampled randomized realizations within each context and preferentially resamples them according to their episode safety scores.PSC can instead draw a new realization, preserving exploration and providing a fallback when no replay samples are available.Source: §2.5 · p. 4
3 Experiments
Experiments progress from controlled ANYmal-D rough-terrain comparisons and safety-prediction ablations to ANYmal-X stair climbing and hardware evaluation. Across these settings, the study compares PSC with baseline curricula and replay-based alternatives.
3.1 ANYmal-D Rough-Terrain Locomotion
- Table 1 reports PSC’s highest mean success across six clean and observation-noise conditions; under clean terrain, its largest LP advantage is 1.38 percentage points at v = 2.Relative to LP, PSC improves success by 0.16 and 0.50 percentage points at clean v = 1 and v = 3; under noisy observations, the gains are 0.55, 1.33, and 0.84 points at v = 1, 2, and 3.Source: §3.1 · pp. 5–6, Table 1 · p. 6
3.2 Safety Prediction and Curriculum Ablations
- 93.05 ± 1.14%: QC-CVaR mean success, exceeding QC-mean at 90.40 ± 2.54% and empirical CVaR at 91.38 ± 3.17%.The critic’s mean prediction correlates with discounted safety return at Spearman ρ = 0.773; its highest-scoring decile has 6.88× the overall collision rate and 2.81× the thigh-or-shank contact rate. Among tested tail masses, α = 0.05 has the highest mean reliability; broadening the readout lowers performance, while α = 0.01 remains closer to PSC.Source: §3.2 · pp. 6–7, Table 2 · p. 7, Table 3 · p. 7
3.3 ANYmal-X Stair Climbing
- 88.45%: PSC’s average teacher success across ANYmal-X stair settings, versus 86.71% for baseline, 86.92% for PLR, and 87.04% for LP.The PSC advantage is largely retained after distillation: the student reaches 87.76 ± 1.70% mean success, and a student distilled without PSC prioritization during distillation reaches 87.52 ± 1.39%.Source: Table 4 · p. 7, §3.3 · p. 7
3.4 Hardware Evaluation
- Hardware evaluation pairs ANYmal-D open-step traversals (100 one-way crossings per seed and method) with production stair trials (50 ascent–descent pairs per policy); Figure 2 contrasts LP and PSC traversals.The figure shows five-frame sequences from matched-view recordings, with red circles marking annotated shank contacts; the supplied results here specify evaluation scope but not outcome values.Source: Table 5 · p. 8, Table 6 · p. 8, Figure 2 · p. 8
4 Related Work
Prior work adapts curricula through task difficulty, learning progress, or failure-related signals, while safety critics typically shape policy behavior or optimization. PSC instead uses predicted future safety exposure to allocate subsequent training experience.
Curriculum learning and adaptive environment sampling
- Curriculum and adaptive-sampling methods prioritize task difficulty, learning progress, uncertainty, or learning potential, whereas PSC reallocates experience using predicted future safety exposure [9] [10] [1] [2] [11] [3] [12] [13] [14] [4] [15] [16].ADD uses a CVaR-derived regret signal to guide a diffusion-based environment generator [16].Source: §4 · pp. 8–9
Risk-aware curricula and failure-driven sampling
- Risk-aware curricula sample low-return or adverse contexts, but PSC learns a shared distribution of future safety cost from rollout states to allocate later experience [18] [19] [21] [22].Related work also uses failure predictors to search for rare evaluation failures and TACL to generate capability-matched trajectories [21] [22].Source: §4 · p. 9
Safety critics and distributional risk
- Unlike safety critics used to constrain policy updates or guide recovery, PSC uses distributional safety predictions to select simulated experience without changing the task reward or PPO loss [23] [24] [25] [30].PSC builds on distributional RL and CVaR [5] [27] [28], but does not use its safety model at deployment.Source: §4 · p. 9
5 Conclusion
PSC allocates locomotion training experience using predicted future safety exposure, and the paper reports reliability gains across evaluated robot settings. The authors also identify limits in its fixed safety cost and predefined environment parameterization.
5 Conclusion
- PSC achieves the highest mean reliability among compared curricula across evaluated ANYmal-D terrain and observation conditions, with ablations favoring learned predictive priorities over empirical tail statistics.Critic diagnostics show that learned scores rank realized safety return and enrich subsequent collision and contact events.Source: §5 · p. 9
- On ANYmal-D hardware, PSC reduces shank-collision incidence by 63% relative to the learning-progress curriculum across three matched seeds, with a reduction in every seed.On the production stair-climbing platform, no shank collisions were observed with PSC in evaluated trials, compared with 33/50 shank-positive baseline trials at similar observed mean traversal speeds.Source: §5 · p. 9
- PSC relies on a hand-specified safety cost and can adapt experience only through exposed environment variables, so untargeted interactions such as foot scuffs may remain outside its safety signal.The authors propose richer multi-mode safety predictions and active discovery or generation of failure conditions as future directions.Source: §5 · p. 9