PaperlayerView exact revision on arXiv

Source-linked AI summary

Generalizing Safety Beyond Collision-Avoidance via Latent-Space Reachability Analysis

Kensuke Nakamura, Lasse Peters, Andrea Bajcsy

arXiv:2502.00935v3cs.ROcs.LG

TL;DR

Robotic safety methods have largely focused on hand-engineered collision avoidance, leaving failures such as spills difficult to specify. The paper introduces Latent Safety Filters, which perform HJ reachability in a generative world model’s latent space and learn failure constraints as classification. Experiments report strong safety monitoring and reduced violations while filtering imitation policies and teleoperation.

  • Problem

    HJ reachability has been practically limited by hand-engineered collision-avoidance constraints and low-dimensional state representations, despite robots facing harder-to-specify failures such as spills.

  • Method

    Latent Safety Filters use a generative world model to analyze raw observations in latent space, classify unsafe states, and approximate HJ reachability for safety-preserving action filtering.

  • Results

    F1 score : 0.984, and the safety controller provides a 63.6% safety failure violation decrease over an imitation-trained base policy.

  • Takeaways & Limitations

    The approach safeguards imitation-learned policies and human teleoperation against complex hazards, including spilling from an opened bag, in simulation and hardware.

  • Takeaways & Limitations

    The safety filter can protect only against outcomes the world model can predict, requiring some unsafe data for effective prediction and control.

Abstract

from arXiv · show

Hamilton-Jacobi (HJ) reachability is a rigorous mathematical framework that enables robots to simultaneously detect unsafe states and generate actions that prevent future failures. While in theory, HJ reachability can synthesize safe controllers for nonlinear systems and nonconvex constraints, in practice, it has been limited to hand-engineered collision-avoidance constraints modeled via low-dimensional state-space representations and first-principles dynamics. In this work, our goal is to generalize safe robot controllers to prevent failures that are hard--if not impossible--to write down by hand, but can be intuitively identified from high-dimensional observations: for example, spilling the contents of a bag. We propose Latent Safety Filters, a latent-space generalization of HJ reachability that tractably operates directly on raw observation data (e.g., RGB images) to automatically compute safety-preserving actions without explicit recovery demonstrations by performing safety analysis in the latent embedding space of a generative world model. Our method leverages diverse robot observation-action data of varying quality (including successes, random exploration, and unsafe demonstrations) to learn a world model. Constraint specification is then transformed into a classification problem in the latent space of the learned world model. In simulation and hardware experiments, we compute an approximation of Latent Safety Filters to safeguard arbitrary policies (from imitation- learned policies to direct teleoperation) from complex safety hazards, like preventing a Franka Research 3 manipulator from spilling the contents of a bag or toppling cluttered objects.

I. INTRODUCTION

The paper extends HJ reachability beyond collision avoidance by analyzing hard-to-model failures in learned latent spaces. Latent Safety Filters use world-model representations to classify failures and automatically filter unsafe actions across simulation and hardware tasks.

  • Open-world robot safety includes nuanced failures such as overflowing liquids, breaking dishes, and spilling bag contents, not only collisions.
  • HJ reachability identifies states doomed to failure and computes safety-preserving policies, but practical robotics applications have struggled with non-collision constraints.
  • F1 score : 0.984 and a 63.6% safety failure violation decrease over an imitation-trained base policy demonstrate the approach’s reported safety-monitoring and control performance.
  • Latent Safety Filters transform hard-to-model constraint specification into latent-space classification while operating on high-dimensional observations without explicit recovery demonstrations.
  • The method computes tractable reachability approximations in a world model’s latent imagination, reducing the need for additional unsafe online interaction data.
  • Across navigation, cluttered manipulation, and hardware tasks, Latent Safety Filters steer policies away from failures while minimizing incompletion more effectively than soft constraints or constrained MDPs.
  • Hardware experiments show minimal teleoperation correction, preservation of a performant imitation policy, safer suboptimal imitation, and generalization across bag colors and backgrounds.

III. LATENT SAFETY FILTERS

Latent Safety Filters perform HJ-style safety analysis in a generative world model’s latent space, using learned failure classifiers and predicted transitions. A discounted learning approximation makes the high-dimensional latent reachability problem tractable, but assurances remain tied to training-data coverage.

  • The framework maps raw observations such as RGB images into a world-model latent space for safety analysis and converts nuanced constraints into classification.
  • World models encode observations into latent states and predict future latent states conditioned on actions, without requiring privileged state access or a specific architecture.
  • The failure classifier learns a latent unsafe set from labeled safe and unsafe observations, representing failures as Flatent = {z | ℓµ(z) < 0}.
  • Latent reachability replaces analytic dynamics or high-fidelity simulation with a world model’s latent imagination, including expected transitions for stochastic models.
  • Exact latent reachability is intractable in high dimensions, so the method adds γ ∈[0, 1) discounting to obtain a learning-based approximation with a convergent Bellman backup.
  • The resulting safety assurance applies only to outcomes represented by the data used to train the world model and failure classifier.

IV. SIMULATION RESULTS

Simulation experiments evaluate whether latent safety filters can recover performant safety-preserving policies from partial observations across increasingly complex tasks and safety specifications.

  • The simulations assess latent safety filters on two vision-based tasks without access to ground-truth dynamics, states, or constraints.

A. How Close Does Latent Safety Get to Privileged Safety?

The latent safety filter is evaluated against privileged safety on a Dubins’ car collision-avoidance benchmark, using learned observation representations rather than ground-truth state and dynamics. Its image-based policy closely matches the privileged baseline, while incomplete action coverage makes its unsafe-set estimate pessimistic.

  • Setup: The benchmark compares latent and privileged HJ safety filters on a discrete-time Dubins’ car avoiding a circular obstacle.The robot state is [px, py, θ], with three discrete angular-velocity actions and a ground-truth obstacle radius of 0.5 m.
  • Setup: LatentSafe learns its safety components from offline observation-action data, whereas PrivilegedSafe uses ground-truth state, dynamics, and margin information.The latent setup trains a world model from 2,000 RGB-image and heading trajectories, while the privileged baseline uses the known model and state.
  • Results: The runtime monitor evaluates alignment between latent or privileged value-function signs and the exact ground-truth value function.The benchmark reports monitor classification quality over all three Dubins’ car state dimensions.
  • Results: LatentSafe maintains safety for 240/250 (96%) initial states, compared with 246/250 (98.4%) for PrivilegedSafe.The evaluation uses initial states where the exact state-based controller can guarantee safety and measures each learned policy’s ability to remain outside the failure set.
  • Ablation: With a world model trained only on straight and left-turn actions, LatentSafe pessimistically labels states requiring a right turn as unsafe.The ablation shows that incomplete action coverage biases the learned model’s understanding of which states can be kept safe.

B. Can Latent Safety Scale to Visual Manipulation?

In visual manipulation, Latent Safety Filters use latent-space reachability to identify when interactions will inevitably topple obstacles and to filter task actions while preserving completion. The approach is evaluated against constrained-MDP filtering in simulation.

  • Safety specification: The task permits touching, pushing, and tilting red blocks, but treats either block toppling as a safety failure.The experiment uses privileged simulator information only to determine whether a block is within 1 radian of the ground plane.
  • Compared methods: LatentSafe uses the same world model and failure classifier as SQRL but applies a different safety-filtering objective.SQRL learns a risk critic and filters actions above a manually tuned risk threshold.
  • Qualitative results: The latent unsafe set detects that the robot is doomed at t = 14, whereas the failure classifier identifies failure only after the blocks fully fall at t = 28.This distinguishes presently visible failure from states that already guarantee future failure.
  • Quantitative evaluation: In 50 randomized rollouts, the study reports success, constraint violation, and incompletion rates for unshielded Dreamer, LatentSafe, and SQRL.Success requires lifting the green block without toppling a red block; incompletion means avoiding violation but failing to lift the block.
  • Quantitative results: SQRL’s incompletion rate rises from 4% at ϵrisk = 0.1 to 70% at ϵrisk = 0.05, while LatentSafe overrides the nominal policy only when needed.The authors report that SQRL marginally improves safety despite this sensitivity, whereas LatentSafe significantly reduces constraint violations while still succeeding.

V. HARDWARE RESULTS: PREVENTING HARD-TO-MODEL ROBOT FAILURES

Hardware experiments test whether Latent Safety Filters can prevent spilling from an opened bag using only visual and proprioceptive observations. LatentSafe overrides unsafe teleoperator motions while allowing actions judged safe.

  • Challenge: The problem is partially observed because the robot cannot directly recover the Skittles’ positions inside the bag from image observations and proprioception.Even privileged-state modeling would require characterizing complex interactions among the manipulator, bag, and contents.
  • Setup: The world model uses third-person and wrist-mounted camera views, while training data contain random actions, non-spilling demonstrations, and spilling demonstrations.The dataset includes 1,300 trajectories: 1,000 random, 150 successful, and 150 spilling trajectories.
  • Results: LatentSafe prevents a sharp pull-up motion that would spill the Skittles and also slows side-to-side motion while allowing safe open-end lifting.The filter therefore distinguishes among different bag-manipulation motions rather than blocking all interaction.

A. Shielding Human Teleoperators

LatentSafe shields human teleoperation by overriding motions predicted to cause spills while allowing safe grasps and movements to proceed. The filter operates from one-step world-model rollouts at 15 Hz.

  • The safety filter uses a one-step world-model rollout of the unshielded teleoperator’s proposed action and operates at 15 Hz.Hardware experiments use ϵ = 0.3.
  • LatentSafe automatically overrides a teleoperator’s sharp pull-up motion that would spill an opened bag of Skittles.
  • DiffusionAdv frequently spills approximately 85% of the bag, whereas DiffusionAdv + LatentSafe spills less than 5% in all but one constraint-violating rollout.
  • LatentSafe does not activate for a safe grasp at the bag’s top, allowing the teleoperator to complete the task safely.

B. Shielding Autonomous Imitation-Learned Policies

The same Latent Safety Filter is evaluated on weak and strong diffusion imitation-learned policies. It substantially reduces failures for the weak policy while preserving the strong policy’s performance without unnecessary overrides.

  • Shielding a Weak IL Policy: LatentSafe reduces DiffusionAdv constraint violations by 63.6% compared with DiffusionAdv alone.LatentSafe still fails 26.4% of the time, but three of four failures spill only one skittle.
  • Shielding a Weak IL Policy: DiffusionAdv alone spills in 73.4% of autonomous trials and frequently spills approximately 85% of the bag.
  • Shielding a Weak IL Policy: LatentSafe minimizes both failure rate and spill severity when the base imitation-learned policy is erroneous.
  • Shielding a Strong IL Policy: DiffusionOpt and DiffusionOpt + LatentSafe each exhibit one violation involving a single spilled skittle across the first 15 trials.
  • Shielding a Strong IL Policy: The same Latent Safety Filter can shield both weak and strong base policies without changing the safety representation or fallback controller.

C. Testing Out-of-Distribution Generalization of Latent Safety

LatentSafe generalizes from a red Skittles bag and wooden background to differently colored Skittles bags and a novel background, but not reliably to dynamically different M&M bags.

  • The filter is trained only on a red Skittles bag with a wooden tabletop background and is tested by shielding a replayed unsafe demonstration.
  • M&M bags are both visually and dynamically out of distribution, appearing stiffer and more papery than Skittles bags.
  • LatentSafe prevents spills for green and purple Skittles bags and a novel background, with similar filtering behavior across bag colors.
  • LatentSafe fails to prevent spills for Classic and Peanut M&M bags, even though it activates before spilling for the brown Classic bag.The authors attribute the gap to transition-model generalization across different bag dynamics.

VI. RELATED WORK

The work extends safety filtering and latent-space control by combining backward HJ reachability with generative world-model representations. This enables runtime monitoring and recovery for failures that are difficult to specify in handcrafted state spaces, while retaining important data and model limitations.

  • Safety Filtering for Robotics: Safety filters monitor task policies and override them with safe actions when the system approaches an unsafe state.
  • Latent Space Control: The approach uses backward reachability to identify states that inevitably lead to failure and compute both a runtime monitor and recovery policy.
  • Computing Safety Behaviors from Offline Data: Unlike approaches restricted to privileged states or handcrafted specifications, latent HJ reachability broadens the failures robots can safeguard against.
  • Limitations: The safety filter can protect only against outcomes that the world model predicts, requiring some unsafe data to model those outcomes effectively.
  • Conclusion: The conclusion reports protection against hard-to-specify spills in simulation and hardware for generative imitation-learned policies and human teleoperation.

APPENDIX

The appendix documents the world-model, reachability-learning, and policy-training configurations used across simulation and hardware experiments.

  • World-model configurations: The RSSM represents latent state as z := {h_t, x_t}, combining deterministic recurrent and stochastic components.It is trained with latent-consistency and observation-reconstruction objectives.
  • Hyperparameter references: The appendix provides separate hyperparameter tables for Dreamer, DINO-WM, DDQN HJ Reachability, DDPG HJ Reachability, and Diffusion Policy.These tables organize the configurations referenced across the experiments.
  • World-model configurations: DINO-WM uses frozen DINOv2 patch tokens and a transformer to predict future tokens from recent actions and observations.The model also encodes two camera views, actions, and proprioception for next-step prediction.
  • Reachability and policy training: The experiments use DDQN for discrete Dubins-car actions and DDPG for continuous simulation and hardware manipulation actions.The listed implementations use a three-layer MLP for the Dubins-car Q-function and standard discounted Bellman training for DDPG.
  • Reachability learning: The reachability classifier applies tanh to its output to smooth the loss landscape while preserving similar treatment of sufficiently safe or unsafe states.The thresholding procedure evaluates these states similarly through tanh(ℓ_μ(z)).
  • Generalization evidence: DINOv2 PCA visualizations represent bags similarly despite changes in bag or background color, supporting qualitative cross-instance consistency in the embedding space.The appendix presents this as qualitative evidence rather than an in-depth generalization study.

B. Common Questions

The common-questions section explains how safety labels are obtained and qualifies the evidence for policy generalization across visually different but semantically equivalent instances.

  • How labels are obtained: Hardware safety labels identify one transition frame after which all observations violate the constraint, while earlier observations remain safe.This labeling procedure covered 1,300 trajectories and took the lead author about 2.5 hours.
  • How labels are obtained: Manual labeling is currently intensive, although visually apparent failures may be easier for non-experts to label than functional failure representations.The authors suggest foundation models could eventually automate annotation and that only a smaller data subset may need labeling.
  • How the policy generalizes: The paper does not provide an in-depth study of world-model or policy generalization capabilities.Instead, it offers qualitative evidence concerning DINOv2 representations across semantically equivalent instances.
  • How the policy generalizes: Across different bag and background colors, DINOv2 PCA visualizations show candy bags represented consistently, suggesting similar embeddings.The evidence is limited to the first three PCA components and does not establish broader generalization behavior.
Loading 2502.00935v3…