PaperlayerView exact revision on arXiv

Source-linked AI summary

CrossSafe: Towards Cross-Embodiment Latent Safety Filters

Ihab Tabbara, Yuxuan Yang, Hussein Sibai

arXiv:2609.28984v1cs.ROcs.AIcs.LGcs.SYeess.SY

TL;DR

Generalist manipulation policies often use common task-space actions, yet those actions can produce different whole-body motions and safety outcomes across robot morphologies. CrossSafe learns a shared HJ safety filter over a morphology-aware latent representation; it reduces nominal collision rates in-distribution and on a held-out embodiment.

  • Problem

    Common task-space actions can induce different whole-body motions across robot morphologies, making the same action safe for one embodiment but unsafe for another.

    Source: §I · p. 1

  • Method

    CrossSafe jointly learns a shared Hamilton–Jacobi safety value function and safety-maximizing policy over a morphology-aware latent state.

    Source: §I · pp. 1–2

  • Results

    CrossSafe reduces nominal-planner collision rate (CR) from 64.1% to 39.8% on in-distribution embodiments (InD) and to 49.8% on the out-of-distribution embodiment (OOD).

    Source: §I · p. 2, §VI · p. 7

  • Takeaways & Limitations

    A shared safety filter can generalize without finetuning to embodiments excluded from safety-filter training, and training on more embodiments improves generalization.

    Source: §I · p. 2

Abstract

from arXiv · show

Cross-embodiment learning has shown that a single model, such as a vision-language-action (VLA) model, can learn state representations and manipulation skills that can be applied across heterogeneous robots to accomplish various tasks. We hypothesize that the same holds for safety enforcement. The reasoning required to satisfy a safety constraint, such as detecting an obstacle, recognizing that it should be avoided, and selecting a safe abstract action, is largely shared across robots. What differs across embodiments is how the abstract safe action is realized: morphology, kinematics, and dynamics determine which actions are safe and feasible. Consequently, the same action can be safe for one robot and unsafe for another. This is especially important for generalist manipulation policies that operate in a common end-effector action space without explicitly capturing how safety depends on the robot's morphology and kinematics. We propose embodiment-conditioned safety filtering, in which a Hamilton-Jacobi reachability-based value function and its corresponding safety-maximizing policy are shared across robots. Using a morphology-aware latent representation of the robot and its environment, we perform Hamilton-Jacobi reachability analysis directly in latent space so that the learned safety concepts can generalize across embodiments while remaining explicitly conditioned on each robot's morphology and kinematics. We evaluate our approach across five bimanual robot embodiments and five manipulation tasks with whole-body collision-avoidance constraints. Our results show that a single policy, jointly trained across five manipulation tasks and four embodiments, exhibits zero-shot generalization to a held-out embodiment, reducing the nominal policy's collision rate. They also show that training using more embodiments improves generalization.

I. INTRODUCTION

CrossSafe asks whether safety filters can share learned reasoning across robot embodiments while preserving morphology-specific information needed to choose safe actions. It combines a shared Hamilton–Jacobi safety filter with a morphology-aware latent representation and reports zero-shot transfer to held-out embodiments.

I. INTRODUCTION

  • The same end-effector action can produce different whole-body motions across robots, so an action safe for one embodiment may be unsafe for another.Differences in link lengths, joint axes, joint limits, and configurations affect the safe control needed to avoid failure.Source: §I · p. 1
  • Existing safeguards either correct a given policy using embodiment-specific costs or compute safe actions in each robot’s native configuration space, rather than jointly learning a shared safety value function and policy.CrossSafe targets embodiment-conditioned safety filtering with one learned HJ value function.Source: §I · p. 1
  • CrossSafe learns a shared Hamilton–Jacobi value function and safety-maximizing policy over a latent state that preserves safety-relevant information about each robot’s morphology.The representation augments HoloBrain-0 with per-link safety features and geometry-aware attention to scene patches.Source: §I · pp. 1–2
  • CrossSafe reduces nominal collision rates on unseen embodiments without finetuning, and training across four embodiments improves generalization over single-embodiment training.The evaluation spans five bimanual embodiments and five tasks.Source: §I · p. 2

A. Cross-embodiment generalist policies

  • Cross-embodiment policies share task knowledge through common action representations, but zero-shot task representations alone may omit morphology information needed for safety-critical control.The related-work discussion contrasts embodiment-invariant task representations with safety needs that depend on robot morphology.Source: A. Cross-embodiment generalist policies · p. 2

B. Latent safety filters and safe control

Prior latent safety-filter work extends safety tools across observations, constraints, environments, or specifications, but does not train a shared safety value function explicitly conditioned on robot embodiments. The section reviews Hamilton–Jacobi reachability and its learned actor–critic approximation.

B. Latent safety filters and safe control

  • Related safety-filter advances generalize across observations, constraints, environments, or specifications, but not through a shared value function explicitly conditioned on robot embodiments.EmbodiSteer uses embodiment-specific kinematics and collision costs, while Any-Body Guard recomputes safe sets from each robot’s kinematic model at runtime.Source: B. Latent safety filters and safe… · p. 3

III. PRELIMINARIES

  • Hamilton–Jacobi reachability computes an optimal value function for avoiding a failure set, whose negative-valued states form the backward reachable set of inevitably unsafe states.The value function is characterized as a fixed point of a Bellman equation.Source: §III · p. 3
  • The HJ policy selects the action maximizing the value of the next state, thereby seeking to avoid the failure set.The policy is defined as an argmax over actions of the value at the resulting state.Source: §III · p. 3
  • Reinforcement-learning actor–critic methods approximate HJ reachability for higher-dimensional systems with continuous actions by learning a safety critic from rollout transitions.The stated target combines the current failure function with a discounted next-state critic value.Source: §III · p. 3

IV. MORPHOLOGY-AWARE LATENT SAFETY FILTERING

CrossSafe shares an HJ safety critic and policy over a morphology-aware latent state that can accommodate embodiments with different numbers of links and joints. It supplements pretrained robot tokens with per-link safety features, including motion and normalized geometry.

IV. MORPHOLOGY-AWARE LATENT SAFETY FILTERING

  • CrossSafe’s shared HJ critic and safe policy operate on a morphology-aware latent state and are designed to handle embodiments with different numbers of links and joints.The encoder builds on frozen pretrained HoloBrain-0 vision and robot-state encoders.Source: §IV · p. 3

A. Frozen HoloBrain-0

  • HoloBrain-0 produces scene tokens for the environment and robot tokens for link configurations, using camera geometry and kinematic structure.Its robot-state encoder embeds link poses and gripper states while using attention informed by the robot’s kinematic graph.Source: A. Frozen HoloBrain-0 · p. 3

B. Augmenting safety-related features

  • Because HoloBrain-0’s single-observation robot tokens omit link velocities, CrossSafe augments each token with safety-related motion and morphology features.The listed features include linear and angular velocity, scale-normalized position, orientation, normalized kinematic-tree depth, and arm identity.Source: B. Augmenting safety-related features · p. 4
  • Link velocities are expressed in a common world frame because identical joint velocities can produce different link motions across morphologies.Positions are measured relative to the arm base and normalized by reach, while normalized depth supports comparison across embodiments.Source: B. Augmenting safety-related features · p. 4
  • The method combines frozen link tokens with safety-related features to create the sequence used as the latent state for filtering.The safety features are constructed per arm from its URDF and measured joint angles and velocities.Source: B. Augmenting safety-related features · p. 4

C. Geometry-aware manipulator–scene attention

CrossSafe connects per-link robot tokens to scene tokens through geometry-aware cross-attention, then propagates information across link tokens. The attention incorporates link-to-patch distance and direction, preserving directional information alongside scene features.

C. Geometry-aware manipulator–scene attention

  • Cross-attention lets each link token incorporate scene content, weighting patches by both feature similarity and their relative position to that link.Link queries attend to scene keys and values, while scene tokens remain unchanged.Source: C. Geometry-aware manipulator–scene… · p. 4
  • The attention-weighted direction from each link to scene patches is retained alongside scene features, explicitly preserving directional information in the latent representation.The geometry-aware attention bias uses the distance and direction between a link and a scene patch.Source: C. Geometry-aware manipulator–scene… · p. 4
  • Self-attention propagates information across link tokens after the cross-attention updates, yielding the latent state.The method applies three blocks of cross-attention, self-attention, and feed-forward processing.Source: C. Geometry-aware manipulator–scene… · p. 4

D. Cross-embodiment safety critic and safe policy

The safe policy maps latent states to joint actions, while the critic evaluates candidate actions using action-conditioned link and scene representations. Shared token processing produces link safety scores that are aggregated with a soft minimum.

D. Cross-embodiment safety critic and safe policy

  • The safe policy maps latent state to per-joint Gaussian actions, then applies tanh and each joint’s action limit; grippers remain under nominal control.A shared linear head outputs each joint’s mean and log standard deviation.Source: D. Cross-embodiment safety critic and… · p. 5
  • The critic augments each link token with its candidate joint displacement, Jacobian-based linear and angular motion, and a controlled-joint indicator before projection into token space.The action-conditioned features form an 8D vector; gripper tokens retain predicted arm motion but have zero joint displacement and control indicator.Source: D. Cross-embodiment safety critic and… · p. 5
  • Geometry-aware cross-attention uses post-action link positions to relate links to scene patches, then shared token processing yields link safety scores aggregated by a soft minimum.Self-attention shares scene information across link tokens.Source: D. Cross-embodiment safety critic and… · p. 5

A. Experimental setup

The evaluation uses five bimanual robot embodiments and five manipulation tasks with a shared obstacle-based collision-avoidance constraint. Models train jointly across tasks and selected embodiments using warmup trajectories and online data.

A. Experimental setup

  • Five bimanual embodiments and five tasks are evaluated with the same static obstacle, placed along the nominal path in every evaluation scene.During training, the obstacle is on the nominal path with probability 0.7; the nominal cuRobo planner excludes it.Source: A. Experimental setup · p. 5
  • Collision constraint h(s) is the minimum signed distance between the obstacle and robot or grasped object, using enclosing spheres and an obstacle bounding box.The approximations can indicate overlap without physical contact.Source: A. Experimental setup · p. 5

B. Training pipeline

  • Each model trains on all five tasks and its training embodiments, starting from a shared buffer of 500 nominal trajectories and collecting online data at the same rate.The 500 warmup trajectories comprise 20 per embodiment-task pair and include collision and collision-free episodes; models use the same hyperparameters.Source: B. Training pipeline · p. 5
  • Training uses soft actor-critic adapted to HJ reachability, with batch size 64, learning rate 3 × 10−4, discount factor 0.9, and temperature 0.4.Each model trains for two days on one NVIDIA A40 GPU.Source: B. Training pipeline · p. 5

C. Online safety filtering

The online filter evaluates the nominal controller synchronously, while the experiments compare CrossSafe with unfiltered control, ablations, and an X-VLA-based variant. Evaluation reports collision, success, intervention, and force metrics across leave-one-embodiment-out folds and specialist/generalist models.

C. Online safety filtering

  • At 25 Hz, the nominal controller and safety filter run synchronously; the full inference pipeline takes 73.0 ± 1.9 ms on an NVIDIA A40.The encoders take 59.8 ± 1.4 ms and added trainable modules take 13.1 ± 0.3 ms per step.Source: C. Online safety filtering · p. 5

D. Baselines and ablations

  • The baselines include the unfiltered nominal controller and ablations removing safety-feature augmentation, geometry-aware cross-attention, or both.X-VLA-Safe replaces HoloBrain-0 with X-VLA and uses per-arm tokens and end-effector actions converted to joint displacements by inverse kinematics.Source: D. Baselines and ablations · pp. 5–6

E. Metrics and evaluation protocol

  • Fig. 2 names five tasks—bread, container, blocks, burger and fries, and bowls—and five embodiments, from Aloha-AgileX through UR5-WSG.Source: Figure 2 · p. 6
  • Evaluation reports collision rate, success rate, intervention rate, and contact force over 50 episodes per task–embodiment pair.Collision rate measures physical simulator contact, while contact force averages episode-level median force over contact frames and assigns zero to collision-free episodes.Source: E. Metrics and evaluation protocol · p. 6
  • Each method uses five leave-one-embodiment-out folds; each fold trains on four embodiments and evaluates on those four plus its held-out embodiment.Table I aggregates 20 InD and 5 OOD task–embodiment pairs per model; Table II contrasts five generalists with five specialists.Source: E. Metrics and evaluation protocol · p. 6

VI. RESULTS

CrossSafe lowers collision rates against the nominal controller on most evaluated task–embodiment pairs, including held-out embodiments, while results vary by robot and task. Ablations and specialist comparisons indicate that geometry-aware attention, safety-feature augmentation, and training across embodiments matter to generalization.

VI. RESULTS

  • 64.1% nominal CR falls to 39.8% InD and 49.8% OOD with CrossSafe; it lowers CR on 21/25 InD and 19/25 OOD task–embodiment pairs.Force also falls from 179.6 N to 113.6 N InD and 102.9 N OOD.Source: Table I · p. 7, §VI · p. 7
  • Compared with X-VLA-Safe, CrossSafe has lower mean CR and Force and higher mean SR in both InD and OOD evaluations.Against nominal, CrossSafe cuts CR by 37.9% InD and 22.3% OOD, while X-VLA-Safe reduces it by 21.4% and 16.1%, respectively.Source: Table I · p. 7, §VI · p. 7
  • 30 OOD generalist-versus-specialist comparisons favor generalists on CR (49.8% vs. 54.4%), SR (29.4% vs. 27.0%), and Force (102.9 N vs. 144.8 N).InD mean SR is higher for generalists (23.7% vs. 18.7%), while mean CR is slightly higher (39.8% vs. 37.6%).Source: Table II · p. 7, §VI · pp. 7–8
  • On held-out embodiments, CrossSafe reduces CR by 14.3 percentage points and Force from 179.6 N to 102.9 N, intervening on 9.2% of control steps.For the held-out seven-DoF-per-arm Franka-Panda, mean CR falls from 66.0% to 60.4% while mean SR rises from 40.4% to 45.2%.Source: §VI · p. 7
  • In the held-out Franka-Panda fold, all three ablations raise mean CR above the nominal controller, compared with CrossSafe’s reduction from 66.0% to 60.4%.The ablations’ mean CRs are 76.0%, 79.2%, and 66.4%, respectively.Source: §VI · p. 7

VII. CONCLUSION

  • CrossSafe shares a safety value function and policy using a variable-length link-token representation, but as a learned filter it gives no formal guarantee and collision rates remain relatively high.The authors identify probabilistic calibration, broader embodiment and task coverage, and hardware validation as future work.Source: §VII · p. 8

APPENDIX

The appendix specifies the simulated control setting, robot and task suite, obstacle placement, and collision-based failure function used in evaluation.

1) Control frequency:

  • Control runs at 25 Hz over a 250 Hz physics timestep, with episodes capped at 450 control ticks (18 seconds).The critic receives the nominal joint displacement planned for the next 0.04-second control interval, or the final configuration if less than one tick remains.Source: 1) Control frequency · p. 10

2) Embodiments:

  • The experiments use five bimanual robot embodiments, with seven actuated joints per arm for Franka-Panda and six for each of the other four.Joint action limits come from each robot’s URDF.Source: 2) Embodiments · p. 10

3) Tasks:

  • The task suite contains five RoboTwin 2.0 manipulation tasks, retaining benchmark objectives and scene randomization settings except for adding one static obstacle per scene.Background randomization is disabled.Source: 3) Tasks · p. 10

4) Obstacle placement:

  • A static box obstacle is placed along or beside the pick-to-place segment, with its along-segment position sampled using ρ from 0.22 to 0.48.Collection samples on-path placement with probability 0.7; all evaluation scenes are on-path.Source: 4) Obstacle placement · p. 10

5) Failure function:

  • The failure function is the signed distance from the robot system to the obstacle, measured using moving-link collision spheres and any grasped object.The obstacle is represented as an oriented bounding box.Source: 5) Failure function · p. 10

B. Safety critic and policy: implementation details

The implementation trains a safe actor and twin critics with an SAC adaptation to Hamilton-Jacobi reachability, while collecting shared warmup and online trajectories.

1) Soft actor-critic training:

  • Twin critics regress a discounted avoid target, and the SAC-adapted actor loss includes entropy regularization annealed from 0.2 to 0.02 over 20,000 gradient steps.The target networks use Polyak averaging with τ = 0.005.Source: 1) Soft actor-critic training · p. 10

2) Safe actor head:

  • A shared linear actor head maps each actuated link token to an action mean and log standard deviation, then samples, squashes, and scales actions by per-joint limits.Log standard deviation is clamped to [−5, 0], and actions are scattered into the action vector by joint index.Source: 2) Safe actor head · p. 10

3) Data collection:

  • The shared nominal-controller warmup buffer contains 500 trajectories, retaining 20 successful episodes for each of 25 task–embodiment pairs.Each training split samples only trajectories from its training embodiments; training alternates 1024 gradient steps with three fresh episodes.Source: 3) Data collection · p. 10

4) Parameter count:

  • Trainable parameter counts exclude the frozen HoloBrain-0 encoders, which receive no gradient.The passage points to Table III for the counts but does not state their values.Source: 4) Parameter count · p. 10

5) CrossSafe training algorithm:

  • The appendix identifies Algorithm 1 as the procedure for training the safe actor and critic.No further algorithm steps are specified in the supplied passage.Source: 5) CrossSafe training algorithm · p. 10

6) Hyperparameters:

The section specifies shared experimental settings and deterministic evaluation, then describes training and the in-distribution and held-out rate tables. The tables report collision and success rates across task–embodiment pairs.

6) Hyperparameters:

  • Methods, ablations, folds, and specialists share hyperparameters, differing only in the safety filter’s training-embodiment pool; Table III inventories trainable parameters.The parameter inventory excludes frozen HoloBrain-0.Source: 6) Hyperparameters · p. 10, Table III · p. 11

1) Scene determinism:

  • Each task–embodiment cell has 50 episodes, evaluations use identical scenes, and reruns reproduce results; failures trigger deterministic scene redraws.The safe policy is evaluated deterministically using its pre-activation mean without sampling.Source: 1) Scene determinism · pp. 10–11
  • Training runs 1024 gradient steps per round, updates the encoder and critic from batches of 64, updates the actor, and collects three rollout episodes.Rollouts use the filter with probability 0.8; otherwise they use the nominal controller with random action perturbations.Source: 1) Scene determinism · p. 11

2) Extended results:

  • Tables V–VI report collision and success rates for each task–embodiment pair, covering in-distribution and held-out evaluations.Table V averages four leave-one-embodiment-out models per cell; Table VI uses the single model that excluded that embodiment from safety-filter training.Source: 2) Extended results · p. 11, Table V · p. 12, Table VI · p. 13
  • Table V compares the unfiltered cuRobo planner, leave-one-embodiment-out X-VLA-Safe and CrossSafe models, and embodiment-specific CrossSafe specialists.Its task average spans five embodiments, while its overall average covers all 25 task–embodiment pairs.Source: Table V · p. 12
  • Table VI reports held-out-embodiment rates for single excluded-from-training X-VLA-Safe and CrossSafe models and specialists averaged across four models.The table presents collision and success rates as CR/SR percentages.Source: Table VI · p. 13
Loading 2609.28984v1…