Not an Accident: Exploiting Safety Filter Inadequacy in Humanoid Robots

1 Introduction

• Safety filters can output controls that satisfy their optimization problem yet fail to maintain safety when task- and environment-specific parameters are configured improperly.The paper identifies this parameter-related inadequacy as a security vulnerability that attackers can exploit to cause collisions.Source: §1 · pp. 1–2
• 1.86x times as many attacks as the random-search baseline were discovered by SIREN, with successful attacks against every evaluated filter.The evaluation covered seven representative filters in simulation and on a Unitree G1 humanoid robot.Source: §1 · p. 2

2.1 Robotic Security

• Humanoid security research has examined communication, authentication, privacy, and software risks, while humanoid-specific control and safety-algorithm vulnerabilities have received little attention.Humanoids also have articulated, high-dimensional bodies and configuration-dependent collision geometry.Source: §2.1 · p. 2

2.2 Robot Dynamics

• The paper models a humanoid as a control-affine system whose state may include joint positions, joint velocities, and base pose, with admissible control inputs affecting its motion.Depending on the abstraction, controls may specify joint velocities or accelerations.Source: §2.2 · p. 2

2.3 Safety Filter Algorithms

• A safety filter modifies a nominal controller's reference action when needed to satisfy safety requirements, with its escape-rate parameters influencing intervention strength and deviation from that action.Optimization-based filters impose constraints on the evolution of safety indices; different filter families use different escape rates.Source: §2.3 · p. 3

Attack Scenario

• An insertion is an intermediate goal that makes an otherwise successful original task fail after the robot completes the insertion and resumes the original goal.The two goals are individually legitimate, but their composition can place the robot where its safety filter prevents successful task completion.Source: §3 · p. 5
• The attack assumes a task-insertion mechanism exists, while the particular application-layer mechanism for inserting tasks is outside the work's scope.New tasks may be scheduled among existing tasks according to priorities, deadlines, or execution costs.Source: §3 · pp. 4–5

Non-Attack Scenario

• Failures directly caused by the original task, regardless of insertion, are excluded because they are readily attributable to that task.Source: §3 · p. 5

Threat Model

• The attacker is assumed to know the filter configuration, environment geometry, and target task, and may evaluate candidate insertions offline before attacking.The attacker interacts at attack time only through the legitimate high-level task interface, without privileged access, sensor spoofing, or direct joint-level control.Source: §3 · p. 5

4.1 Safety Filters Failures

• Safety filters can fail in two ways: a too-small escape rate may not avert collision, while a too-large rate may exceed available control authority and make optimization infeasible.Both failure modes depend on the humanoid’s state when the filter engages.Source: §4.1 · pp. 5–6
• Positive clearance at engagement does not ensure safety: adverse approach dynamics can consume it before the response reverses motion, even while the safety-index condition is met.For higher-relative-degree filters, clearance may continue decreasing while the filter drives the safety index downward.Source: §4.1 · p. 6
• A small escape rate is not inherently unsafe; failure arises when it combines with an unfavorable engagement state, which an inserted goal can induce before the constraint activates.Early engagement may leave enough clearance for a relatively small escape rate to work.Source: §4.1 · p. 6
• Because required recovery and available control authority vary with robot configuration and environment, one fixed parameter setting cannot adequately cover all humanoid states; current filters also lack proactive failure avoidance.The filters compute only the next control step and do not account for future states or optimization feasibility.Source: §4.1 · p. 7

5 Exploiting Safety Failures Adversarially

• Attackers exploit state-dependent safety failures by using high-level goals to place the humanoid in a state where its subsequent task triggers a filter failure.For reaching tasks, the accepted Cartesian goal space is substantially lower-dimensional than the full humanoid state space.Source: §5 · p. 7

5.1 Attack Formalization

• A successful inserted goal is executable on its own, while the original task succeeds normally but fails after that intermediate goal is executed.The inserted goal need not approach an obstacle or cause a violation itself; it induces a state that makes the later task unsafe.Source: §5.1 · p. 8
• The attack searches authorized task inputs rather than directly manipulating the robot’s high-dimensional internal state, relying on the nominal controller to realize the state transition.The resulting trajectory uses legitimate robot software, truthful safety-filter observations, and a valid command.Source: §5.1 · p. 8

5.2 Finding Inserting Goals

• CEM focuses closed-loop simulation trials on admissible goals likely to induce failure, using candidate scores to guide later samples toward failures or collision states.Confirmed failures outrank near misses, while smaller collision clearance raises the score within an outcome class.Source: §5.2 · p. 8

6 Implementation

• SIREN is implemented as a Python module in the SPARK v1.0 simulation framework.The implementation includes an adaptation layer and experiment analysis.Source: §6 · p. 8
• The core implementation comprises approximately 1,400 lines of code, and the complete codebase comprises approximately 6,800 lines.The complete count includes the adaptation layer and experiment analysis.Source: §6 · p. 8
• The search algorithm takes the initial state, original goal, goal space, filter, simulator, budget, and batch size, and outputs ranked inserted goals and confirmed attacks.Figure 1 presents SIREN’s CEM-based search algorithm.Source: §6 · p. 9, Figure 1 · p. 9

7.1 Safety Filter Vulnerability Presence

• The controlled vulnerability study sampled SPARK tasks and searched inserting goals and obstacle placements, counting an attack only when the original task succeeded without insertion and failed with it.Candidate goals were enumerated in a 30 × 30 × 30 cm3 region at 5 cm intervals; obstacle candidates were selected using swept volumes of the original and attacked trajectories, then validated by simulation.Source: §7.1 · p. 9
• The enumeration is sound but not complete: simulation validates every reported attack, but the procedure may miss other successful inserting goals.Its stated purpose is to establish concrete attack instances, not exhaustively find all attacks in an environment.Source: §7.1 · pp. 9–10
• Goal-insertion attacks were feasible against all evaluated safety filters in the tested SPARK tasks.The validated cases preserve success on the original task and first attacked leg, while the safety filter is engaged but fails on the second leg.Source: §7.1 · p. 10, Table 1 · p. 10
• In a representative SSA case, inserting a middle target beside the obstacle led to a right-arm collision on the leg toward the original endpoint.SSA had engaged and adjusted control away from the obstacle, but the configured escape rate was too slow for the arm's speed and inertia.Source: §7.1 · p. 10
• For SSA, increasing the escape rate first invalidated an insertion attack, then made it valid again when the required rate exceeded available control authority and the optimization became infeasible.The insertion can change which arm point is nearest the obstacle, exposing differences in those points' control authority.Source: §7.1 · p. 10, Figure 3 · p. 10, Figure 4 · p. 11

7.2 Exploitation on a Real Humanoid

• Across the tested upper-body, static-obstacle scenarios, the qualitative failures observed in simulation were reproduced on the physical humanoid.The study selected one randomly chosen scenario per algorithm and replayed its goal sequence and obstacle placement on hardware.Source: §7.2 · p. 11

7.3 Attack Discovery Effectiveness

• SIREN's scalability evaluation measures how many attacks automated search discovers in vulnerable SPARK scenarios, comparing CEM-based SIREN with random search and obstacle-near initialization variants.The dataset contains five vulnerable scenarios per safety filter, 35 total; the supplied evaluation passages describe the protocol but not its comparative outcomes.Source: §7.3 · p. 11

R-SSA R-CBF R-SSS p-SSA

• The real-humanoid demonstration shows the original task avoiding an obstacle, while an inserted intermediate goal leads to collision on the return to the original goal.The case study describes the arm first moving toward the obstacle and then colliding while moving toward the original endpoint.Source: Figure 5 · p. 12
• Search strategies receive an equal budget of 125 simulation trials because simulation time varies and dominates search cost.Each search process costs 61.1 minutes on average.Source: R-SSA R-CBF R-SSS p-SSA · p. 12
• 3,277 attacks were identified by SIREN versus 1,760 by random search across 35 scenarios; both methods found at least one attack in every scenario.The table is captioned as search effectiveness across 35 scenarios.Source: R-SSA R-CBF R-SSS p-SSA · p. 12, Table 3 · p. 12
• Neither near-obstacle initialization variant outperforms its corresponding baseline, and the results provide no evidence that successful inserting goals are concentrated near obstacles.SIREN-NI has a slightly higher hit rate but does not find attacks in every scenario.Source: R-SSA R-CBF R-SSS p-SSA · p. 12
• CEM search converges between two obstacles and identifies 21 attacking points in 125 trials, while points near obstacles are not necessarily attacks.The observation explains why near-obstacle initialization did not improve attack discovery.Source: R-SSA R-CBF R-SSS p-SSA · p. 13

8 Potential Defense and Mitigation

• Safety parameters depend on both task and environment, and inaccurate configurations can create opportunities for attack; no existing method determines correct values for arbitrary deployments.The paper also notes little prior work on relationships between these parameters and task or environmental characteristics.Source: §8 · p. 13
• Prohibiting insertion into already scheduled tasks can prevent their failures and make precisely timed attacks harder, though it may reduce flexibility and efficiency.The paper also discusses a dynamic emergency-stop monitor, since a conventional stop or fixed recovery pose can itself cause a fall or collision during recovery.Source: §8 · p. 13
• Mitigations cannot eliminate the structural vulnerability while filters use only the current state without future-feasibility reasoning or temporary task-reversing maneuvers.The paper identifies predictive safe control with inclusive feasibility guarantees as necessary to eliminate the vulnerability and an open research problem.Source: §8 · p. 13

9 Limitations and Future Work

• The prevalence of successful attacks is unknown, and the paper cannot estimate their probability for arbitrary tasks and environments.The humanoid state space makes quantification challenging, and no principled method currently determines whether an attack exists or when it is feasible.Source: §9 · p. 13
• The attack assumes complete knowledge of the target safety filter, including its algorithm and parameter values.Future work could consider attackers with incomplete information and whether limited probing can infer enough to construct attacks.Source: §9 · p. 13
• The evaluation does not model the complete humanoid software stack, whose perception and SLAM errors may affect attack execution and success.Future work will investigate attacks that remain effective despite inaccuracies in perception, localization, and other system components.Source: §9 · p. 13

10 Conclusion

• Existing safety filter algorithms contain vulnerabilities that attackers can exploit to cause safety violations such as collisions.Source: §10 · p. 13
• The paper presents goal insertion attacks and an automated approach that identifies vulnerable inserting goals.The conclusion characterizes the approach as scalable.Source: §10 · p. 13
• The evaluation confirms that goal insertion attacks exist in realistic scenarios and can compromise real humanoid robots.The paper also discusses potential defenses and mitigation strategies.Source: §10 · p. 13

• The evaluated filters are undergoing industry validation and are not productized or integrated into humanoid robots’ default manufacturing configurations.Source: Ethical Considerations · p. 17
• The work aims to discover critical and adversarial safety issues before humanoids are deployed.Source: Ethical Considerations · p. 17
• The authors plan to identify downstream users who may have deployed affected filters and inform them of the vulnerabilities.Source: Ethical Considerations · p. 17

A Clearance Derivation

• The clearance response is derived from the engagement state under a locally constant achieved escape rate.This assumption is exact during a constant-rate interval and otherwise models the response until clearance recovers or reaches zero.Source: A Clearance Derivation · p. 17
• The clearance expression separates an engagement-state offset, escape-rate-induced recovery, and inherited approach dynamics.The homogeneous response matches clearance derivatives at engagement and decays under the stated stability assumption; repeated roots yield polynomial–exponential terms.Source: A Clearance Derivation · p. 17
• A positive achieved escape rate therefore does not preclude an initial decrease in clearance.Source: A Clearance Derivation · p. 17
• With zero escape rate, clearance tends to dmin − φ(x0); when φ(x0) > dmin, its limiting clearance is negative.Even if initial clearance is positive, continuity implies negative clearance at some finite time.Source: A Clearance Derivation · p. 17
• A sufficiently small positive achieved escape rate can satisfy the safety-index derivative constraint while physical clearance crosses zero.Source: A Clearance Derivation · p. 17