Modular Safety Guardrails Are Necessary for Foundation-Model-Enabled Robots in the Real World

1. Introduction

• Open-ended environments, evolving specifications, environmental uncertainty, and FM stochasticity compound safety risks.Human interaction also brings semantic, intent-alignment, and social-norm requirements that no monolithic mechanism reliably addresses.Source: §1 · p. 1
• Monolithic end-to-end safety solutions remain fragile under distribution shift, evolving requirements, and datasets that rarely contain catastrophic failures.These limitations constrain their robustness and longevity.Source: §1 · p. 1
• External modularity separates safety enforcement from FMs, while internal modularity assigns specialized mechanisms to distinct failure modes.The paper argues this modularity is necessary because safety requirements drift, are hard to specify in advance, and failures can be rare but high impact.Source: §1 · p. 2
• The paper distinguishes action safety, decision safety, and human-centered safety, covering physical constraints, contextual appropriateness, and alignment with human intent and norms.These requirements extend beyond physical feasibility as FMs enter unstructured, human-centered environments.Source: §1 · pp. 1–2
• The paper proposes separate monitoring and evaluation and intervention layers, with the latter using decision-level gating and action-level filtering.The architecture supports cross-layer co-design, independent verification and updates, composability, and coverage across all three safety dimensions.Source: §1 · p. 2

2. Roles of Foundation Models in Robotics

• Perceptual FMs map sensory inputs into semantic representations, supporting open-vocabulary recognition, scene understanding, and affordance prediction without task-specific retraining.This broadens operational scope while raising uncertainty, grounding, and reliability challenges in safety-critical settings.Source: §2 · p. 2
• As reasoning modules, FMs interpret natural-language instructions and compose executable action sequences, often by selecting from predefined robot skills.Examples include Code as Policies, ProgPrompt, PaLM-SayCan, LLM-Planner, and Alpamayo-R1.Source: §2 · p. 2
• End-to-end vision-language-action models map images and language instructions directly to robot actions, unifying perception, grounding, and control in one policy.The π-series instead pairs a pretrained vision-language backbone with a separate action expert for continuous control.Source: §2 · p. 3

3. Safety Definitions and Specifications for FM-enabled Robotics

• The paper uses safety definitions for conceptual categories and safety specifications for explicit constraints a robot must satisfy during operation.The distinctions are framed for FM-enabled robots in human-centered, unstructured environments.Source: §3 · p. 3
• Action safety keeps physical execution within constraints such as collision avoidance, joint limits, and safe force or impedance modulation.These specifications are relatively well understood and largely addressed through model-based control-theoretic approaches.Source: §3 · p. 3
• Decision safety assesses whether proposed behavior is semantically and contextually appropriate, including constraints that low-level physical limits cannot capture.Examples include not placing a soft toy on a hot stove and not pouring coffee too fast.Source: §3 · p. 3
• Human-centered safety concerns whether behavior remains predictable, understandable, and trustworthy to people over sustained interaction.Misaligned expectations, human adaptation, and miscalibrated trust can make behavior seem unsafe even when action- and decision-level constraints are met.Source: §3 · p. 3

4.1. Intrinsic FM and Robot Limitations

• Perceptual and reasoning unreliability threatens decision safety, while hard physical constraints and modeling mismatch threaten action safety.Under distribution shift, vision-language and vision-language-action systems exhibit hallucination, weak spatial grounding, and poorly calibrated uncertainty; generative-output trustworthiness is not well captured by conventional uncertainty measures.Source: §4.1 · p. 4

4.2. Adversarial and Worst-case External Factors

• Prompt- and perception-level attacks can bypass safeguards, while disturbances and unexpected contacts challenge nominal assumptions; this motivates FM-decoupled runtime enforcement.The proposed examples include certified control and safety-filtering layers that enforce physical constraints independently of high-level reasoning.Source: §4.2 · p. 4

4.3. Open-world Deployment

• Compositional, effectively unbounded tasks and contexts make exhaustive specification, testing, and offline validation infeasible.The paper calls for lifecycle strategies spanning prevention, runtime monitoring and mitigation, and post-incident recovery rather than relying only on training or static rules.Source: §4.3 · p. 4

4.4. Human Complexity and Mutual Adaptation

• Ambiguous intent and subjective, personalized safety expectations make fixed constraints difficult to encode and insufficient for sustained human-robot interaction.Mutual adaptation, shifting trust, and evolving social norms can make behavior seem unsafe even when action- and decision-level constraints are satisfied.Source: §4.4 · p. 4

5.1. View A: Safety Should be Embedded in the Model

• Model-internal alignment can shift behavior toward user intent, but it does not provide a non-bypassable runtime boundary and remains vulnerable to distribution shift and novel hazards.When internal failures occur, no external mechanism prevents unsafe actions from being executed.Source: §5.1 · pp. 4–5

5.2. View B: Safety Should be Provided by External Add-ons (Single-perspective or Narrow Subset)

• External add-ons commonly address one safety dimension or a narrow subset rather than integrated coverage across all three dimensions.The section contrasts action-, decision-, and human-centered add-ons.Source: §5.2 · p. 5
• Action-level standards and filters enforce physical constraints but cannot reason about semantic or human-contextual hazards and rely on predefined unsafe sets and sufficiently accurate dynamics models.These assumptions can break down in unstructured FM-enabled robotics settings.Source: §5.2 · p. 5
• Decision-level add-ons screen plans but cannot guarantee physical safety during execution, while human-centered add-ons align behavior with expectations but cannot provide hard enforcement when misalignment occurs.Decision-level approaches can also be vulnerable to hallucinations, adversarial manipulation, incomplete specifications, and distribution shift; human intent may be ambiguous and preferences may change.Source: §5.2 · p. 5

5. Alternative Views on Safety Placement in FM-Enabled Robotics

• Neither model-internal safety nor single-perspective external add-ons suffices alone; the paper argues for modular guardrails spanning action, decision, and human-centered risks.The section frames existing approaches as covering only subsets of safety challenges in open-world deployment.Source: §5 · p. 4, §5.2 · p. 5

6.1. Definition and Design Principles

• A modular safety guardrail is a non-bypassable execution-time layer that mediates proposals before physical execution, monitoring risk and intervening across physical, semantic, and interaction-level safety.The design principles call for independent verification and updates, composition across failure modes, and non-bypassability at execution time.Source: §6.1 · p. 5

6.2. Modular Safety Guardrail Architecture

• The architecture separates risk assessment from enforcement: monitoring evaluates autonomy-stack outputs, while the intervention layer uses decision and action gates.The gates target complementary failure modes across the three safety dimensions.Source: §6.2 · p. 6

6.2.1. MONITORING & EVALUATION LAYER

• Monitoring independently assesses outputs from perception, planning, and control through decoupled channels and explicit execution-relevant representations, producing risk signals across all three safety dimensions.Signals can include physical constraint violations, semantic or contextual risks, and predictability, preference-alignment, or trust indicators.Source: §6.2.1 · p. 6

6.2.2. INTERVENTION LAYER

• The decision gate screens plans for semantic or human-centered risks, while the action gate constrains commands at execution as a non-bypassable physical last line of defense.Separating the gates lets inappropriate plans be rejected upstream while the action gate handles disturbances, tracking error, and other execution risks; the architecture supports auditing, verification, and composition of heterogeneous mechanisms.Source: §6.2.2 · pp. 6–7

6.3. Co-Design Opportunities Enabled by Modular Guardrails and Deployment Examples

• Representation alignment makes monitoring outputs directly actionable by preserving risk structure; coordinated conservatism lets the decision gate condition approval on whether the action gate can enforce safety margins.The paper’s hot-pan example uses a thermal-risk-scaled keep-out region and continues placement when separation remains enforceable, instead of defaulting to a stop.Source: §6.3 · pp. 7–8
• Deployment examples illustrate guardrails across FM configurations, including a household planner whose high-risk or uncertain plans are rejected or sent for clarification while physical constraints are enforced at the action gate.The examples are presented as existence proofs of failure modes that model-internal or single-perspective safety alone cannot address.Source: §6.3 · p. 7

6.4. Scope and Limitation

• The guardrail is not a universal solution, does not make FMs intrinsically safer or guarantee globally optimal decisions, and depends in practice on sufficiently independent safety signals and runtime coordination.FM-based guardrails may share data, design choices, or evaluation pipelines with the autonomy stack, while large-FM reasoning can delay the decision gate and require conservative action-gate fallbacks.Source: §6.4 · p. 8

• A broader ecosystem of composable modules should target distinct failure modes, uncertainty sources, and human contexts, with clear interfaces and benchmarks spanning all three safety dimensions.Benchmarks should include rare catastrophic scenarios that current datasets systematically underrepresent.Source: §7 · p. 8
• Principled co-design should specify what safety-relevant information modules produce, how it is represented, and how it flows bidirectionally across the autonomy stack.This can help upstream components avoid repeatedly proposing actions that downstream gates reject or heavily modify.Source: §7 · p. 8
• The research agenda is to standardize interfaces, expand modular guardrail capabilities, and develop co-design principles for composable, updateable, deployment-ready safety mechanisms.Source: §7 · p. 8

• For open-vocabulary mobile manipulation, monitoring cross-checks detections with depth and temporal consistency, while action gates enforce collision constraints despite unreliable perception.OOD detection flags unfamiliar or ambiguous inputs, and the decision gate is optional because the FM produces no plans.Source: A. Additional Deployment Examples for… · p. 14
• A pose-uncertainty ellipsoid preserves localization-error direction and magnitude, letting the action gate tighten margins anisotropically and the decision gate check workspace feasibility.The example contrasts this with a scalar confidence score that would force a blunt stop/go choice.Source: A. Additional Deployment Examples for… · p. 14
• For an end-to-end VLA policy, monitoring combines token-level uncertainty with an independent scene check, while the action gate projects commands onto safe sets or triggers fallback.The decision gate is bypassed; this setup mainly enforces action safety, with limited decision safety through uncertainty signals.Source: A. Additional Deployment Examples for… · p. 14
• In cluttered VLA grasping, mapping uncertainty to a spatial risk field lets the action gate tighten margins and slow motion only near high-risk regions.The example describes safe grasping without a hard stop.Source: A. Additional Deployment Examples for… · p. 14