PaperlayerView exact revision on arXiv

Source-linked AI summary

Ensuring Safe Physical AI in Urban Mobility via Hazard-Informed Synthesized Envelopes

Alexei Odinokov, Rostislav Yavorskiy

arXiv:2608.14481v1cs.ROcs.AI

TL;DR

Urban mobility safety is a system-level, cross-layer property that conventional component-level or reactive approaches do not fully represent. This paper introduces hazard-informed safety envelopes spanning symbolic, spatial, and dynamic models, with runtime enforcement to support predictive, physically executable, and operationally safe behavior.

  • Problem

    Urban mobility safety is a system-level property that evolves across design layers rather than a static attribute of one software component.

  • Method

    The framework propagates hazard-informed safety constraints across symbolic, spatial, and dynamic layers and enforces them at runtime through a Physical AI Harness.

  • Results

    The learned envelope supports predictive assessment of approaching unsafe-operation boundaries, proactive intervention, interpretable validation, and physically executable runtime decisions.

  • Takeaways & Limitations

    A consistent safety objective can flow through the autonomy stack while runtime enforcement preserves the validity of learned constraints during deployment.

Abstract

from arXiv · show

As heterogeneous robotic systems deploy across diverse urban zones, maintaining safety amid complex human-robot interactions remains a critical challenge. We present a unified framework that bridges systematic hazard analysis and runtime enforcement using hazard-informed safety envelopes. Rather than treating safety as a static constraint isolated within individual software modules, we introduce a cross-layer safety transformation process spanning symbolic, spatial, and dynamic world models. We show how this representation naturally interfaces with physical AI runtime harnesses to guarantee safe urban mobility.

I. INTRODUCTION

Urban mobility robotics is expanding from controlled industrial settings into public spaces, where heterogeneous systems create distinct safety, operational, and regulatory challenges. The paper frames safety as a cross-layer property and proposes hazard-informed urban safety envelopes linking hazard analysis with runtime enforcement.

  • Emerging Urban Mobility Applications: Urban robotics now includes sidewalk delivery robots, 24/7 autonomous vehicles and robotaxis, and nighttime road-maintenance robots.These systems operate in different urban zones and support delivery, passenger transport, and infrastructure inspection or repair.
  • Safety Challenges: Deploying heterogeneous robots alongside humans poses severe safety, operational, and regulatory challenges across distinct urban zones.Each domain uses a different safety paradigm to prevent physical harm and minimize conflict.
  • Cross-Layer Safety: Safety is a system-level feature that evolves as information and decisions flow through different layers of system design.A pedestrian can be represented as a vulnerable road user, dynamic obstacle, or moving object with position and velocity while preserving the same safety objective.
  • Cross-Layer Safety: The paper treats safety as a cross-layer property whose representation changes with abstraction level while its underlying objective remains constant.This perspective connects symbolic, planning, and control representations of the same harmful-interaction-avoidance objective.
  • Proposed Framework: The proposed framework bridges systematic hazard analysis and runtime enforcement through hazard-informed urban safety envelopes defining safe operating conditions for autonomous robots and vehicles.The envelopes represent the range of conditions under which autonomous systems can safely interact with their environments.

II. URBAN SAFETY ENVELOPE

The urban safety envelope uses hazard-informed synthetic scenarios to learn predictive safety margins across urban driving conditions. It supports proactive runtime responses and traceable safety validation by linking learned behavior to explicitly defined hazards.

  • Hazard-informed learning: The framework generates synthetic training scenarios around explicitly defined assets, exposure modes, and hazard scenarios where safety margins are reduced.This extends training beyond normal driving data toward safety-critical conditions.
  • Hazard-informed learning: Systematic variation of pedestrians, cyclists, occlusions, weather, and traffic density produces situations ranging from safe operation toward safety limits.Models observe gradual transitions instead of learning only from rare collision events.
  • Predictive enforcement: When safety margins become too small, the system can reduce speed, increase following distance, select an alternative trajectory, or transfer control to a higher-level safety controller.These responses maintain sufficient safety margins before dangerous situations develop.
  • Predictive enforcement: The learned models continuously estimate proximity to predefined safety-constraint violations and identify when the vehicle approaches the boundary of safe operation.This enables predictive safety assessment rather than reacting only after dangerous situations emerge.
  • Interpretability and validation: Because each scenario originates from an explicitly defined hazard, learned behavior is traceable to assets, vulnerabilities, and hazard scenarios for transparent validation and certification.This traceability provides stronger engineering evidence for safety validation and certification.

III. SAFETY ACROSS HIERARCHICAL WORLD MODELS · A. Symbolic Safety Layer

Urban mobility safety is organized across hierarchical world models because no single representation supports strategic decisions, trajectory planning, and real-time control simultaneously. The symbolic safety layer represents missions, regulations, legal constraints, and ethics through constraints on permissible actions and relationships between entities.

  • III. SAFETY ACROSS HIERARCHICAL WORLD MODELS: Autonomous urban mobility uses hierarchical world models, with each layer maintaining a specialized environmental representation suited to its function.The hierarchy supports strategic decision making, trajectory planning, and real-time vehicle control.
  • III. SAFETY ACROSS HIERARCHICAL WORLD MODELS: Each world-model layer defines its own safe-operation notion, so urban safety emerges from consistent interaction among multiple safety abstractions across the control hierarchy.
  • A. Symbolic Safety Layer: The symbolic layer models the environment through entities, relationships, and operational rules rather than physical geometry.It addresses mission objectives, traffic regulations, legal constraints, and ethical requirements.
  • A. Symbolic Safety Layer: Symbolic safety constraints require pedestrian priority, traffic-signal obedience, restricted-area avoidance, and compliance with operational policies.The environment is represented as a structured network of permissible and prohibited actions.
  • A. Symbolic Safety Layer: Pedestrians are represented as vulnerable road users whose presence imposes behavioral constraints on the autonomous system.
  • A. Symbolic Safety Layer: In the symbolic relation graph, safety rules are visualized as directed edges such as Yield and Stop between entities.This graph expresses rule relationships within the symbolic representation.

B. Spatial Safety Layer

The spatial safety layer transforms symbolic objectives into geometric motion plans over maps encoding free space, obstacles, traversability costs, and predicted traffic motion. Safety is enforced through spatial relationships and occupancy-based trajectory generation.

  • Spatial Safety Layer: The layer converts symbolic objectives into geometric motion plans using spatial maps of free space, obstacles, traversability costs, and predicted traffic motion.These environmental and traffic representations support motion planning at the spatial level.
  • Spatial Safety Layer: Safety is encoded through collision probability, minimum clearance distances, visibility constraints, and safe trajectory generation.The framework expresses safety through explicit spatial relationships.
  • Spatial Safety Layer: Dynamic objects are represented by predicted occupancy rather than semantic meaning, so pedestrians become dynamic obstacles influencing safe-path selection.Their future motion directly affects the chosen path.
  • Spatial Safety Layer: The continuous environment is discretized into a grid to compute a safe geometric path.This grid-based representation implements the spatial mapping shown in Fig. 3.

C. Dynamic Safety Layer

The dynamic safety layer governs vehicle–environment interaction through continuous dynamics, actuator limits, and physical constraints. It maintains stable, physically achievable commanded trajectories while modeling pedestrians as moving objects for real-time safe control.

  • C. Dynamic Safety Layer: The dynamic safety layer represents the world through continuous dynamics, actuator limits, and physical constraints governing vehicle–environment interaction.
  • C. Dynamic Safety Layer: Safety requires stable behavior while respecting steering, braking, acceleration, tire-road friction, and actuator limits.
  • C. Dynamic Safety Layer: The controller ensures commanded trajectories remain physically achievable rather than enforcing traffic regulations or navigation goals.
  • C. Dynamic Safety Layer: Pedestrians are abstracted as moving objects defined by position, velocity, and relative motion, enabling real-time computation of safe control actions.

IV. SAFETY TRANSFORMATION ACROSS LAYERS · A. Forward Transformation: From Rules to Actions

The framework transforms urban safety from symbolic rules into geometric constraints and physically feasible commands across hierarchical world models. Real-world feedback propagates upward, enabling closed-loop adaptation of the safety envelope.

  • IV. SAFETY TRANSFORMATION ACROSS LAYERS: Three world models represent the same physical environment at different abstraction levels, enabling safety refinement from symbolic objectives to low-level physical commands.The hierarchy supports downward propagation of safety information across layers.
  • IV. SAFETY TRANSFORMATION ACROSS LAYERS: Real-world feedback travels upward through the hierarchy, updating representations at each layer for closed-loop adaptation.The safety transformation therefore operates bidirectionally rather than only from rules toward actions.
  • A. Forward Transformation: From Rules to Actions: The symbolic layer encodes mission objectives, traffic regulations, ethical principles, and operational policies as permissible high-level behaviors.Examples include yielding at crosswalks, respecting speed limits, and avoiding restricted zones.
  • A. Forward Transformation: From Rules to Actions: The spatial layer translates symbolic constraints into geometric safety margins and collision-free trajectories using occupancy, clearance distances, and predicted agent motion.It reasons about the environment geometrically rather than specifying only abstract behavioral rules.
  • A. Forward Transformation: From Rules to Actions: For pedestrian yielding, the spatial planner requires a minimum lateral distance of 0.5m and stopping before the crosswalk when arrival is within 2s.The planner computes a path satisfying these spatial margins.
  • A. Forward Transformation: From Rules to Actions: The dynamic layer converts planned trajectories into steering, braking, and acceleration commands that respect actuator limits, tire-road friction, and vehicle dynamics.Safety at this stage requires stability and reference-trajectory tracking within prescribed bounds.

B. Feedback Transformation: From Execution to Awareness · C. Emergence of the Urban Safety Envelope

Feedback transforms execution data into updated dynamic and spatial plans, while the urban safety envelope emerges as the intersection of symbolic, spatial, and dynamic constraints. Hazard-informed synthetic scenarios support learning this envelope so systems can anticipate safe-operation boundaries and maintain safety through coherent cross-layer propagation.

  • B. Feedback Transformation: From Execution to Awareness: Sensor feedback updates the dynamic model with vehicle behavior, disturbances, and obstacles, enabling the planner to revise trajectories when they are no longer collision-free.The feedback loop connects execution observations to spatial replanning, including responses to suddenly braking vehicles.
  • B. Feedback Transformation: From Execution to Awareness: Persistent or significant execution deviations are transformed into awareness of changing conditions and potential safety-boundary violations.The supplied passage identifies persistent or significant deviations as a trigger within the feedback transformation, though its final mechanism is truncated.
  • C. Emergence of the Urban Safety Envelope: The urban safety envelope is the intersection of propagated symbolic rules, spatial collision-free criteria, and dynamic feasibility bounds.A state is safe only when it satisfies all three constraint classes simultaneously.
  • C. Emergence of the Urban Safety Envelope: Hazard-informed synthetic scenarios enable the system to learn the safety envelope and anticipate approach to its safe-operation boundary.The training data is systematically generated around known hazard types.
  • C. Emergence of the Urban Safety Envelope: A sidewalk delivery robot at an intersection must yield to pedestrians, stop before a blind corner with a buffer, and decelerate smoothly within torque limits.This example instantiates the safety transformation across symbolic, spatial, and dynamic layers.
  • C. Emergence of the Urban Safety Envelope: When a pedestrian appears closer than expected, feedback updates the dynamic state, the planner recalculates a tighter stop trajectory, and the controller adjusts braking force.The example illustrates continuous safety transformation across layers during execution.
  • C. Emergence of the Urban Safety Envelope: The hierarchy assigns each layer an appropriate computational abstraction while coherently propagating safety information to keep the overall system safe.Fig. 5 summarizes this cross-layer safety transformation.

V. RUNTIME SAFETY ENFORCEMENT THROUGH THE PHYSICAL AI HARNESS · A. Projection

The Physical AI Harness preserves safety semantics during real-time execution by supervising AI-generated decisions against robot capabilities and environmental requirements. Its Projection mechanism constrains learned commands to the urban safety envelope before physical execution.

  • V. RUNTIME SAFETY ENFORCEMENT THROUGH THE PHYSICAL AI HARNESS: Real-time machine-learning decisions require an additional enforcement mechanism to remain consistent with safety constraints established during system design.Correct perception, planning, and control alone do not ensure that generated outputs preserve those constraints during execution.
  • V. RUNTIME SAFETY ENFORCEMENT THROUGH THE PHYSICAL AI HARNESS: The Physical AI Harness is a runtime software layer that governs AI-model execution within robotic systems.It differs from conventional middleware by continuously supervising decisions rather than primarily coordinating software-component communication.
  • V. RUNTIME SAFETY ENFORCEMENT THROUGH THE PHYSICAL AI HARNESS: The harness supervises AI-generated decisions for compatibility with robot physical capabilities and environmental operational requirements.This supervision addresses both the robot’s capabilities and the conditions of its operating environment.
  • V. RUNTIME SAFETY ENFORCEMENT THROUGH THE PHYSICAL AI HARNESS: Projection, Isolation, and Transfer form three complementary mechanisms addressing distinct aspects of runtime safety.Together, these mechanisms define the Physical AI Harness approach described in the section.
  • A. Projection: Projection constrains machine-learning outputs before execution by mapping predicted commands onto a predefined region of acceptable behavior.Arbitrary actions are therefore not passed directly to the robotic system.
  • A. Projection: In the urban framework, the acceptable region is the urban safety envelope derived from hazard analysis and safety envelope learning.The envelope defines the safety margins used to evaluate steering, speed, lane-change, and trajectory commands.
  • A. Projection: Commands violating envelope constraints are modified or rejected before reaching the controller.Projection is consequently the final validation step between learned decision making and physical execution.

B. Isolation · C. Transfer

Isolation protects essential vehicle functions from competing AI workloads through dedicated resources and execution budgets, while Transfer hands authority to verified fallback control as safety margins decline or operating conditions become unsafe.

  • B. Isolation: Autonomous vehicles run perception, localization, planning, communication, and control components concurrently, while machine-learning models may compete for resources and degrade safety-critical functions.
  • B. Isolation: Isolation allocates dedicated computational resources and enforces execution budgets so intensive AI workloads can operate alongside conventional control algorithms.This separation preserves deterministic behavior required for real-time control.
  • B. Isolation: Isolation prevents computational-performance degradation from propagating into unsafe vehicle behavior within the urban safety framework.
  • C. Transfer: AI models may encounter unexpected weather, sensor failures, infrastructure damage, or unseen traffic scenarios outside their operational design domain.These conditions can invalidate assumptions established during training.
  • C. Transfer: Transfer hands authority from the AI model to a verified fallback controller when the system approaches unsafe operating conditions.Fallbacks include emergency braking, reduced-speed operation, minimal-risk maneuvers, or a conventional rule-based controller.
  • C. Transfer: Transfer triggers when the estimated safety margin falls below an acceptable threshold, intervening proactively as the vehicle approaches the safe-operation boundary.The system intervenes before an imminent collision rather than waiting for one.

D. Relationship to the Urban Safety Envelope · VI. CONCLUSION

The framework connects hazard analysis, safety-envelope learning, and runtime enforcement across the autonomous system’s operational lifecycle. Its proof-of-concept implementation uses NVIDIA simulation, world-model, and digital-twin technologies for closed-loop validation.

  • D. Relationship to the Urban Safety Envelope: Hazard analysis identifies situations defining the urban safety envelope, while envelope learning estimates proximity to unsafe operation.The Physical AI Harness then enforces decisions that remain physically executable and operationally safe during deployment.
  • D. Relationship to the Urban Safety Envelope: The Physical AI Harness provides runtime enforcement of safety knowledge established during system design.It preserves the safety constraints produced by the hazard-informed methodology throughout deployment.
  • D. Relationship to the Urban Safety Envelope: Safety assurance forms a continuous chain spanning hazard identification, synthetic scenario generation, machine learning, runtime monitoring, and physical execution.This preserves safety during model development and throughout the autonomous system’s complete operational lifecycle.
  • VI. CONCLUSION: Safety information flows continuously through the autonomy stack, changing representation across abstraction levels while preserving a consistent safety objective.Runtime enforcement keeps learned safety constraints valid during deployment.
  • VI. CONCLUSION: The proof-of-concept implementation leverages NVIDIA world models and simulation technologies to validate the proposed framework in practice.Its closed-loop validation pipeline integrates complementary platforms across robotics simulation, synthetic data generation, and urban digital-twin construction.
  • VI. CONCLUSION: NVIDIA Isaac Sim and Isaac Lab provide physically accurate environments for training and evaluating autonomous agents.They serve as the primary robotics simulation and learning frameworks in the validation pipeline.
  • VI. CONCLUSION: NVIDIA Cosmos world foundation models generate high-quality synthetic data and simulate urban scenarios from nominal traffic to safety-critical edge cases.This supports diverse scenario coverage within the closed-loop validation pipeline.
  • VI. CONCLUSION: NVIDIA Omniverse provides infrastructure for building high-fidelity digital twins of urban environments.It supplies the digital-twin foundation for the closed-loop validation pipeline.
Loading 2608.14481v1…