PaperlayerView exact revision on arXiv

Source-linked AI summary

RT-Safe: Benchmarking Agent Safety in Real-Time Embodied Environment

Tianruo Rose Xu, Jiawei Ren, Yichi Yang, Zhaoxu Zheng, Lianhui Qin

arXiv:2610.09294v1cs.AIcs.LG

TL;DR

Embodied safety matters because physical failures can cause injury or hardware damage, and the world keeps changing while an agent reasons. RT-SAFE evaluates urban navigation under those real-time conditions. Across eight VLMs, high task completion coexists with sharply lower safe success and more collisions in real time; offline RL also reduces collision rates.

  • Problem

    Embodied agents can cause physical harm, and their surroundings may change during inference, making safety depend on both the decision and when it executes.

    Source: §1 · p. 1

  • Method

    RT-SAFE benchmarks urban navigation with moving actors, hazards, and traffic rules while the simulation advances during inference and action execution.

    Source: §1 · p. 2

  • Results

    Across eight VLMs, hard-setting real-time execution raises collisions 12.3× and lowers safe success from 19.8% to 0.7%, while task completion remains similar to static evaluation.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    The findings show that task completion alone does not establish safe completion, and offline RL can reduce collisions while trading off task completion, progress, and collision avoidance.

    Source: §3.2 · p. 5, §5 · p. 10

Abstract

from arXiv · show

Rapid progress in AI agents has brought growing attention to agent safety, with extensive evaluation focused on digital environments. As agents move into the physical world, embodied safety becomes increasingly important: failures can cause human injury and costly hardware damage. Beyond selecting safe actions, embodied agents must also operate under real-time constraints: the physical world does not pause while an agent reasons. As pedestrians move and vehicles approach during inference, an action that appears safe at observation time may become unsafe before execution. Real-time embodied safety therefore depends on both decision quality and decision latency. We introduce RT-SAFE, a simulated urban benchmark for evaluating embodied-agent safety under real-time constraints. RT-SAFE combines navigation tasks with moving actors, environmental hazards, and traffic rules, while allowing the world to evolve throughout inference and action execution. Across eight VLMs, agents achieve high task completion yet almost never complete safely: in the hardest setting, only 0.7% of episodes finish without a safety event. More strikingly, matched static and real-time evaluations yield task completion rates of 91.3% and 94.1%, respectively, while real-time execution increases collisions by $12.3\times$. These results reveal that standard task success can mask substantial safety failures, and that decision latency itself can become a source of physical risk. Finally, we show that RT-SAFE can support offline RL training and substantially reduce collision rates while achieving strong task completion.

1 INTRODUCTION

Embodied-agent safety must account for physical harm and the fact that the world changes while an agent reasons. RT-SAFE evaluates this timing challenge and finds that high task completion can coexist with substantial safety failures.

1 INTRODUCTION

  • Embodied failures can cause collisions, human injury, or hardware damage, so task completion alone is insufficient to evaluate safety.Digital-agent safety benchmarks have examined unsafe tool use, harmful task execution, and vulnerabilities to adversarial instructions.Source: §1 · p. 1
  • Because pedestrians, vehicles, and signals can change during inference, safety depends on both the decision and when it becomes executable.The resulting risks include safety events during inference and actions based on stale observations.Source: §1 · p. 1
  • Existing embodied benchmarks often abstract away inference latency, motivating evaluation of safety throughout deliberation and execution under matched paused and real-time conditions.The cited related work includes a benchmark that freezes simulation during model decisions and studies of delayed control.Source: §1 · pp. 1–2
  • RT-SAFE keeps the world evolving during inference and action execution, evaluating urban navigation amid moving actors, hazards, and traffic rules.The simulator records events during both intervals and distinguishes collisions incurred while deciding from those incurred while acting.Source: §1 · p. 2, Figure 1 · p. 2, Figure 2 · p. 3
  • In hard environments, real-time execution raises collisions 12.3× and reduces safe success from 19.8% to 0.7%, despite similar task completion.Task completion is 91.3% in static evaluation and 94.1% in real time; inference-time contacts account for most real-time collisions.Source: §1 · p. 2

2 RT-SAFE

RT-SAFE combines urban navigation with a continuously evolving simulation and explicit safety-event recording. Its interface, event rules, and metrics separate destination arrival from safe completion and characterize decision behavior.

2.1 TASK AND REAL-TIME EXECUTION

  • RT-SAFE evaluates safe arrival along ordered sidewalk and crosswalk subgoals, recording collisions, hazards, and traffic violations separately from destination arrival.Earlier safety events remain counted even if the agent later reaches the destination.Source: §2.1 · p. 3, §2 · p. 3
  • Real-time mode advances the world during inference and action execution, while static mode pauses it during inference; collisions are classified as passive or active by interval.Inference time includes model response time and recorded agent-side processing time; pedestrians, vehicles, and signals may change before an action executes.Source: §2.1 · p. 3

2.2 ENVIRONMENT AND AGENT INTERFACE

  • The benchmark uses 36 routes across five city maps, with easy, medium, and hard conditions defined by nested 60%, 80%, and 100% actor subsets.Routes vary in length, turns, crossings, and exposure to moving actors; scenes include hazards, obstacles, and controlled vehicles.Source: §2.2 · pp. 3–4

2.3 SAFETY EVENTS AND FEEDBACK

  • Safety events cover contacts, trip/oil/water interactions, and traffic violations, with simulator feedback and records supporting evaluation and training rewards.Active contacts incur a 6 s recovery delay; vehicle collisions terminate episodes immediately. Traffic violations and any resulting collision are recorded separately.Source: §2.3 · pp. 4–5

2.4 EVALUATION METRICS

  • Evaluation separates arrival success from event-free safe success and reports collisions by type and interval, hazards, violations, path efficiency, latency, action choices, and route progress.Response latency multiplied by decisions per episode serves as a proxy for cumulative inference exposure.Source: §2.4 · p. 5

3 EXPERIMENTS

Across the experiments, high task completion coexists with poor safety in evolving scenes, and matched static-versus-real-time evaluations expose large safety losses. The section also examines reasoning-effort tradeoffs and offline RL training for reducing collisions.

3.1 EXPERIMENTAL SETUP

  • The benchmark evaluates eight VLMs across easy, medium, and hard conditions, with 36 routes per difficulty and 108 episodes per model.Models use the same prompts, observations, and action interface.Source: §3.2 · p. 5, §3.1 · p. 5

3.2 RQ1: HOW SAFELY DO VLM AGENTS NAVIGATE IN REAL TIME?

  • Across eight VLMs, task success is 94.4% but safe success is 0.7%; latency × decisions explains 93.4% of collision-count variation, versus 78.4% for latency alone.Inkling combines long response times with few decisions and fewer collisions than the exposure-based fit predicts, while Grok combines long responses with short moves and the highest collision count.Source: §3.2 · pp. 5–6, Figure 3 · p. 6

3.3 RQ2: HOW DOES AGENT PERFORMANCE DIFFER BETWEEN STATIC AND REAL-TIME EVALUATION?

  • Task completion is 91.3% static versus 94.1% real-time, while safe success drops from 19.8% to 0.7% and collisions rise 12.3×.Real-time collisions average 40.68 per episode versus 3.31 static; approximately 84% of real-time collisions occur during inference.Source: §3.3 · p. 6, Figure 4 · p. 7
  • In real time, 4 m moves fall from 32.7% to 15.2%, while 1 m moves rise from 7.2% to 14.8% and 2 m moves from 48.1% to 59.8%.The shift persists for six of eight models after controlling for closer pedestrians and increased collision feedback.Source: §3.3 · p. 7, Figure 4 · p. 7

3.4 RQ3: HOW DOES REASONING EFFORT AFFECT NAVIGATION AND SAFETY?

  • Increasing reasoning effort from provider default to Higher changes task success from 94.1% to 94.8%, but collisions rise from 40.7 to 62.0 per episode and safe success falls from 0.7% to 0.0%.Moving from Medium to Higher increases response latency and worsens the weighted safety score for every model.Source: §3.4 · p. 7
  • Fable’s route progress rises from 1.69 to 1.88 m per decision between Medium and Higher effort, while latency increases from 5.8 to 51.9 s.Across models, provider-default-to-Higher effort slightly reduces active collisions from 6.4 to 5.9 per episode but raises passive collisions from 34.3 to 56.1.Source: §3.4 · pp. 7–8

3.5 IMPROVING REAL-TIME SAFETY DECISION MAKING

  • Offline RL uses an IQL-style expectile value-learning objective with a CQL regularizer, while BC uses the same backbone with cross-entropy loss.BC sampling at temperature 1.0 avoids some indefinite waits at signalized crossings that occur when selecting the highest-logit action.Source: §3.5 · pp. 8–9
  • The “expert + RL data” policy achieves 87.5% success, tied with behavior cloning, and 8.9× and 3.4× fewer collisions per 100 m than prompting and behavior cloning.Its SPL is 0.870; Table 1 reports offline RL evaluation on 16 tasks from two held-out maps.Source: §3.5 · p. 8, Table 1 · p. 9

4 RELATED WORK

Prior work evaluates digital-agent safety, embodied hazards, dynamic navigation, and latency-aware control, but RT-SAFE focuses on safety during benign navigation while general-purpose VLMs act through a shared discrete interface. Its protocol measures safety as the environment evolves during inference and execution, without a separate reactive policy revising actions.

Safety Benchmarks for Digital and Embodied Agents.

  • RT-SAFE combines urban collisions, environmental hazards, and traffic violations, recording whether collisions occur during inference or execution and comparing paused and real-time protocols.This comparison tests whether safety measured with a frozen inference interval carries over to an evolving environment.Source: §4 · p. 9
  • Prior safety benchmarks span tool-use risks, harmful execution, hazardous instructions, planning, and procedural or cumulative risks; ReactHuman pauses simulation during inference.RT-SAFE instead focuses on safety during benign navigation as the world evolves throughout measured inference delays.Source: §4 · p. 9

Real-Time Embodied Planning and Latency-Aware Control.

  • Other work addresses dynamic navigation, driving, asynchronous interaction, delayed control, and timing-sensitive decisions, which RT-SAFE complements by jointly analyzing latency, decision frequency, and action selection.The analysis includes whether benefits of greater reasoning effort compensate for safety risks accumulated while deciding.Source: §4 · pp. 9–10
  • Unlike approaches that overlap inference with execution or use reactive policies, RT-SAFE evaluates general-purpose VLMs through a shared discrete navigation interface without revising selected actions.Measured response times enter the evolving simulation as part of the agent’s physical exposure.Source: §4 · p. 10

5 CONCLUSION

RT-SAFE finds that similar task success in hard static and real-time evaluation can coexist with sharply lower safe success and more collisions. Its appendix documents the benchmark’s urban routes and actors, agent interface and timing, and safety-event logging.

5 CONCLUSION

  • In hard environments, real-time evaluation yields similar task success to static evaluation, but collisions increase 12.3× and safe success falls from 19.8% to 0.7%.More productive decisions can coexist with greater collision exposure during inference; offline RL reward design produces different tradeoffs among task completion, progress, and collision avoidance.Source: §5 · p. 10

A BENCHMARK IMPLEMENTATION DETAILS

  • The appendix documents RT-SAFE’s environment and actors, agent interface and control, safety-event detection, and metrics and episode records.It organizes implementation details across Sections A.1–A.4.Source: A BENCHMARK IMPLEMENTATION DETAILS · p. 16

A.1 ENVIRONMENT SETUP AND ACTORS

  • Five city maps contain 36 routes with varied lengths, turns, crossings, and hazard exposure; easy, medium, and hard conditions activate nested 60%, 80%, and 100% actor subsets.Matched comparisons use the same initial scene configuration and simulation seed.Source: §A.1 · p. 16

A.2 AGENT INTERFACE AND CONTROL

  • Agents receive state, subgoal, recent interaction history, traffic rules, and images, then choose among 16 movement, turning, or waiting actions; real-time planning intervals expose the evolving world.Planning time includes prompt construction, model call, response receipt, and parsing, while actions begin only after the full response is processed.Source: §A.2 · p. 16

A.3 SAFETY RULES AND DETECTION

  • Active contacts incur a 6 s penalty; active building contacts also trigger a 3 s rollback interval, and three consecutive building-collision decisions terminate an episode.Trip hazards add a 6 s recovery interval; oil halves the next move’s speed, while recorded water events do not alter movement in the reported implementation.Source: §A.3 · p. 20
  • The simulator records event feedback and timing, distinguishing passive contacts during planning from active contacts during execution and recovery.Collision counts use engine-counter changes and blocked-move flags; hazard triggers are polled after actions, while traffic violations can stage conflict vehicles.Source: §A.2 · p. 16, §A.3 · pp. 19–20

A.4 EVALUATION METRICS AND RECORDS

RT-SAFE reports task completion and safety separately, with event counts, action behavior, and aggregation conventions specified for interpreting results.

A.4 EVALUATION METRICS AND RECORDS

  • The three-difficulty mean is the arithmetic mean of easy, medium, and hard episode means, and event counts include successful and failed episodes.Rates use corresponding episodes; passive share is total passive contacts divided by total contacts, not the fraction of episodes with a passive event.Source: §A.4 · p. 20
  • Hazard interactions and traffic violations are counted separately from collisions, and a violation and its resulting collision remain distinct recorded events.Conflict-vehicle impacts and launches are campaign totals; their ratio measures impacts per launched vehicle.Source: §A.4 · p. 20
  • RT-SAFE’s safety-event records include each event’s detection rule and consequence, while model abbreviations and provider-specific effort settings follow the reporting conventions in Table 3.Displayed totals may differ slightly from sums of components because of rounding.Source: §A.4 · p. 20, Table 2 · p. 21, Table 3 · p. 21

B EXPERIMENTAL CONFIGURATION AND DETAILED RESULTS

The appendix groups detailed results by research question and specifies model configurations, comparison units, and reporting conventions. Comparisons hold routes and seeds fixed within conditions, while effort labels and model membership vary by provider and analysis.

B EXPERIMENTAL CONFIGURATION AND DETAILED RESULTS

  • Results are grouped by research question: RQ1 reports difficulty summaries and hard-setting breakdowns, RQ2 compares static and real-time execution, and RQ3 compares reasoning effort.Success rates, safe success rates, and action shares are percentages; event and decision counts are per episode unless noted.Source: B EXPERIMENTAL CONFIGURATION AND… · p. 21

B.1 MODEL CONFIGURATION AND REPORTING

  • The action interface, route suite, and simulation seed remain fixed within each comparison, while RQ2 uses mode-specific timing instructions.Response times include the model’s serving interface and should not be interpreted as model-intrinsic compute requirements.Source: §B.1 · p. 21
  • Table 4 reports mean ± standard deviation across three simulation seeds on eight fixed hard real-time routes per model.Each seed-level metric is averaged over the eight routes; collision counts are per episode unless otherwise specified.Source: Table 4 · p. 22
  • RQ1 averages the same 36 routes across three difficulties; RQ2 compares each model on those routes in hard static and real-time settings, and RQ3 compares effort settings on hard real-time routes.Unreported values are dashes, not zeros or substitutes from another condition.Source: §B.1 · p. 22
  • Effort labels are provider-specific, and every model has three reported conditions; the default–Higher aggregate includes all eight models, with 288 episodes per condition.Sol contributes its low and max conditions; provider-default effort is the lowest rung for Sol and the middle rung for other models.Source: §B.1 · p. 22

B.2 VARIANCE AND STABILITY ANALYSIS

Across three seed-level route averages, collision-count variability was lower for Sol and Gemini than for Inkling and DeepSeek. This variation motivates caution when interpreting small performance differences.

B.2 VARIANCE AND STABILITY ANALYSIS

  • Sol and Gemini show relatively low collision-count variability, while Inkling and DeepSeek fluctuate more across seeds.The evaluation covers 96 episodes on eight hard real-time routes across three environment seeds.Source: §B.2 · p. 22
  • Three environment seeds resample actors and traffic consequences while keeping routes, prompts, model settings, and decision budgets fixed.The main run uses seed 0; seeds 1 and 2 provide replications.Source: §B.2 · p. 22
  • Because aggregate stability differs by model, small performance differences warrant caution.Table 4 reports means and standard deviations across the three seed-level route averages.Source: §B.2 · p. 22

B.3 RQ1: DIFFICULTY AND MODEL BEHAVIOR

Across eight models, mean response latency is positively associated with collisions, though individual model profiles differ. The section reports difficulty-averaged real-time behavior and sets out the matched static-versus-real-time evaluation and its reporting conventions.

B.3 RQ1: DIFFICULTY AND MODEL BEHAVIOR

  • The full performance results cover eight models across three difficulties, with 108 episodes per model and 864 overall.Each model completes 36 routes at each difficulty; legitimate task failures remain included.Source: §B.3 · p. 22, Table 5 · p. 23
  • Mean response latency is positively associated with collisions across eight models (R2 = 0.784, p = 0.0035), though Inkling has comparatively few collisions despite slower responses.Sonnet is fastest and has the fewest collisions; DeepSeek combines long response times with a high collision count.Source: §B.3 · p. 22, Figure 7 · p. 23
  • Radar profiles average action-based counts across difficulties and normalize each axis by its maximum across all eight models; radar area is not a composite performance measure.Movement measures commanded distance, while waiting excludes inference and recovery delays.Source: §B.3 · p. 23, Figure 7 · p. 23
  • The exposure proxy is mean response latency multiplied by mean decisions, and the regressions use eight model-level points to test total collisions, not passive collisions alone.The proxy excludes agent-side processing overhead, and the regressions do not establish causation.Source: §B.3 · pp. 23–24

B.4 RQ2: STATIC VERSUS REAL-TIME

  • The static-versus-real-time comparison matches 36 hard routes per model and holds initial configurations and action interfaces fixed, while the world advances only in real-time mode during inference.Prompts contain different timing instructions; tables report aggregate and per-model completion, safety, and decision behavior.Source: §B.4 · p. 24, Table 11 · p. 26, Table 12 · p. 26
  • Static-versus-real-time action-share reporting distinguishes executed-action shares, including parser fallbacks, from valid-choice shares averaged equally across models.Figure 4c and Table 14 use valid choices; aggregate executed-action shares pool actions across models.Source: §B.4 · p. 24, Table 8 · p. 25, Table 10 · p. 25

B.4.1 ACTION SELECTION ACROSS EVALUATION MODES

Commanded 4 m move shares are lower in real time than static mode, including after standardizing on shared prior-step context. The adjusted changes remain negative for at least six models across tested clearance-bin definitions.

B.4.1 ACTION SELECTION ACROSS EVALUATION MODES

  • The 4 m share falls from 32.7% to 15.2% overall and from 50.8% to 9.1% for Grok between static and real-time modes.Shares pool valid choices within each model and mode, then average the eight model-level proportions equally; they measure commanded moves, not realized displacement.Source: §B.4.1 · p. 27
  • The adjustment matches coarse histories of recent crowding and collision feedback, not complete simulator states or visible actors.Only strata observed in both modes contribute to the standardization.Source: §B.4.1 · p. 27
  • Adjusted changes are negative for six models using the primary 200/400 cm cut points, and the same six remain negative under 100/300 and 250/500 cm alternatives.Astra also becomes negative under these alternatives, while Sol remains positive.Source: §B.4.1 · p. 27

B.5 RQ3: REASONING EFFORT

Across all eight models, higher reasoning effort slightly raises task success while collisions rise and safe success falls. The section also defines its weighted event score and route-progress measure, and describes the expert data used for offline training.

B.5 RQ3: REASONING EFFORT

  • Model–effort conditions use the same 36 hard real-time routes and fixed prompts and interfaces, but effort labels are provider-specific rather than equivalent reasoning budgets.The world advances during inference and action execution.Source: §B.5 · p. 27, Table 15 · p. 28
  • Task success changes from 94.1% to 94.8% from provider-default to higher effort, while collisions increase from 40.7 to 62.0 per episode and safe success falls from 0.7% to 0.0%.The comparison includes all eight models and 288 episodes per condition; failed episodes remain included.Source: §B.5 · p. 28, Table 16 · p. 29
  • Net progress includes successful and failed episodes, counts backward movement and rollback against progress, and stops at the last locatable post-action state when terminal-collision telemetry is unavailable.Safety-event totals and recorded inference exposure retain terminal events and intervals beyond the progress-metric window.Source: §B.5 · p. 29
  • The safety score uses the negative per-episode penalty C + H + 10V, where C, H, and V count collisions, hazard interactions, and traffic violations.The multiplier 10 is a chosen severity weight, not an empirically calibrated conversion to physical harm, and no weight-sensitivity analysis is reported.Source: §B.5 · pp. 29–30

C.1 OFFLINE DATASET AND EXPERT POLICY

  • The offline dataset uses an A*-based expert that plans with privileged simulator state, but training observations for behavior cloning and offline RL are non-privileged.The expert uses the same fixed 3 s decision delay as RL evaluation; 105 episodes (4632 transitions) are collected for training and 12 episodes (403 transitions) for validation.Source: §C.1 · p. 30

C.2 OFFLINE RL AND BC ARCHITECTURES

Offline RL and behavior cloning share a VLM encoder but use different prediction heads and training objectives. The policies are evaluated with a fixed three-second latency; reward components and training hyperparameters are specified separately.

C.2 OFFLINE RL AND BC ARCHITECTURES

  • The offline RL and BC models use Qwen3.5-4B with LoRA adapters and a shared last-token state representation, but use different output heads.Offline RL predicts Q-values and state values; BC predicts action logits.Source: §C.2 · p. 30
  • Both policies center the final hidden representation using an EMA of its training-time mean, reusing that mean at evaluation.Source: §C.2 · p. 30
  • Offline value learning optimizes over training-set transitions, with expectile regression parameterized by τ and α weighting CQL regularization.The objective also uses a discount factor γ and a stop-gradient operator.Source: §C.2 · p. 30
  • At inference, offline RL selects the action with the highest predicted Q-value, using a target Q network whose parameters are smoothed with EMA.The Q and value networks share the VLM backbone and differ only in their output heads.Source: §C.2 · p. 30
  • Offline RL and BC training and evaluation use a fixed three-second latency, matching the latency used to collect the offline dataset.The VLM encodes the observation using the last-token hidden representation.Source: §C.2 · p. 31

C.3 REWARD FUNCTIONS AND HYPERPARAMETERS

C.4 EVALUATION PROTOCOL

The offline RL comparison evaluates policies on 16 tasks across two held-out maps and two environment seeds, using the easy real-time setting and fixed three-second latency.

C.4 EVALUATION PROTOCOL

  • Evaluation covers 16 tasks on held-out maps RT15 and RT20, repeated across two environment seeds, in the easy real-time setting with fixed three-second latency.Episodes are marked failed for stagnation if the agent makes no single-step progress of at least 25 cm toward the subgoal over the last 100 decisions.Source: §C.4 · p. 31

Related papers

Loading 2610.09294v1…