PaperlayerView exact revision on arXiv

Source-linked AI summary

Benchmarking Obstacle-Conditioned Safety in Household Robot Navigation

Hyunwoong Bae, Jisu Han, Jiyeon Choi, Dohoon Kim, Taesup Moon

arXiv:2610.34674v1cs.ROcs.AIcs.CV

TL;DR

Safe household navigation requires motion to reflect obstacle safety relevance and proximity, a capability existing evaluations do not test. OSMA-Bench and SSI assess this behavior in matched scenarios. Evaluated VLM- and VLA-based policies do not consistently adapt, while separating parameter inference from code generation improves VLM adaptation.

  • Problem

    Existing evaluations do not test whether semantic safety distinctions are reflected in continuous motion, despite the need to adapt clearance and motion to obstacle type and proximity.

    Source: §1 · pp. 1–2

  • Method

    OSMA-Bench pairs scenarios that vary obstacle type while holding spatial conditions fixed, and SSI measures whether clearance and proximity-dependent motion follow the expected caution ordering.

    Source: §1 · p. 2

  • Results

    Evaluated VLM- and VLA-based policies do not consistently adapt motion to obstacle semantics; separating VLM parameter inference from code generation improves adaptation, while VLA interventions yield limited improvements.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    The results identify a semantic-to-motion grounding gap: safety-relevant distinctions do not reliably guide executed motion in the evaluated VLM- and VLA-based policies.

    Source: §1 · p. 2, §5 · p. 10

    The cited SSI result is based on only 6 of 35 cells satisfying the CFSR-I criterion and should be interpreted cautiously.

    Source: §4.2 · p. 6

Abstract

from arXiv · show

Safe household navigation requires robots to regulate clearance and speed according to what an obstacle is and how close it is, without unnecessary conservatism. Yet existing evaluations do not test whether semantic safety distinctions are reflected in continuous motion. To that end, we introduce OSMA-Bench, an Obstacle-Conditioned Safe Motion Adaptation Benchmark built on spatially matched scenario sets that differ only in obstacle type. We also propose the \emph{Semantic Sensitivity Index} (SSI), which measures whether clearance and distance-dependent speed and smoothness follow the caution ordering of obstacles. Evaluating VLM-based and vision-language-action (VLA) policies on the benchmark, we find that neither consistently adapts its motion to obstacle safety, even with explicit safety prompts. Our analysis shows that the policies can correctly distinguish obstacles by safety relevance but fail to reflect this distinction in the motion parameters or actions they produce. For VLM-based policies, which recognize the obstacle and then generate a navigation program, adding an explicit parameter-inference step between these two stages improves adaptation. These findings expose a gap in \emph{semantic-to-motion grounding}, highlighting the need to translate semantic caution into appropriately safe motion.

1 INTRODUCTION

Household robots need to adapt clearance and motion to both obstacle safety relevance and proximity, while avoiding needless conservatism. OSMA-Bench and SSI test this semantic-to-motion grounding, and evaluations find that current VLM- and VLA-based policies do not consistently adapt their motion.

1 INTRODUCTION

  • Safe household navigation requires more clearance and slower motion near a child than a trash bin, with further slowing as the robot gets closer.Uniform conservatism toward every obstacle would needlessly sacrifice efficiency.Source: §1 · p. 1
  • Semantic-to-motion grounding requires inferring caution from obstacle semantics and translating it into clearance, proximity-dependent speed, and executed motion.The paper asks whether VLMs and VLAs exhibit this capability in their executed trajectories.Source: §1 · p. 1
  • Existing evaluations assess social navigation or embodied safety, but do not test whether diverse obstacles’ safety distinctions shape continuous motion.The benchmark addresses this gap with comparisons that vary obstacle identity while holding route geometry fixed.Source: §1 · pp. 1–2
  • OSMA-Bench pairs spatially matched scenarios with SSI to test obstacle-conditioned motion, finding that evaluated VLM- and VLA-based policies do not consistently adapt, even with safety prompts.For VLMs, separating parameter inference from code generation improves adaptation; VLA prompting and action-selection interventions yield limited recovery.Source: §1 · p. 2

2 RELATED WORKS

Prior work evaluates social navigation, safety decisions, and robot planning, but does not establish whether motion adapts to diverse obstacle types under matched spatial conditions.

Safety Evaluation in Embodied Robotics

  • Social- and human-aware navigation evaluate executed trajectories and human interactions, while embodied-safety evaluations assess safety judgments, actions, or task requirements [Tsoi et al., 2020; Biswas et al., 2022; Li et al., 2024; Dong et al., 2026; Fang et al., 2026; Sermanet et al., 2025; Chen et al., 2025; Lu et al., 2026; Pu et al., 2026; Zhang et al., 2025b; Huang et al., 2025].The cited evaluations cover distinct parts of safety, including visual reasoning, unsafe-action detection, and adversarial robustness.Source: §2 · pp. 2–3
  • Existing paradigms focus on people, safety judgments, or procedural actions rather than paired tests that vary obstacle identity while fixing spatial configuration.Thus, they do not directly evaluate whether diverse obstacles’ safety implications shape executed motion.Source: §2 · p. 3

Foundation Models for Robot Planning and Control

3 OSMA-BENCH BENCHMARK

OSMA-Bench provides controlled, trajectory-level comparisons for diagnosing obstacle-conditioned safe motion adaptation. It varies obstacle identity and spatial conditions while holding other task conditions constant, then evaluates task-level and motion-level safety.

3 OSMA-BENCH BENCHMARK

  • OSMA-Bench enables reproducible, trajectory-level paired comparisons by varying obstacle identity, route geometry, and layout while holding other task conditions constant.The benchmark is built on RoboCasa1.Source: §3 · p. 3

3.1 TASK DESIGN

  • The 625 scenarios combine 18 obstacles in three caution tiers with seven route geometries and five kitchen layouts to test adaptation across spatial configurations.Matched comparisons hold obstacle location fixed; the tiers are High (Living), Moderate (Fragile), and Low (Robust), in descending order of expected caution.Source: §3.1 · pp. 3–4, Table 1 · p. 3

3.2 EVALUATION METRICS

  • The evaluation separates task-level safety from motion-level safety because reaching the goal without collision does not show whether motion adapts to obstacle caution.For within-policy-group motion comparisons, SSI uses the intersection of tasks all models complete successfully without collision (CFSR-I).Source: §3.2 · p. 4, §3.2.1 · p. 4, §3.2.2 · p. 5

3.2.2 MOTION-LEVEL SAFETY METRIC

  • Motion features combine clearance with proximity-normalized speed, acceleration, and jerk measured within a 1.25 m obstacle boundary.Mean and maximum values of the normalized quantities capture general motion and transient peaks; minimum clearance is measured over the full trajectory.Source: §3.2.2 · pp. 4–5
  • SSI compares sign-aligned motion-feature means across matched layout–route cells and obstacle caution tiers, then aggregates tier-order agreement across features.It ranges from −1 for complete reversal to +1 for complete agreement; zero indicates equal numbers of agreeing and disagreeing pairs, or potentially cancellation.Source: §3.2.2 · p. 5

4 EXPERIMENT RESULTS

Across three policy groups, the evaluation tests whether foundation-model policies adapt motion to obstacle caution, then probes how semantic knowledge becomes control. The evaluated policies show limited or inconsistent obstacle-conditioned safe motion, while separating control-value inference from task-code generation improves VLM-based adaptation.

4.1 EXPERIMENTAL SETUP

  • The study evaluates VLA, VoxPoser, and PIVOT policies in closed loop, using matched comparisons to assess executed motion across obstacle types.The VLM policies use code generation or waypoint selection, while VLA policies generate actions from visual observations and language instructions.Source: §4.1 · p. 6

4.2 RQ1: DO FOUNDATION MODELS GENERATE OBSTACLE-CONDITIONED SAFE MOTION?

  • No evaluated policy reaches the ideal combination of collision-free task success and perfect agreement with the expected caution ordering.VLA policies have low CFSR/TSR and negative SSI; VLM-based policies fare better on task-level safety but often have SSI near zero, while VoxPoser is consistently positive.Source: §4.2 · pp. 6–7, Figure 3 · p. 7

4.3 RQ2: CAN VLMS DISTINGUISH AMONG OCTS, AND HOW CAN THESE DISTINCTIONS BE TRANSLATED INTO EXECUTED MOTION?

  • For Qwen3.5-9B, isolated value inference reaches value-level SSI 0.98, while separating value inference from code generation raises value-level SSI to 0.97 but improves executed SSI by only +0.01.Explicit safety prompting raises value-level SSI to 0.54–0.58 and improves executed SSI by +0.01 to +0.13; jointly generating task code may hinder inference of obstacle-appropriate control values.Source: §4.3 · p. 8, Table 2 · p. 8
  • Because trajectories never enter their assigned velocity regions, a three-step decomposition conditions the velocity-region setting on clearance and improves SSI and value-level SSI.The inferred velocity-region radius satisfies RV ≥ RC + Rbase in 59.0% of tasks; effective grounding requires ordered control values configured to affect execution.Source: §4.3 · p. 8, Figure 4 · p. 8

4.4 RQ3: CAN EXTERNAL GUIDANCE IMPROVE VLA SAFETY, AND DOES THE VLA BACKBONE RETAIN OBSTACLE SAFETY KNOWLEDGE?

  • The VLA study tests prompt guidance and rollout selection, and finds that safety-relevant information remains accessible in the backbone but is not reliably expressed in executed motion.A reconstructed GR00T N1.5 backbone reaches value-level SSI 0.573, and a linear probe distinguishes 18 obstacle identities with 61.7% accuracy.Source: §4.4 · pp. 9–10

5 CONCLUSION

The paper presents OSMA-Bench and SSI to evaluate whether motion reflects obstacle caution, and finds that current VLM- and VLA-based policies do not adapt consistently. It concludes that semantic distinctions must be translated into appropriate control parameters and executed motion.

5 CONCLUSION

  • OSMA-Bench uses spatially matched obstacle-type comparisons, while SSI measures whether clearance and proximity-normalized motion follow expected caution ordering.VLM- and VLA-based policies do not consistently adapt motion; separating control-value inference from task-code generation improves VLM-based adaptation, while VLA guidance yields limited improvement.Source: §5 · p. 10

A LIMITATIONS AND FUTURE WORK

The paper identifies boundaries in the benchmark’s task setting, motion metric, model coverage, caution-tier interpretation, and spatial matching. Its task variants also combine obstacle caution tiers with route and kitchen-layout axes.

A LIMITATIONS AND FUTURE WORK

  • The study is limited to simulation navigation; manipulation and physical-platform evaluation are proposed as possible next steps.SSI covers a fixed set of motion features and omits behaviors such as approach direction and legibility; the authors suggest human-aligned VLM evaluators for more holistic assessment.Source: A LIMITATIONS AND FUTURE WORK · p. 15
  • The evaluated policies are compact open-weight VLM-based and VLA models; adding frontier models, classical planners, and RL policies would broaden coverage.Caution tiers encode relative conservatism within this benchmark, not a universal ordering across contexts; obstacle size may also partly confound clearance differences despite matched positions.Source: A LIMITATIONS AND FUTURE WORK · p. 15

B OSMA-BENCH TASK DETAILS

  • OSMA-Bench varies obstacle caution tier alongside route geometry and kitchen layout, preserving matched comparisons across tiers.Route geometry varies local spatial relationships, while kitchen layouts vary global configurations.Source: B OSMA-BENCH TASK DETAILS · p. 15

B.1 SPATIAL AXES: ROUTE GEOMETRY AND LAYOUT VARIANTS

The benchmark varies local route geometry and global kitchen layout while retaining matched obstacle-tier comparisons. Results across visual styles show outcome variation, and the appendix reports evaluation costs.

B.1 SPATIAL AXES: ROUTE GEOMETRY AND LAYOUT VARIANTS

  • The benchmark uses seven start–goal routes to vary path length, direction, and turning structure, and five kitchen layouts to vary global spatial configuration.It excludes narrow layouts with too little clearance variation and layouts whose central island forces the same detour across obstacle tiers.Source: §B.1 · p. 15
  • Across styles, the three-step policy has the highest SSI under both INDUSTRIAL and MODERN 1, while appearance changes collision-free outcomes.SSI changes by up to 0.08 between styles; under the three-step policy, 24 successful INDUSTRIAL tasks fail in MODERN 1 and 9 change in the opposite direction.Source: §B.1 · pp. 15–16, Table 4 · p. 17

B.2 COMPUTATIONAL COST

  • Across five models, median per-task wall-clock time is 119 s for VoxPoser and 170 s for PIVOT; the 9,375-rollout VoxPoser evaluation took approximately 16 wall-clock hours.The reported PIVOT timings include text-LMP response caching across tasks and seeds.Source: §B.2 · p. 17

C EVALUATION METRIC DETAILS

The appendix specifies the notation and metrics used to evaluate navigation performance and whether motion reflects the expected caution ordering across obstacle tiers. Its metric design compares matched layout–route cells and distinguishes local obstacle-proximal behavior from global route choice.

C EVALUATION METRIC DETAILS

C.1 NOTATION

  • Distance-based metrics use only safety-relevant obstacles, with surface-to-surface distance measured from the robot to each obstacle at each timestep.Source: §C.1 · p. 17

C.3 CAUTION TIERS

  • Living, fragile, and robust obstacles form high-, moderate-, and low-caution tiers, respectively, as an evaluation hierarchy rather than a universal safety law.The tiers reflect harm severity and collision likelihood; residual obstacle-size effects on passage width remain a limitation.Source: §C.3 · pp. 17–18

C.4 MOTION FEATURES

  • Motion features include full-trajectory minimum clearance and obstacle-proximal speed, acceleration, and jerk ratios, each summarized by means and maxima.The ratios are measured within a shared 1.25 m boundary; jerk is estimated from a smoothed position fit because third finite differences amplify jitter.Source: §C.4 · p. 20
  • Distance-normalized speed and smoothness ratios are expected to decrease with increasing caution because the same motion leaves less stopping distance and can increase contact risk near obstacles.Source: §C.4 · p. 20

C.5 NAVIGATION PERFORMANCE METRICS

  • Normalized path length compares executed distance with the shortest collision-free reference path, while collision steps, first-contact velocity, and normalized traversal time complement success rates.Path length is excluded from SSI because it reflects global route choice and can obscure local tier-dependent adaptation.Source: §C.5 · p. 21

C.6 SEMANTIC SENSITIVITY INDEX (SSI)

  • SSI sign-aligns motion features so larger values mean more conservative behavior, then measures Kendall’s τ ordering of tier means within matched layout–route cells.It averages over eligible cells and features; SSI = 0 is the expected value for a tier-indifferent policy, while a negative score indicates systematically reversed ordering.Source: §C.6 · pp. 21–23
  • The primary SSI analysis uses tasks all models in a policy group complete successfully without collision, with broader task subsets also reported for comparison.Source: §C.6 · p. 22

D POLICY IMPLEMENTATION DETAILS

The policy implementations cover direct-action VLA systems and VLM systems that produce navigation plans or waypoints. VLM outputs are converted into robot motion through a shared waypoint follower, and the appendix identifies the settings used for the later results.

D.1 VLA

  • The evaluated VLA checkpoints generate action sequences from language instructions, RGB observations, and robot state, then execute the predicted chunks before querying again.The checkpoints are RoboCasa-adapted releases evaluated without additional fine-tuning; their differing training and execution settings mean the comparison does not isolate architecture effects.Source: §D.1 · pp. 23–24
  • VLA inputs include three camera views and proprioception, but no top-down images, ground-truth obstacle positions, or caution-tier labels by default.Source: §D.1 · p. 23

D.2.1 CODE GENERATION: VOXPOSER

  • VoxPoser has the VLM generate code that instantiates affordance, avoidance, rotation, and velocity settings, which a planner uses to produce a waypoint path.Its per-obstacle control parameters include clearance and speed scaling within a specified distance; the mobile-base adaptation uses 2D maps and A* planning.Source: §D.2.1 · pp. 25–26

D.2.2 WAYPOINT SELECTION: PIVOT

  • PIVOT iteratively selects and refines waypoint candidates over three rounds, then separately queries the VLM for a segment-wise speed profile.Its waypoint prompt includes obstacle motion context and anticipated short-horizon swept regions.Source: §D.2.2 · p. 26

D.2.3 SHARED ACTION OUTPUT AND EXECUTION

  • A shared holonomic waypoint follower converts VoxPoser paths or PIVOT waypoints and speed settings into low-level base commands; neither planner directly predicts the action vector.The follower uses robosuite’s BASIC composite controller and sets arm, gripper, and torso entries to zero.Source: §D.2.3 · p. 26

E MAIN RESULTS

  • The appendix states that qualitative and quantitative results are presented in the following subsections.Source: E MAIN RESULTS · p. 26

E.1 QUANTITATIVE RESULTS

Across policy groups, task completion, collision avoidance, and semantic adaptation do not consistently align. VoxPoser shows the strongest combination of avoidance, efficiency, and semantic sensitivity, but its adaptation remains incomplete.

E.1 QUANTITATIVE RESULTS

  • VLA policies show limited task reliability and no positive mean semantic sensitivity: CFSR-I SSI ranges from −0.595 to −0.262.Their TSR ranges from 28.1% to 69.2% and CFSR from 11.4% to 39.2%; seed variability limits conclusions about the strength of the negative ordering.Source: §E.1 · pp. 26–27, Table 12 · p. 27
  • PIVOT improves CFSR/TSR to 77.8%–89.5%, but mean SSI remains negative for every evaluated model after excluding Cosmos-Reason2-8B.PIVOT also has higher NPL and NPT than corresponding VoxPoser backbones, and robot-domain specialization does not resolve the adaptation limitation.Source: §E.1 · p. 27, Table 13 · p. 28
  • VoxPoser combines CFSR/TSR of 94.9%–99.5% with positive SSI of 0.258–0.561 after excluding Cosmos-Reason2-8B, and lower NPL and NPT than corresponding PIVOT backbones.Limits remain: Cosmos-Reason2-8B has CFSR/TSR of 96.0% but TSR of 52.5%, while Qwen3.5-4B has 99.5% CFSR/TSR and 286.8 average contact steps.Source: §E.1 · p. 27
  • Across the three policy groups, task completion, collision avoidance, and semantic adaptation remain distinct capabilities; even VoxPoser SSI stays below 0.6.For VLM-based policies on CFSR-I, clearance increases from low to moderate caution but decreases from moderate to high caution.Source: §E.1 · pp. 27–28, Table 13 · p. 28

E.2 QUALITATIVE RESULTS

  • In one matched case, VoxPoser–RoboBrain2.5-8B-NV avoids collision on a compact route but its clearance rises by 0.248 m from low to moderate caution and falls by 0.080 m from moderate to high.Its route and collision results align with lower NPL (1.375 versus 1.701) and higher CFSR (94.3% versus 71.0%) than PIVOT with the same backbone.Source: §E.2 · p. 28

E.3 BREAKDOWN OF FAILED EXECUTIONS

The failure analysis separates unsuccessful navigation into target-generation, pipeline, orientation, and execution problems. The appendix also describes a velocity-region threshold and a modular parameter-inference procedure intended to make requested motion values affect execution.

E.3 BREAKDOWN OF FAILED EXECUTIONS

  • VLA success rates are especially low on Routes E and F: 0.0–12.2% and 0.0–2.4%, respectively, versus 38.7–93.7% on the other routes combined.Doors and people are outside the original RoboCasa NavigateKitchen target distribution, suggesting limited generalization to those destinations.Source: §E.3 · pp. 28, 30, Table 14 · p. 30
  • Across 7,500 VLA rollouts, all 3,748 failures reached the evaluation horizon, and 3,739 ended outside the 0.5 m goal radius.The analyzed final-segment behaviors include stagnation, movement without net progress, and an incomplete approach at timeout; the illustrated failures had no recorded obstacle contact.Source: §E.3 · p. 30, Figure 8 · p. 31
  • Among VLM navigation failures, 82.9% were unresolved targets that could not be grounded to scene objects; 17.1% were empty targets or invalid planner code, preventing trajectory rollout.For get affordance map(’stove’), the map has no positive cells, so the task terminates before trajectory rollout.Source: §E.3 · pp. 30–31
  • Among PIVOT task failures, 71.5% are pipeline interruptions, 18.2% terminal-orientation failures, and 10.3% failures to reach the goal.All terminal-orientation cases occur on Route F, while all 177 logged failures to reach the goal have planned endpoints within tolerance, indicating failure during execution.Source: §E.3 · p. 31

F.1 EXPERIMENT DESIGN

  • The appendix’s isolated value-level probe asks for control values on each of the 625 tasks, while motion SSI evaluates executed trajectories and averages Kendall τ across seven features.Motion statistics use the common collision-free successful task intersection for the compared methods; TSR and CFSR instead use all planned tasks.Source: §F.1 · pp. 31–32

F.2 WHEN THE VELOCITY REGION IS TRAVERSED

  • The assigned velocity affects execution only when RV ≥ RC + Rbase; the measured Rbase is 36 cm across 2,454 rollouts.Without the threshold, the original VoxPoser and safety-prompt variants do not traverse the velocity region, and the two-step inference also places RV too close to RC.Source: §F.2 · p. 32
  • Tasks meeting the velocity-region threshold spend 94–100% of their closest-approach segment inside it, while tasks below the threshold almost never enter it.The paper notes that Rbase depends on the platform footprint and can be recovered by the same measurement.Source: §F.2 · p. 32

F.3 MODULAR DESIGN

  • The three-step design infers clearance, then speed and velocity radius, before inserting those values into the generated program’s obstacle-map calls.A final clamp raises RV to at least RC + Rbase before execution; safety-prompt variants append guidance without changing the rest of the pipeline.Source: §F.3 · p. 33

F.4 FULL RESULTS

For Qwen3.5-9B, the three-step decomposition improves ordering of mean motion features and makes requested speed affect executed motion. Clearance remains the unresolved dimension: the requested ordering largely fails to appear in executed minimum distance.

F.4 FULL RESULTS

  • Mean-feature ordering improves under the three-step method: µv, µa, and µj reach +0.89, +0.92, and +0.90, versus +0.52, +0.41, and +0.39 for baseline.Peak-feature ordering changes little, while dmin scores lower under decomposition than baseline (+0.31 versus +0.49).Source: §F.4 · p. 33, Table 16 · p. 34
  • Under decomposition, requested speeds take effect and µv for High obstacles roughly halves to 0.49, 0.65, and 0.85 s−1.In baseline and safety-prompt variants, velocity regions are not traversed and µv separates obstacle tiers only weakly.Source: §F.4 · p. 34, Table 17 · p. 34
  • Executed dmin is the feature the decomposition does not reorder, despite the requested control values and their corresponding executed features being compared by obstacle tier.The passage identifies the residual distance-axis gap as the subject of the following section.Source: §F.4 · p. 34, Table 17 · p. 34

F.5 RESULTS ON A SMALLER MODEL

The smaller model responds differently to safety prompts than the 9B model, with gains accompanied by lower task success. The remaining distance-axis gap is traced to the planner’s objective and specific table-top obstacles, while very low requested speeds account for three-step failures.

F.5 RESULTS ON A SMALLER MODEL

  • For Qwen3.5-4B, safety prompts raise SSI from +0.33 to +0.59 with Per obstacle and +0.53 with Both, while three-step decomposition reaches +0.44.These SSI gains coincide with lower TSR and CFSR; the CFSR-I for Collision risk contains only 217 of 625 tasks, so scores reflect that selection.Source: §F.5 · pp. 34–35, Table 18 · p. 35, Table 19 · p. 35

F.6 RESIDUAL GAP ON THE DISTANCE AXIS

  • Requested clearance is correctly ordered in 34 of 35 cells, but executed dmin reproduces that order in only one cell (3%), below the 17% chance expectation.The request is met in 56% of tasks, and a request difference greater than 20 cm survives as about 9 cm in the trajectory.Source: §F.6 · pp. 35–36
  • Four Moderate table-top obstacles account for much of the inversion: their dmin stays near 0.6 m because the planner avoids the table footprint, not the object on it.Without those obstacles, the Moderate mean falls to 0.40 m and executed order holds in 62% of cells; A* minimizes path length outside inflated obstacles without rewarding additional standoff.Source: §F.6 · p. 36

F.7 FAILURE CASES OF THE THREE-STEP METHOD

  • The three-step method loses 3.0 points of TSR against baseline; all 22 failures are High-OCT tasks requesting 20% nominal speed or less.The robot stalls beside the obstacle after a slow pass, and doubling the per-waypoint step budget recovers only six failures.Source: §F.7 · p. 36, Table 20 · p. 36

F.8 REPRODUCTION

  • The released three-step method includes the Step 1 and Step 2 prompt templates so probe and rollout questions match.Reported runs use the same task list, layouts, seed, and camera setup.Source: §F.8 · p. 36

G.1 EXTERNAL GUIDANCE EXPERIMENTS

The experiments vary safety instructions and add a rollout selector to test whether external guidance changes navigation outcomes. Candidate contact outcomes often agree across rollouts, while the selector’s comparison with uniform selection varies by model and candidate count.

G.1 EXTERNAL GUIDANCE EXPERIMENTS

  • Five instruction conditions varied the task goal, generic avoidance guidance, obstacle naming, caution emphasis, or behavioral guidance, with each condition run once on 625 tasks using seed 0.The caution and behavioral conditions each add a distinct sentence to the obstacle-naming condition; none specifies a caution tier or tier-specific clearance or speed.Source: §G.1 · p. 37
  • The selector generated K candidate rollouts from saved states, asked a verifier to choose or reject them, then executed the chosen candidate or the first candidate after rejection or invalid selection.The baseline and selector differ in policy random-number-generator reseeding, so their comparison does not isolate candidate selection’s effect.Source: §G.1 · p. 37
  • Across selection conditions, 86.5–92.2% of recorded decisions had the same contact outcome for every candidate; at K = 16, all candidates contacted in 11.6% of 9B and 13.6% of 35B decisions.These are decision-level contact outcomes during candidate lookahead, not trajectory or image similarity.Source: §G.1 · p. 37
  • Differences from uniform candidate selection were +8.4 percentage points for 9B at K = 4, −9.5 for 35B at K = 8, and −8.9 for 35B at K = 16; intervals for the other three conditions included zero.The score compares the executed candidate’s collision-free status with the expected score under uniform candidate selection.Source: §G.1 · pp. 37–38

G.1.1 DIVERSITY OF SAMPLED ROLLOUTS

Sampled rollouts showed limited diversity, and alternative candidate inputs did not improve observed collision-free performance. Reconstructed GR00T N1.5 inferred obstacle-conditioned control values unevenly, while separately trained classifiers could distinguish obstacle identities from action-head inputs.

G.1.1 DIVERSITY OF SAMPLED ROLLOUTS

  • Median mean pairwise distance stayed 0.100–0.104 m across conditions; near obstacles, only 3.5%–7.2% of decisions offered opposite-direction candidates, though small differences could change contact outcomes.Increasing K raised maximum pairwise distance and endpoint spread while leaving mean pairwise distance nearly unchanged.Source: §G.1.1 · p. 39, Table 22 · p. 39
  • Candidate images were highly similar (SSIM 0.972–0.974; differing pixels 1.60%–1.66%), although small trajectory differences could still change contact outcomes.The image measures include shared scene content and do not establish equivalent motion or contact outcomes.Source: §G.1.1 · p. 39, Table 23 · p. 40
  • On fixed candidate sets, collision-free selection was 58.1% for 9B and 51.5% for 35B, below the 59.1% uniform-selection expectation.This comparison covers decisions at states visited by the 9B system, not complete-task success.Source: §G.1.1 · p. 40
  • Across 625 seed-0 tasks, neither alternative input format improved CFSR over the original grid, and the comparisons established neither significant differences nor equivalence.CFSR was 44.64% for the grid, 44.00% for the path image, and 43.84% for overlays.Source: §G.1.1 · pp. 40–41, Table 25 · p. 42

G.2.1 BACKBONE RECONSTRUCTION

  • The text-decodable GR00T N1.5 reconstruction preserved available checkpoint parameters and added Qwen3-1.7B decoder layers 13–28.Weight comparisons supported correspondence with public checkpoints, with the vision patch embedding an exception.Source: §G.2.1 · p. 43

G.2.2 VALUE-LEVEL RESULTS.

  • Reconstructed GR00T N1.5 reached value-level SSI 0.573, versus 0.978 and 0.990 for the two Qwen models; its clearance ordering was weaker than its velocity ordering.Its mean clearance did not increase consistently across caution tiers, and combining velocity with clearance as v/d reduced SSI to 0.342.Source: §G.2.2 · p. 43, Table 26 · p. 43

G.3 OBSTACLE INFORMATION AVAILABLE TO THE ACTION HEAD

  • Separate classifiers achieved 75.3% vision-encoder and 61.7% action-head-input accuracy; these are not original action-head recognition or risk-use scores.The classifiers were evaluated on the same fixed 162 observations, with nine examples for each of 18 obstacle types.Source: §G.3 · pp. 44–45, Table 27 · p. 44
  • The obstacle-feature analysis was limited to repeated observations from one model, one route, and three layouts.The analysis also notes that classifiers may use scene cues associated with obstacle identity and that accuracy differences do not directly quantify information lost during processing.Source: §G.3 · p. 45

Related papers

Loading 2610.34674v1…