Risk-Aware Semantic Grounding for Trustworthy LLM-Based Robot Planning

1 Introduction

• LLM outputs can be syntactically plausible yet semantically incorrect, including hallucinated objects, wrong room assignments, unsupported assumptions, and ambiguous grounding.In human-centered environments, such errors may reduce task performance and lead to unsafe behavior or loss of user trust.Source: §1 · pp. 1–2
• Existing approaches improve planning reliability but focus on generating better plans rather than deciding whether a plan should be generated at all.They remain vulnerable to ambiguous instructions, nonexistent entities, and semantically inconsistent requests.Source: §1 · p. 2
• The framework assesses ambiguity, hallucination, and semantic conflict before planning, then decides whether an instruction should be executed, clarified, or rejected.TRUST-NAV covers standard navigation tasks and risk-inducing instruction scenarios.Source: §1 · p. 2
• The work formulates trustworthy planning as risk estimation, separates reliability assessment from plan generation, and reports improved ambiguity detection and semantic-conflict rejection with competitive planning performance.Its contributions also include a decision mechanism that executes, clarifies, or rejects instructions and the TRUST-NAV benchmark.Source: §1 · p. 2

2.1 Semantic Mapping for Robot Navigation

• Semantic maps add object, region, and relational information, but prior approaches do not verify whether new instructions are consistent with the map.The cited work ranges from SLAM with object detection to structured maps and open-vocabulary scene graphs.Source: §2.1 · p. 3

2.2 Language-Guided Navigation

• R2R and ALFRED evaluate multi-step instructions, while modular language-to-waypoint systems still assume instructions are valid and prioritize task completion.Source: §2.2 · p. 3

2.3 LLM-Based Task Planning in Robotics

• LLM planners such as SayCan and Inner Monologue improve flexibility but generally assume well-posed instructions and existing referenced entities.Other cited approaches generate structured programs or policy code, or integrate perception and language.Source: §2.3 · p. 3

2.4 Trustworthy and Uncertainty-Aware LLM Planning

• Prior work calibrates uncertainty or evaluates safety [12], whereas this framework scores ambiguity, hallucination, and semantic conflict to choose execute, clarify, or reject before planning.TRUST-NAV targets grounding failures rather than physical danger.Source: §2.4 · p. 4

2.5 Decoupled Planning Architectures

• The framework extends decoupled planning by inserting a rule-based, training-free risk-assessment stage between language understanding and motion execution.The stage can be tuned on a small validation set to isolate risk gating from the planner and map representation.Source: §2.5 · p. 4

3 Methodology

• RA-SGF adds a pre-execution risk-assessment agent and decision layer to instruction-following pipelines, producing a plan or a clarification or rejection when grounding is unreliable.The architecture estimates grounding risk before planning to support execute, clarify, or reject decisions.Source: §3 · pp. 4–5, Figure 1 · p. 5

3.1 Risk Assessment Agent

• An independent LLM scores ambiguity, absent map entities, and instruction–map contradictions, then combines the scores into weighted risk R.Experimental weights are wa = 0.30, wh = 0.40, and wc = 0.30; they were selected on a small validation subset, not optimized on the test benchmark.Source: §3.1 · pp. 5–6

3.2 Decision Layer

• The deterministic decision layer checks per-risk hard ceilings before aggregate soft thresholds, then selects execute, clarify, or reject.Hard ceilings address clear-cut failures, while soft thresholds handle borderline cases; the parameters are adjustable for deployment risk tolerance.Source: §3.2 · p. 6

3.3 Planner Agent

• After execution is approved, the planner uses map tools, flexible name matching, and nearest-first greedy ordering for unordered multiple targets, then returns a structured decision, plan, and response.The nearest-first rule selects the closest remaining target to the robot’s current position at each step.Source: §3.3 · pp. 6–7

4.1 Experiment Environment: Smart Home Case

• The apartment spans 12 × 8 meters, contains eight functional regions, and places the robot at a fixed, known starting position near the entrance.Its rooms contain representative furniture and household objects used as navigation and task targets.Source: §4.1 · p. 8

4.2 TRUST-NAV Benchmark

• TRUST-NAV contains 206 natural-language instructions across five categories, with the first two assessing standard planning and the rest assessing risk-aware decisions.The categories cover single-step and multi-step planning, ambiguity, nonexistent entities, and semantically conflicting object-room combinations.Source: §4.2 · pp. 8–9
• The benchmark assesses planning accuracy alongside uncertainty identification, hallucination prevention, and semantic consistency enforcement.The source code, benchmark, and evaluation scripts are publicly available.Source: §4.2 · p. 9

5 Results and Discussion

• The evaluation first measures executable navigation performance, then assesses trustworthiness-aware decision making.All methods use OpenAI GPT-5.4-mini with the same configuration.Source: §5 · pp. 9–10

Baseline 1: Nearest Semantic Object Planner (NSOP)

• NSOP reduces each instruction to one navigation goal by matching instruction tokens to a semantic-map object type and selecting the nearest instance.It ignores sequencing, object attributes, and intermediate constraints, serving as a lower-bound reference for a geometry-light strategy.Source: Baseline 1: Nearest Semantic Object… · p. 10

Baseline 2: Rule-Based Sequential Global Planner (RBSGP)

• RBSGP splits instructions using temporal or conjunctive cues, selects matching object-property pairs when available, and otherwise chooses the nearest object of the identified type.It returns ordered goals but cannot detect ambiguity, hallucinated entities, or semantic inconsistencies.Source: Baseline 2: Rule-Based Sequential… · p. 10

Baseline 3: Simple LLM Planner (SLLmP)

• SLLmP uses an LLM to decompose instructions into ordered navigation targets from object identities, types, and properties, without geometric information or feasibility checks.This baseline isolates semantic reasoning without explicit spatial grounding.Source: Baseline 3: Simple LLM Planner (SLLmP) · p. 10

Baseline 4: Tool-Augmented LLM Planner Agent (TA-LLmPA)

• TA-LLmPA iteratively uses tools with the full geometric map to retrieve and filter objects and compute distances before returning ordered goals.Unlike the proposed method, it has no explicit risk quantification, per-dimension scoring, or threshold-based gating.Source: Baseline 4: Tool-Augmented LLM Planner… · p. 11

• Planning Accuracy (PA) measures the fraction of executable queries whose generated plan exactly matches the ground-truth object sequence.Source: §5.2 · p. 11
• Ambiguity Detection Rate (ADR) measures the fraction of ambiguous instructions correctly classified as requiring clarification.Source: §5.2 · p. 11
• Hallucination Rejection Rate (HRR) measures the fraction of hallucinated instructions correctly rejected.Source: §5.2 · p. 11
• Semantic Conflict Rejection Rate (SCR) measures the fraction of semantically inconsistent instructions correctly rejected.Source: §5.2 · p. 12
• Decision Accuracy (DA) measures correctly predicted decisions as a fraction of all benchmark queries.The framework assigns instructions to execute, clarify, or reject.Source: §5.2 · p. 12

5.3 Planning Performance

• SLLmP achieves 80.33% multi-step accuracy; TA-LLmPA achieves 100.00% single-step and 65.57% multi-step, while RA-SGF achieves 91.18% and 42.62%, respectively.Table 2 and Fig. 3 summarize planning performance on executable navigation tasks.Source: §5.3 · p. 12, Table 2 · p. 12, Figure 3 · p. 13
• Planning accuracy decreases as task complexity increases.The proposed framework prioritizes reliable pre-execution decisions rather than maximizing execution rates.Source: §5.3 · p. 12
• Planning accuracy alone does not fully capture system reliability because RA-SGF may clarify or reject instructions before planning.The next evaluation therefore considers trustworthiness-oriented decisions.Source: §5.3 · p. 12

5.4 Evaluation under Risk-Inducing Scenarios

• TRUST-NAV assesses whether planners recognize when ambiguous, unsupported, or semantically inconsistent instructions should not be executed.Generating a plan alone is insufficient when execution may be inappropriate.Source: §5.4 · p. 13

Decision Accuracy (DA)

• Decision accuracy measures selection of the appropriate execute, clarify, or reject action, rather than correctness of the generated plan.The measure is reported across all TRUST-NAV categories.Source: Decision Accuracy (DA) · p. 13
• SLLmP achieves the highest overall decision accuracy (84.95%), while RA-SGF performs best on the two failure modes most closely related to trustworthy grounding.RA-SGF is designed to prioritize safe pre-execution decisions over maximizing overall execution rates.Source: Decision Accuracy (DA) · p. 13

Ambiguity Detection Rate (ADR)

• RA-SGF achieves the highest ADR (87.80%), outperforming SLLmP (78.05%) and TA-LLmPA (58.54%).Explicit ambiguity modeling lets the system request clarification instead of committing to an arbitrary grounding.Source: Ambiguity Detection Rate (ADR) · p. 13

Hallucination Rejection Rate (HRR)

• TA-LLmPA performs best on HRR (92.50%), while RA-SGF reaches 82.50% and SLLmP 67.50%.The paper suggests direct semantic-map tool access may make hallucinated entities easier to detect, limiting the gains from explicit risk modeling here.Source: Hallucination Rejection Rate (HRR) · p. 14

Semantic Conflict Rejection Rate (SCR)

• RA-SGF achieves the highest SCR (96.67%), compared with 86.67% for both TA-LLmPA and SLLmP.The paper attributes this result to semantic-conflict analysis that detects logically inconsistent instructions before planning.Source: Semantic Conflict Rejection Rate (SCR) · p. 14
• ADR and SCR support the hypothesis that explicit risk-aware grounding is especially valuable for ambiguous or internally inconsistent instructions.The paper frames these as cases where generating an incorrect plan may be worse than declining execution.Source: Semantic Conflict Rejection Rate (SCR) · p. 14

6 Conclusion

• The framework evaluates semantic-grounding reliability before execution through dedicated ambiguity, hallucination, and semantic-conflict analysis.This contrasts with conventional architectures focused primarily on generating executable navigation plans.Source: §6 · p. 14
• TRUST-NAV includes standard navigation tasks and risk-inducing instruction scenarios to evaluate planning and trustworthiness-aware decision making.Source: §6 · p. 14
• RA-SGF achieves the highest ambiguity detection rate (ADR, 87.80%) and semantic conflict rejection rate (SCR, 96.67%).Conventional LLM planners perform strongly on valid navigation tasks but remain vulnerable to ambiguous, unsupported, and semantically inconsistent requests.Source: §6 · p. 15
• The results support assessing trustworthy systems by whether they recognize when action should not be taken, as well as by whether they act correctly.The paper identifies this as an evaluation philosophy for LLM-based robotics.Source: §6 · p. 15
• TRUST-NAV covers grounding-related failures in a static indoor environment, not perception errors, dynamic scenes, or long-horizon embodied tasks.The benchmark is also limited to a single semantic environment, so findings are not a complete evaluation of trustworthy embodied intelligence.Source: §6 · p. 15
• Future work will explore adaptive threshold calibration, richer semantic consistency models, more diverse benchmark environments, and integration with real-world robots in dynamic settings.Source: §6 · p. 15