PaperlayerView exact revision on arXiv

Source-linked AI summary

RACO: Reliability-Aware Coarse-Goal Optimization for Inspection-Oriented UAV Vision-Language Navigation

Sen Wang, Yiming Sun, Jiaxuan He, Pengfei Zhu

arXiv:2608.22678v1cs.ROcs.AIcs.CV

TL;DR

Inspection-oriented UAV-VLN needs valid inspection-region arrival and correct object confirmation, but existing coarse-to-fine policies may trust drifting coarse goals. The paper introduces LG-UVI and RACO to evaluate and correct these failures, with RACO improving SR over reproduced HETT by 9.53 and 7.98 percentage points on validation-unseen and test-unseen.

  • Problem

    Inspection-oriented UAV-VLN requires valid viewing regions and correct object confirmation while avoiding similar distractors, exposing limits of coordinate-level metrics and unreliable coarse goals.

  • Method

    LG-UVI adds object-centric inspection annotations and diagnostics, while RACO verifies coarse-goal hypotheses with candidate anchors and applies scale-adaptive terminal refinement.

  • Results

    9.53 and 7.98 percentage points: RACO increases SR over reproduced HETT on validation-unseen and test-unseen, respectively, while improving inspection-region arrival and reducing false verification risk.

  • Takeaways & Limitations

    Reliability-aware coarse-to-fine correction complements existing UAV-VLN policies beyond coordinate-level navigation success.

  • Takeaways & Limitations

    All learned gates, scorers, thresholds, candidate-selection rules, and execution bounds are frozen before unseen-split evaluation after calibration on designated training and calibration records.

Abstract

from arXiv · show

UAV vision-language navigation (UAV-VLN) is commonly evaluated as goal reaching, but inspection-oriented deployment requires the agent to stop within a valid inspection region and avoid falsely confirming visually or semantically similar distractors. This requirement exposes a key weakness in existing coarse-to-fine UAV-VLN policies: the coarse goal predicted before local refinement is often treated as reliable, although it may drift toward plausible but incorrect object regions and limit the ability of the local stage to recover. To systematically evaluate this problem, we introduce LG-UVI, an object-centric inspection evaluation setting derived from CityNav/CityRefer. LG-UVI extends standard UAV-VLN episodes with target objects, hard distractors, type-aware inspection regions, and diagnostics for inspection-region arrival and object-level confirmation. To address this inspection-oriented setting, we further propose RACO, a reliability-aware adaptive coarse-to-fine navigation framework. Instead of treating the predicted coarse goal as a fixed waypoint, RACO views it as a runtime hypothesis and uses object-level candidate anchors to check and correct coarse localization before Stage 1 and at the Stage 1-to-Stage 2 boundary. RACO also applies scale-adaptive terminal refinement to handle terminal near-miss cases using runtime-observable geometric and anchor-based evidence. Under a unified online evaluation protocol, RACO improves SR over the reproduced HETT baseline by 9.53 and 7.98 percentage points on validation-unseen and test-unseen, respectively. It also improves inspection-region arrival and reduces false verification risk, showing that coarse-goal reliability optimization is an effective complement to existing coarse-to-fine UAV-VLN policies.

Introduction

Inspection-oriented UAV-VLN requires valid object inspection states, not merely coordinate-level arrival, because coarse-goal drift and terminal near-misses can produce incorrect or invalid confirmations. LG-UVI evaluates these failures, while RACO corrects unreliable coarse goals and refines bounded terminal errors.

  • Standard navigation metrics do not fully capture inspection-oriented deployment, which requires suitable viewing regions, inspectable final states, and avoidance of similar distractors.
  • HETT failures commonly involve unreliable coarse localization, same-category distractor drift, and terminal near-misses, including a substantial 20–35 m residual-distance band.
  • LG-UVI extends CityNav and CityRefer with target objects, same-category candidates, hard distractors, type-aware inspection regions, and inspection-specific diagnostics.
  • RACO treats the coarse goal as a runtime hypothesis and verifies or corrects it with object-level candidate anchors before Stage 1 and at the Stage 1-to-Stage 2 transition.
  • RACO uses scale-adaptive terminal inspection refinement to repair bounded near-miss errors using runtime-observable geometric and anchor-based features.
  • 9.53 and 7.98 percentage points: RACO increases SR over reproduced HETT on validation-unseen and test-unseen, respectively, while improving path efficiency and inspection diagnostics.

Related Work

Prior UAV-VLN work largely emphasizes coordinate-level navigation, while related embodied systems study semantic grounding and object interaction. LG-UVI and RACO extend this direction by separating inspection-region arrival from object confirmation and revising unreliable coarse and terminal states.

  • Indoor and aerial VLN benchmarks primarily evaluate whether agents reach navigation goals efficiently using coordinate-level metrics.
  • Recent aerial VLN systems improve city-scale realism, simulation, reasoning, memory, and foundation-model-based planning, but generally retain coordinate-level success criteria.
  • Object-centric embodied tasks require semantic-category, instance, and task-specific grounding beyond metric goal locations, as illustrated by REVERIE and ALFRED.
  • Application-oriented aerial language systems explore open-vocabulary grounding, terminal delivery, mission generation, onboard navigation, and interactive execution.
  • LG-UVI separates inspection-region arrival from object-level confirmation, while RACO revises unreliable coarse and terminal states through object-level candidate anchors.

LG-UVI Benchmark

LG-UVI converts coordinate-level UAV-VLN into object-centric inspection evaluation by adding type-aware regions, candidate objects, hard distractors, and diagnostics. RACO’s framework overview shows reliability correction and terminal refinement operating on a two-stage backbone.

  • LG-UVI extends CityNav and CityRefer episodes so agents must stop in a suitable viewing region and associate the final state with the intended target object.
  • Each episode adds object IDs, semantic types, geometry, type-aware inspection regions, same-category candidates, and hard distractors to the preserved navigation data.
  • Candidate pools contain up to 32 objects, including the target, nearest same-type candidates, and deterministic additional same-type negatives; hard distractors are nearest same-type objects.
  • Cars use 20 m center-radius inspection regions, whereas buildings, ground regions, and parking areas use 20 m contour-buffer regions.
  • Zone-SR, OSA, ISR, and FVR measure inspection-region arrival, object confirmation, combined inspection success, and false verification of hard distractors.
  • RACO augments the two-stage backbone with stage-aware coarse-goal reliability correction and scale-adaptive terminal inspection refinement.

Method

RACO extends a two-stage UAV-VLN backbone with runtime reliability checks that ground coarse goals in object anchors and correct unreliable handoffs. It also performs bounded, scale-adaptive terminal refinement using runtime geometric and anchor evidence.

  • Framework: RACO keeps the original two-stage navigation backbone while acting on the raw coarse goal as a runtime hypothesis rather than a fixed waypoint.It operates on the backbone's coarse goal, Stage 1 handoff state, and terminal state.
  • Object-Level Candidate Anchors: Object-level candidate anchors provide geometry-aware references for correcting coarse localization and refining the terminal state.Anchors encode spatial compatibility, semantic consistency, and distinctiveness among neighboring candidates.
  • Object-Level Candidate Anchors: The runtime anchor set contains no episode-specific target identities or hard-distractor labels and differs from the ground-truth-inclusive evaluation pool.This keeps inference anchors distinct from the candidate pool used only for evaluation.
  • Stage-Aware Coarse-Goal Reliability Correction: RACO separates intervention detection from anchor selection before Stage 1 and at the Stage 1-to-Stage 2 boundary.Reliability gates assess whether correction is needed, while separately parameterized modules choose the correction target.
  • Stage-Aware Coarse-Goal Reliability Correction: If the Stage 1 handoff is unreliable, RACO replans toward a selected runtime anchor before entering Stage 2, subject to tolerance and iteration limits.The replanning process is bounded and stops when the anchor is reached within fixed tolerance or the preset iteration limit is reached.
  • Scale-Adaptive Terminal Inspection Refinement: The terminal policy uses runtime-observable residual, geometry, anchor, ambiguity, replanning, and state-anchor features and executes online before final metrics.The learned policy is trained before evaluation and applied after Stage 2.
  • Scale-Adaptive Terminal Inspection Refinement: Terminal refinement maps residual distance to a bounded geometry-based move and uses a learned discrete scale, including abstention, to adjust near-miss states.No refinement is applied outside the specified near-miss range; the scale set is {0, 0.5, 0.75, 1.0, 1.25}.

Experiments

RACO is evaluated on LG-UVI under a unified online protocol against reproduced HETT and matched ablations. It improves navigation and inspection outcomes, with gains attributed to runtime correction, terminal refinement, and type-aware anchor filtering.

  • Experimental setup: RACO is evaluated with standard navigation metrics and inspection-oriented metrics across validation-seen, validation-unseen, and test-unseen splits.Evaluation uses NE, SR, OSR, SPL, Zone-SR, OSA, ISR, and FVR; thresholds are selected on held-out data, while evaluation splits are reserved for testing.
  • Main results: RACO increases SR from 18.06 to 27.59 on Validation Unseen and from 25.73 to 33.71 on Test Unseen versus reproduced HETT.NE also decreases on both unseen splits, while OSR and SPL follow the same trend.
  • Main results: RACO improves SR over RACO-Base by 5.23–8.20 percentage points across the three splits, showing benefits beyond the LG-UVI-trained backbone.On Validation Unseen, SR rises from 19.39 to 27.59 and OSR from 34.63 to 41.71.
  • Inspection diagnostics: RACO improves Zone-SR by 6.08 and 6.14 percentage points and reduces FVR by 19.51 and 15.09 points on Validation Unseen and Test Unseen.OSA and ISR remain mixed across splits, leaving reliable object-level target confirmation as a limitation.
  • Ablation study: Combining pre-stage and stage-boundary correction raises Validation Unseen SR from 19.39 to 23.43 over RACO-Base, exceeding either module alone.The two modules address different errors: unreliable initial goals and poor handoff states before local refinement.
  • Ablation study: Type-aware anchor filtering improves SR, OSR, and Zone-SR on both validation splits, indicating that geometric proximity alone is insufficient for anchor selection.The language-derived type cue helps reject nearby objects that do not match the instruction.
  • Supplementary results: Scale-adaptive refinement outperforms no-terminal and fixed-step variants, while Full RACO exceeds the strongest map-aware heuristic by 3.36 Test-Unseen SR points.RACO is more sensitive to object-position noise than to moderate deletion or category corruption, and median Test-Unseen path-length change is zero.
  • Qualitative analysis: Qualitative cases show RACO correcting unreliable coarse goals before Stage 1, poor handoffs before Stage 2, and bounded terminal near-misses after refinement.Ground-truth target geometry is used only for visualization and distance annotations.

Conclusion

LG-UVI exposes object-level inspection failures in two-stage UAV-VLN, and RACO addresses them through candidate-anchor correction and scale-adaptive terminal refinement. RACO improves navigation and inspection-region arrival while reducing false verification, although object-level confirmation remains difficult.

  • Conclusion: LG-UVI adds object-centric metadata and diagnostic metrics to CityNav/CityRefer, revealing failures from unreliable coarse goals and terminal near-misses.These failures can leave agents outside valid inspection regions or associated with visually similar distractors.
  • Conclusion: RACO corrects coarse goals before Stage 1 and at the Stage 1-to-Stage 2 boundary, then applies scale-adaptive terminal refinement to near-miss cases.The framework builds object-level candidate anchors for reliability-aware coarse-to-fine navigation.
  • Conclusion: Experiments show that RACO improves navigation performance and inspection-region arrival while reducing false-verification risk compared with reproduced HETT.Object-level target confirmation remains difficult, motivating stronger grounding and verification for visually similar distractors.

S1 Additional Method Details

RACO’s supplementary method details define target-agnostic runtime anchors, learned reliability gates, bounded replanning, and scale-adaptive terminal refinement using online-observable features.

  • Runtime Anchor Construction: RACO constructs at most 32 nearby runtime anchors around the predicted coarse goal, prioritizing language-compatible object types and using centers or contour-derived references.The runtime map is target-agnostic and excludes episode-specific target and distractor labels.
  • Pre-stage Goal Adapter: The pre-stage adapter assigns keep, positive-correction, or ambiguous supervision classes from frozen-backbone success and correction labels.Candidate ranking treats target objects or references within 20 m of the target as positive during offline supervision.
  • Pre-stage Goal Adapter: The pre-stage gate and candidate ranker use weighted binary HistGradientBoosting models with fixed configurations and calibrated acceptance thresholds.Calibration enforces a no-harm constraint on originally successful episodes and selects interventions using proxy success, net rescue, and anchor precision.
  • Boundary Correction: At the Stage 1-to-Stage 2 boundary, a learned gate uses trajectory, endpoint, candidate, and score statistics while excluding ground-truth error and rescue labels at runtime.When activated, the executor selects an admissible anchor within 80 m and replans for at most eight additional coarse steps, stopping within 8 m of the anchor.
  • Scale-adaptive Terminal Inspection Refinement: Terminal refinement is considered only for 20 ≤ dT ≤ 35 m and applies distance-dependent geometric moves toward the runtime inspection anchor.The tested scale-adaptive policy improves Validation Seen SR from 32.23 to 35.55 and Validation Unseen SR from 23.43 to 27.59.

S3 Map-Aware Controls and Object-Map Robustness

Map-aware controls show that RACO’s gains exceed tested non-learning heuristics and remain sensitive mainly to object-reference localization under synthetic map perturbations.

  • Map-aware controls: Full RACO exceeds the strongest tested map-aware heuristic by 3.36 percentage points, showing that map access and geometric rules alone do not account for the gain.The heuristics use the same backbone, annotation-derived scene prior, anchor space, and geometric execution bounds but remove learned reliability and utility models.
  • Object-map robustness: Synthetic object-map perturbations produce relatively small SR changes for object deletion and moderate category corruption, but larger drops after 5 or 10 m position perturbations.These tests use synthetic perturbations of an annotation-derived scene prior rather than a learned or online-predicted object map.
  • Runtime behavior and model cost: Full RACO’s average Test Unseen path length increases by 3.98 m relative to RACO-Base, while 2,883 of 5,281 episodes retain unchanged path length.The reported activation analysis indicates selective rather than universal trajectory modification.
  • Runtime behavior and model cost: The candidate-anchor selector contains 2,683,303 parameters, while the selector and three lightweight decision modules require approximately 13.2 MB of storage.

S5 Formal Definitions of LG-UVI

LG-UVI formalizes inspection success through region arrival, object association, joint inspection success, and hard-distractor verification diagnostics averaged across evaluation episodes.

  • Episode quantities: LG-UVI defines each episode using a terminal position, target object, predicted associated object, type-aware inspection region, and hard-distractor set.
  • Evaluation procedure: All four inspection-oriented diagnostics are averaged over the same N evaluation episodes.Explicit object candidates use their predicted identifiers; otherwise, LG-UVI applies its deterministic same-type association rule.
  • Diagnostic metrics: Zone-SR measures inspection-region arrival, while OSA measures object association accuracy.
  • Diagnostic metrics: ISR requires both correct inspection-region arrival and correct object association, whereas FVR measures association with a hard distractor.
Loading 2608.22678v1…