PaperlayerView exact revision on arXiv

Source-linked AI summary

SP-Guide: Enhancing Aerial Vision-Language Navigation with Spatial Priors

Haoxiang Tang, Dong Zhou, Shen Tan, Minxuan Jiang, Xitong Li

arXiv:2610.32697v1cs.CVcs.AI

TL;DR

AVLN asks UAVs to reach language-described targets from partial observations, but aligning view-centered imagery with globally anchored maps is difficult. SP-Guide combines spatially constrained cross-modal reasoning with terminal target refinement. It is reported to outperform conventional baselines on a large-scale aerial navigation benchmark.

  • Problem

    AVLN must ground language in partial egocentric observations and global maps whose view-centered and world-centered coordinate semantics differ.

    Source: §I · p. 1

  • Method

    SP-Guide is a transformer-based AVLN framework that combines spatially constrained cross-modal reasoning with terminal target refinement.

    Source: §I · p. 1

  • Results

    SP-Guide consistently outperforms conventional baselines on a large-scale aerial vision-language navigation benchmark.

    Source: §I · p. 2

  • Takeaways & Limitations

    SP-Guide extends spatial-prior reasoning to cross-modal consistency by constraining attention with distances in a shared physical frame.

    Source: B. Spatial Priors in Vision Transformers · p. 2

Abstract

from arXiv · show

Aerial vision-language navigation (AVLN) requires a UAV to reach a language-described target from partial egocentric observations while reasoning over large outdoor scenes. Existing approaches typically fuse view-centered RGB observations and globally anchored history maps as generic visual embeddings, leaving cross-modal spatial alignment to be learned from sparse navigation supervision. We present SP-Guide, a transformer- based goal-prediction framework that addresses this weakness of alignment through a Spatial Prior Cross-Modal Encoder, an Instruction Enhanced Decoder, and a Goal Convergence Filter for terminal target refinement. Experiments implemented on the CityNav benchmark show that SP-Guide improves success rate over the representative baselines, with the largest gains on unseen environments. These results indicate that physically grounded spatial priors provide an effective inductive bias for robust AVLN.

I. Introduction

AVLN combines partial egocentric observations with large-scale spatial reasoning, but mismatched visual and map coordinates complicate grounding and terminal decisions. SP-Guide adds spatially constrained cross-modal reasoning and terminal refinement, and is reported to outperform conventional baselines.

  • Rapid viewpoint shifts, small distant targets, and the need for long-range spatial consistency make aerial navigation challenging.Source: §I · p. 1
  • View-centered observations and globally anchored maps have different coordinate semantics, forcing sparse supervision to teach correspondence, instruction relevance, and goal geometry together.Source: §I · p. 1
  • Noisy progress estimates can cause early stopping, overshooting, or oscillation, so better one-step prediction alone may not improve closed-loop navigation.Source: §I · p. 1
  • SP-Guide strengthens local-to-global alignment and terminal target selection while preserving standard preprocessing and pretrained modality encoders.Source: §I · pp. 1–2
  • SP-Guide consistently outperforms conventional baselines on a large-scale aerial vision-language navigation benchmark.Source: §I · p. 2
  • The Goal Convergence Filter aggregates recent goal and progress predictions to produce a more reliable final target and stopping decision.Source: §I · p. 2

A. Aerial Vision-Language Navigation

Aerial vision-language navigation extends language-guided navigation to geographically large outdoor scenes. CityNav supplies real-world urban data, demonstrations, and a geographic semantic map with stable spatial cues and instruction-mentioned landmarks.

  • Indoor continuous VLN provides reference baselines and highlights the importance of coupling local perception, spatial memory, and language, while aerial settings add larger scenes and more severe viewpoint changes.Source: A. Aerial Vision-Language Navigation · p. 2
  • CityNav provides a large-scale real-world urban AVLN dataset with language descriptions, human demonstration trajectories, and a geographic semantic map.Source: A. Aerial Vision-Language Navigation · p. 2

B. Spatial Priors in Vision Transformers

Vision transformers have incorporated spatial structure through tokenization, locality, hierarchy, and positional biases. SP-Guide addresses the gap between single-lattice priors and AVLN’s need to align observations and maps across coordinate systems.

A. Task Formulation

The task asks a UAV to follow a natural-language destination instruction using partial observations and action history. Success is determined by the final position’s proximity to the global target.

  • A UAV receives an instruction describing a target and selects discrete actions from partial observations conditioned on the instruction and observation history.Source: A. Task Formulation · p. 2
  • An episode succeeds when the final UAV position lies within a predefined tolerance radius of the global target.Source: A. Task Formulation · p. 2

B. SP-Guide Overview

SP-Guide combines a Spatial Prior Cross-Modal Encoder, an Instruction Enhanced Decoder, and a Goal Convergence Filter; its encoder builds shared visual memory while injecting spatial constraints between local observations and global map tokens.

  • SP-Guide comprises a Spatial Prior Cross-Modal Encoder (SPCE), an Instruction Enhanced Decoder (IED), and a Goal Convergence Filter (GCF).Source: B. SP-Guide Overview · p. 2
  • SPCE builds shared visual memory from multimodal observations and injects spatial constraints between local observation tokens and global map tokens.Source: B. SP-Guide Overview · p. 2
Loading 2610.32697v1…