Source-linked AI summary
AirAnchor: Bridging Local and Global Spatial Information for Zero-Shot Aerial Vision-and-Language Navigation
Shanwei Fan, Bin Zhang, Zhiwei Xu, Yingxuan Teng, Siqi Dai, Lin Cheng, Guoliang Fan
TL;DR
Zero-shot Aerial VLN must ground language in both immediate local geometry and long-horizon global spatial context, but existing methods generally use only one scale. AirAnchor bridges these scales with spatial anchors, persistent object memory, and an integrated navigation agent, and experiments show it outperforms existing zero-shot baselines. Its evaluation and sensing assumptions leave robustness to real-world noise and dynamic environments open.
Problem
Existing zero-shot Aerial VLN methods typically represent spatial information at either a local or global scale, limiting access to both immediate geometric grounding and long-horizon spatial context.
Method
AirAnchor uses query-driven object and directional anchors for local Egocentric Anchor Graphs, persistent object memory for global landmark priors, and an agent that integrates both.
Results
AirAnchor outperforms existing zero-shot baselines, with ablations validating the effectiveness of its designed components.
Takeaways & Limitations
Spatial anchors provide a shared basis for jointly using local grounding and global memory in zero-shot Aerial VLN navigation.
Takeaways & Limitations
Evaluation is limited to simulated AerialVLN-S environments, while sensing and geometric-grounding errors can propagate into navigation and persistent memory.
Abstract
from arXiv · showhide
Aerial Vision-and-Language Navigation requires drones to follow natural-language instructions and navigate through complex urban environments. Accurate navigation relies on both local and global spatial information, which support immediate action grounding and long-horizon path planning, respectively. However, existing zero-shot methods typically operate at a single spatial scale, relying either on local representations constructed online from current observations or on global memories built offline from historical experience. To address this limitation, we propose AirAnchor, a new paradigm that bridges local and global spatial information through spatial anchors and integrates both into a shared navigation framework, enabling comprehensive spatial grounding for decision-making. AirAnchor consists of three core components: (1) Query-Driven Spatial Anchor Grounding, which identifies decision-relevant anchors from visual observations and organizes them into local spatial representations; (2) Persistent Object Spatial Memory, which incrementally maintains an object knowledge base as persistent global spatial memory and retrieves landmark-related spatial priors; and (3) a Spatially-Informed Navigation Agent, which explicitly integrates both local and global spatial information into an agentic framework for decision-making. Extensive experiments on AerialVLN demonstrate that AirAnchor substantially outperforms existing zero-shot baselines, validating the effectiveness and efficiency of the proposed paradigm.
1 Introduction
AirAnchor addresses the limitations of single-scale zero-shot Aerial VLN by bridging local spatial grounding and global spatial memory through decision-relevant spatial anchors. Its modular navigation system integrates these representations for spatially grounded decision-making.
- Aerial VLN requires UAVs to follow natural-language instructions through complex 3D aerial environments, but generic MLLMs struggle with precise metric and directional relations.
- Existing zero-shot methods use either online local representations or offline global memories, leaving immediate action grounding and long-horizon planning at separate spatial scales.Local representations provide geometric evidence for immediate actions, whereas global representations support broader spatial reasoning.
- AirAnchor distills decision-relevant spatial information into object and directional anchors that support local spatial representations and persistent global memory.Object anchors represent objects and spatial attributes; directional anchors encode cues associated with directions.
- Query-Driven Spatial Anchor Grounding identifies anchors from current observations, computes spatial cues from depth, and organizes them into an Egocentric Anchor Graph.
- AirAnchor combines anchor grounding, persistent object memory, and an MLLM-based navigation agent that jointly uses local and global information in a closed-loop framework.The contributions describe this as a three-component modular system for zero-shot navigation.
- Extensive experiments show that AirAnchor outperforms existing zero-shot baselines, while ablations validate the effectiveness of its designed components.
2 Related Works
Related work in Vision-and-Language Navigation spans learned cross-modal and history-based methods, newer MLLM-based agents, and zero-shot approaches using local representations, global memory, or modular designs. Aerial VLN extends this problem into continuous 3D aerial environments.
- Vision-and-Language Navigation began with the Room-to-Room benchmark and later expanded to continuous environments with low-level control.
- Early VLN methods emphasized cross-modal representation learning and history modeling, while recent methods use vision-language-action models, dual-system architectures, and zero-shot agents.
- Aerial VLN extends VLN to continuous 3D aerial environments, with learning-based methods using trajectory supervision and zero-shot methods using spatial representations, global memory, or modular designs.
3 Method
AirAnchor combines query-driven local spatial anchors, persistent global object memory, and an agentic navigation framework for spatially grounded Aerial VLN decisions. Its method represents relevant local geometry sparsely, retrieves landmark priors globally, and jointly uses both information sources for navigation.
- Overview: AirAnchor decomposes navigation into local anchor grounding, persistent object memory, and spatially informed agentic decision-making.The local representation is an Egocentric Anchor Graph, while the global representation is maintained through a Spatial Object Knowledge Base.
- Query-Driven Spatial Anchor Grounding: Query-Driven Spatial Anchor Grounding identifies decision-relevant object and directional anchors from current observations and organizes their spatial cues into an Egocentric Anchor Graph.Object anchors represent important objects; directional anchors specify ray-casting directions through pixel coordinates.
- Query-Driven Spatial Anchor Grounding: The local representation exposes sparse semantic-geometric references instead of constructing an exhaustive geometric map, providing lightweight and interpretable grounding for navigation decisions.Each anchor stores semantic and geometric attributes, while UAV-anchor edges encode spatial cues such as bearing, height difference, and distance.
- Persistent Object Spatial Memory: Persistent Object Spatial Memory maintains a hierarchical Spatial Object Knowledge Base that updates object instances and retrieves landmark-related spatial priors.The SOKB uses type-instance organization, bounded semantic and visual banks, coarse-to-fine association, and geometry filtering.
- Persistent Object Spatial Memory: The memory associates incoming anchors using semantic, geometric, and visual evidence, then selects landmark candidates by matching their spatial cues with instruction-implied relations.Matched instances are fused by merging spatial extents and updating semantic and visual banks; unmatched anchors create new instances.
- Spatially-Informed Navigation Agent: The Spatially-Informed Navigation Agent integrates local EAG information and global landmark priors within an agentic framework for efficient, spatially grounded navigation.Instruction subtasks and referenced landmarks support focused long-horizon planning, while the MLLM performs skill-level decisions and updates progress and plans.
4 Experiments
AirAnchor is evaluated on AerialVLN-S using standard navigation metrics, zero-shot and learning-based baselines, component ablations, and analyses of its spatial memory and navigation behavior. It consistently outperforms zero-shot aerial baselines, with gains linked to complementary local grounding, persistent landmark priors, and specialized skills.
- Experimental Setup: AirAnchor is evaluated on AerialVLN-S using SR, OSR, NE, and SDTW, with statistical, learning-based, and zero-shot baselines.The benchmark includes 17 compact scenes; the low-level planner uses A∗ search, and zero-shot baselines share the same MLLM backbone.
- Overall Performance: AirAnchor outperforms all zero-shot aerial baselines across Seen and Unseen validation splits, with relative SR gains of 39.1% and 48.4% over FineCog-Nav.It also improves OSR by 37.5% / 43.8%, SDTW by 48.1% / 52.2%, and reduces NE by 17.4% / 20.9% on Seen / Unseen.
- Overall Performance: AirAnchor shows minor degradation from Validation Seen to Unseen, indicating robust performance across diverse aerial scenes.Its gains in OSR and NE are consistent with combining local metric grounding and persistent landmark priors, while SR and SDTW gains reflect skill-level planning and progress reflection.
- Ablation Study: Removing object anchors reduces SR and SDTW by 9.4% and 12.5% and increases NE by 7.1 m, while removing directional anchors reduces SR by 3.1%.The ablation attributes directional anchors to ray-based geometry and object anchors to semantic and metric grounding of instruction-relevant landmarks.
- Ablation Study: Replacing SOKB with Flat KB reduces SR by 12.5% and increases NE by 10.3 m, while SOKB reduces duplicate instances by 74.0% and improves retrieval accuracy by 18.6%.SOKB stores 20.9% fewer instances while preserving 9.8% more correct unique objects.
- Ablation Study: Using Pixel Navigation alone reduces SR by 15.6% and increases NE by 11.1 m; View Rotation has the largest impact among specialized skills.Altitude Adjustment and Path Backtracking also contribute to performance, while the agent coordinates local and global information during long-horizon episodes.
5 Conclusion
The paper concludes that AirAnchor bridges local grounding and global memory through spatial anchors in an MLLM-powered agentic framework. Its local Egocentric Anchor Graph, persistent Spatial Object Knowledge Base, and closed-loop navigation agent together demonstrate efficacy and robustness.
- Conclusion: AirAnchor bridges local grounding and global memory through spatial anchors within an MLLM-powered agentic framework.The paradigm combines local Egocentric Anchor Graph reasoning with persistent global object memory and a Spatially-Informed Navigation Agent.
6 Limitations
AirAnchor's limitations concern perception reliability, cold-start memory, skill flexibility, evaluation scope, and computational overhead. These constraints bound robustness beyond the evaluated simulated setting.
- Depth, object-localization, and geometric-grounding errors can propagate into the EAG and later navigation decisions.Incorrect object associations may also accumulate in the persistent SOKB and affect later retrieval.
- Global memory benefits depend on previously accumulated scene knowledge; unseen landmarks may be absent from the SOKB.Until sufficient object knowledge is collected, the agent relies primarily on local observations.
- The predefined four-skill controller and low-level planner may be insufficient for more complex maneuvers or dynamic environments.
- Evaluation is limited to simulated AerialVLN-S environments, leaving robustness to real-world sensing noise and dynamic environments unvalidated.
- Using multiple foundation models, including an online MLLM, introduces additional computational and communication overhead.
A Implementation Details
AirAnchor grounds sparse, decision-relevant anchors from RGB-D observations, converts them into local geometric cues, and maintains persistent object memory through matching and reliability-aware updates.
- Anchor Querying: At each decision step, the anchor-query MLLM receives an egocentric RGB observation and navigation context, then queries exactly K = 3 spatial anchors.The MLLM determines the relative numbers of object and directional anchors from the current navigation context.
- Object Anchors: Object anchors are localized with Grounded SAM, filtered by reliable depth, and represented through compressed 2.5D geometry.Objects without positive-depth pixels are discarded; long-range masks may instead be degraded into directional anchors.
- Egocentric Anchor Graph: Anchor geometry is expressed relative to the UAV in a z-up world frame and organized as UAV-anchor edges in the Egocentric Anchor Graph.The representation includes angular, vertical, horizontal-distance, and 3D-distance cues for object and directional anchors.
- Persistent Object Memory: The SOKB matches incoming object anchors using type compatibility, geometric gating, semantic similarity, and visual similarity before fusion or instance creation.The matching design uses an asymmetric footprint-coverage term because one aerial observation may cover only part of a larger persistent object.
- Memory Updates: Persistent instances update geometry and confidence using observation reliability and association consistency, while bounded banks retain reliable and diverse attributes.Low-diversity repeated observations receive low utility, whereas reliable evidence from distinct viewpoints is retained.
A.2.7 Memory Lifecycle During Evaluation
During evaluation, AirAnchor maintains scene-specific persistent memory, retrieves landmarks per active subtask, selects parameterized navigation skills, and reflects on progress after execution.
- Memory Lifecycle: Each scene has a separate SOKB initialized empty and updated with object anchors acquired during navigation.When evaluation returns to a scene, its accumulated memory is restored under the canonical episode order.
- Subtask Planning: The instruction is decomposed into ordered subtasks, with landmark retrieval repeated whenever the agent advances to a new subtask.Only one subtask is active at a time, and its ordered subgoals support automatic-backtracking thresholds.
- Skill-Level Planning: The agent combines EAG information, global landmark priors, and skill parameters in a single navigation call rather than separately predicting low-level distances or magnitudes.
- View Rotation: View Rotation gathers eight uniformly distributed yaw observations, then supports coarse direction selection and fine-grained angular reasoning.The observations are panoramic references rather than candidate actions, and execution uses the nearest available 15° rotation primitive.
- Path Backtracking: Path Backtracking reuses the current subtask's stored transition chain and is limited to at most two executions per subtask.Retracing stored transitions does not invoke the local A∗ planner.
- Local Execution: A∗ constructs temporary local free-space representations only for Pixel Navigation and Altitude Adjustment, using depth observations capped at the reliable sensing boundary.View Rotation executes directly, while failed local paths return control to the high-level planner for a new closed-loop decision.
- Progress Reflection: After a skill executes, the agent recenters the EAG at the new pose and compares anchor relations before and after motion to expose spatial changes.Completion advances to the next subtask or triggers STOP after the final subtask; otherwise the episode ends at the global loop budget.
B.1 Additional Quantitative Analysis
The supplied passage identifies the evaluation setting as the AerialVLN-S Validation Seen split under the main experiments' evaluation protocol.
- Additional quantitative analysis uses the AerialVLN-S Validation Seen split under the same evaluation protocol as the main experiments.
B.1.1 Impact of the Number of Spatial Anchors
AirAnchor balances spatial-anchor coverage against redundancy and noise. The analysis selects K = 3 as a compact, complementary setting for navigation decisions.
- A small K provides insufficient spatial references for navigation.
- An excessively large K introduces less relevant objects, noisy spatial cues, and a larger EAG context that can dilute attention.
- K = 3 provides complementary spatial references while avoiding excessive redundancy and noise in the EAG.
B.1.2 Memory Construction Mechanism
The memory analysis favors online, task-conditioned construction over frozen scene-level initialization. Persistent memory is most useful when built from the same task-relevant anchors that support local grounding.
- Memory construction: Online Persistent memory improves SR from 8.7 to 9.6 and SDTW from 3.4 to 4.0 while reducing NE by 8.2 m versus frozen pre-rendered memory.
- Memory construction: AirAnchor conditions object-anchor construction on the active subtask, navigation progress, and current plan, biasing memory toward useful landmarks.
- Memory construction: Repeated observations refine stored geometry, semantic descriptions, and visual evidence through the same coarse-to-fine association mechanism.
- Memory construction: Pre-rendered initialization adds only 0.3 SR points and 0.2 SDTW points over Online Persistent while decreasing NE by 1.6 m.
- Memory construction: The results support progressively constructing global spatial knowledge from task-relevant object anchors rather than maintaining a static scene representation.
B.1.3 Robustness to Episode Ordering
Persistent memory remains robust to episode-order randomization. SR and SDTW change little, while larger NE variation reflects cold-start differences in accumulated same-scene landmarks.
- Randomizing episode order leaves SR and SDTW close to the canonical results.
- NE varies more because episode order changes which same-scene landmarks have already been accumulated when an episode is encountered.
- Semantic, visual, and geometric consistency lets repeated landmarks be consolidated across trajectories regardless of encounter order.
B.1.4 MLLM Efficiency Analysis
AirAnchor concentrates richer spatial reasoning in skill-level decisions rather than repeated primitive-action calls. This yields substantially lower workload than FineCog-Nav while retaining strong navigation performance, though anchor-based reasoning has documented failure boundaries.
- Workload comparison: SPF requires 30.3 MLLM calls and 21,725 tokens per episode through a lightweight reactive waypoint policy.
- Workload comparison: FineCog-Nav requires 513.4 calls and 390,697 tokens per episode because it repeatedly processes perception, reasoning, decision, and memory modules.
- Workload comparison: AirAnchor uses 45.8 calls and 68,059 tokens per episode while achieving the highest SR of 9.6.
- Workload comparison: Relative to FineCog-Nav, AirAnchor reduces MLLM invocations by 91.1% and total token consumption by 82.6%.
- Failure boundaries: If anchor selection omits a critical landmark or motion direction, its geometry is unavailable to subsequent reasoning for that decision step.
- Failure boundaries: Incomplete or inaccurate anchor evidence can cause premature or delayed subtask completion, propagating into trajectory deviations near the destination.