PaperlayerView exact revision on arXiv

Source-linked AI summary

AirGroundVLN: A Large-Scale Benchmark for Goal-Oriented Air-Ground Collaborative Vision-and-Language Navigation

Zhenxuan Zeng, Qingle Wu, Wei Suo, Maojia Wu, Bairong Zhang, Hangzheng Yu, Peng Wang

arXiv:2610.10421v1cs.RO

TL;DR

Goal-oriented air–ground VLN lacks large-scale benchmarks and must handle cross-view historical context and complementary but asymmetric aerial and ground perception. AirGroundVLN provides a benchmark and AG-CoNAV combines collaborative memory with regional-to-local planning. AG-CoNAV achieves the best success rate across the reported splits, and the benchmark establishes a reference testbed for future work.

  • Problem

    Goal-oriented air–ground VLN lacks large-scale, diverse benchmarks, and collaboration must address cross-view historical context and asymmetric aerial and ground observability.

    Source: §1 · p. 2

  • Method

    AirGroundVLN supplies episodes, environment splits, and aerial-visibility evaluation, while AG-CoNAV combines SACM historical context retrieval with AGRLP aerial guidance and ground planning.

    Source: §1 · pp. 2–3

  • Results

    AG-CoNAV achieves the best SR across all splits; against the training-free Air-Ground model with the same Qwen3.5-2B backbone, seen-split SR rises from 4.05% to 32.64%.

    Source: §5.3 · p. 8

  • Takeaways & Limitations

    AirGroundVLN establishes a testbed and reference baselines for future research on heterogeneous air–ground collaborative VLN.

    Source: §6 · p. 10

Abstract

from arXiv · show

Goal-oriented Vision-and-Language Navigation (VLN) requires agents to locate and reach targets described in natural language without prescribed routes. Air--ground collaboration is valuable for tasks requiring both wide-area search and fine-grained localization. However, systematic study of goal-oriented air--ground collaborative VLN remains limited by the lack of large-scale, diverse benchmarks and two core challenges: 1) substantial differences between aerial and ground views, together with useful observations becoming unavailable as navigation proceeds, make it difficult to maintain spatially consistent context across platforms and over time; and 2) asymmetric spatial observability makes ground perception locally detailed but spatially limited and aerial perception broad but locally coarse, limiting the reliability of single-platform planning. To address these limitations, we introduce AirGroundVLN, a benchmark containing 10,281 navigation episodes and 955 target instances across 19 Unreal Engine environments, with seen/unseen splits and an aerial-visibility protocol for systematic evaluation. Alongside the benchmark, we propose AG-CoNAV, a trainable reference framework comprising two key components: Spatiotemporally Anchored Collaborative Memory (SACM) and Aerial-Guided Regional-to-Local Planning (AGRLP). SACM maintains and retrieves spatially consistent historical context across aerial and ground observations. Meanwhile, AGRLP combines regional aerial guidance with fine-grained ground navigation. Extensive experiments demonstrate the effectiveness of AG-CoNAV and establish AirGroundVLN as a comprehensive benchmark for future exploration.

1 INTRODUCTION

Air–ground collaborative VLN combines aerial wide-area search with ground-level target localization, but the setting lacks large-scale benchmarks and must address cross-view historical context and asymmetric perception. AirGroundVLN and AG-CoNAV provide a benchmark and trainable reference framework for studying these challenges.

1 INTRODUCTION

  • Air–ground VLN needs collaboration because aerial platforms provide broad context while ground platforms offer detailed local perception and physical access.Observations may become unavailable during navigation, and either platform alone can be insufficient for reliable navigation.Source: §1 · p. 2
  • AirGroundVLN provides 10,281 episodes, 955 target instances, and 19 Unreal Engine environments, with seen/unseen splits and aerial-visibility evaluation.Episodes pair aerial and ground observations with goal-oriented instructions rather than prescribed intermediate actions.Source: §1 · pp. 2–3
  • AG-CoNAV combines SACM for spatially consistent historical context with AGRLP for regional aerial guidance and locally executable ground planning.The framework is presented as a trainable reference for the two collaborative navigation challenges.Source: §1 · pp. 2–3
  • Systematic evaluations across environment generalization and aerial-visibility conditions establish reference performance and demonstrate AG-CoNAV’s effectiveness.The benchmark is accompanied by training-free and trainable baselines.Source: §1 · p. 3

2 RELATED WORK

Prior VLN benchmarks extend from route following toward goal-oriented navigation, including aerial settings, while most still focus on one platform. Multi-agent work has begun to explore heterogeneous air–ground collaboration, but dedicated trainable air–ground collaborative VLN remains underexplored.

2.1 VISION-AND-LANGUAGE NAVIGATION BENCHMARKS

  • VLN benchmarks span indoor route following, continuous and outdoor navigation, and goal-oriented target search, including goal-oriented aerial language guidance.Most benchmarks still focus on a single embodied platform.Source: §2.1 · p. 3
  • Large-scale goal-oriented air–ground VLN and dedicated trainable methods for this setting remain underexplored.AirGroundBench and AGC-VLN are described as limited in scale or scope relative to this gap.Source: §2.1 · p. 3, §2.2 · p. 3

2.2 MULTI-AGENT COLLABORATIVE NAVIGATION

  • Collaborative navigation includes homogeneous ground and aerial teams, language-conditioned multi-agent work, and emerging heterogeneous UAV–UGV settings.Examples include cooperative object search and language-conditioned air–ground navigation.Source: §2.2 · p. 3

3 AIRGROUNDVLN BENCHMARK

AirGroundVLN defines closed-loop goal-oriented navigation in which a UGV reaches a language-specified target with complementary UAV perception. Its construction combines instructions, air–ground trajectories, visibility labels, and quality assurance, yielding a diverse benchmark with multiple evaluation splits.

3.1 TASK DEFINITION

  • The UGV is the primary target-reaching platform, while the UAV supplies complementary aerial perception and global spatial context.Instructions specify target category, appearance, surrounding context, and coarse direction without prescribing intermediate actions.Source: §3.1 · p. 4

3.2 BENCHMARK CONSTRUCTION

  • Benchmark construction generates goal-oriented instructions and platform-specific A∗ reference trajectories, annotates aerial visibility, and applies human quality assurance.UGV paths respect road topology and traversability; UAV paths are generated from start to end poses.Source: Figure 2 · p. 4, §3.2 · p. 5

3.3 DATASET ANALYSIS

  • AirGroundVLN contains 10,281 air–ground collaborative episodes across 19 scenes and 955 target instances, extending benchmark scale for this setting.Its environments include urban areas, towns, industrial areas, and other complex outdoor scenes.Source: §3.2 · p. 4, §3.3 · p. 5
  • UAV reference trajectories average 79.0 m and UGV trajectories 90.9 m, while instructions average 82 words and targets span six semantic categories.The longer average ground paths reflect ground-topology and traversability constraints.Source: §3.3 · p. 5, Figure 3 · p. 6
  • The benchmark provides train, val-seen, test-seen, and test-unseen splits, with test episodes categorized as aerial-visible or aerial-occluded.The test-unseen split evaluates generalization to unseen environments.Source: §3.3 · pp. 5–6

4 METHOD

AG-CoNAV uses SACM to preserve spatially consistent context across aerial and ground observations over time, then uses AGRLP to coordinate regional aerial guidance with fine-grained ground navigation.

4.2 AERIAL-GUIDED REGIONAL-TO-LOCAL PLANNING

  • AGRLP uses predicted aerial and ground waypoints to pair regional aerial guidance with locally executable ground planning and target approach.The ground platform selects a reachable sub-goal and computes a collision-free local trajectory, then verifies target candidates against the instruction and surrounding context.Source: §4 · p. 6, §4.2 · pp. 7–8

4.1 SPATIOTEMPORALLY ANCHORED COLLABORATIVE MEMORY

  • SACM compresses observations into memory tokens with spatial anchors and dispersion, then retrieves historical evidence using spatial and temporal priors.The dispersion-aware spatial term allows more positional variation for tokens representing broader regions.Source: §4.1 · p. 7

5 EXPERIMENTS

Experiments compare AG-CoNAV with training-free and trainable baselines across AirGroundVLN conditions, then ablate its modules and SACM priors. AG-CoNAV achieves the best success rate across splits, while the ablations report gains from combining its components and priors.

5.1 EXPERIMENTAL SETUP

  • AG-CoNAV freezes the Qwen3.5-2B backbone, trains LoRA adapters with memory and waypoint modules, and uses a 5 m success-distance threshold.Experiments follow the stated evaluation protocol.Source: §5.1 · p. 8

5.2 BASELINES

  • Baselines include random navigation, zero-shot VLMs and AGC-VLN, plus UGV-only, UGV with a global BEV map, and an adapted AeroDuo.The training-based comparisons examine global spatial context and heterogeneous air–ground collaboration.Source: §5.2 · p. 8

5.3 MAIN RESULTS

  • AG-CoNAV achieves the best SR across all splits; on seen episodes, it raises SR from 4.05% to 32.64% and SPL from 2.38% to 29.85% versus the same-backbone training-free Air-Ground model.The comparison covers seen and unseen environments and aerial-visible and aerial-occluded conditions.Source: §5.3 · p. 8, Table 2 · p. 9

5.4 ABLATION STUDIES

  • Combining SACM and AGRLP yields the best seen-split ablation performance, with 32.64% SR and 29.85% SPL.AGRLP alone raises the baseline SR from 25.68% to 26.30%, while SACM alone reaches 31.39% SR; the combined variant reaches 32.64%.Source: §5.4 · p. 9, Table 3 · p. 9
  • Using both SACM priors reaches 32.64% SR on the seen split, compared with 29.83% for the temporal prior alone and 31.19% for the spatial prior alone.AGRLP is enabled for all variants, and the passage reports consistent improvements in other navigation metrics when both priors are combined.Source: §5.4 · p. 9, Table 4 · p. 10

6 CONCLUSION

The conclusion recaps AirGroundVLN and AG-CoNAV, reporting extensive evaluations and positioning the benchmark as a testbed for future heterogeneous air–ground collaborative VLN research.

6 CONCLUSION

  • AirGroundVLN contains 10,281 episodes and 955 target instances across 19 Unreal Engine environments, while AG-CoNAV combines SACM with AGRLP.The benchmark includes an aerial-visibility evaluation protocol; the authors report that extensive evaluations validate AG-CoNAV and establish strong baselines.Source: §6 · p. 10

AI USE STATEMENT

The authors report using generative AI to assist with instruction generation and manuscript editing, with generated instructions manually reviewed and verified.

AI USE STATEMENT

  • Generative AI assisted with instruction generation and manuscript language editing; the authors manually reviewed and verified all generated instructions.The authors state they take responsibility for the final content.Source: AI USE STATEMENT · p. 10

A EVALUATION PROTOCOL AND METRICS

Evaluation separates seen and unseen environments and aerial-visible and aerial-occluded targets, while instructions identify targets through appearance, context, and coarse direction without prescribing routes. Episodes undergo multi-stage human review before retention.

A EVALUATION PROTOCOL AND METRICS

  • Seen evaluation uses new episodes in 17 training environments, unseen evaluation uses 2 held-out environments, and test episodes are also split by aerial target visibility.Seen episodes vary start poses and relative target directions; unseen episodes test generalization to held-out environments and target instances.Source: A EVALUATION PROTOCOL AND METRICS · p. 14

B.1 INSTRUCTION CONSTRUCTION

  • The construction pipeline selects distinctive targets, describes their appearance and surroundings separately, then synthesizes those cues into a goal-oriented instruction.GPT-5.5 generates target and context descriptions from multi-view observations; annotators review candidate targets for consistent description and identification.Source: §B.1 · pp. 14–15
  • Instructions combine a target description, surrounding context, and one of eight coarse directions, without prescribing intermediate actions.Start locations are selected by annotators to satisfy local traversability constraints; the relative direction is computed from start and target coordinates.Source: §B.1 · pp. 14–15

B.2 AERIAL-VISIBILITY ANNOTATION.

  • Aerial visibility is manually labeled according to whether the target is directly observable from above or hidden by scene elements.Occluders include roofs, overhangs, and tree canopies.Source: §B.2 · p. 16

B.3 HUMAN QUALITY ASSURANCE.

  • Of 14,697 constructed episodes, 10,281 were retained after review, including 3,906 corrected and re-verified episodes.Review checks target identifiability, instruction consistency, trajectories, and visibility annotations; unresolved ambiguous or invalid samples are discarded.Source: §B.3 · p. 16

C ADDITIONAL DATASET STATISTICS

The dataset statistics describe varied target categories, appearance cues, and aerial visibility. The accompanying method passages specify waypoint supervision and a final approach procedure distinct from benchmark success evaluation.

C ADDITIONAL DATASET STATISTICS

D.1 TRAINING OBJECTIVE.

  • Waypoint training combines Gaussian heatmap distribution supervision with an auxiliary constraint on the predicted distribution’s expected coordinate.The aerial and ground objectives are jointly optimized, with separate weights for coordinate-level supervision and the aerial objective.Source: §D.1 · p. 17

D.2 TARGET APPROACH AND TERMINATION

  • After target confirmation, the UGV approaches the estimated target position using a 1.5 m stopping radius and a 20-iteration budget.The controller’s stopping radius is distinct from benchmark success, which evaluates the final UGV pose against the ground-truth target position.Source: §D.2 · p. 17

E IMPLEMENTATION DETAILS

The implementation specifies training settings and a common target-verification and final-approach procedure. Training-free baselines use current aerial and ground observations to propose waypoints and verify target matches.

E IMPLEMENTATION DETAILS

  • AG-CoNAV freezes the Qwen3.5-2B backbone and trains LoRA adapters; its memory bank holds up to 16 historical air–ground entries.The listed setup uses an initial learning rate of 2×10−5 and a maximum navigation budget of 200 steps.Source: E IMPLEMENTATION DETAILS · p. 17
  • All methods share target verification and final approach, with a 1.5 m stopping radius and at most 20 approach steps.Source: E IMPLEMENTATION DETAILS · p. 18
  • Training-free waypoint models use the instruction and current ground and aerial BEV observations to predict the next UGV and UAV waypoints.A verifier matches candidate appearance and surrounding context to the instruction and produces a confidence score used for termination.Source: E IMPLEMENTATION DETAILS · p. 18

F QUALITATIVE NAVIGATION ANALYSIS

Across three illustrated episodes, AG-CoNAV reaches the target with reasonable path execution, including in unseen environments with aerial occlusions. The qualitative comparisons also describe failure patterns for zero-shot and static-map baselines.

F QUALITATIVE NAVIGATION ANALYSIS

  • AG-CoNAV reaches target locations across three representative episodes, including unseen cases, and maintains localization and executable planning in unseen aerial-occluded settings.The trajectory visualizations compare AG-CoNAV with Qwen3.5-2B, GPT-5.5, and UGV + Global BEV Map against ground-truth paths.Source: F QUALITATIVE NAVIGATION ANALYSIS · p. 19, Figure 12 · p. 20, Figure 14 · p. 21
  • Zero-shot baselines often choose incorrect directions at intersections, while the static global-map baseline mistargets or fails during execution.The passage attributes the zero-shot errors to long-range spatial reasoning difficulty and the map baseline’s issues to missing fine-grained local detail.Source: F QUALITATIVE NAVIGATION ANALYSIS · p. 19, Figure 12 · p. 20
Loading 2610.10421v1…