PaperlayerView exact revision on arXiv

Source-linked AI summary

A-PAIR: A Benchmark and Identity-Consistent Grounding Framework for Air-Ground Cross-View Referring Person Detection

Zhoupeng Guo, Xinjie Yao, Yunqi Zhu, Zhihe Fan, Siqi Zhao, Jianjun Chen, Yichen Dong, Yan Fan, Pengfei Zhu

arXiv:2608.27997v1cs.CVcs.MM

TL;DR

Existing single-view grounding methods do not require ground and aerial predictions to identify the same person. The paper introduces A-PAIR, FARA, and ICRG for paired air-ground referring, and reports that ICRG raises pair F1 from 16.65% to 22.28%.

  • Problem

    Existing methods predict or evaluate views independently, without requiring ground and aerial predictions to identify the same individual.

    Source: Language-Guided Person Detection · p. 2, Introduction · p. 2

  • Method

    The paper introduces the 22,137-sample A-PAIR benchmark, FARA for semi-automatic annotation, and ICRG combining factorized grounding, candidate-completeness supervision, and cross-view consistency calibration.

    Source: Introduction · p. 2

  • Results

    22.28% pair F1, up from GroundingDINO-T's 16.65%, is reported for ICRG on the image-disjoint test split.

    Source: Main Results Validate Paired Reasoning · p. 6

  • Takeaways & Limitations

    The findings support treating AGCV-RPD as a coupled detection and identity-consistency problem rather than two independent single-view grounding tasks.

    Source: Introduction · p. 2

Abstract

from arXiv · show

Air-ground cross-view referring person detection is a necessary component in the language-to-perception-to-control chain of collective embodied intelligence, grounding a language command into the same physical target before ground and aerial agents can coordinate downstream actions. Existing referring expression comprehension and open-vocabulary grounding methods do not jointly account for cross-view identity consistency, making them insufficient for Air-Ground Cross-View Referring Person Detection (AGCV-RPD), which involves similar pedestrian distractors, weak aerial appearance cues, and cross-view identity consistency. To study this problem, we introduce Air-Ground Paired Identity-Aware Referring (A-PAIR), the first comprehensive AGCV-RPD benchmark, containing 22,137 cross-view referring samples. To construct A-PAIR efficiently, we propose Factorized Annotation and Referential Alignment (FARA), a semi-automatic annotation framework that generates factorized referring descriptions and identity-consistency supervision at reduced cost. We propose Identity-Consistent Referring Grounding (ICRG), a framework that combines factorized referential grounding, candidate-completeness supervision, and cross-view consistency calibration for joint air-ground pair selection. ICRG improves ground, aerial, and pair-level detection over strong baselines, increasing pair F1 from 16.65% to 22.28%. These results show that AGCV-RPD requires paired detection and identity-consistent reasoning.

Air-Ground Cross-View Referring Person Detection

The supplied passages provide the author list but no substantive claims about the section.

Air-Ground Cross-View Referring Person Detection

Introduction

AGCV-RPD requires selecting the same language-referred person in paired ground and aerial views, coupling detection with cross-view identity consistency. The paper introduces A-PAIR and ICRG to study and address this task.

Introduction

  • Single-image grounding and standalone identity matching do not solve AGCV-RPD, which couples language-based region proposals across views with same-person consistency amid similar pedestrians and weak aerial appearance cues.The task also involves repeated spatial layouts and plausible per-view predictions that may refer to different people.Source: Introduction · pp. 1–2
  • ICRG unifies factorized referential grounding, candidate-completeness supervision, and cross-view identity-consistency calibration to reduce ambiguity, retain weakly described targets, and suppress mismatched pairs.Factorized supervision separates identity-stable appearance semantics from view-dependent scene-spatial semantics.Source: Introduction · p. 2
  • A-PAIR is introduced as the first comprehensive AGCV-RPD benchmark, with 22,137 cross-view referring samples across 7,588 ground images and 3,891 aerial images.It derives from G2APS and includes paired images, same-identity boxes, structured text, and auxiliary identity-consistency labels.Source: Introduction · p. 2
  • FARA is a semi-automatic annotation pipeline combining VLM annotation, keyframe propagation, and quality control for cross-view descriptions and identity supervision.Source: Introduction · p. 2

Related Work

Existing language-guided and cross-view methods address image-level grounding, place matching, or person matching separately; A-PAIR evaluates paired ground–aerial language alignment with identity consistency.

Language-Guided Person Detection

  • Language-guided detection spans context and relation modeling, transformer-based direct-box or text-conditioned detection, and region-language pre-training.Related work also includes text-based and full-image person search (Li et al. 2017), plus multimodal and small-object grounding for dialogue, coordinate generation, and tiny targets (Goto et al. 2025; Zhan et al. 2026).Source: Language-Guided Person Detection · p. 2
  • Existing language-guided methods remain single-view, while AGCV-RPD requires ground and aerial predictions to identify the same individual.A-PAIR jointly evaluates language-region alignment and cross-view identity consistency.Source: Language-Guided Person Detection · p. 2

Cross-View Air-Ground Perception

  • Cross-view air-ground research includes scene geo-localization, person re-identification, language-aware detection, and drone-view person or referring detection.Examples include geo-localization (Workman, Souvenir, and Jacobs 2015; Hu et al. 2018; Zhu, Yang, and Chen 2021), re-identification (Luo et al. 2019; Sun et al. 2018; He et al. 2021), and drone-view datasets VisDrone and RefDrone (Sun et al. 2026).Source: Cross-View Air-Ground Perception · p. 2
  • A-PAIR addresses paired-referring gaps with ground-aerial images, same-identity boxes, auxiliary identity labels, and pair-level evaluation.Geo-localization matches places, re-identification assumes given crops, and drone-view referring lacks paired ground evidence for identity verification.Source: Cross-View Air-Ground Perception · p. 2

The A-PAIR Benchmark

A-PAIR provides paired ground–aerial referring samples with shared identity supervision and supports language grounding across both views. Its annotation and evaluation design addresses the need to select the same target in both modalities.

Data Source

  • A-PAIR derives from G2APS and retains paired observations sharing at least one identity, yielding 7,588 ground–aerial pairs, 7,588 unique ground images, and 3,891 unique aerial images.Each sample has an appearance-full and scene-spatial expression, enabling full-, spatial-, and factorized-prompt evaluation; boxes use xywh.Source: Data Source · p. 2

FARA annotation

  • FARA builds each referring expression from a reusable pid-level appearance phrase and a view-specific spatial phrase, with bounded keyframe copying or fallback phrases.Appearance phrases use stable visible cues, while spatial phrases describe scene-relative relations; automated checks validate annotations and a completion pass fills missing appearance profiles.Source: FARA annotation · p. 3
  • FARA exports 32,016 ground and 34,108 aerial training annotations plus 377,402 identity pairs, including 50,741 positives.Negative pairs substitute a different-pid person from the same paired images; the official split assigns whole connected components sharing an image to one split.Source: FARA annotation · p. 3

Dataset Statistics and Comparison

  • A-PAIR combines language descriptions with paired aerial–ground images, same-identity boxes, and paired evaluation, capabilities existing benchmark categories do not jointly provide.The benchmark comparison identifies gaps in reference, aerial, cross-view, and ReID benchmarks.Source: Dataset Statistics and Comparison · p. 3, Table 1 · p. 4
  • 22,137 cross-view referring samples are split into 15,497 training, 2,213 validation, and 4,427 test samples.Each sample pairs ground–aerial images with a target identity; Table 2’s columns count unique identities, image pairs, and images by view.Source: Dataset Statistics and Comparison · p. 3, Table 2 · p. 4

Evaluation Metrics

  • Pair accuracy and pair F1 are the primary cross-view competence indicators because a pair is correct only when both views are correct.Under the single-target protocol, every other person in the image is a negative.Source: Evaluation Metrics · pp. 3–4

Method

ICRG selects a ground–aerial person pair by combining language-based grounding, all-person candidate completion, and auxiliary identity-consistency calibration. It requires the selected boxes to identify the same target in both views.

Method

  • ICRG combines factorized grounding, candidate completion, and cross-view consistency to select final ground and aerial boxes.The framework builds candidate pools from referring boxes and all-person detections, then calibrates candidate pairs before selection.Source: Method · p. 4

Task Definition

  • A pair is correct only when both view detections identify the target, making pair correctness stricter than single-view correctness.Each A-PAIR sample pairs ground and aerial images with an appearance-full expression, a scene-spatial expression, and the target identity.Source: Task Definition · p. 4

Factorized Grounding

  • Factorized grounding unions candidates from appearance-full and scene-spatial expressions, using identity semantics and spatial context as complementary cues.Appearance can support precise detection when visible, while scene-spatial cues can improve recall in crowded scenes with ambiguous appearance; the union defers identity choice to cross-view fusion.Source: Factorized Grounding · pp. 4–5

Candidate Completeness

  • Candidate-completeness supervision trains an all-person detector to add missed targets to each view’s candidate pool for later filtering.The detector is recall-oriented: referring and consistency scores can filter the spurious boxes it introduces.Source: Candidate Completeness · p. 5

Cross-View Consistency

  • Cross-view consistency scores whether ground and aerial candidates depict the same person, calibrating language-compatible pairs rather than retrieving independently of language.A ResNet-50 crop encoder learns from cross-view pairs labeled by shared identity and can suppress pairs whose boxes match the expression but show different people.Source: Cross-View Consistency · p. 5

ij , sid(gi, aj)

The final stage selects an identity-consistent ground–aerial candidate pair by maximizing a fused score. Its non-negative fusion weights are chosen on validation data, rather than learned by gradient descent.

ij , sid(gi, aj)

  • The method selects the ground–aerial candidate pair with the highest fused score, combining detection and referring confidences with an identity-consistency term.The predicted pair is the arg-max of the fused score.Source: Fusion and Inference · p. 6, ij , sid(gi, aj) · p. 6

Fusion and Inference

  • Fusion weights are selected by validation-split grid search to maximize pair accuracy, keeping the fusion interpretable and avoiding overfitting the small test set.Learnable components are trained on the training split; fusion weights are fixed after validation selection, and the test split is evaluated once.Source: Fusion and Inference · p. 6

Experiments

On A-PAIR, ICRG improves pair F1 over the strongest single-view baseline, while diagnostic and ablation results show that aerial recall, ambiguous pedestrian descriptions, and cross-view identity consistency remain central challenges. Factorized grounding, candidate completeness, and consistency calibration address these challenges at different stages of paired detection.

Setup

  • Experiments use 15,497 training, 2,213 validation, and 4,427 test samples, with methods trained on the training split and evaluated once on the test split unless noted.Reported metrics cover per-view instance-level and image-level performance and pair-level performance.Source: Setup · p. 6

IoU-Quality Diagnosis Reveals Core Challenges

  • Only 12.54% of test samples are pair-level hits, with aerial misses, similar-pedestrian ambiguity, and inconsistent single-view predictions recurring as failure patterns.Many errors select nearby or partially overlapping regions, and view-wise hits do not ensure an identity-consistent pair.Source: IoU-Quality Diagnosis Reveals Core… · p. 6, Figure 4 · p. 6

Main Results Validate Paired Reasoning

Factorized Grounding Reduces Ambiguity

  • Pair F1 improves from 3.60% to 20.88% when factorized grounding alone is enabled.Separating appearance cues for who from scene and spatial cues for where supplies complementary candidates and reduces errors from ambiguous descriptions and similar pedestrians.Source: Factorized Grounding Reduces Ambiguity · p. 6, Table 4 · p. 7

Completeness Improves Aerial Recall

  • Adding candidate completeness to factorized grounding raises aerial F1i from 20.83% to 21.82% and aerial F1img from 34.47% to 35.82%.The full model reaches aerial F1i of 21.89% and aerial F1img of 35.92%; candidate completion safeguards recall for small or under-specified aerial targets, though extra proposals may add distractors.Source: Completeness Improves Aerial Recall · p. 7, Table 4 · p. 7

Consistency Calibration Improves Pairing

  • Adding consistency calibration to factorized grounding plus candidate completeness raises pair Acc from 12.24% to 12.54% and pair F1 from 21.82% to 22.28%.Calibration reranks existing ground–aerial candidate pairs by identity compatibility rather than generating candidates; missed aerial targets and similar distractors still limit pair accuracy.Source: Consistency Calibration Improves Pairing · p. 7, Table 4 · p. 7

Conclusion

A-PAIR frames language-guided person detection as a joint ground-aerial decision requiring both boxes to match the query and the same identity. ICRG combines referential grounding, candidate-completeness supervision, and cross-view consistency calibration, improving pair-level detection over strong single-view baselines.

Conclusion

  • 22.28% pair F1, up from 16.65%, as ICRG improves ground-view, aerial-view, and pair-level detection over strong single-view baselines.The results demonstrate the value of explicitly modeling cross-view identity consistency for paired referring detection.Source: Conclusion · p. 7
  • A-PAIR pairs ground-aerial observations with same-identity boxes, factorized language expressions, all-person detection labels, and identity-consistency pairs.It formulates language-guided person detection as a joint decision requiring both predicted boxes to match the query and correspond to the same individual.Source: Conclusion · p. 7
  • ICRG addresses ambiguity and cross-view identity mismatches by unifying factorized referential grounding, candidate-completeness supervision, and cross-view consistency calibration in a query-to-pair framework.The conclusion contrasts this joint approach with single-view referring methods that process the two views independently.Source: Conclusion · p. 7
  • Future work will extend A-PAIR to temporal air-ground observations, more crowded scenes, and broader environmental diversity.Source: Conclusion · p. 7

Related papers

Loading 2608.27997v1…