Source-linked AI summary
A-PAIR: A Benchmark and Identity-Consistent Grounding Framework for Air-Ground Cross-View Referring Person Detection
Zhoupeng Guo, Xinjie Yao, Yunqi Zhu, Zhihe Fan, Siqi Zhao, Jianjun Chen, Yichen Dong, Yan Fan, Pengfei Zhu
TL;DR
AGCV-RPD asks whether language can identify the same person across paired ground and aerial views, a setting complicated by similar pedestrians and weak aerial appearance cues. The paper introduces A-PAIR with FARA and proposes ICRG, whose joint grounding and identity-consistency approach raises pair F1 from 16.65% to 22.28%.
Problem
Existing grounding methods do not jointly require language-region alignment and cross-view identity consistency for paired ground-aerial detection.
Method
The paper constructs A-PAIR with FARA and uses ICRG to combine factorized grounding, candidate-completeness supervision, and cross-view consistency calibration.
Results
22.28% pair F1, up from 16.65% for GroundingDINO-T, is reported for ICRG on the image-disjoint test split.
Takeaways & Limitations
AGCV-RPD requires joint paired detection and identity-consistent reasoning rather than independent single-view grounding.
Abstract
from arXiv · showhide
Air-ground cross-view referring person detection is a necessary component in the language-to-perception-to-control chain of collective embodied intelligence, grounding a language command into the same physical target before ground and aerial agents can coordinate downstream actions. Existing referring expression comprehension and open-vocabulary grounding methods do not jointly account for cross-view identity consistency, making them insufficient for Air-Ground Cross-View Referring Person Detection (AGCV-RPD), which involves similar pedestrian distractors, weak aerial appearance cues, and cross-view identity consistency. To study this problem, we introduce Air-Ground Paired Identity-Aware Referring (A-PAIR), the first comprehensive AGCV-RPD benchmark, containing 22,137 cross-view referring samples. To construct A-PAIR efficiently, we propose Factorized Annotation and Referential Alignment (FARA), a semi-automatic annotation framework that generates factorized referring descriptions and identity-consistency supervision at reduced cost. We propose Identity-Consistent Referring Grounding (ICRG), a framework that combines factorized referential grounding, candidate-completeness supervision, and cross-view consistency calibration for joint air-ground pair selection. ICRG improves ground, aerial, and pair-level detection over strong baselines, increasing pair F1 from 16.65% to 22.28%. These results show that AGCV-RPD requires paired detection and identity-consistent reasoning.
Air-Ground Cross-View Referring Person Detection
The supplied passage identifies the paper’s authors.
- Zhoupeng Guo is listed as the first author.
- Xinjie Yao, Yunqi Zhu, Zhihe Fan, and Siqi Zhao are also listed among the authors.
- Jianjun Chen, Yichen Dong, Yan Fan, and Pengfei Zhu complete the displayed author list.
Introduction
AGCV-RPD requires selecting the same referred person in paired ground and aerial views, beyond independent single-view grounding. The paper introduces A-PAIR and ICRG to address ambiguity, weak aerial cues, and identity consistency.
- Introduction: AGCV-RPD requires ground and aerial predictions to identify the same physical person, not merely match language independently in each view.
- Introduction: The task is challenged by similar pedestrians, weak appearance cues for small aerial targets, and cross-view identity consistency.
- Introduction: ICRG combines factorized referential grounding, candidate-completeness supervision, and auxiliary cross-view identity-consistency supervision in a unified query-to-pair pipeline.
- Introduction: A-PAIR is presented as a comprehensive AGCV-RPD benchmark, while FARA provides a semi-automatic annotation framework for cross-view descriptions and identity supervision.
- Introduction: A-PAIR contains 22,137 cross-view referring samples over 7,588 unique ground images and 3,891 unique aerial images.
Related Work
Prior work supplies single-view grounding, cross-view place matching, or crop-based person re-identification, but does not solve paired language-guided identity decisions.
- Language-Guided Person Detection: Language-guided person detection has developed context-aware, transformer, region-language pre-training, person-search, multimodal, and small-object approaches.
- Language-Guided Person Detection: Existing methods remain single-view and do not require ground and aerial predictions to identify the same individual.
- Cross-View Air-Ground Perception: Geo-localization matches places and re-identification matches given crops, whereas A-PAIR evaluates paired ground-aerial referring with language alignment and identity consistency.
The A-PAIR Benchmark
A-PAIR is a paired air-ground benchmark built from cross-view observations, with factorized language descriptions, identity supervision, and evaluation protocols that assess both views jointly.
- Benchmark composition: A-PAIR supplies appearance-full and view-specific scene-spatial expressions, enabling full-, spatial-, and factorized-prompt evaluation within one protocol.FARA constructs paired cross-view records from verified G2APS boxes and generates factorized descriptions for the target identity.
- Annotation and supervision: FARA augments paired records with all-person detection labels and same-identity positive and different-identity negative pairs for cross-view supervision.The training export contains 32,016 ground and 34,108 aerial annotations, plus 377,402 identity pairs including 50,741 positives.
- Benchmark composition: 22,137 cross-view referring samples pair ground–aerial images with a target identity, split into 15,497 training, 2,213 validation, and 4,427 testing samples.The benchmark includes 7,588 unique ground images, 3,891 unique aerial images, and 7,588 shared-identity ground-aerial pairs.
- Benchmark comparison: Unlike existing benchmarks, A-PAIR combines language descriptions, paired aerial–ground views, same-identity boxes, and paired evaluation protocols.This combination provides a data foundation for bridging the two modalities while maintaining cross-view consistency.
- Evaluation: Pair-level predictions are correct only when both views are correct, making Pair Acc and Pair F1 the primary indicators of cross-view competence.The evaluation also reports instance-level and image-level F1 and accuracy for each view.
Method
ICRG jointly selects ground-aerial person pairs by combining factorized language grounding, all-person candidate completion, and auxiliary cross-view identity consistency.
- ICRG processes paired ground and aerial images with factorized referring expressions to predict one identity-consistent box pair.It builds candidate pools, calibrates candidate pairs, and selects the final ground and aerial boxes.
- Factorized Grounding: Factorized grounding separates stable appearance semantics from view-dependent scene-spatial cues, producing complementary candidates for each view.Appearance supports precise detection when visible, while scene-spatial cues improve recall amid ambiguity; their union defers identity selection to fusion.
- Candidate Completeness: Candidate-completeness supervision uses an all-person detector to enumerate every person, allowing overlooked targets to enter each view’s candidate pool.The detector is recall-oriented, while later referring and consistency scores filter spurious boxes.
- Cross-View Consistency: Cross-view identity consistency calibrates language-compatible candidates by comparing ground and aerial crop embeddings for the same physical person.The auxiliary signal bridges the street-level and aerial appearance gap and can suppress pairs whose boxes depict different people.
- Fusion and Inference: Inference constructs referring and all-person candidates in both views, scores candidate pairs with detection, referring, and identity-consistency cues, and returns the highest-scoring pair.Algorithm 1 summarizes this candidate-generation and pair-scoring procedure.
ij , sid(gi, aj)
ICRG evaluates every ground-aerial candidate pair with fused detection, referring, and identity-consistency evidence, then selects the pair with the maximum score.
- Inference scores each candidate pair using fused detection, referring, and identity-consistency cues.The candidate pair is evaluated through the fused score in Eq. (11).
- The fusion weights are selected by validation grid search to maximize pair accuracy rather than learned by gradient descent.This keeps fusion interpretable and avoids overfitting the small test set.
- The predicted pair is the arg-max candidate pair under the fused score, with learnable components trained on the training split and fusion weights fixed afterward.The test split is evaluated once after validation-based weight selection.
Experiments
Experiments show that paired detection is substantially harder than single-view localization, while ICRG improves pair selection by combining factorized grounding, candidate completeness, and cross-view consistency.
- Protocol: The experiments use 15,497 training, 2,213 validation, and 4,427 test samples under the A-PAIR evaluation protocol.Methods are trained on the training split and evaluated once on the test split unless noted otherwise.
- IoU-Quality Diagnosis: 12.54% of A-PAIR test samples are pair-level hits with IoU ≥0.5 in both views, indicating that correct paired detection is harder than obtaining plausible individual-view boxes.Failure analysis identifies aerial detection, similar pedestrians, and identity mismatch across views as recurring challenges.
- Main Results: 22.28% pair F1 is achieved by ICRG versus 16.65% for GroundingDINO-T, the strongest single-view baseline.Figure 5 shows ICRG attaining the best tradeoff between ground-view detection and pair-level performance.
- Factorized Grounding: 20.88% pair F1 results when factorized grounding is enabled alone, compared with 3.60% without it.Separating appearance from scene-spatial cues supplies complementary candidates for ambiguous descriptions and similar pedestrians.
- Candidate Completeness: Adding candidate completeness raises aerial F1i from 20.83% to 21.82% and aerial F1img from 34.47% to 35.82% over the factorized-only setting.The full model reaches 21.89% aerial F1i and 35.92% aerial F1img, indicating improved target availability alongside possible distractor introduction.
- Consistency Calibration: Adding consistency calibration increases pair Acc from 12.24% to 12.54% and pair F1 from 21.82% to 22.28%.Consistency calibration reranks existing ground–aerial candidate pairs rather than generating new candidates.
Conclusion
The paper introduces AGCV-RPD, the A-PAIR benchmark, FARA annotation, and ICRG for joint air-ground person detection. ICRG improves ground, aerial, and pair-level detection over strong single-view baselines, raising pair F1 from 16.65% to 22.28% and supporting explicit cross-view identity consistency.
- Benchmark: A-PAIR is a benchmark of paired ground-aerial observations with same-identity boxes, factorized language expressions, all-person labels, and identity-consistency pairs.The benchmark supports joint cross-view decision-making rather than independent single-view grounding.
- Framework: ICRG unifies factorized referential grounding, candidate-completeness supervision, and cross-view consistency calibration in a query-to-pair pipeline.These components address ambiguity, candidate recall, and identity-consistent pair selection.
- Results: 22.28% pair F1 is obtained by ICRG, increasing from 16.65% for strong single-view baselines.The reported improvement spans ground-view, aerial-view, and pair-level detection.
- Conclusion: The results demonstrate the value of explicitly modeling cross-view identity consistency for paired referring detection.Future work extends A-PAIR toward temporal observations and more crowded scenes.