Source-linked AI summary
GrabVG: Graph-Attentive Binding for Visual Grounding in UAV Imagery
Chaowei Wang, Yan Di, Jingjun Sun, Baozhe Liu, Jiaxu Tian, Yuheng Li, Guangqian Guo, Shan Gao
TL;DR
Crowded UAV imagery makes visually similar objects difficult to distinguish, while existing alignment and relation modeling can miss candidate-level differences and relevant local structure. GrabVG narrows the search to text-aligned hypotheses before binding appearance and topological evidence with graph attention, achieving 67.31% and 80.34% Acc@0.5 on AerialVG and AerialSense, respectively, with gains of 10.55 and 8.76 percentage points over corresponding baselines.
Problem
Crowded UAV scenes contain many small, visually similar objects, while existing alignment and dense relation methods may inadequately encode candidate differences and relevant local context.
Method
GrabVG separates preattentive hypothesis search from graph-attentive feature binding, using text-aligned candidate filtering and sparse graph attention over appearance and topological evidence.
Results
67.31% and 80.34% Acc@0.5 on AerialVG and AerialSense outperform corresponding baselines by 10.55 and 8.76 percentage points, respectively.
Takeaways & Limitations
GrabVG provides an effective and efficient search-before-binding formulation for visual grounding in crowded UAV imagery.
Takeaways & Limitations
Training uses pseudo-annotation supervision, and appends the ground-truth referent box when retained teacher predictions provide no positive match.
Abstract
from arXiv · showhide
Visual grounding in Unmanned Aerial Vehicle (UAV) imagery aims to localize a target object in complex bird's-eye-view scenes according to a natural language description. However, the abundance of small, densely distributed, and visually similar objects creates high visual redundancy, while repetitive local configurations give rise to strong topological ambiguity. Existing approaches mainly focus on visual--language feature alignment or dense contextual interaction, yet they struggle to distinguish subtle inter-instance differences and effectively exploit spatial topological structures, leading to inaccurate grounding in highly crowded scenarios. To address these challenges, we propose $\textbf{GrabVG}$, a novel visual grounding framework inspired by human visual search. GrabVG explicitly decomposes grounding into two sequential stages: $\textit{preattentive hypothesis search}$ and $\textit{graph-attentive feature binding}$. Specifically, we first generate a compact set of reliable object hypotheses through distillation-guided proposal induction and text-aware hypothesis filtering, substantially reducing background distractions and semantic mismatches. These hypotheses are then organized into a sparse graph, where language-guided intra-instance visual cues and inter-instance topological relationships are jointly bound and propagated via graph attention, enabling efficient spatial reasoning and accurate target localization. Extensive experiments on AerialVG and AerialSense show that GrabVG achieves a favorable accuracy--speed trade-off, reaching 67.31$\%$ and 80.34$\%$ Acc@0.5 and outperforming the corresponding baselines by 10.55 and 8.76 percentage points, respectively.
1. Introduction
GrabVG reformulates crowded UAV visual grounding as a search-before-binding process that first narrows candidate hypotheses and then binds appearance and topological evidence through graph attention. This design addresses visual redundancy and topological ambiguity while achieving strong accuracy with competitive inference speed.
- Motivation: UAV grounding is difficult because many small, densely distributed objects have similar appearances and repetitive spatial configurations.These properties create high visual redundancy and strong topological ambiguity, limiting appearance-only and topology-only identification.
- Limitations: Existing methods rely mainly on vision–language alignment or relation modeling, but may miss subtle candidate-level differences amid numerous intermediate queries.The introduction identifies insufficient candidate discrimination as a limitation of current cross-modal and relational approaches.
- Framework: GrabVG separates preattentive hypothesis search from graph-attentive feature binding, following a progressive human visual-search process.Guided Search motivates narrowing likely locations first, while Feature Integration Theory motivates focused binding of distributed visual features.
- Hypothesis search: Distillation-guided proposal induction and text-aware hypothesis filtering construct a compact, expression-aligned candidate space before referent selection.The mechanisms transfer expression-aligned proposals from an external teacher pipeline and filter likely background queries.
- Results: 67.31% Acc@0.5 on AerialVG and 80.34% Acc@0.5 on AerialSense exceed corresponding baselines by 10.55 and 8.76 percentage points, respectively.GrabVG maintains competitive inference speed while delivering this accuracy–speed trade-off.
2. Related Work
Related work spans proposal-based, one-stage, and transformer-based visual grounding, with later methods emphasizing visual–language alignment, relational reasoning, and semantic–geometric modeling. Remote-sensing and UAV grounding additionally motivates reasoning-centric and graph-based approaches, while GrabVG is positioned as a sparse, fixed-depth alternative.
- Visual Grounding: Visual grounding methods have progressed from proposal ranking to direct region regression, with subsequent work refining visual–language correspondence through language-guided and dynamic mechanisms.Early proposal-based methods use appearance, location, and contextual cues, whereas one-stage and transformer-based methods regress referred regions from cross-modal representations.
- UAV and Remote-Sensing Visual Grounding: Remote-sensing grounding addresses scale variation, cluttered backgrounds, high-resolution imagery, and small targets through improved alignment, relational reasoning, and semantic–geometric modeling.ProVG uses a progressive survey–locate–verify strategy to sequentially inject global, relational, and attribute cues into dense visual features.
- Reasoning-Centric Remote-Sensing Grounding: Reasoning-centric methods such as RSGround-R1 and Geo-R1 use supervised reasoning traces and reinforcement fine-tuning but require extra supervision or rollout-based optimization and incur autoregressive inference overhead.GrabVG instead performs fixed-depth differentiable reasoning over sparse candidate graphs.
- Relation Modeling for Visual Grounding: Relational grounding methods distinguish visually similar candidates using contextual comparisons, expression decomposition, language-guided graph attention, cross-modal relation graphs, and scene-graph alignment.The supplied passage notes that these relations are typically modeled through graph- or component-based relational mechanisms.
3. Method
GrabVG decomposes UAV visual grounding into explicit candidate construction and graph-based comparison. It induces and filters expression-aligned hypotheses, then binds appearance and local topology before selecting the referent.
- Overview: GrabVG constructs candidate hypotheses before graph-based comparison instead of directly decoding from an unconstrained set of visual tokens.A DETR-style proposal generator produces initial hypotheses, which are progressively filtered and bound before final referent selection.
- Preattentive Hypothesis Search: Distillation-guided proposal induction uses offline expression-aligned pseudo annotations to supervise multiple foreground hypotheses rather than only the annotated referent.The ground-truth referent box is appended during training when no retained teacher prediction meets the matching criterion.
- Preattentive Hypothesis Search: Text-aware hypothesis filtering uses language context to retain the top-K foreground candidates before the more expensive binding operations.The auxiliary filtering features estimate relevance, while the selected original decoder features are forwarded to feature binding.
- Graph-Attentive Feature Binding: Intra-instance appearance binding predicts language-aware, box-relative sampling locations and aggregates multi-scale visual features for each hypothesis.The anchors are conditioned on the referring expression, and differentiable bilinear sampling fuses features across visual levels.
- Graph-Attentive Feature Binding: Inter-instance topological binding performs edge-aware graph attention over a symmetric k-nearest-neighbor graph with self-loops and geometry-conditioned messages.The geometric descriptor encodes relative displacement, distance, direction, and log-scale differences, modulated by a pooled sentence representation.
4. Experiments
GrabVG is evaluated on AerialVG and AerialSense using Top1, Top5, and mIoU, achieving state-of-the-art grounding accuracy and effective cross-scenario generalization. Ablations show that proposal induction, appearance binding, sparse graph connectivity, and early filtering jointly improve localization and efficiency.
- Results on AerialVG: 67.31% Top1, 89.43% Top5, and 53.34% mIoU establish GrabVG’s state-of-the-art performance on AerialVG.GrabVG improves Top1 over OTA-Det by 12.41 percentage points and over AerialVG by 17.28 percentage points.
- Results on AerialSense: 80.34% Top1 and 72.00% mIoU show that GrabVG generalizes effectively to diverse UAV scenarios in AerialSense.It surpasses ReX-Omni and AerialVG by 8.87 and 8.76 percentage points in Top1, respectively.
- Component-wise Contribution: 62.84% Top1 follows proposal induction from the 56.76% PropVG baseline, while appearance binding adds a further 2.75-point improvement.The proposal module uses expression-aligned pseudo annotations to produce a more reliable, better-covered hypothesis space.
- Appearance-Binding Design: 65.59% Top1 and 87.89% Top5 are achieved by triple-ring anchors with scales [0.5, 1.0, 1.2], outperforming single- and double-ring layouts.The layout covers each hypothesis’s interior, boundary, and immediate surroundings.
- Topology and Propagation Depth: 67.31% Top1 from the 6-NN graph exceeds dense interaction at 63.98% and Delaunay triangulation at 66.21%, while four layers provide the best balance.Six layers reach the highest Top1 of 67.96% but reduce Top5; eight layers degrade both metrics.
- Inference Efficiency: 28.51 FPS makes GrabVG substantially faster than AerialVG at 13.65 FPS and close to OTA-Det at 32.00 FPS.Filtering raises speed from 25.09 FPS without filtering by reducing candidates processed by the binding modules.
5. Conclusion
GrabVG separates candidate-space construction from evidence-based referent selection for visual grounding in crowded UAV imagery. It combines text-aware candidate filtering with adaptive visual sampling and sparse graph attention over appearance and geometric-neighborhood information.
- GrabVG separates candidate-space construction from evidence-based referent selection in crowded UAV imagery.This two-stage design first establishes candidates and then selects the referent using evidence.
- Expression-aligned pseudo-annotation supervision and text-aware filtering establish a manageable candidate pool.
- Adaptive visual sampling and sparse graph attention compare local appearance with geometric neighborhood information.