PaperlayerView exact revision on arXiv

Source-linked AI summary

OBB-PBD: Extending Parallel Box Decoding to Oriented Bounding Boxes for Referring Expression Grounding in Aerial Imagery

Chang Woo Song, Yeong Jun Koh

arXiv:2610.35450v1cs.CV

TL;DR

Grounding in overhead imagery requires oriented boxes, while existing PBD covers only four-coordinate horizontal boxes. This paper extends the block-aligned interface to six-slot oriented boxes and evaluates it with diagnostic controls. On AVVG, the extension improves rotIoU over the zero-shot baseline on the complete test split.

  • Problem

    PBD’s released block-aligned interface covers four-coordinate horizontal boxes, leaving its extension to six-slot oriented boxes unexamined despite arbitrary object orientations in overhead imagery.

    Source: §1 · p. 2

  • Method

    The study extends PBD to a six-slot oriented-box interface and validates the implementation with diagnostic probes and controlled evaluation.

    Source: §1 · p. 2, §6 · p. 15

  • Results

    OBB-PBD improves rotIoU by 0.178 ± 0.012 over the zero-shot baseline, from 0.1253 to 0.303, on the complete AVVG test split across three seeds.

    Source: §4.1 · p. 11, §6 · p. 15

  • Takeaways & Limitations

    Diagnostic probes can reveal decoding defects that benchmark accuracy or a decreasing training loss may not expose.

    Source: §1 · p. 2, §6 · p. 15

    The study does not compare head-to-head with joint OBB regression and does not claim superiority over that architecture family.

    Source: §5.2 · p. 14

Abstract

from arXiv · show

Referring expression grounding in overhead imagery requires oriented bounding boxes rather than horizontal ones, since aerial objects appear at arbitrary orientations. Most grounding models emit box coordinates sequentially; Parallel Box Decoding (PBD) instead predicts all geometry slots in one forward pass, but only for four-coordinate horizontal boxes. We extend this block-aligned interface to a six-slot oriented representation using a double-angle parameterization. Rather than assuming correctness from a decreasing loss curve, we verify the extension with image/prompt-shuffle controls and visibility/leakage probes, which show that the corrected model depends on both the image and the prompt rather than on prompt statistics alone. On the aerial vehicle visual grounding (AVVG) benchmark, whose targets are genuinely oriented, the extended model improves rotated intersection over union (rotIoU) from 0.1253 for the zero-shot baseline to 0.303 +/- 0.012 (mean +/- standard deviation over three training seeds) across the complete 6139-record test split spanning all 120 images. Learning oriented boxes rather than their axis-aligned envelopes adds 0.055 +/- 0.008 rotIoU beyond a matched control (per-seed 95% confidence intervals all excluding zero), whereas block expansion alone yields no rotIoU gain distinguishable from zero.

1. Introduction

Overhead grounding must handle small, dense, variably oriented objects and expressions that rely on relationships, while horizontal boxes fit elongated rotated objects poorly. This study extends PBD from four-coordinate horizontal boxes to six-slot oriented boxes and treats verification and controlled decomposition as part of evaluating the extension.

Motivation

  • In overhead imagery, dense scenes, small targets, relational expressions, and arbitrary object orientations make grounding difficult, while horizontal boxes fit elongated rotated objects poorly.Oriented boxes represent center, size, and orientation for objects with meaningful aspect ratios and headings.Source: §1 · p. 2

Research gap

  • PBD predicts every box coordinate slot together in one forward pass, but its released four-coordinate interface left extension to six-slot oriented boxes unexamined.The study also notes that joint-regression OBB architectures are a different design and are not compared head-to-head.Source: §1 · p. 2

Verification

  • A block-alignment failure can silently hide image and prompt tokens from coordinate slots while training loss decreases, motivating targeted diagnostic probes.The study treats verification as part of the extension rather than relying on the loss curve alone.Source: §1 · p. 2

Contributions

  • The study’s contributions are a structurally validated six-slot PBD extension, diagnostic probes for block-parallel extensions, and a bounded AVVG result on the complete test split.Checkpoint selection was kept disjoint from that test split.Source: §1 · p. 2

Evaluation plan

  • A three-step decomposition separates the joint effects of domain adaptation and decoding-path change, six-slot format alone, and learning oriented geometry.The paper reports scope and limitations separately.Source: §1 · p. 3

2. Related Work

Prior work establishes text-conditioned grounding, remote-sensing grounding, oriented aerial detection, and structured coordinate decoding as related but distinct lines. This study positions its contribution as an empirically validated extension of PBD to oriented boxes, with accuracy and latency characterized under a controlled protocol.

2.1. Visual Grounding on Natural Images

  • Natural-image grounding methods establish text-conditioned detection, but the cited systems do not address oriented geometry or parallel block-aligned coordinate decoding.The passage situates these methods in a lineage from referring-expression benchmarks and DETR to grounded pretraining and multimodal models.Source: §2.1 · p. 3

2.2. Remote-Sensing Visual Grounding

  • Remote-sensing grounding has progressed from horizontal-box benchmarks to models and collections supporting oriented grounding, including GeoChat and GeoGround.Other remote-sensing models cover grounding alongside additional tasks or use segmentation; their reported results are not directly comparable to AVVG because datasets, splits, and metrics differ.Source: §2.2 · pp. 3–4

2.3. Oriented Object Detection in Aerial Imagery

  • Aerial detection research motivates oriented geometry, but its detection-focused methods and metrics are distinct from text-conditioned grounding and this study’s rotated-IoU protocol.The passage surveys oriented annotations and detector approaches, including methods addressing angle-regression discontinuity.Source: §2.3 · p. 4

2.4. Structured and Parallel Coordinate Decoding

  • LocateAnything’s PBD predicts a box’s geometry slots together in one block-aligned pass, whereas its released formulation covers horizontal boxes; SATGround uses a separate joint-regression design.This study does not compare against SATGround head-to-head under a matched protocol.Source: §2.4 · p. 4

2.5. Positioning

  • The contribution is an empirically validated extension of LocateAnything-style block-aligned decoding to orientation, with accuracy and latency characterized under a controlled protocol.GeoChat and GeoGround already establish oriented-box grounding capability, while LocateAnything provides the structured decoding interface.Source: §2.5 · p. 5

3. Materials and Methods

The method extends parallel box decoding from horizontal to oriented boxes using six geometry slots, and validates both the representation and the model’s access to image and prompt inputs. It evaluates the adapted model on AVVG with direct decoding and complementary diagnostic checks.

3.1. Task Formulation

  • The six-slot representation encodes center, size, and orientation with double-angle terms, and canonicalizes boxes to w ≥ h before encoding.The shared uniform codec quantizes coordinates to 1001 levels; signed orientation values are rescaled before encoding.Source: §3.1 · p. 5

3.2. Parallel OBB Decoding

  • OBB-PBD extends LocateAnything’s four-slot corner block by replacing corner coordinates with center-and-size and adding two orientation slots.The <box> trigger starts a single forward pass that predicts the six geometry tokens together; the closing token follows at the next step.Source: §3.2 · pp. 5–6

3.3. Backbone Corrections Required by the Extension

  • A training-path prefix-length bug hid image and prompt tokens from coordinate slots despite falling loss; recovering the length from mask tokens restored visibility.The first slot’s attention mask changed from 7 of 894 keys, none image or prompt tokens, to all 894.Source: §3.3 · p. 6

3.4. Training Configuration

  • Training combines next-token cross-entropy with differentiable soft-argmax spatial, circular-consistency, and directional losses; its normalized directional gate excludes boxes with normalized side ratio below 1.05, including those near the image’s aspect ratio (about 16:9).Keeping seed 1’s initialization and data order, the physical-aspect-ratio gate scored rotIoU 0.3008 (95% CI [0.2799, 0.3228]) versus 0.3124 for the otherwise identical normalized-gate run; the paired difference was −0.012 (per-record image-cluster CI [−0.019, −0.004]). This interval reflects test-set sampling only, and the single physical-gate run cannot separate the gate’s effect from training randomness.Source: §3.4 · pp. 7–8
  • Training uses AVVG alone, with 421 training and 74 validation images disjoint from the 120-image test split, adapting LocateAnything-3B with LoRA and a full projector.The language model and last six MoonViT blocks use LoRA; the multimodal projector is trained as a full module.Source: §3.4 · pp. 7–9

3.5. Evaluation Protocol and Checkpoint Selection

  • Evaluation uses direct decoding and selects each trained model’s checkpoint by validation rotIoU; the primary metric is polygon-based rotated IoU.The zero-shot baseline instead uses the upstream generate() path, and the ground-truth HBB envelope is a reference rather than a formal upper bound.Source: §3.5 · pp. 9–10

3.6. Diagnostic Checks

  • Diagnostic checks shuffle images and prompts and probe visibility and leakage, testing visual dependence, prompt dependence, and whether ground-truth tokens enter training inputs.Qualitative overlays also distinguish geometric errors from referring failures such as selecting a neighboring vehicle.Source: §3.6 · p. 10
  • Illustrative overlays show OBB-PBD capturing vehicle tilt where the axis-aligned baseline cannot, as well as both models missing a neighboring-car reference.The three examples were selected to illustrate contrasting behaviors, not sampled at random.Source: §3.6 · p. 11

4. Results

On the complete AVVG test split, OBB-PBD improves rotated overlap over the zero-shot baseline. Matched controls find no distinguishable rotIoU gain from block expansion alone, while orientation learning improves rotated alignment; image-shuffle diagnostics support image dependence, and parallel decoding is faster than autoregressive decoding.

4.1. Main Quantitative Results

  • 0.178 ± 0.012 rotIoU improvement over the zero-shot baseline (0.1253 to 0.303) on the complete 6139-record, 120-image AVVG test split; this is a cross-path comparison.The baseline uses upstream generate(), whereas OBB-PBD uses the direct decoder.Source: §4.1 · p. 11

4.2. HBB-Envelope Control: Isolating the Format-Switch Effect from the Orientation-Learning Effect

  • The six-slot format-switch step’s rotIoU interval included zero in all seeds, while orientation learning added +0.046 to +0.060 rotIoU with intervals excluding zero.The orientation-learning comparison includes the tighter extent permitted by an oriented box.Source: §4.2 · p. 12, Table 3 · p. 13

4.3. Image-Dependence Diagnostic

  • Across all three seeds, matched-condition rotIoU was 45.79–50.96 times image-shuffled rotIoU, supporting image dependence under the validated pairing.After excluding shuffled records that received their own image, the residual matched a model-free location-and-size prior.Source: §4.3 · p. 13

4.4. Efficiency and Limitations of the Result

  • Parallel decoding took 329.82 ms versus 568.65 ms autoregressively for six coordinates (1.72× faster), while dense-scene referring failures remain the main accuracy limitation.The shared prefix, including the vision encoder, dominates both paths’ cost; subjective prompts can also be unresolved from box geometry alone.Source: §4.4 · pp. 13–14

5. Discussion

The decomposition attributes a consistent rotIoU gain to learning oriented geometry, while block expansion alone shows no distinguishable gain. The discussion limits the evidence to the validated AVVG/UAV setting and notes caveats in the comparisons and angle metric.

5.1. Decomposition of Format and Orientation Effects

  • Learning orientation adds 0.055 ± 0.008 rotIoU over a matched control, while block expansion alone shows no distinguishable rotIoU gain.The orientation gain includes both angle and tighter extent; the format-switch comparison's 95% CI includes zero in every seed.Source: §5.1 · p. 14
  • The adaptation and decoding-path step accounts for most of the axis-aligned localization gain, while orientation learning improves rotated alignment but can lower aabbIoU by up to 0.021 versus the HBB-envelope control.The interval for that aabbIoU difference excludes zero in seed 3.Source: §5.1 · p. 14

5.2. Limitations

  • The evidence is limited to the validated AVVG/UAV setting and does not establish generalization across datasets, sensors, object scales, or annotation formats, or the optimality of OBBs or PBD.Source: §5.2 · p. 14
  • The study does not compare head-to-head with localization-token/MLP joint-regression architectures such as SATGround [9], so it makes no superiority claim over joint OBB regression.Such a comparison would require reimplementing another architecture family under this study's backbone, data, and evaluation protocol.Source: §5.2 · p. 14
  • Angle MAE averages over all successfully decoded samples, including those with rnorm < 1.05, so it is not restricted to samples optimized by the directional loss.On AVVG's non-square images, the gate selects boxes near the image's aspect ratio; included samples can still have large periodicdistance errors.Source: §5.2 · p. 14

6. Conclusions

The paper extends LocateAnything's Parallel Box Decoding interface to six-slot oriented boxes and evaluates the extension with diagnostic controls and matched comparisons. It identifies follow-up validation and comparison work, while limiting its conclusions to the studied AVVG/UAV setting.

6. Conclusions

  • The paper extends Parallel Box Decoding from four-coordinate horizontal boxes to six-slot oriented boxes and evaluates the extension under a diagnostic protocol.Matched controls distinguish the contribution of learning oriented boxes from that of the larger coordinate block.Source: §6 · p. 15
  • Future work should validate loss weights with multi-seed, equal-budget ablations and compare against localization-token/MLP joint-regression architectures under a matched protocol.Source: §6 · p. 15
  • The validated protocol has not been extended beyond AVVG/UAV data to other sensors, object scales, or grounding annotation formats.Source: §6 · p. 15
  • The authors state that code, training configuration, evaluation protocol, and trained checkpoints will be made available with the specified licenses.Source: §6 · p. 15

Appendix A. Upstream Implementation Defects

Table A1 catalogs corrections to the pretrained LocateAnything-3B code, with one correction specific to coordinate blocks larger than four and others affecting fine-tuning generally.

Appendix A. Upstream Implementation Defects

  • U1 applies only when the configuration has more than four coordinate slots.Source: Appendix A · p. 16
  • U2–U5 affect any fine-tuning of this backbone.Source: Appendix A · p. 16
  • Table A1 lists corrections made to the pretrained LocateAnything-3B code.Source: Table A1 · p. 16

Appendix B. Per-Seed Results

Appendix B supplies per-seed results and paired arm differences, with uncertainty estimated by image-cluster bootstrap intervals. Its dagger-marked comparison captures a joint effect rather than fine-tuning alone.

Appendix B. Per-Seed Results

  • Tables A2 and A3 report per-seed arm results and paired differences, using 95% image-cluster bootstrap CIs from 120 test images and 10,000 draws.The three training seeds use distinct data-order seeds but the same image-level split.Source: Appendix B · p. 16, Table A3 · p. 17
  • The zero-shot baseline has rotIoU 0.1253 [0.1073, 0.1460] and aabbIoU 0.1512 in Table A2.Source: Table A2 · p. 16
  • The dagger-marked comparison is the joint effect of domain adaptation and a decoding-path change, not fine-tuning alone.Source: Appendix B · p. 17

Related papers

Loading 2610.35450v1…