PaperlayerView exact revision on arXiv

Source-linked AI summary

SonarVoxNet: Diver Detection in 3D Bounding Box using 3D Sonar

Eugene Park, Jiwon Lee, Seyoung Kan, Trung Dong, Xiaomin Lin, Jane Shin

arXiv:2610.01644v1cs.RO

TL;DR

FLS loses elevation needed to estimate diver orientation, and existing detectors do not represent free pitch and roll. SonarVoxNet adapts voxel and center-based detection to 3D sonar with full rotation, alongside the Diver3D labeled dataset. Ablations identify the yaw-only-to-full-SO(3) transition as the dominant factor in accurate detection.

  • Problem

    FLS discards elevation, and existing 3D sonar diver detection and labeled datasets do not support full orientation in diverse non-upright poses.

    Source: §I · pp. 1–2

  • Method

    SonarVoxNet adapts voxel encoding and an anchor-free center-based head to 3D sonar, using continuous 6D rotation, and the paper introduces the Diver3D dataset.

    Source: §I · p. 2

  • Results

    AP3D@0.35 rises from 0.328 to 0.538 on the dense backbone and from 0.319 to 0.515 on the sparse backbone with full-SO(3) rotation rather than yaw-only.

    Source: B. Ablation Study · p. 6

  • Takeaways & Limitations

    The results show that full-body diver orientation is recoverable from sparse 3D sonar alone, laying groundwork for future diver pose estimation and diver-robot interaction.

    Source: §VII · p. 8

    Diver3D has low positive density, averaging 1.3 divers per non-empty frame, leaving foreground supervision scarce.

    Source: F. Limitations and Future Work · p. 8

Abstract

from arXiv · show

Autonomous underwater vehicles (AUVs) assisting human divers must continuously track not only the diver's 3D position but also their full-body orientation. However, vision-based perception is unreliable underwater, and forward-looking sonar -- despite being widely used -- discards the elevation information needed for orientation estimation, posing a fundamental limitation. Recently commercialized 3D sonar preserves elevation but produces sparse, noisy returns, and existing detectors are built for dense LiDAR data and for targets that remain upright and rotate only about the yaw axis (e.g., vehicles, pedestrians), making them unable to represent a freely pitching and rolling diver. To address this gap, we present two contributions. First, SonarVoxNet adapts a voxel-based encoder and an anchor-free center-based detection head to 3D sonar data, replacing the conventional yaw-only rotation representation with a continuous 6D rotation parameterization to predict full 9-DoF oriented bounding boxes -- to our knowledge, the first 3D sonar diver detector to do so. Second, Diver3D is the first public 3D sonar dataset with full 3D orientation labels for divers in diverse, non-upright poses, collected at a natural cave-diving site. Through controlled ablations over the backbone and detection head, we show that the dominant factor behind accurate 3D sonar-based diver detection is the transition from yaw-only rotation to full-SO(3) rotation. This transition substantially improves detection accuracy and reduces orientation error. These results demonstrate that full-body diver orientation is recoverable from 3D sonar alone, laying the groundwork for future work on diver pose estimation and diver-robot interaction.

I. INTRODUCTION

AUV diver assistance requires tracking body orientation, but FLS loses elevation and existing 3D sonar detectors cannot represent freely pitching and rolling divers. SonarVoxNet and Diver3D address these model and data gaps, and ablations identify full-SO(3) rotation as the dominant factor in accurate 3D sonar detection.

Motivation

  • FLS projects 3D structure onto a 2D acoustic plane and discards elevation, which is needed to estimate a diver’s 3D orientation.The loss can be tolerable for rigid seabed objects, but not for divers whose orientation in the water column matters.Source: §I · p. 1
  • Existing dense-sensor detectors poorly fit sparse, noisy sonar returns and assume yaw-only rotation, unlike divers who pitch and roll continuously.A yaw-only detector can localize a diver while misrepresenting out-of-plane orientation relevant to human-robot interaction.Source: §I · p. 1

Contributions

  • SonarVoxNet adapts voxel encoding and an anchor-free center-based head to sparse, noisy 3D sonar, replacing yaw-only output with continuous 6D rotation for 9-DoF boxes.The paper presents it as the first 3D sonar diver detector of this kind.Source: §I · p. 2
  • Diver3D is the first public 3D sonar dataset with full 3D diver-orientation labels, collected in diverse poses at a natural cave-diving site.It comprises 2 hours, 37,010 frames, 34,568 annotated instances, and 36 scenes.Source: §I · p. 2
  • Full-SO(3) rotation, rather than backbone choice, is the dominant factor behind accurate 3D sonar diver detection.The ablation also finds that orientation-preserving sonar augmentation yields additional gains in detection accuracy and orientation fidelity.Source: §I · p. 2

II. RELATED WORKS

Prior 3D sonar detection addressed generic underwater objects, while existing detection heads encode only yaw and cannot represent a diver’s full rotation. SonarVoxNet uses voxel processing and a center-based head to adapt detection to sparse diver targets and regress geometry from a BEV representation.

II. RELATED WORKS

  • SonarPoint is the only prior 3D sonar detection work described, and it targets generic underwater objects rather than divers.This paper addresses diver detection with full 3D orientation and introduces a public dataset labeled for that task.Source: §II · p. 2
  • Voxel-based backbones discretize point clouds into a regular grid for efficient convolutional processing, motivating the paper’s VoxelNet-style encoding.The related-work discussion contrasts this with point-based methods that scale poorly with point count.Source: §II · p. 2
  • Existing methods assume upright targets with yaw-only orientation, so none of the five compared baselines can represent a diver’s continuous pitch and roll.SonarVoxNet instead predicts a full continuous 3D rotation.Source: §II · p. 2

A. Overall Architecture

  • SonarVoxNet quantizes sonar points into voxels, encodes them, collapses the z-axis into a 2D feature map, and refines that map into a BEV representation.The BEV feature is the shared representation consumed by the detection head.Source: A. Overall Architecture · p. 2
  • The center-based head predicts sparse divers as local heatmap peaks and regresses their geometry directly, avoiding fixed anchor priors.The architecture attaches this head to the BEV representation; its targets are typically one to two divers per frame.Source: A. Overall Architecture · pp. 2–3
  • The regressed geometry and location form 3D boxes that undergo causal geometry refinement focused on out-of-BEV-plane geometry.The refinement is intended to fit boxes to divers freely hovering underwater.Source: A. Overall Architecture · p. 3

B. Detection Head

The detection head predicts diver centers, dimensions, and full 3D rotations, producing 9-DoF oriented boxes. Training combines center heatmap classification with regression of box geometry and rotation.

B. Detection Head

  • The head predicts each diver’s center, height, dimensions, and full 3D rotation, yielding a 9-DoF oriented bounding box.Center recovery combines heatmap peaks with sub-pixel offsets and a separate height estimate.Source: B. Detection Head · p. 3
  • Unlike yaw-only boxes, the head regresses a continuous 6D rotation representation that maps to a full SO(3) rotation matrix.The representation is described as continuous and singularity-free over SO(3).Source: B. Detection Head · p. 3

C. Training Objective

  • Training combines a Gaussian focal heatmap loss with L1 regression of center offsets, height, log-dimensions, and the 6D rotation target.The regression targets are supervised at ground-truth center cells.Source: C. Training Objective · p. 3
  • The regression loss normalizes over the number of ground-truth center cells, using a minimum count of one.At inference, the predicted 6D component is converted to a valid rotation matrix by Gram–Schmidt orthonormalization.Source: C. Training Objective · p. 3

D. Causal Geometry Refinement

A causal inference-time stage refines box geometry using only current and past observations. It smooths center and rotation estimates and applies a shrink-only temporal size update, without changing detection recall.

D. Causal Geometry Refinement

  • The refinement stage associates detections into short tracks and recursively updates box geometry using observations only up to the current frame.Center and rotation use interacting multiple-model filters, while size uses a conservative shrink-only estimate.Source: D. Causal Geometry Refinement · pp. 3–4
  • The size update is restricted to shrinking the box relative to its causal temporal estimate.The rule takes the minimum of the current size and the estimate over frames 1:t.Source: D. Causal Geometry Refinement · p. 4
  • Because refinement neither promotes sub-threshold candidates nor removes detections, it affects box geometry while recall remains bounded by the detector.The paper treats refinement as a secondary inference-time contribution.Source: D. Causal Geometry Refinement · p. 4

A. Data Collection

Diver3D data were collected at Blue Grotto, an underwater cave system in Florida, across seven field sessions.

A. Data Collection

  • Blue Grotto recordings span natural and human-altered underwater settings, including divers, fish, robots, platforms, pipes, cave passages, and calibration targets.The footage was captured across seven separate field sessions.Source: A. Data Collection · p. 4

B. Labeling Process

Diver3D labels each diver with a full-body oriented box, including plausible body extent for reliably identified partial observations. The dataset provides diverse tilt examples, fixed scene-wise splits, and a defined sonar-grid and model-evaluation setup.

B. Labeling Process

  • Annotators place a full-body box in sonar-centric coordinates, labeling plausible complete extent when a diver is reliably identifiable but partly observed.Orientation uses a diver-centric right-handed body frame, with the torso-forward axis and shoulder-to-shoulder axis defining its orientation.Source: B. Labeling Process · p. 4

C. Analysis of the Dataset

V. EXPERIMENTS

  • The scene-wise split assigns 21 scenes to training, 7 to validation, and 8 to test, with validation for checkpoint selection and test reserved for reporting.Fixed scene identifiers prevent leakage between temporally adjacent observations.Source: C. Analysis of the Dataset · p. 4, §V · p. 4
  • The implementation crops sonar points to a fixed 3D region and voxelizes it into a 120 × 100 × 10 grid before voxel encoding and sparse-convolution processing.Experiments use fixed decoding and checkpoint-selection settings, and report means and population standard deviations over two random seeds.Source: §V · pp. 4–5

VI. RESULTS

The results section defines detection and geometric error measures for evaluation, then combines detection, localization, scale, and full-3D orientation into the Sonar–Diver Detection Score (SDS).

Evaluation Metrics

  • Detection quality is measured with IoU-based average precision in 3D and bird’s-eye view, while matched true positives also report translation and scale errors.These metrics are AP3D, APBEV, ATE, and ASE.Source: §VI · p. 5
  • AOE3D measures full-3D geodesic orientation error, folding only the cuboid’s yaw π ambiguity because diver up/down and front/back are distinguishable.The paper treats AOE3D as central because yaw-only error misses out-of-plane pose.Source: §VI · p. 5
  • SDS assigns half its score to detection and distributes the rest across localization, scale, and full-3D orientation.It omits velocity and attribute terms because those annotations are unavailable.Source: §VI · p. 5

A. Overall Performance

SonarVoxNet outperforms the evaluated baselines on Diver3D, and controlled ablations identify full-SO(3) rotation as the dominant architectural factor. The study also compares backbone and rotation-parameterization choices.

A. Overall Performance

  • SonarVoxNet without augmentation averaged improvements of 0.249 AP3D@0.35, 0.215 AP3D@0.50, and 0.131 SDS over all baselines on Diver3D.The comparison used test-set evaluations of LiDAR-domain detectors trained for 20 epochs without augmentation.Source: A. Overall Performance · p. 5

B. Ablation Study

  • Replacing the anchor head with a center head improves top-down detection, but not 3D box quality because it still regresses yaw-only orientation.The controlled matrix varies dense or sparse 3D backbones and anchor-yaw, center-yaw, or center-full-SO(3) heads without augmentation.Source: B. Ablation Study · p. 5, Table III · p. 6
  • AP3D@0.35 rises from 0.328 to 0.538 (dense) and 0.319 to 0.515 (sparse) with full-SO(3), while APBEV barely moves.The gains widen as the IoU criterion tightens, reaching nine- to fifteen-fold improvement at IoU@0.5.Source: B. Ablation Study · p. 6
  • AOE3D falls from 65–71° to 48–51° with full-SO(3), while yaw-only AOE remains essentially unchanged across the three heads.The comparison locates the orientation improvement in out-of-plane rotation that yaw-only AOE omits.Source: B. Ablation Study · p. 6
  • Dense and sparse full-SO(3) models perform within seed noise (SDS 0.576 vs. 0.574); sparse runs at 57 vs. 48 FPS.Sparse is level or better on strict AP3D@0.50 and every orientation, scale, and translation term; its speed is an advantage for embedded underwater platforms.Source: B. Ablation Study · p. 6
  • Across rotation targets, no representation leads on all three metrics; 6D is adopted for strict AP3D@0.5, continuity, and avoiding 9D’s inference-time SVD.All representations yield 45°–56° AOE3D, and near-random close-range estimates indicate a sensor-resolution error floor rather than a parameterization-set floor.Source: B. Ablation Study · p. 6

C. Performance over Multiple Divers

Test-set performance is reported by diver count, using ROC-AUC where AP3D cannot be computed for empty scenes. Augmentation addresses sparse positive supervision with an annealed sampling curriculum and yields substantial gains.

C. Performance over Multiple Divers

D. Data Augmentation

  • AP3D@0.35 rises 0.515→0.703 (+37%) and at strict IoU = 0.5, 0.233→0.359 (+54%), while SDS rises 0.574→0.673 with augmentation.The gains are reported on the Diver3D test set.Source: D. Data Augmentation · pp. 6–7, Table V · p. 7
  • Ground-truth sampling enriches sparse per-frame positives by reusing real diver returns, while the dataset has 1.3 divers per non-empty frame.The training frames are redundant, and a quarter are empty.Source: D. Data Augmentation · p. 7
  • The best augmentation curriculum targets four divers for the first 65% of training, then decreases the target by one each epoch while retaining one inserted target.Aspect-preserving placement is constrained by the sonar view; this floor-1 curriculum outperforms constant-four and zero-floor schedules.Source: D. Data Augmentation · p. 7

E. Causal Geometry Refinement

Causal geometry refinement improves the geometry and overlap quality of detections already produced, without adding detections. The paper also identifies data, annotation, and modeling boundaries and concludes that full-body orientation is recoverable from sparse 3D sonar.

E. Causal Geometry Refinement

  • AP3D gains rise from +7.16% at IoU 0.30 to +30.08% at 0.50 after refinement, lifting SDS from 0.673 to 0.719.Refinement improves translation, scale, and full-3D orientation metrics on the same box sets; its residual regressors are fit on validation and the improvements hold on both seeds.Source: E. Causal Geometry Refinement · p. 7, Table VI · p. 7
  • The inference-time refinement adjusts geometry only for existing detections, so it does not promote sub-threshold candidates or manufacture recall.Center and rotation residuals pass the declared gate; learned size refinement fails it by eroding AP3D at looser IoU thresholds.Source: E. Causal Geometry Refinement · p. 7

F. Limitations and Future Work

  • The study uses a coarse z-axis grid, has scarce foreground supervision, faces uncertain full-rotation labels, and leaves acoustic-specific modeling open.Diver3D has low positive density despite a frame count comparable to established LiDAR benchmarks.Source: F. Limitations and Future Work · p. 8

VII. CONCLUSION

  • The authors conclude that the yaw-only-to-full-SO(3) transition, rather than backbone or rotation-parameterization choice, dominates accurate 3D detection.They report full-body diver orientation as recoverable from sparse 3D sonar alone.Source: §VII · p. 8

Related papers

Loading 2610.01644v1…