PaperlayerView exact revision on arXiv

Source-linked AI summary

Spiking Neural Networks for Energy-Efficient Object Detection in Forward-Looking Sonar Imagery

Gwenevere Frank, Gert Cauwenberghs

arXiv:2608.22072v1cs.CVeess.SP

TL;DR

AUVs need sustained object detection under fixed power budgets, but existing FLS detectors are energy-intensive and prior work has not evaluated SNNs for sonar imagery. This paper benchmarks directly trained SpikeYOLO across three FLS datasets against CNN baselines and finds substantially lower theoretical energy with competitive accuracy and stronger speckle-noise robustness. The results support SNN suitability for selected FLS detection settings, while performance remains dependent on acoustic-return complexity.

  • Problem

    The paper addresses whether SNNs can provide energy-efficient object detection for FLS imagery on power-constrained AUVs, a setting not evaluated by prior work.

  • Method

    The paper benchmarks fully spiking SpikeYOLO, trained with surrogate gradients, against conventional detector baselines across three FLS object-detection datasets using explicit theoretical energy accounting.

  • Results

    SpikeYOLO achieves comparable or near-comparable accuracy to YOLOv8m across the evaluated datasets while using less theoretical energy, including 3.3× lower energy on UATD.

  • Takeaways & Limitations

    SNNs appear best suited to sparse, high-contrast FLS returns but remain competitive on more complex acoustic detection tasks.

  • Takeaways & Limitations

    SNN performance is constrained by acoustic-return complexity: on UATD, which has more classes and complex signatures, it scores below YOLOv8m.

Abstract

from arXiv · show

Autonomous underwater vehicles (AUVs) are increasingly important tools in industries ranging from research, to energy, to defense. AUVs are power-constrained platforms operating in remote environments with fixed battery capacities, where propulsion competes with compute and sensors for power over lengthy mission durations. AUVs frequently operate in dark or turbid waters where optical sensing is of limited value, and rely on sonar as their primary sensing modality. Convolutional neural networks (CNNs) are the state-of-the-art solution for object detection in forward-looking sonar imagery, but are energy expensive (e.g. YOLOv8m: 322 mJ/inference). Spiking neural networks (SNNs) rely on binary spike activations and thus sparse accumulate-only operations, allowing them to be remarkably energy efficient, particularly when paired with dedicated neuromorphic hardware. The sparse, high-contrast structure of forward-looking sonar (FLS) returns is structurally matched to spike coding in a way that optical imagery is not. No prior work has assessed the suitability of SNNs for object detection in FLS imagery. SpikeYOLO, a fully spiking network trained with surrogate gradients, was benchmarked against state-of-the-art CNN baselines on three FLS object detection datasets. Key results: SpikeYOLO T=2 achieves 3.3$\times$ lower theoretical compute energy on UATD (97 vs 322 mJ) at competitive accuracy (0.529 mAP@0.5:0.95 vs. YOLOv8m's 0.575); SpikeYOLO matches YOLOv8m on mAP@0.5 and outperforms YOLO-SONAR and Fast R-CNN baselines on the sparse Marine-Debris-FLS dataset at 4.4$\times$ lower energy; SpikeYOLO demonstrates superior robustness to multiplicative speckle noise (3.0% degradation at $σ{=}0.4$ vs. 8.9% for YOLOv8m), outperforming YOLOv8m outright at $σ{=}0.6$, directly relevant to real-world FLS deployment.

1. Introduction

AUVs must sustain sonar-based object detection during long, power-constrained missions, while FLS imagery poses distinctive sensing challenges. The paper motivates SNNs as a potentially energy-efficient match for sparse, high-contrast sonar returns and evaluates SpikeYOLO across three FLS datasets.

  • Power-constrained AUV operation: AUVs rely on fixed internal batteries to power propulsion, sensors, and computation during lengthy missions.Survey-class AUVs carry 16–17 kWh, while hotel loads consume tens to hundreds of watts.
  • Power-constrained AUV operation: Object detection must continue throughout dives for obstacle avoidance, mine detection, victim localization, and habitat characterization.These missions routinely exceed 10 hours, and state-of-the-art detectors consume hundreds of millijoules per inference.
  • FLS sensing challenges: FLS supports detection in turbid, unlit waters but contains sparse, high-contrast returns alongside speckle noise, acoustic shadows, multipath reverberation, and limited labels.CNNs such as YOLOv8m and RT-DETR-L are state-of-the-art for FLS detection.
  • SNN motivation: SNNs replace conventional multiply-accumulate operations with sparse accumulate-only processing using binary spike events.Accumulate-only operations cost roughly 5× less energy than multiply-accumulate operations in 45 nm CMOS.
  • Study contribution: The paper provides the first FLS evaluation of SNN object detection, benchmarking SpikeYOLO against strong baselines across UATD, Marine-Debris-FLS, and WHFLS.SpikeYOLO T=2 uses 3.3× less theoretical compute energy than YOLOv8m on UATD while retaining competitive accuracy.

2. Related Work

Prior FLS detectors achieve strong performance but generally treat inference energy as outside the deployment problem. SNN research offers sparse, event-driven alternatives, and this paper extends directly trained SNN detection to FLS with explicit energy accounting.

  • FLS object detection: FLS detection is complicated by inhomogeneous resolution, duplicate multipath echoes, acoustic shadows, and ambiguous target returns.These properties can reduce resolution, duplicate objects, or obscure targets.
  • FLS object detection: Earlier CNN-based FLS detectors demonstrated successful detection, including Fast R-CNN deployment on an NVIDIA Jetson at 0.52 s per frame.The cited water-tank study reported average precision ranging from 0.752 to 0.881.
  • FLS object detection: Modern YOLO- and transformer-based FLS detectors adapt attention and feature-extraction mechanisms to sparse, high-contrast sonar imagery.YOLOv8 adaptations reported 98.2% mAP@0.5 on the URPC2021 competition dataset.
  • Deployment gap: Prior FLS studies did not treat inference energy as a deployment constraint and assumed GPU-based inference rather than power-limited embedded AUV compute.This leaves the energy implications of FLS detection unaddressed in the cited work.
  • Deployment gap: AUV hotel loads directly affect maximum range and mission duration, while embedded Jetson sonar processing can consume 10–15 W.This makes detector compute relevant to long-duration deployment budgets.
  • Energy-efficient computation: FPGA accelerators, INT8 quantization, and pruning reduce overhead but retain conventional ANN computation dominated by multiply-accumulate operations.The paper contrasts these approaches with SNNs, which change the underlying operation type and exploit sparsity.
  • Energy-efficient computation: Prior SNN work found improved robustness under small training sets and noisy inputs, properties relevant to scarce labels and pervasive speckle noise in FLS.Direct surrogate-gradient training was developed to avoid accuracy losses associated with ANN-to-SNN conversion.
  • Neuromorphic hardware: Neuromorphic processors such as TrueNorth and Loihi provide scalable hardware platforms for sparse, programmable SNN execution.TrueNorth reported 46 billion synaptic operations per second per watt, while Loihi supports programmable on-chip learning rules.

3. Method

The method evaluates fully spiking and conventional detectors across three FLS datasets, using COCO initialization, dataset-specific fine-tuning, and theoretical operation-energy estimates. SpikeYOLO is trained end-to-end with surrogate gradients and evaluated against CNN and transformer baselines under a common retraining protocol.

  • 3.1. Datasets: Three public FLS datasets span different sonar platforms, environments, and object categories: UATD, Marine-Debris-FLS, and WHFLS.UATD contains 10 shape classes; WHFLS contains victim, boat, and plane classes and includes real ocean clutter and environmental noise.
  • 3.2.1. SpikeYOLO: SpikeYOLO is a fully spiking detector trained end-to-end with surrogate gradient descent using the YOLOv8 Backbone/Neck/Head structure.Its C2f modules are replaced by SNN-Block-1 and SNN-Block-2, and Integer LIF neurons provide integer-valued training activations while retaining accumulate-only inference.
  • 3.2.2. Energy Model: Inference energy is estimated with the Horowitz 45 nm CMOS model, assigning 4.6 pJ to each 32-bit MAC.YOLOv8m has 70.02 GMACs, yielding an estimated 322 mJ per inference regardless of dataset.
  • 3.2.2. Energy Model: SpikeYOLO inference energy is modeled as T · ¯s · NOPS · EAC, with T timesteps, measured mean activation ¯s, 60.93 synaptic operations per timestep, and accumulate-only operation cost.Mean activation is measured separately on each test set because spike rates depend on the input distribution.
  • 3.3. CNN Baselines: CNN baselines include YOLOv8m and RT-DETR-L, while published Faster R-CNN, YOLO-SONAR, and Fast R-CNN results provide additional comparisons.Retrained models are initialized from COCO checkpoints and fine-tuned for 50 epochs per dataset; published comparison results were not retrained here.

4. Results

Across UATD and Marine-Debris-FLS, SpikeYOLO remains competitive with conventional detectors while using substantially less theoretical inference energy.

  • UATD Detection Performance: 0.529 mAP@0.5:0.95 trails YOLOv8m’s 0.575 on UATD, while SpikeYOLO uses 3.3× less theoretical compute energy.The corresponding energy values are 97 mJ versus 322 mJ.
  • UATD Detection Performance: Figure 3 reports UATD mAP@0.5 and mAP@0.5:0.95 by network and characterizes SNN accuracy as competitive.
  • Marine Debris Detection: 0.987 mAP@0.5 matches YOLOv8m on Marine-Debris-FLS, with SpikeYOLO using 4.4× less energy.SpikeYOLO also outperforms YOLO-SONAR and Fast R-CNN on both reported metrics.
  • Marine Debris Detection: Figure 4 reports Marine-Debris-FLS mAP@0.5 and mAP@0.5:0.95 by network, with SpikeYOLO matching YOLOv8m on mAP@0.5.

4.3. WHFLS Detection Performance

On WHFLS, SpikeYOLO nearly matches the strongest detectors on mAP@0.5 while substantially reducing theoretical inference energy.

  • WHFLS Detection Performance: 0.9950 mAP@0.5 is achieved by both SpikeYOLO and RT-DETR-L, versus 0.9949 for YOLOv8m.On mAP@0.5:0.95, SpikeYOLO scores 0.861 ± 0.001, below YOLOv8m’s 0.872±0.004 and RT-DETR-L’s 0.893±0.001.
  • WHFLS Detection Performance: 6.2× less energy is used by SpikeYOLO than YOLOv8m on WHFLS, and 9.2× less than RT-DETR-L.The estimated energy values are 52 mJ, 322 mJ, and 476 mJ, respectively.
  • WHFLS Detection Performance: Table 5 reports WHFLS detection results, with bold entries marking the best per column and mAP@0.5:0.95 values averaged over three independent seeds.
  • WHFLS Detection Performance: Figure 5 compares WHFLS mAP@0.5 and mAP@0.5:0.95 across networks, showing SpikeYOLO ahead of YOLOv8m on mAP@0.5 at 0.9950.

4.4. Inference Energy

The paper estimates inference energy from MAC and accumulate operations, with SpikeYOLO’s energy additionally determined by measured dataset-specific spike rates.

  • Inference Energy: 322 mJ per forward pass is estimated for YOLOv8m from 69.9 GMACs and EMAC = 4.6 pJ.SpikeYOLO energy is estimated using NOPS = 60.93 and dataset-varying mean spike activation.
  • Inference Energy: 3.3× lower theoretical energy is attributed to SpikeYOLO at T=2 through accumulate-only operations.
  • Inference Energy: The energy comparison uses theoretical CMOS estimates with EMAC = 4.6 pJ and EAC = 0.9 pJ.
  • Inference Energy: Mean spike activation is measured separately on each FLS test set at T=2 rather than imported from COCO literature values.The reported activation is the mean I-LIF integer output per neuron per timestep, ranging from 0 to 4.

4.5. Timestep Ablation

SpikeYOLO’s timestep ablation finds T=2 to be the practical operating point: accuracy remains comparable to larger timestep counts while energy is lower, and the model is also evaluated under speckle noise.

  • Timestep Ablation: Accuracy drops noticeably at T=1, while T ∈{2, 4, 8} remains flat across all three datasets.Inference energy scales linearly with T.
  • Timestep Ablation: Figure 7 plots mAP@0.5:0.95 against inference timesteps T for SpikeYOLO on all three FLS datasets.The caption describes accuracy as relatively flat across T ∈{2, 4, 8}, with energy scaling linearly and T=2 Pareto-optimal.
  • Timestep Ablation: T=2 achieves the same accuracy as larger T at half the energy of T=4, so it is used for the paper’s comparisons.
  • Robustness to Speckle Noise: 3.0% degradation at σ = 0.4 is observed for SpikeYOLO, compared with 8.9% for YOLOv8m without retraining.At σ = 0.6, SpikeYOLO outperforms YOLOv8m with 0.491 versus 0.484 mAP@0.5:0.95.
  • Robustness to Speckle Noise: Figure 8 evaluates mAP@0.5:0.95 under multiplicative speckle noise on the UATD test set without retraining.At σ=0.6, the fully-spiking model outperforms the ANN baseline outright.

4.7. Low-Data Regime

SpikeYOLO is more robust than YOLOv8m when UATD training data is reduced, leading at both 25% and 50% data fractions.

  • Low-data performance: 5.8% greater mAP@0.5:0.95 at 25% data and 4.4% greater mAP@0.5:0.95 at 50% data were achieved by SpikeYOLO over YOLOv8m.Results are means over two random seeds for reduced-data settings.
  • Evaluation protocol: UATD low-data results used two random seeds, whereas 100% results used three-seed retrains.The table reports mean ± standard deviation for the reduced-data ablation.
  • Low-data performance: SpikeYOLO led YOLOv8m at 25% and 50% of UATD training data, while YOLOv8m recovered at 100%.The comparison is shown as mAP@0.5:0.95 versus training-data fraction.

4.8. Per-Class and Qualitative Analysis

Per-class and qualitative UATD analyses show consistent difficulty patterns across architectures: simple returns are detected more accurately, while complex geometries produce lower-confidence or missed detections.

  • Per-class analysis: Plane achieves the highest per-class AP@0.5:0.95 across models, while square cage and cylinder are consistently challenging.The ranking is broadly consistent regardless of architecture, indicating object-property-driven difficulty in sonar imaging.
  • Qualitative analysis: Both SpikeYOLO T=2 and YOLOv8m reliably localize planes but degrade on ROV, cylinder, and square cage.The difficult classes receive lower-confidence detections and occasional missed objects.

5. Discussion

SNN performance depends on the structure of FLS returns: sparse, high-contrast datasets favor SNNs, while complex UATD signatures narrow but do not eliminate their accuracy gap. Energy estimates also depend on memory assumptions.

  • Dataset-dependent performance: SNNs achieve comparable or greater accuracy than CNN baselines on Marine-Debris-FLS and WHFLS while using less energy.They occasionally match the transformer-based RT-DETR-L on these sparse, high-contrast datasets.
  • Dataset-dependent performance: On UATD, SNNs score close but below YOLOv8m while requiring less energy, consistent with UATD’s more complex acoustic signatures.
  • Training strategy: 0.359 ± 0.037 mAP@0.5:0.95 was achieved by SpikeYOLOv8 after fine-tuning on UATD, below SpikeYOLO’s 0.529 and YOLOv8m’s 0.575.The result suggests ANN-to-SNN conversion is less effective for FLS detection than end-to-end surrogate-gradient training.
  • Energy-estimation scope: Energy comparisons assume on-chip SRAM weight storage and omit memory-movement costs.The discussion identifies these assumptions as part of the energy-estimation scope.

6. Conclusion

SpikeYOLO provides lower-energy FLS object detection with competitive accuracy, especially on sparse, high-contrast datasets, and shows improved robustness to speckle noise and reduced training data.

  • Energy and accuracy: 3.3× lower theoretical inference energy was achieved on UATD by SpikeYOLO than YOLOv8m, at 0.529 versus 0.575 mAP@0.5:0.95.The corresponding energy values are 97 versus 322 mJ.
  • Energy and accuracy: 0.861 versus 0.872 mAP@0.5:0.95 was achieved on WHFLS at the same 3.3× energy advantage.
  • Energy and accuracy: 0.781 versus 0.801 mAP@0.5:0.95 was achieved on Marine-Debris-FLS while SpikeYOLO used less energy.
  • Robustness: 3.0% versus 8.9% degradation at σ=0.4 shows lower speckle-noise sensitivity for SpikeYOLO than YOLOv8m, with SpikeYOLO outperforming at σ=0.6.SpikeYOLO also degrades less when trained on random subsamples of the training data.
Loading 2608.22072v1…