Source-linked AI summary
Towards Deployable Underwater Vessel Classification
Abishek Soti, Thura Pyae Sone, Naqib Ibnul, Htoo Htet Aung, Henry Zhong, Gregory Cohen, Ying Xu
TL;DR
Autonomous passive-acoustic platforms face tight power, computation, and communication constraints, motivating compact underwater vessel classifiers designed around acoustic representations. The study evaluates multiple representations and compact models, finding strong scores on the available ShipsEar split but more moderate test performance with recording-level DeepShip partitioning.
Problem
Whether strong compact-model performance on the available ShipsEar split persists for unseen recordings is uncertain because source-recording provenance cannot be reconstructed.
Source: §1 · p. 1Method
The study evaluates conventional and auditory-inspired acoustic representations with lightweight classifiers and compact CNNs, using recording-level DeepShip partitioning and statistical pooling in its compact convolutional model.
Source: §1 · p. 1, §3.2 · p. 2Results
On the available ShipsEar split, the two-layer STFT CNN reaches 0.9918 macro F1; on recording-level DeepShip, the compact four-layer CNN reaches test macro F1 up to 0.7226, while ResNet18 has lower validation macro F1 than the Mel CNN.
Source: §1 · p. 1, §4 · p. 4, §3.7 · p. 4Takeaways & Limitations
The results support emphasizing acoustic representation and evaluation design before increasing classifier capacity, and identify compact, representation-driven approaches as a practical direction for deployable underwater acoustic classification.
Source: §4 · p. 4ShipsEar source-recording identities cannot be reconstructed sufficiently to verify recording-level independence, and exploratory sweeps used the provided split during configuration and epoch selection.
Source: §3.4 · p. 3
Abstract
from arXiv · showhide
We propose a compact underwater acoustic classification framework combining multi-representation feature engineering, temporal statistical pooling, and compact convolutional architectures designed for acoustic time-frequency and cochlear representations. We investigate multiple conventional and auditory-inspired representations and first evaluate lightweight classifiers and Conventional Neural Networks (CNNs) on ShipsEar dataset. On the provided split, a two-layer CNN achieves a macro F1 of 0.9918, while a Radial Basis Function Support Vector Machine (RBF-SVM) reaches 0.9883. However, source-recording provenance cannot be reconstructed, preventing verification of recording-independent generalisation. We therefore evaluate on DeepShip dataset using recording-level partitioning before segmentation. Under this protocol, a 157K-parameter compact CNN achieves a test macro F1 of 0.7226, while an 11.17M-parameter ResNet18 provides no improvement in validation performance under the matched setting. These results demonstrate the importance of representation-aware feature and model design, together with rigorous recording-level evaluation, for classification performance and deployability in compact underwater acoustic systems.
1. INTRODUCTION
The study tests whether acoustic representations and compact models can support underwater vessel classification, while examining whether near-ceiling ShipsEar scores persist when recordings are separated before segmentation. It reports strong ShipsEar performance but identifies recording provenance as a limit on interpreting those scores as generalisation to unseen recordings.
1. INTRODUCTION
- The study evaluates conventional STFT, Mel, MFCC, and CQT features alongside gammatone filterbanks and CARFAC, using classical classifiers, CNNs, and shallow Transformers.The representations are compared to assess whether their acoustic characteristics affect classification without requiring greater model complexity.Source: §1 · p. 1
- 99.10% accuracy with an approximately 183K-parameter STFT two-layer CNN on the available ShipsEar split, versus 98.50% with a 21.61M-parameter Mobile ViT using 0–3 kHz LOFAR.The comparison is reported across the available standard ShipsEar split and the earlier LOFAR evaluation.Source: §1 · p. 1
- ShipsEar source-recording provenance is insufficient to verify recording-independent generalisation, so DeepShip recordings are partitioned before segmentation for an unseen-recording evaluation.The ShipsEar results may overestimate generalisation if acoustically similar segments from one recording appear across partitions.Source: §1 · p. 1
- On recording-level DeepShip evaluation, the compact CNN achieves higher validation macro F1 than ResNet18 despite using approximately 71× fewer parameters, while performance varies across representations.Grad-CAM analysis also reveals class-dependent frequency relevance in the compact CNN’s predictions.Source: §1 · p. 1
2. DATASETS AND EVALUATION PROTOCOL
The evaluation uses two underwater-vessel datasets with different partitioning constraints: ShipsEar has a fixed split of pre-segmented clips, while DeepShip recordings can be separated before segmentation. The DeepShip protocol reserves test recordings for final evaluation and uses validation macro F1 for model selection.
2.1. ShipsEar and DeepShip
- ShipsEar contains about 3.1 hours from 90 recording sessions and 11 vessel types grouped into five classes; DeepShip has 609 readable recordings, about 47.2 hours, and four vessel classes.ShipsEar audio was sampled at 64 kHz and resampled to 16 kHz; DeepShip recordings cover Cargo, Passenger, Tanker, and Tug.Source: §2.1 · p. 1
2.2. Partitioning and Evaluation
- The processed ShipsEar dataset provides a fixed train–test split of 1,778 and 445 pre-segmented five-second clips.Source: §2.2 · p. 1
- DeepShip recordings are assigned to training, validation, and test partitions before segmentation, yielding 26,933, 3,266, and 3,457 five-second segments, respectively.Validation macro F1 guides model selection and early stopping; the test partition is reserved for final evaluation.Source: §2.2 · p. 2
3. REPRESENTATIONS AND MODELS
The section evaluates acoustic representations, temporal pooling, and compact models, then tests performance on ShipsEar and recording-separated DeepShip. It also examines class-specific spectral relevance and whether a larger model improves DeepShip generalisation.
3. REPRESENTATIONS AND MODELS
- The study compares STFT, Mel, MFCC, CQT, Gammatone, and CARFAC representations, including CARFAC's BM, IHC, and LI outputs.The representations capture different spectral and temporal characteristics of underwater vessel acoustics.Source: §3.1 · p. 2, Figure 1 · p. 2
3.1. Acoustic Representations and Statistical Pooling
- Mean–standard-deviation pooling reduced ShipsEar STFT input dimensionality from 60,800 to 256 while improving test macro F1 from 0.9641 to 0.9883.The pooled vector concatenates each channel's temporal mean and standard deviation.Source: §3.1 · p. 2, Table 1 · p. 2
3.2. Neural Models and Model Capacity
- The compact four-layer CNN applies statistical pooling to learned feature maps before classification; the ShipsEar Patch Transformer instead processes local patches as token embeddings.The higher-capacity ResNet18 provides a convolutional comparison under the DeepShip protocol.Source: §3.2 · p. 2
3.4. ShipsEar as a Diagnostic Benchmark
- 0.9918 macro F1 for the two-layer STFT CNN and 0.9883 for the pooled STFT RBF-SVM were near-ceiling ShipsEar results, but source-recording independence could not be verified.The paper treats these results as diagnostic benchmarks, not unbiased estimates for unseen recordings.Source: Table 2 · p. 3, §3.4 · p. 3
3.5. DeepShip Performance with Recording-Level Partitioning
- 0.7226 was the highest observed test macro F1 on DeepShip, for Mel; BM had the highest validation macro F1, 0.6766.DeepShip partitions kept source recordings separate before segmentation, and representation scores varied across the evaluated configurations.Source: Table 3 · p. 3, §3.5 · pp. 3–4
- Fivefold grouped cross-validation for Mel RBF-SVM yielded mean validation macro F1 of 0.6200 ± 0.0217 at segment level and 0.6974 ± 0.0309 after recording-level aggregation.The train–validation gap averaged 0.3196, and the gap persisted across folds.Source: §3.5 · p. 4
3.6. Grad-CAM Analysis
- The compact four-layer CNN's class-wise Grad-CAM profiles differed: Passenger emphasized very low frequencies, Tug lower-to-mid frequencies, Tanker showed broader relevance, and Cargo was comparatively sparse.Profiles averaged relevance over time and correct test predictions within each class.Source: Figure 2 · p. 4, §3.6 · p. 4
3.7. Effect of Model Capacity on DeepShip
- 11.17M-parameter ResNet18 reached validation macro F1 of 0.6110, below 0.6519 for the approximately 157K-parameter Mel CNN under the matched DeepShip protocol.The best-effort reproduction of a published ResNet18 baseline reached 62.13% held-out accuracy, treated as diagnostic because held-out performance was monitored during stopping.Source: §3.7 · p. 4
4. CONCLUSION
Recording-level evaluation makes compact-model performance more moderate than the available ShipsEar split suggests, while a much smaller CNN outperforms ResNet18 on DeepShip validation. The results support prioritising acoustic representations and evaluation design over simply increasing classifier capacity.
4. CONCLUSION
- 0.7226 test macro F1 is the compact four-layer CNN's observed DeepShip score; the 11.17M-parameter ResNet18 has lower validation macro F1 than the approximately 157K CNN.The comparison uses the matched 5-second DeepShip protocol.Source: §4 · p. 4
- On ShipsEar, the two-layer STFT CNN reached 0.9918 macro F1 and the pooled STFT RBF-SVM 0.9883, but unavailable provenance prevents recording-independent interpretation.Source: §4 · p. 4
- Conventional and auditory-inspired representations both provide useful vessel-discriminative information, while statistical pooling lets comparatively small classifiers use it.Source: §4 · p. 4
- Grad-CAM shows class-dependent spectral relevance within the STFT CNN, and the findings support compact, representation-driven approaches as a practical direction for deployable classification.Source: §4 · p. 4