Source-linked AI summary
Efficient Passive Acoustic Monitoring of Killer Whales Using a Two-Stage Detection and Ecotype Classification Cascade
Daniela Ruiz, Manuel Castellote, Zhongqi Miao, Carl Chalmers, Bruno Demuro, Rahul Dodhia, Pablo Arbelaez, Juan M. Lavista
TL;DR
Killer whale PAM needs accurate, real-time monitoring despite severe class imbalance and deployment shift. The paper proposes a lightweight two-stage ResNet cascade for detection and ecotype classification, with abstention and active-learning adaptation. It improves benchmark cascade performance, adapts Stage 1 to Puget Sound, and runs faster than real time, while continuous-stream recall and full target-domain cascade evaluation remain open.
Problem
Killer whale PAM must scale beyond manual annotation while maintaining accurate, real-time detection under background-noise dominance, class imbalance, and deployment shift.
Method
A lightweight ResNet-18 cascade detects killer whale vocalizations, classifies confident detections into five ecotypes with abstention, and adapts Stage 1 using active learning on Puget Sound data.
Results
The cascade reaches seven-class macro-F1 0.933 versus 0.919 for a single-stage model, while active learning increases Puget Sound detection F1 from 0.405 to 0.755 across manually verified detection windows.
Takeaways & Limitations
The pipeline supports task-specific ecotype recognition, adaptation to a new acoustic domain, and faster-than-real-time killer whale monitoring.
Takeaways & Limitations
Continuous-stream detection performance remains unevaluated, and adaptation and evaluation of Stage 2 and the complete cascade await target-domain ecotype-labeled detections.
Abstract
from arXiv · showhide
Passive acoustic monitoring of killer whales is particularly important for conservation of the endangered Southern Resident killer whale population, but requires accurate models that can operate in real time under severe class imbalance and deployment shift. We propose a lightweight ResNet-based two-stage cascade that first detects killer whale vocalizations and then classifies confident detections into five eastern North Pacific ecotypes, abstaining on ambiguous calls. We train and evaluate the pipeline on the DCLDE 2027 dataset, where the detector achieves 0.960 macro-F1 and the classifier 0.958, outperforming frozen Perch 2.0 embeddings on the five-ecotype benchmark. By separating detection from ecotype recognition, the end-to-end cascade improves seven-class macro-F1 from 0.919 for a single-stage model to 0.933, with the largest gain on the rare OKW ecotype. To assess transfer beyond the benchmark, we use active learning to adapt the Stage 1 to the acoustic environment of Puget Sound, WA, increasing killer whale detection F1 from 0.405 to 0.755 on manually verified detection windows. Finally, each stage processes a 3 s window in approximately 1.4 ms on an NVIDIA H100, enabling faster than real time inference. These results demonstrate that the proposed two-stage cascade pipeline enables reliable killer whale detection and classification, adaptation to new acoustic domains, and real-time monitoring for conservation applications.
1 Introduction
Killer whale PAM is essential for scalable, timely conservation monitoring, especially for endangered Southern Residents, but deployment requires accurate, efficient models robust to imbalance and acoustic shift.
- Reliable monitoring is a conservation priority because the Southern Resident population is listed as Endangered under the U.S. ESA.
- PAM provides continuous, weather-independent, non-invasive observations across the large spatial and temporal scales occupied by killer whale populations.
- Automatic detection and ecotype classification are essential because hydrophone networks generate thousands of recording hours annually, making manual annotation infeasible.
- Deployment streams are dominated by background noise, requiring high recall, sufficient precision, and robustness to acoustic conditions underrepresented in annotated datasets.
- The proposed lightweight two-stage pipeline detects killer whale vocalizations, classifies confident detections into five ecotypes, abstains on ambiguous calls, and adapts Stage 1 using Puget Sound data.
2 Related Work
Related work spans supervised spectrogram models, large-scale pretrained audio encoders, and early killer whale detection and ecotype-classification systems.
- Spectrogram-based image-classification architectures, including BirdNET and lightweight CNN backbones, support large-scale passive acoustic monitoring.
- Self-supervised audio encoders such as BEATs, AudioMAE, EAT, AVES, Bird-MAE, animal2vec, and AVEX learn transferable representations from unlabeled audio.
- Perch 2.0 shows that large-scale supervised learning remains competitive on standardized bioacoustic benchmarks.
- Early killer whale recognition emphasized binary detection, while later work addressed Pacific Northwest ecotype classification using DCLDE data and transfer learning.
3 Dataset
The dataset combines heterogeneous eastern North Pacific recordings with filtered, annotation-centered three-second windows, standardized spectrogram inputs, and leakage-controlled splits for separate detection and ecotype tasks.
- Source Corpus: The DCLDE 2027 corpus is the largest publicly available dataset for killer whale PAM and aggregates recordings from diverse providers and systems.
- Source Corpus: 1,065.8 hours of audio contain only 42.5 annotated hours, underscoring the dataset’s sparse labeling and heterogeneous annotation coverage.
- Source Corpus: Filtering uncertain, nonlocalized, and zero-duration annotations removes 1,532 events, yielding 206,042 annotated events.
- Windows and Spectrogram Representation: Recordings are resampled to 24 kHz and converted into cached log-mel spectrograms spanning 200 Hz–12 kHz.
- Windows and Spectrogram Representation: Training uses fixed 3 s windows centered on annotated events because over 99% of labeled events are shorter than 3 s.
- Dataset Splits: Stage 1 uses 612,761 KW, Bio, and NonBio windows, whereas Stage 2 uses 216,940 ecotype-labeled KW windows.
- Dataset Splits: Both stages share held-out recordings, with source-file grouping preventing leakage and enabling cascade results to be interpreted alongside individual-stage performance.
4 Method
The method uses a sequential detector–classifier cascade with calibrated abstention, compact ResNet-18 models, benchmark comparisons, and active-learning adaptation for deployment-domain shift.
- Cascade: Stage 1 classifies each log-mel spectrogram as NonBio, Bio, or KW, forwarding only predicted KW windows to Stage 2.
- Cascade: Stage 2 assigns one of five ecotypes or outputs Unassigned when calibrated confidence falls below its abstention threshold.
- Architecture: Both stages use lightweight ResNet-18 backbones adapted for single-channel mel spectrograms.
- Optimization: Stage 2 uses inverse-frequency weighted cross-entropy because SRKW outnumbers OKW by approximately 7.5:1.
- Efficiency: The models require approximately 11.2M parameters and achieve about 1.4 ms batch-1 latency per 3 s window on an NVIDIA H100.
- Baselines: Perch 1.0 and Perch 2.0 are evaluated through frozen embeddings and multinomial logistic regression at window and recording levels.
- Evaluation: The end-to-end benchmark cascade forwards predicted KW windows through both stages and compares against a single-stage seven-class ResNet-18.
- Domain Adaptation: Active learning adapts Stage 1 using expert-validated detections, original training samples, and simulated vessel-noise masking while fine-tuning only the final residual block and classification head.
5 Results
On the DCLDE test set, the two-stage system performs strongly for detection and ecotype classification, improves seven-class performance over a single-stage model, and adapts substantially to Puget Sound through active learning. Calibration and deployment-domain results also expose limitations in confidence transfer and continuous-stream evaluation.
- Stage 1 Detection: 0.960 macro-F1 and 0.992 macro average precision were achieved by the Stage 1 detector on the held-out DCLDE test set.Per-class F1 ranged from 0.956 for KW to 0.962 for NonBio.
- Stage 2 Ecotype Classification: 0.958 macro-F1 was achieved by the Stage 2 ecotype classifier across five ecotypes, with every ecotype exceeding 0.93 F1.The rare OKW class remained most challenging but reached 0.939 F1.
- Comparison with Perch: 0.958 versus 0.933 and 0.913 was the window-level macro-F1 for ResNet-18 compared with Perch 2.0 and Perch 1.0, respectively.At recording level, ResNet-18 achieved 0.925 versus 0.873 for Perch 2.0.
- End-to-End Cascade: 0.921 macro-F1 resulted when Stage 2 used Stage 1 predictions instead of ground-truth detections, compared with 0.958 under ground-truth detections.Missed KW windows and forwarded non-KW windows reduced cascade performance by 0.020 and 0.017, respectively.
- End-to-End Cascade: 58% of incorrectly forwarded non-KW windows were rejected by the calibrated confidence threshold.The threshold illustrates how abstention can limit propagation of uncertain predictions during deployment, although abstention was not evaluated in the benchmark cascade.
- End-to-End Cascade: 0.933 versus 0.919 was the seven-class macro-F1 for the cascade compared with a single-stage ResNet-18.The rare OKW class increased from 0.842 to 0.930, while the remaining classes differed by less than 0.013.
- Deployment-Domain Adaptation: 0.405 to 0.556 was the Stage 1 detection F1 increase after the first active-learning iteration on Puget Sound evaluation data.Precision reached 100% while recall fell to 38.5%, producing a more conservative detector.
- Deployment-Domain Adaptation: 0.755 was the KW F1 achieved by the final Puget Sound detector after additional verified data, shipnoise augmentation, and reduced fine-tuning learning rate.Recall increased from 38.5% to 61.5% relative to v2 while precision remained 97.6%.
6 Conclusion
The lightweight two-stage cascade supports real-time killer whale monitoring, performs strongly on the DCLDE benchmark, and improves detection through active learning under Puget Sound domain shift.
- 0.933 vs. 0.919 seven-class macro-F1 shows the cascade outperforming a single-stage model, with the largest gain for rare OKW, 0.930 vs. 0.842.Stage 1 reaches 0.960 macro-F1 and Stage 2 reaches 0.958 across five ecotypes.
- 2.5 percentage points at the window level and 5.2 points at the recording level separate the ecotype classifier from Perch 2.0.The comparison demonstrates the value of task-specific supervision for ecotype recognition.
- Stage 1 detection errors primarily limit cascade performance, while abstention suppresses a substantial fraction of false detections forwarded to ecotype classification.Future work includes continuous-stream validation, Stage 2 adaptation, complete-cascade evaluation, and site-specific abstention-threshold calibration.
- Active learning increased Puget Sound killer whale precision from 0.28 to 0.98 while improving F1 across successive deployment iterations.The adaptation used expert-validated detections and site-specific background noise under severe domain shift.
- Each stage uses 11.2M parameters and 5.009 GMACs per 3-second window, with approximately 1.4 ms batch-1 inference on an NVIDIA H100 NVL.Edge-device performance remains unevaluated, but model execution is faster than real time on the reported hardware.