PaperlayerView exact revision on arXiv

Source-linked AI summary

A Perch out of Water: Few-Shot Detection of Bryde's Whale Vocalisations in False Bay

G. Tonitz, C. M. Geldenhuys, T. R. Niesler

arXiv:2610.29572v1eess.AS

TL;DR

Automated baleen-whale call detection often lacks the large annotated datasets conventional systems require. This paper tests fixed pretrained bioacoustic embeddings with a prototypical few-shot classifier, adapting Perch’s input and allowing multilabel detection. The adapted approach improves Antarctic results, while Perch outperforms WhaleVAD on unseen False Bay Bryde’s whale calls.

  • Problem

    Automated baleen-whale call detectors generally require large annotated datasets that are unavailable for many species and populations.

    Source: §I · p. 1

  • Method

    The study compares fixed pretrained Perch and WhaleVAD embeddings with a prototypical few-shot classifier, adapting Perch’s input representation and testing a multilabel formulation.

    Source: §I · p. 1

  • Results

    On False Bay calls, Perch achieves macro AUC-ROC 0.987 against WhaleVAD’s 0.859; on ATBFL, the two modifications yield a 9.1% absolute improvement and macro AUC-ROC of 0.739 at k = 10.

    Source: §VII · p. 9

  • Takeaways & Limitations

    General-purpose bioacoustic embeddings paired with a lightweight few-shot classifier offer a practical starting point for discovering and monitoring call types with scarce or absent annotations.

    Source: §VII · p. 9

Abstract

from arXiv · show

The automated detection of baleen whale calls generally requires large annotated datasets, which are not available for many species and populations. We investigate whether pretrained bioacoustic embeddings, combined with a prototypical few-shot classifier, allow such calls to be detected and classified from only a handful of labelled examples. Two embedding networks are compared: Perch, a general-purpose bioacoustic model trained predominantly on birdsong, and WhaleVAD, a task-specific blue- and fin-whale call detector. Neither model is fine-tuned, and at most ten labelled examples per call type are assumed. Evaluating on Antarctic blue- and fin-whale recordings, we find that the default Perch input representation is poorly matched to low-frequency whale calls, and that increasing the frame length while restricting the mel filterbank to the frequency range occupied by the calls improves the AUC-ROC by 5.8% absolute when ten examples per call type are available. Allowing the classifier to detect each call type independently, so that segments may contain several calls, affords a further 3.3% improvement and a macro AUC-ROC of 0.739, representing a combined gain of 9.1% absolute. Both embeddings are applied to Bryde's whale vocalisations recorded in False Bay, South Africa, which neither model has encountered during pre-training. Here Perch outperforms WhaleVAD for every call type, achieving a macro AUC-ROC of 0.987 against 0.859, despite having a predominantly terrestrial training set. These results indicate that general-purpose bioacoustic embeddings, paired with an input representation matched to the calls of interest, can support whale call detection for species and populations for which large annotated datasets do not exist.

I. INTRODUCTION

Automated whale-call detection is valuable for monitoring populations with scarce annotations, but deep-learning detectors generally require large labelled datasets. This paper tests fixed pretrained embeddings with few-shot classification and adapts Perch’s input representation for low-frequency calls.

I. INTRODUCTION

  • Few labelled datasets constrain automated baleen-whale call detection, including for vulnerable South African inshore Bryde’s whales with fewer than 1,000 individuals.The paper reports no deep-learning detector for Bryde’s whale calls in False Bay.Source: §I · p. 1
  • The study compares fixed Perch and WhaleVAD embeddings on Antarctic blue- and fin-whale recordings and unseen False Bay Bryde’s whale vocalisations.Neither model is trained or fine-tuned for these experiments.Source: §I · p. 1
  • Perch’s default input representation poorly captures low-frequency calls, so the proposed alternative lengthens frames and concentrates mel filters in the call band.This trades time resolution for frequency resolution.Source: §I · p. 1
  • The proposed multilabel one-vs-background prototypical classifier allows a segment to contain more than one call type.Source: §I · p. 1
  • On unseen False Bay Bryde’s whale calls, Perch transfers better than WhaleVAD despite its predominantly birdsong-based training.Source: §I · pp. 1–2

II. BACKGROUND

Few-shot metric-based classification uses a small labelled support set to classify novel sounds from embeddings. The paper describes fixed pretrained embeddings and the WhaleVAD detector used in its comparisons.

A. Prototypical networks

  • Few-shot learning targets recognition of novel classes from very few labelled examples, making it relevant when data collection or annotation is difficult.Source: A. Prototypical networks · p. 2
  • A prototypical classifier represents each class by the mean embedding of its support examples and assigns queries to the nearest prototype without trainable classifier parameters.Here, the pretrained embedding model remains fixed.Source: A. Prototypical networks · p. 2

B. WhaleVAD

  • WhaleVAD is a lightweight per-frame blue- and fin-whale call detector with a convolutional and recurrent architecture.It was trained on ATBFL using focal loss and stochastic negative mini-batch undersampling to address class imbalance.Source: B. WhaleVAD · p. 2

C. Perch

The study uses Perch’s general-purpose bioacoustic embeddings alongside a review of automated baleen-whale detection, including the limited work on False Bay Bryde’s whales.

C. Perch

  • Perch converts five-second audio segments into log-mel spectrograms and uses an EfficientNet-B3 backbone to produce 1536-dimensional embeddings.The experiments use the embedding network while leaving its pretrained weights fixed.Source: C. Perch · p. 2

A. Detection and classification of baleen whale calls

  • Prior baleen-whale detection has progressed from signal-processing methods to deep neural networks, but these systems typically require large annotated datasets.Source: A. Detection and classification of… · p. 2
  • False Bay Bryde’s whale studies used hidden Markov models or energy detection followed by human classification; no deep-learning detector had been developed for this population.The cited work also includes CNN detection of a North Pacific Biotwang call, not reported in False Bay.Source: A. Detection and classification of… · pp. 2–3

B. Few-shot sound event detection

Few-shot sound event detection has been studied in challenge settings that remain predominantly terrestrial. This paper evaluates systems on Antarctic whale recordings and False Bay Bryde’s whale data.

B. Few-shot sound event detection

  • Few-shot sound event detection frames the task as finding target sounds in continuous recordings from a handful of annotated examples, with prototypical methods prominent in challenge results.The DCASE challenge added dedicated few-shot bioacoustic tasks from 2021 to 2024.Source: B. Few-shot sound event detection · p. 3
  • Prior challenge data remained predominantly terrestrial, and marine vocalisations were among the most difficult for submitted systems.Earlier pretrained-embedding evaluations used curated extracts rather than detection in continuous passive-acoustic recordings.Source: B. Few-shot sound event detection · p. 3

IV. DATA

  • The ATBFL evaluation uses disjoint validation recordings from Casey 2017 and Kerguelen 2015, covering three collapsed blue- and fin-whale call classes.Annotated calls account for 3.49% of recording duration, and bmabz calls are the most abundant and longest.Source: §IV · p. 3
  • The False Bay dataset contains Bryde’s whale single-tone, multi-tone, burst-tonal, and tonal-downsweep calls in recordings from April 2023.The first recording was strongly annotated in full, adding 1,180 calls to the 1,032 originally annotated; calls occupy 0.38% of its duration, and two recordings contain no TD calls.Source: §IV · pp. 3–4

V. EXPERIMENTAL STRUCTURE

The experiments use fixed-length segments with multilabel annotations when calls overlap, and proceed from segmentation and labelling to model, classifier, input, and evaluation descriptions. The supplied figures show four False Bay call types and summarize call-duration and annotation statistics.

V. EXPERIMENTAL STRUCTURE

  • The experimental workflow proceeds through segmentation and labelling, pretrained embeddings, few-shot classification, Perch input adjustments, and evaluation.The results then compare systems on ATBFL and evaluate both embeddings on False Bay.Source: §V · p. 4

A. Segmentation and labelling

  • Segments are 5 s long with 4 s hops, and receive a call label above 0.02 purity; segments may carry multiple labels.Positive segments are multilabel in 4.15% of ATBFL and 14.66% of False Bay; TD is excluded from False Bay experiments because independent supports cannot be drawn from other recordings.Source: A. Segmentation and labelling · pp. 4–5
  • Table II summarizes minimum, maximum, and average call duration, plus annotation counts for call types in ATBFL and False Bay.Source: Table II · p. 4
  • The False Bay figure presents spectrogram representations of its four Bryde’s whale call types.Source: Figure 1 · p. 4

B. Models

The study compares two pretrained embedding models without training or fine-tuning their weights, then classifies query segments using class prototypes. A one-vs-background variant supports assigning multiple call labels to a segment.

B. Models

  • Neither Perch nor WhaleVAD is trained or fine-tuned for these experiments.Source: B. Models · p. 5
  • Perch produces a 1536-dimensional embedding from 5 s audio sampled at 32 kHz, while WhaleVAD embeddings are mean-pooled from its 256-dimensional recurrent-layer outputs.WhaleVAD is trained on blue- and fin-whale calls; on False Bay it is used as an embedding network, not a detector.Source: B. Models · p. 5

C. Few-shot classifier

  • For each call type and the negative class, the classifier averages k support embeddings into prototypes and compares query embeddings with them using Euclidean distance.The multiclass configuration applies a softmax to the negated distances and selects the highest-probability class.Source: C. Few-shot classifier · p. 5
  • The one-vs-background configuration converts distances into per-class scores, allowing multiple call labels for one segment.A score’s sign indicates whether the query is closer to that call-type prototype or the negative prototype.Source: C. Few-shot classifier · p. 5

D. Perch input representation

The Perch front end is adjusted to better represent low-frequency whale calls, and evaluation uses recording-separated support and query segments. The results section compares these settings and few-shot systems across ATBFL and False Bay.

D. Perch input representation

  • Perch’s default front end discards much of the calls’ fundamental-frequency energy and has poor frequency resolution in the low bands.It uses 20 ms windows and 128 mel bands spanning 60 Hz to 16 kHz, with 50 Hz bin spacing.Source: D. Perch input representation · p. 5
  • The adjusted Perch representation retains 128 mel bands while varying frame length and restricting the mel filterbank’s upper limit to concentrate bands on whale-call frequencies.The frame length is varied from 1024 to 32 768 samples, and the filterbank’s upper limit is lowered from 16 kHz to 1 kHz.Source: D. Perch input representation · p. 5

E. Performance evaluation

  • Evaluation samples supports from recordings other than each query’s recording, with 100 support draws per query and k ranging from 1 to 10.False Bay queries use only the strongly annotated 2023-04-22 recording; scores are macro-averaged across classes and reported as means and standard deviations over draws.Source: E. Performance evaluation · p. 5

VI. RESULTS

  • The results compare Perch input adjustments on ATBFL, few-shot systems against the supervised WhaleVAD topline, and both embeddings on False Bay.Table III reports frame length, mel-filterbank frequency range, macro AUC-ROC, and AP for ATBFL with multiclass classification and k = 5.Source: §VI · p. 5, Table III · p. 6

A. Perch input representation

Restricting the mel filterbank to 0–1000 Hz and lengthening the frame improves Perch’s ATBFL performance, with the best reported setting selected for subsequent experiments.

A. Perch input representation

  • Lowering the filterbank’s lower edge from 60 Hz to 0 Hz while retaining the default frame length improves AUC-ROC by 1.1% absolute, with AP unchanged.The recovered fundamental-frequency energy remains spread over too few mel bands to substantially benefit the model.Source: A. Perch input representation · p. 6
  • 5.4% higher AUC-ROC and 1.6% higher AP than the default configuration are achieved at 8192-sample frames with a 0–1000 Hz filterbank; the scores are 0.690 and 0.162.No further gains are observed beyond 8192 samples; this setting is used for subsequent experiments and applied to False Bay without modification.Source: A. Perch input representation · p. 6
  • Within the 0–1000 Hz filterbank setting, performance improves consistently as frame length increases up to 8192 samples.Source: A. Perch input representation · p. 6

B. ATBFL dataset

On ATBFL, the adjusted input representation improves macro AUC-ROC, while the multilabel configuration achieves the strongest macro AUC-ROC. The combined changes yield a 9.1% absolute gain over unmodified Perch at k = 10, though the fully supervised WhaleVAD topline remains ahead.

B. ATBFL dataset

  • 5.8% absolute macro AUC-ROC gain at k = 10 comes with the adjusted input representation, although bmabz AUC-ROC falls by 3.4% absolute.The bp call accounts for nearly all the gain, rising from 0.783 to 0.950; d improves by 3.9% absolute.Source: B. ATBFL dataset · p. 6
  • 9.1% absolute improvement in macro AUC-ROC over unmodified Perch at k = 10 comes from combining the adjusted input representation and multilabel formulation.The multilabel system reaches macro AUC-ROC 0.739 at k = 10.Source: B. ATBFL dataset · p. 6
  • The multilabel configuration reaches the highest macro AUC-ROC at every tested k, while multiclass retains a small AP edge at k = 10, 0.176 versus 0.174.Multilabel improves AUC-ROC for bmabz and d by 6.2% and 4.4% absolute, respectively, but lowers bp AP from 0.350 to 0.295.Source: B. ATBFL dataset · p. 6
  • All three configurations improve in AUC-ROC and AP as k increases; the multilabel system gains 7.9% of its 9.8% total absolute AUC-ROC improvement between k = 1 and k = 5.The adjusted multiclass configuration has the highest AP, while multilabel has the highest macro AUC-ROC.Source: B. ATBFL dataset · p. 6
  • At k = 10, the fully supervised WhaleVAD topline remains ahead by 7% in AUC-ROC and 19% in AP, despite the few-shot system using at most ten examples per call type.The few-shot system's AP of 0.174 is above chance given that annotated calls occupy 3.49% of ATBFL recordings; AP depends on class prevalence.Source: B. ATBFL dataset · p. 6
  • WhaleVAD retains a clear advantage for bmabz and d on both ATBFL subsets, while the few-shot classifier is superior for bp.The comparison is between the best few-shot system at k = 10 and WhaleVAD trained on the full ATBFL training set.Source: B. ATBFL dataset · p. 6

C. FB dataset

On the False Bay recordings, Perch outperforms WhaleVAD across call types and support-set sizes. Perch’s macro AUC-ROC reaches 0.987 at k = 10, while the authors attribute WhaleVAD’s weaker transfer to its specialization for blue- and fin-whale calls.

C. FB dataset

  • For binary detection, Perch scores AUC-ROC 0.980 and AP 0.844 at k = 10, compared with WhaleVAD’s 0.779 and 0.298.At k = 1, Perch scores 0.958 AUC-ROC and 0.372 AP, versus WhaleVAD’s 0.690 and 0.076.Source: C. FB dataset · pp. 6–7
  • At k = 10, Perch achieves macro AUC-ROC 0.987 and macro AP 0.580, versus WhaleVAD’s 0.859 and 0.246, respectively.Perch outperforms WhaleVAD for every call type and support-set size.Source: C. FB dataset · p. 7, Table V · p. 8
  • Perch’s AUC-ROC exceeds 0.969 for both narrowband call types MT and ST, whereas WhaleVAD’s scores fall to 0.894 and 0.718.The wideband, impulsive BT call is most readily identified by both embeddings, with AUC-ROC scores of 0.998 for Perch and 0.965 for WhaleVAD.Source: C. FB dataset · p. 7
  • The authors attribute WhaleVAD’s weaker transfer to its optimization for blue- and fin-whale calls, unlike Perch’s broader training; Perch’s precision-recall curves are higher over most recall values for all three call types.Neither model had encountered the False Bay data during training.Source: C. FB dataset · pp. 7–8

VII. CONCLUSION

The study tests pretrained bioacoustic embeddings with a prototypical few-shot classifier, using at most ten labelled examples per call type and no fine-tuning. Adjusting Perch’s input representation and allowing multilabel classification improves ATBFL results, while Perch outperforms WhaleVAD on unseen False Bay Bryde’s whale vocalisations.

VII. CONCLUSION

  • The experiments compare Perch, trained predominantly on birdsong, with WhaleVAD, a blue- and fin-whale detector, without fine-tuning either model and using at most ten labelled examples per call type.Classification uses a prototypical few-shot classifier.Source: §VII · p. 8
  • A matched Perch input representation and multilabel classification together improve macro AUC-ROC by 9.1% absolute over unmodified Perch at k = 10.The adjusted front end improves ATBFL macro AUC-ROC and AP by 5.4% and 1.6% absolute at k = 5; the multilabel configuration reaches macro AUC-ROC 0.739 at k = 10.Source: §VII · p. 9
  • On ATBFL, few-shot systems remain below the fully supervised WhaleVAD topline, with 7% and 19% gaps in AUC-ROC and AP at k = 10.WhaleVAD was trained on more than 58,000 annotated blue- and fin-whale calls, whereas Perch had not been exposed to marine audio.Source: §VII · p. 9
  • On False Bay, Perch reaches macro AUC-ROC 0.987 and macro AP 0.580 at k = 10, exceeding WhaleVAD’s 0.859 and 0.246.Both models had not seen this dataset during training, and Perch outperforms WhaleVAD for every call type and support-set size.Source: §VII · p. 9
  • The authors conclude that general-purpose embeddings with a lightweight few-shot classifier are a practical starting point for call types with scarce or absent annotations, provided the input representation suits the target frequencies.Future work includes adapting the embedding network to marine audio, frame-level event detection, and evaluating tonal downsweep calls, which appeared in only one recording.Source: §VII · p. 9
Loading 2610.29572v1…