Source-linked AI summary
Perch 2.0 transfers 'whale' to underwater tasks
Andrea Burns, Lauren Harrell, Bart van Merriënboer, Vincent Dumoulin, Jenny Hamer, Tom Denton
TL;DR
Marine bioacoustic datasets are difficult to collect and label, motivating transfer methods that can work with few examples. This paper evaluates linear probes on Perch 2.0 embeddings across underwater tasks and compares them with other pretrained models. Perch 2.0 generally performs best on uncontaminated marine transfer tasks, although another model leads on one DCLDE task and some comparisons involve training-data overlap.
Problem
Marine bioacoustic data are difficult to collect and verify, while many tasks lack large amounts of human-labeled data.
Method
The study uses few-shot linear probing of Perch 2.0 embeddings and compares them with embeddings from other pretrained bioacoustic models across marine datasets.
Results
Perch 2.0 generally outperforms published comparison models on ReefSet and cetacean species tasks, except DCLDE 2026 Known Bio Species, where BirdNet V2.3 ranks higher.
Takeaways & Limitations
Perch 2.0 embeddings are recommended for developing marine mammal classifiers with few labeled examples.
Takeaways & Limitations
Comparisons are partly confounded by training-data overlap for SurfPerch on ReefSet and GMWM on NOAA PIPAN, while AVES performance may depend on its pooling method.
Abstract
from arXiv · showhide
Perch 2.0 is a supervised bioacoustics foundation model pretrained on 14,597 species, including birds, mammals, amphibians, and insects, and has state-of-the-art performance on multiple benchmarks. Given that Perch 2.0 includes almost no marine mammal audio or classes in the training data, we evaluate Perch 2.0 performance on marine mammal and underwater audio tasks through few-shot transfer learning. We perform linear probing with the embeddings generated from this foundation model and compare performance to other pretrained bioacoustics models. In particular, we compare Perch 2.0 with previous multispecies whale, Perch 1.0, SurfPerch, AVES-bio, BirdAVES, and Birdnet V2.3 models, which have open-source tools for transfer-learning and agile modeling. We show that the embeddings from the Perch 2.0 model have consistently high performance for few-shot transfer learning, generally outperforming alternative embedding models on the majority of tasks, and thus is recommended when developing new linear classifiers for marine mammal classification with few labeled examples.
1 Background: why marine bioacoustics is challenging
Marine bioacoustics is difficult to collect, verify, and model because underwater recordings require specialized deployment and environmental conditions alter sound propagation. Few-shot and transfer learning support rapid classifier development when labeled data are scarce.
- 1 Background: why marine bioacoustics is challenging: Underwater recordings require specialized sensors, water-sealed equipment, and costly deployment by mooring buoys or scuba divers.Low underwater visibility often prevents visual confirmation of the vocalizing species.
- 1 Background: why marine bioacoustics is challenging: Depth, temperature, salinity, bathymetry, and signal energy alter underwater acoustic propagation and detection range.
- 1 Background: why marine bioacoustics is challenging: New models are needed to classify and annotate large underwater recording databases as discoveries and population-specific songs continue to emerge.
- 1 Background: why marine bioacoustics is challenging: Few-shot and transfer learning enable rapid iteration on newly discovered sounds without requiring large amounts of human-labeled data.
- 1 Background: why marine bioacoustics is challenging: This paper evaluates whether broadly trained terrestrial bioacoustic embeddings can represent marine mammal classes well enough for new classifier development.
2 Perch 2.0
Perch is a broad bioacoustics foundation model trained on over 1.5 million labeled recordings spanning more than 14,500 species. Its architecture and auxiliary objectives are designed to produce embeddings that transfer across bioacoustic tasks and unseen species.
- 2 Perch 2.0: Perch was trained on over 1.5 million labeled recordings covering more than 14,500 species, mostly birds but also insects, mammals, and amphibians.Training sources include Xeno-Canto, iNaturalist, the Tierstimmenarchiv, and FSD50K.
- 2 Perch 2.0: Perch uses an EfficientNet-B3 convolutional architecture on log-mel spectrograms with classification, self-distillation, and source-recording prediction losses.These objectives aim to produce embeddings that are linearly separable across bioacoustic tasks.
- 2 Perch 2.0: Perch embeddings have generalized beyond species classification to individual identification, call type or dialect classification, and species absent from training such as bats.
3 Datasets
The evaluation covers three marine validation sets chosen to test transfer beyond focal recordings: NOAA PIPAN, ReefSet, and DCLDE 2026. DCLDE is partitioned into species, ecotype, and known-species tasks, while NOAA PIPAN and ReefSet span diverse underwater sound classes.
- 3 Datasets: The study evaluates Perch 2.0 on NOAA PIPAN, ReefSet, and DCLDE 2026 to assess transfer to passive hydrophone recordings and ecological monitoring tasks.The Watkins database was considered insufficiently representative because it includes focal recordings from ships or land.
- 3 Datasets: DCLDE 2026 contains over 200,000 short annotations from multiple providers and hydrophone systems for distinguishing killer-whale subpopulations and other noises.Annotations average 0.73 seconds, and 99% are shorter than 3.0 seconds.
- 3 Datasets: DCLDE defines species, ecotype, and known-species tasks, with ecotype classification restricted to five orca populations and uncertain killer-whale labels removed from the known-species task.
- 3 Datasets: NOAA PIPAN includes weakly labeled 30-second recordings covering anthropogenic noise, unknown whale species, and six baleen-whale species.
- 3 Datasets: ReefSet contains 1.88-second examples spanning biological reef noises, species or genera, anthropogenic noise, and waves.
4 Method
The study compares pretrained embedding models using repeated few-shot linear probing across marine tasks. Recordings are windowed, pooled when necessary, and evaluated with one-vs-all ROC-AUC, while training-data overlap is explicitly identified.
- 4 Method: The protocol embeds fixed-size recording windows, averages windows into normalized recording-level embeddings when needed, and trains logistic regression with k ∈{4, 8, 16, 32} examples per class.The process is repeated five times with independent training samples, and average one-vs-all ROC-AUC is reported.
- 4 Method: Comparisons include Perch 2.0, Perch 1.0, SurfPerch, GMWM, BirdNet 2.3, AVES-bio, and BirdAVES through tools supported by the Perch Hoplite repository.
- 4 Method: Two comparisons contain training-data overlap: SurfPerch was trained on ReefSet, while GMWM used much NOAA PIPAN audio and lacks several evaluated NOAA classes.DCLDE 2026 is unseen by all models, although GMWM was trained on killer whales.
- 4 Method: NOAA PIPAN classes with fewer than k + 1 recording-level embeddings are dropped, specifically class Bm at k = 16 and classes Bm and Be at k = 32.
- 4 Method: AUC-ROC compares Perch 2.0 and other pretrained models at k ∈{8, 16} samples per class, with the GMWM off-the-shelf classifier reaching 0.612.The comparison evaluates embedding-based few-shot transfer rather than relying directly on model predictions.
5 Results
Perch 2.0 generally transferred best to new marine tasks, except for contaminated evaluations and one DCLDE task. Few-shot probing also substantially improved GMWM performance over its pretrained classification scores.
- 5 Results: The reported results summarize k = 8 and k = 16 in Table 2 and k ∈{4, 8, 16, 32} in Figure 1, using logistic-regression probing with an implementation differing from prior SurfPerch evaluations.The optimizer and regularization differ from the earlier mini-batch Adam procedure.
- 5 Results: Perch 2.0 outperformed published comparison models on ReefSet and all cetacean tasks except DCLDE Known Bio Species, where BirdNet V2.3 ranked first.The comparison excludes cases involving training-data contamination.
- 5 Results: 0.612 AUC-ROC from GMWM classification scores increased to 0.954 when its embeddings were used for few-shot learning on DCLDE.The authors suggest the classification head may overfit to microphone or other training-data characteristics.
6 Discussion - why do models trained on birds work so whale?
The authors attribute strong marine transfer from terrestrial bioacoustic models to scale, fine-grained supervision, and potentially shared sound-production mechanisms across taxa. They conclude that these models can support efficient marine bioacoustic modeling, while pooling choices may still affect some results.
- 6 Discussion - why do models trained on birds work so whale?: Large models trained on many species may learn detailed acoustic features that transfer to out-of-domain marine tasks despite little underwater audio.The proposed explanation combines neural scaling, difficult bird-classification supervision, and broad class coverage.
- 6 Discussion - why do models trained on birds work so whale?: Shared myoelastic-aerodynamic sound production across birds and marine mammals may partially explain transfer across these taxa.The paper presents this as a possible explanation rather than an established mechanism.
- 6 Discussion - why do models trained on birds work so whale?: Strong few-shot transfer suggests pretrained terrestrial models could improve the efficiency of diverse marine bioacoustic tasks in agile-modeling workflows.The authors also note that AVES performance on low-example tasks might improve with modified embedding pooling.
7 Appendix
Appendix visualizations compare embedding geometry across models on five killer-whale ecotypes. Perch 2.0 shows especially strong separation of difficult ecotype pairs relative to several alternatives.
- 7.1 Additional tSNE Visualizations: The Google multispecies whale model embeddings do not show strong linear separability, consistent with the quantitative results in Table 2.The visual comparison includes GMWM, SurfPerch, Perch 1.0, Perch 2.0, BirdNet V2.3, AVES-bio, and AVES-Bird.
- 7.1 Additional tSNE Visualizations: AVES-bio and BirdAVES show greater class entanglement, especially for southern resident killer whales, than the other evaluated models.Other models separate the southern resident ecotype more distinctly.
- 7.1 Additional tSNE Visualizations: Perch 2.0 appears to provide the best boundary between northern resident and transient killer-whale ecotypes.The AVES models do not separate these two ecotypes well.