Source-linked AI summary
Rate-Agnostic Bioacoustics: Heterogeneous Multi-Taxa Classification with Continuous Filterbanks and Fourier Neural Operators
Stefano Ciapponi, Francesco Ardan Dal Rı, Nicola Conci, Elisabetta Farella
TL;DR
Fixed-rate spectral representations require heterogeneous bioacoustic recordings to be resampled, with fixed input rates imposing Nyquist limits relevant to ultrasonic calls. The paper combines native-rate SFI filtering with an FNO backbone and progressive temporal-scale fusion; SFI-FNO outperforms the compared frontends and reaches .906 accuracy on the evaluated corpus.
Problem
Fixed-rate spectral representations require resampling across heterogeneous recording rates, and fixed input rates can impose Nyquist limits on ultrasonic calls.
Source: §1 · p. 1Method
The paper couples a native-rate SFI continuous-frequency filterbank with an FNO backbone that models log-frequency-time features across progressively fused temporal scales.
Source: §1 · p. 1Results
.906 accuracy, .921 balanced accuracy, and .899 Macro-F1 make SFI-FNO the best overall configuration, while SFI consistently improves over MEL48 and MAX across backbones.
Source: §4 · p. 3Takeaways & Limitations
The SFI frontend retains actual bandwidth while providing a common representation without requiring sampling-rate standardization.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Conventional bioacoustic classification models rely on fixed-rate spectral representations, requiring recordings acquired at heterogeneous sampling rates to be resampled before analysis. We propose a Sampling-Frequency-Independent (SFI) frontend that processes each recording directly at its native sampling rate, coupled with a Fourier Neural Operator (FNO) backbone featuring progressive temporal-scale fusion. This framework avoids fixed-rate resampling and high-frequency information loss while producing fixed-size representations across sampling rates. Mild training-time sampling-rate (\textit{sr}) augmentation further improves robustness to unseen rate variations. Evaluated on a multi-taxa corpus comprising 84 classes and 60 sampling rates, the proposed SFI-FNO configuration outperforms fixed-rate and corpus-maximum-rate baselines, achieving .906 accuracy, .921 balanced accuracy, and a Macro-F1 score of .899.
1. INTRODUCTION
Bioacoustic recordings span highly heterogeneous sampling rates, while fixed-rate spectral inputs can require resampling, retraining, or model switching. The paper proposes native-rate SFI filtering with an FNO backbone to process this variability without discarding available spectral information.
1. INTRODUCTION
- Fixed-rate STFT and Mel representations tie matrix dimensions and feature alignment to sampling frequency, forcing resampling, retraining, or model switching across rates.The cited passage describes field recordings ranging from 300 Hz to over 250 kHz.Source: §1 · p. 1
- Models operating at 16–32 kHz impose Nyquist limits that matter for taxa whose calls extend into the ultrasonic range.The paper motivates processing heterogeneous native rates without sacrificing available spectral information.Source: §1 · p. 1
- The proposed SFI continuous-frequency filterbank operates at native rates and feeds log-frequency-time representations to an FNO backbone with progressive scale fusion.The approach is evaluated against fixed-rate and corpus-maximum Mel frontends and CNN and MBConv backbones.Source: §1 · p. 1
2. METHODOLOGY
The methodology builds rate-independent multi-scale features from native-rate audio, then models and fuses their frequency-time structure into a fixed-dimensional representation.
2.1. Sampling-Frequency-Independent Frontend
- The SFI frontend processes audio at its arbitrary native sampling rate without resampling and constructs representations across multiple temporal scales.Continuous log-frequency responses are adapted by a frequency-response residual MLP, while a Nyquist validity mask and dyadic processing handle rate-dependent bands.Source: §2.1 · p. 2
2.2. Fourier Neural Operator Backbone
- A 2D FNO applies global spectral transformations over the log-frequency-time grid, followed by masked global pooling into a fixed-dimensional vector.The block combines learned Fourier modes with a local 1×1 transformation, GroupNorm, GELU, and dropout.Source: §2.2 · p. 2
2.3. Scale Fusion and Latent Projection
- The temporal pyramid processes 16, 64, and 256 ms scales progressively, pooling hidden maps to each next scale's frame length before convolutional fusion.Dual masked global pooling combines mean and peak responses; a linear projection with LayerNorm maps the result to a 192-dimensional latent embedding.Source: §2.3 · p. 3
3. EXPERIMENTAL SETUP
The benchmark combines four taxonomic datasets with heterogeneous sampling rates and class distributions. A 3 × 3 suite compares three frontends and three backbones, with an additional rate-augmentation robustness run.
3.1. Datasets
- The corpus combines marine mammals, European birds, frogs and toads, and bats, excluding species with fewer than 15 samples.The datasets span different taxonomic groups, spectral ranges, and class distributions; Table 1 summarizes corpus statistics after filtering.Source: §3.1 · p. 3, Table 1 · p. 3
3.2. Model Configurations and Hyperparameters
- A 3 × 3 suite pairs SFI, MEL48, and corpus-maximum-rate frontends with FNO, CNN, and MBC backbones.All configurations use K = 128 frequency bins and the same window-hop pairs; MEL48 is resampled to 48 kHz and MAX to the maximum corpus rate.Source: §3.2 · p. 3
- SFI-FNO was retrained across all five folds with random sampling-rate augmentation from 0.75–1.05× at p = .85.Training used AdamW for up to 80 epochs, with validation Macro-F1 for scheduling and early stopping.Source: §3.2 · p. 3
4. RESULTS
SFI improves over MEL48 and MAX across backbones, with SFI-FNO best overall. Rate augmentation substantially reduces SFI-FNO’s sensitivity to test-rate perturbations while maintaining Macro-F1 across perturbations.
Rate Robustness
- Figure 3 plots the number of corpus files at each individual native sampling rate on a logarithmic scale.The section describes a benchmark with heterogeneous native sampling rates.Source: §4 · p. 3, Figure 3 · p. 4
Classification Results
- .906 accuracy, .921 balanced accuracy, and .899 Macro-F1 make SFI-FNO the best overall configuration; SFI improves over MEL48 and MAX across backbones.Relative to MEL48, SFI raises accuracy by 5.2, 5.7, and 4.7 points for CNN, MBC, and FNO, respectively.Source: §4 · p. 3, Table 2 · p. 4
Rate Robustness
- Rate augmentation reduces the mean perturbation-induced Macro-F1 change from −.291 ± .074 to −.004 ± .005 for SFI-FNO.Perturbed test recordings use 95%, 90%, or a random 85–95% rate factor; the evaluation retains recordings whose occupied bandwidth remains representable after perturbation.Source: §4 · pp. 3–4, Table 3 · p. 4
- With rate augmentation, SFI-FNO maintains Macro-F1 between .865 and .869 across test-rate perturbations.The unaugmented comparison is strongly rate-sensitive; the reported Table 3 summarizes Macro-F1 under perturbation as mean ± standard deviation.Source: §4 · p. 4, Table 3 · p. 4
5. DISCUSSION AND CONCLUSIONS
The discussion frames heterogeneous sampling rates as a trade-off between spectral preservation and computation, and reports that SFI improves over resampling baselines. The authors conclude that native-rate processing with global frequency-time modeling may support larger-scale heterogeneous monitoring.
5. DISCUSSION AND CONCLUSIONS
- Cross-taxa benchmark results should be interpreted as evidence about robustness across heterogeneous taxa, not directly compared with dataset-specific classifiers.The cited AVES-Bio, Perch 2.0, and specialized Watkins-only results use different evaluation scopes; Table 4 reports SFI-FNO performance per dataset over five folds.Source: §5 · p. 4, Table 4 · p. 4
- SFI consistently improves over MEL48 and corpus-maximum-rate resampling, while SFI-FNO uses sample-rate-dependent shortcuts that rate augmentation largely suppresses.Augmentation reduces mean perturbation-induced Macro-F1 change from −.291 ± .074 to −.004 ± .005 while preserving overall performance.Source: §5 · p. 4
- Moderate-rate downsampling reduces processing cost but discards content above Nyquist, whereas corpus-maximum-rate resampling preserves bandwidth but enlarges lower-rate recordings.SFI retains each recording’s actual bandwidth and provides a common representation without sampling-rate standardization.Source: §5 · p. 4
- The framework combines native-rate spectral processing with global frequency-time modeling, providing a basis for larger-scale heterogeneous monitoring and foundation-model-like training.The authors identify unsupervised and semi-supervised learning as possible future directions.Source: §5 · p. 4