OpenWhistle: A Large-Scale Longitudinal Dataset and Benchmark of Bottlenose Dolphin Vocalizations

1 Introduction

• Broad bioacoustic corpora are shallow per species, while dolphin datasets are small, fragmented, and largely unavailable publicly.Studying vocal learning, identity, social coordination, and temporal change requires deep longitudinal data from known individuals.Source: §1 · pp. 1–2
• OpenWhistle pairs approximately 180,000 whistles over five years from five known individuals with 8,354 expert labels and protocols for detection and whistle-type classification.The resource preserves contiguous sequences from interacting individuals.Source: §1 · p. 2
• The single-species longitudinal corpus complements broad-coverage benchmarks and supports label-efficient learning and temporal-shift evaluations.The paper also identifies vocal exchanges, interaction dynamics, temporal drift, and vocal development as future research directions.Source: §1 · p. 2
• The released pipeline supports scalable whistle detection, segmentation, and type categorization from continuous passive acoustic recordings.It is presented as a practical recipe for constructing large dolphin acoustic datasets.Source: §1 · p. 2
• The OpenWhistle-pretrained Wav2Vec2.0 model outperforms general bioacoustic models on both whistle detection and classification.The authors report that in-domain pretraining learns effective dolphin-whistle representations.Source: §1 · p. 2

2.1 Dolphin Vocalizations and Communication

• Signature whistles are stable, individually distinctive calls used for recognition and social bonds, while non-signature whistles are more variable and less understood.Dolphins produce and imitate signature whistles; they can constitute up to 70% of whistles in natural settings.Source: §2.1 · pp. 2–3

2.2 Existing Datasets

• Large bioacoustic datasets such as Xeno-Canto and BirdSet offer broad taxonomic coverage but are typically shallow per species and focus on detection or species classification [34].These characteristics make them less suited to studying the structure of a single species’ communication system.Source: §2.2 · p. 3
• Existing dolphin resources trade off scale, fine-grained labels, temporal coverage, or openness, with no resource combining these properties at scale.Some datasets provide detailed contours but only a few thousand whistles; others offer longer coverage with limited annotations or restricted access.Source: §2.2 · p. 3
• OpenWhistle provides raw audio, fine-grained whistle-type annotations, and reproducible evaluation, and is described as the only such resource tested for self-supervised models.A more recent effort [32] uses spectrogram images and coarse binary annotations.Source: §2.2 · p. 3

3 Data Collection

• Recordings were collected at Dolphin Reef, where a resident pod includes five dolphins, including a Tursiops aduncus female who joined sporadically in 2019.The site is a delimited marine area open to the sea on the northern Gulf of Aqaba.Source: §3 · p. 3
• Fixed hydrophones continuously capture underwater audio while dolphins move freely within the area and can exit to the open sea.Dolphins often leave to forage when the site is closed or human activity is low.Source: Figure 1 · p. 4, §3 · p. 4
• The semi-natural setting combines realistic acoustic conditions and social interactions with stable individual identity and multi-year continuity.The authors describe this combination as supporting analyses that need both ecological realism and individual-level resolution.Source: §3 · p. 4

4.1 Annotation Pipeline

• The whistle-presence CNN achieved 96.52% precision and 97.99% recall on a held-out set of 16,708 segments.It uses spectrograms and VGG16-based architecture; consecutive detections are concatenated, and segments less than 6 seconds apart are merged into sequences.Source: §4.1 · p. 4, Figure 2 · p. 5
• The annotation pipeline combines contour-similarity clustering with expert visual review to assign whistles to 10 types.ARTwarp uses dynamic time warping; the final categories comprise seven signature and three non-signature whistle types.Source: §4.1 · pp. 4–5

4.2 Pretraining Dataset

• The pretraining corpus contains about 114 hours and 180,000 whistles across 33,267 sequences, recorded over five years from five identified individuals.Sequences average 12.95 seconds, with a 4.11-second mean interval between whistle segments; overlapping vocalizations and environmental sounds are preserved.Source: §4.2 · p. 5

4.3 Expert-annotated Set

• The annotation pipeline categorized 8,354 whistles into 10 categories: 7,624 signature whistles (91.3%) across 7 types and 730 non-signature whistles (8.7%) across 3 types, serving as ground truth for downstream tasks.The distribution is highly imbalanced, reflecting natural production frequencies, with a few dominant signature whistles and several rare categories.Source: §4.3 · p. 6
• Annotated whistles average 0.84 seconds in duration and have a mean SNR of 13.24 dB, above the full pretraining corpus.Manual curation favors clear, minimally overlapping vocalizations.Source: §4.3 · p. 6

5.1 Tasks

• Classification assigns isolated whistles to six well-represented categories in a balanced set of 3,000 instances, with mean accuracy as the metric.Categories include signature and non-signature whistle types; less-represented categories are excluded.Source: §5.1 · p. 6
• Detection labels 0.5 s recording segments with present whistle types or background, using 400 instances per type across seven classes and 2,800 background segments.Performance is measured by mean average precision (mAP).Source: §5.1 · p. 7, Figure 4 · p. 7

5.2 Evaluation Protocol

• Evaluation uses frozen segment representations with logistic-regression probes and 70%/15%/15% train, validation, and test splits grouped by recording session.Session-level grouping prevents acoustic context from a session appearing across splits while maintaining similar label distributions.Source: §5.2 · p. 7
• Bootstrap resampling estimates uncertainty by sampling the test set with replacement 1,000 times and reporting the mean and standard deviation.The regularization parameter C is selected using the validation set.Source: §5.2 · p. 7

6.1 Models and Baselines

• The comparison probes classical acoustic features, Biolingual, AVES-core, AVES-bio, and OpenWhistle-trained Wav2Vec2.0 using logistic regression on frozen representations.The probe’s inverse regularization strength is tuned over C ∈ {0.1, 1.0, 10.0}.Source: §6.1 · pp. 7–8

6.2 Results

• 81.1% classification accuracy and 75.6 mAP detection with OpenWhistle-trained Wav2Vec2.0 exceed AVES-bio’s 75.1% classification accuracy and 65.0 mAP detection, but both tasks remain unsolved.The reported comparison appears in Table 2.Source: Table 2 · p. 8, §6.2 · p. 8
• OpenWhistle-trained Wav2Vec2.0 exceeds the strongest handcrafted baseline by 25.5 accuracy points in classification and 27.9 mAP in detection, and also surpasses off-the-shelf pretrained models.The authors describe this as the first application of large-scale self-supervised pretraining directly on dolphin vocalization data.Source: §6.2 · p. 8
• The remaining errors leave room to improve detection and classification, which underpin downstream analyses of dolphin communication.The paper describes OpenWhistle as both a pretraining resource and a challenging benchmark for tracking progress.Source: §6.2 · p. 8

7 Conclusion

• OpenWhistle’s large-scale whistle corpus was collected over five years from a stable pod of known individuals.The conclusion describes the setting as a natural marine environment.Source: §7 · p. 8
• An expert-labeled subset enables controlled evaluation, while scale, continuous longitudinal coverage, and detailed annotation distinguish the resource from typically small, short-term, or closed datasets.The resource supports studying dolphin communication in a single-species setting.Source: §7 · p. 9
• OpenWhistle supports self-supervised representation learning, and its detection and classification tasks probe fine-grained discrimination among structurally similar whistles linked to individual identity.The reported performance gains alongside remaining errors indicate that the tasks probe non-trivial acoustic structure rather than superficial cues.Source: §7 · p. 9
• The released dataset, processing pipeline, and evaluation protocol provide a foundation for studying vocal sequences, repertoire change over time, and interaction dynamics.The paper relates interaction-dynamics analysis to.Source: §7 · p. 9

8 Future Work

• Planned releases will add audio, extracted whistles, and expert labels across more periods of the five-year span, while future work will explore contextual and video data.The expanded annotations are intended to support more robust evaluation and study of temporal variability and less frequent whistle types.Source: §8 · p. 9

• Recordings from one site and five dolphins constrain dataset diversity, so results should be validated on independent groups.Source: §9 · p. 9
• Uneven recording density, concentrated times of day, and a full 2022 gap leave dolphin vocal activity non-uniformly sampled; models may reflect overrepresented conditions and behaviors.Source: §9 · p. 9
• The detector’s 97.99% recall leaves an estimated ∼3,700 whistles uncaptured, with low-SNR vocalizations more likely missed; an absent whistle type may still have been produced.Source: §9 · p. 9
• Expert labels cover only five months of the five-year recording window, so performance on that subset may not generalize to the full dataset.Source: §9 · p. 9
• The annotated dataset is highly imbalanced, with some categories below 100 examples, limiting rare-type evaluation and potentially biasing models toward frequent categories.Source: §9 · p. 9
• The benchmark tests whistle-type detection and classification, not semantic interpretation or communicative meaning; evaluating signal function requires behavioral, social, and contextual annotations.Source: §9 · pp. 9–10

10 Ethics and Broader Impact

• Passive, hidden hydrophones recorded in a semi-natural environment without training or constraining dolphins; human interaction was voluntary and dolphin-initiated.Source: §10 · p. 10

A.1 Equipment and Recording Protocol

• Three hydrophones sampled at 96 kHz across 7,495 sessions, yielding 6,271.78 hours of usable audio from 12 November 2019 to 28 March 2024.Source: §A.1 · p. 14

A.2 Dolphins and Individual Metadata

• Figure 5 distinguishes recording-period dolphins in red from absent dolphins whose signature whistles appear in the dataset, shown in blue.Source: Figure 5 · p. 14
• The recording-period pod included four resident Tursiops truncatus ponticus and intermittently visiting Tursiops aduncus female Yosefa, whose signature whistle was identified using SIGID.Source: §A.2 · p. 14
• Resident dolphins’ identity, family relationships, and signature whistles are linked in the metadata, supporting analyses of vocal identity and social structure.Source: §A.2 · p. 14

B Dataset Construction Pipeline

• Positive detections are segmented and split into branches: grouped whistle sequences form the pretraining corpus, while F0 estimation, ARTwarp categorization, and expert review produce annotated labels.Source: B Dataset Construction Pipeline · pp. 14–15, Figure 6 · p. 15
• The VGG16-based detector classifies 0.4-second spectrogram windows; sequence summaries group positive windows across gaps up to 6 seconds, retaining sequences of 2–20 seconds.Source: B Dataset Construction Pipeline · p. 15
• Without retraining, the detector scores F1 0.904 on the WMMSD binary clip benchmark, 0.907 on the WMMSD delphinid-versus-clear-noise proxy benchmark, and 0.966 on a DCLDE proxy subset.Source: B Dataset Construction Pipeline · p. 15
• Dolphin-specific CREPE estimates F0 contours, which ARTwarp categorizes using dynamic time warping and an adaptive resonance theory network.Source: B Dataset Construction Pipeline · pp. 15–16

C Pretraining Setup

• Wav2Vec2.0 is pretrained on OpenWhistle, with its feature encoder adapted to 44.1 kHz audio to preserve relative temporal resolution.Source: C Pretraining Setup · p. 16

D Additional Analysis of Whistle-Type Classification

• The confusion matrix shows strong overall classification but more frequent confusion between Nana’s and Yosefa’s signature whistles, whose spectrograms share similar frequency contours; examples also show within-type modulation variation.Source: D Additional Analysis of Whistle-Type… · pp. 16–17, Figure 7 · p. 16