Source-linked AI summary
Interpreting Dolphin Vocal Sequences via Multiple Sequence Alignment
Daniel Kohlsdorf, Denise Herzing, Thad Starner
TL;DR
Understanding recurring structure in dolphin vocalizations matters for studying the linguistic complexity and social structures of wild pods. The paper adapts ClustalW to continuous acoustic embeddings using Gaussian kernel similarity, producing alignments that reveal shared temporal structure. Feature representation strongly affects alignment quality, and the reported alignments expose patterns difficult to see in standard spectrograms.
Problem
Researchers seek recurring structure in large dolphin vocalization databases to investigate communication, linguistic complexity, and social structures.
Source: §1 · p. 1Method
The framework adapts ClustalW to continuous dolphin acoustic embeddings, replacing discrete substitution scores with Gaussian kernel similarity to visualize shared temporal structure.
Source: §1 · p. 2, §10 · p. 13Results
MFCC and Wav2vec alignments have gap percentages above 97%, while Perch and Aggression whisper small yield a more balanced structural representation.
Source: §8 · p. 11, §10 · p. 13Takeaways & Limitations
The alignments reveal shared structural patterns that are difficult to detect through standard spectrogram inspection, offering a tool for comparative acoustic analysis.
Source: §8 · p. 11, §10 · p. 13The pipeline uses pre-segmented recordings with isolated vocalizations; continuous recordings with overlapping vocalizers pose source-separation and sequence-boundary challenges.
Source: §8 · p. 12
Abstract
from arXiv · showhide
Dolphin communication understanding is essential for uncovering the linguistic complexity and social structures of wild pods. We adapt the ClustalW bioinformatics algorithm to analyze continuous acoustic data, treating vocalizations as high-dimensional spectral feature vectors. By replacing discrete scoring with a continuous Gaussian kernel similarity measure, our framework generates Multiple Sequence Alignment (MSA) visualizations that reveal shared structural patterns. These alignments highlight temporal motifs such as synchronized burst pulses in aggressive contexts that are difficult to detect through standard spectrogram inspection.
1 Introduction
Researchers use machine learning to find recurring patterns in large dolphin vocalization databases, with the aim of uncovering linguistic complexity and understanding wild dolphins’ social structures. This paper applies sequence alignment to dolphin acoustic data and presents initial structural patterns across behavioral contexts.
1 Introduction
- Machine learning is used to identify recurring patterns in extensive vocalization databases as a route to studying communication and social structure.The introduction also presents future animal-signal communication as a possible application, including for conservation and monitoring.Source: §1 · p. 1
- Wild Dolphin Project recordings of spotted dolphins in the Bahamas span more than 30 summers and include play, nursing, foraging, and aggression.Aggressive behavior is characterized by synchronized burst pulses.Source: §1 · pp. 1–2
- Sequence alignment is adapted from bioinformatics to identify structural organization in dolphin acoustic data, with initial patterns examined across behavioral contexts.The paper introduces pairwise and multiple sequence alignments and applies ClustalW to vocalizations, including aggression.Source: §1 · p. 2
2 Ethical Considerations
The paper adopts a listener-first approach to understanding dolphin signals while recognizing that direct communication could carry ethical risks. Its analysis focuses on aggression, and it excludes playback of agonistic signals.
2 Ethical Considerations
- Direct communication using synthetic dolphin signals could disrupt social structures, cause psychological stress, or alter natural foraging and mating behaviors.The authors advocate a non-invasive, listener-first approach and say future bidirectional communication should prioritize dolphin autonomy and wild behavioral integrity.Source: §2 · p. 2
- The analysis focuses on aggression and excludes playback of agonistic signals, including synchronized burst pulses.The authors say reproducing these patterns risks triggering intra-pod conflict or long-term social displacement; the aim is human recognition of social states, not manipulation through artificial signaling.Source: §2 · p. 2
3 Related Work
Related work spans machine-learning classification and monitoring, probabilistic sequence modeling, and alignment-based approaches to acoustic structure. The paper positions its contribution within sequence alignment and treats encoder choice as separate from adapting alignment to continuous acoustic embeddings.
3 Related Work
- Machine-learning research includes vocalization classification, detection of behaviors and sequences, caller identification, signature-whistle tracking, and conservation monitoring.Examples include DNN classification of bottlenose dolphin whistle types linked to foraging, mating, or socializing [3], and monitoring anthropogenic noise impacts.Source: §3 · p. 2
- Prior sequence modeling uses HMMs to describe sound evolution and unsupervised denoising autoencoders to reconstruct clean vocalizations from synthetically corrupted inputs.The described decoder uses deconvolution, upsampling, and an LSTM stack to preserve long-range sequence dependencies.Source: §3 · p. 2
- ClustalW and other alignment approaches are presented as ways to identify conserved motifs and structural syntax across variable communicative exchanges.The paper builds on sequence-alignment and generative paradigms to seek latent syntactic organization in cetacean communication.Source: §3 · p. 2
- Alternative audio encoders include self-supervised speech models and cross-species bioacoustic foundation models, which the paper describes as potential drop-in replacements for its Whisper encoder.Encoder choice is described as orthogonal to the paper’s focus on adapting sequence alignment to continuous acoustic embeddings.Source: §3 · p. 2
4 Working with Audio Data
The paper represents spectrograms as sequences of spectral feature vectors and describes whistles, burst pulses, and echolocation clicks by their distinct visual patterns.
4 Working with Audio Data
- A spectrogram is represented as a time-ordered sequence of spectral feature vectors.Researchers use spectrograms to visualize audible dolphin communication data.Source: §4 · p. 2
- Whistles appear as single lines, burst pulses as multiple parallel lines, and echolocation clicks as thin vertical lines spanning a wide frequency range.The text distinguishes sustained whistle tones, rhythmic burst pulses, and broadband clicks used for biological sonar.Source: §4 · p. 3
5 Alignment Algorithms
This section presents a dolphin-audio alignment pipeline that adapts progressive multiple sequence alignment to embedding sequences, using continuous acoustic-frame similarity. It combines spectral clustering, column-informed affinities, and temporal smoothing to visualize repeated motifs while preserving horizontal correspondence.
5 Alignment Algorithms
- The pipeline windows each spectrogram, embeds its segments into a temporally ordered sequence, then progressively aligns the per-recording sequences into shared columns for visualization and clustering.Each window is processed independently, and the embeddings are concatenated in temporal order.Source: §5 · p. 3, Figure 3 · p. 3
5.1 What is an Alignment
- In an MSA, gaps stretch or shrink sequences so motifs align in shared columns; aligning more than two sequences produces a profile.Each column contains either an original value or a gap for each sequence.Source: §5.1 · pp. 3–4
5.2 Constructing Pairwise Alignments
- Pairwise alignment uses dynamic programming: DTW minimizes cumulative distance, Levenshtein minimizes edit costs, and Needleman–Wunsch maximizes similarity scores with gap penalties.DTW typically uses Euclidean distance; Levenshtein assigns unit insertion and deletion costs.Source: §5.2 · p. 4
5.3 Constructing Multiple Sequence Alignments and Progressive Alignment
- Progressive MSA builds a guide tree from all-pairs similarities or distances, then merges sequences and profiles into shared columns, inserting gaps to preserve motifs across recordings.Hierarchical clustering joins the most similar sequences first, and progressive merging continues toward the root of the guide tree.Source: §5.3 · pp. 4–5
5.3.1 Early Stopping and Multiple Alignment Clusters.
- Because one alignment can obscure dissimilar vocal groups, early stopping at a distance threshold or target cluster count yields independent alignments and supports analysis at multiple granularities.The distance-percentile threshold adapts to dataset variability; resulting cluster assignments can support downstream behavioral or social-context comparisons.Source: §5.3.1 · pp. 5–6
5.4 Dolphin Vocalization Alignment
- Gaussian-kernel scoring adapts progressive alignment to continuous acoustic frames, and the resulting MSAs reveal temporal correspondences and repeated motifs difficult to detect by spectrogram inspection alone.The kernel assigns higher similarity to nearby feature vectors and lower similarity to distant ones, replacing discrete substitution scores.Source: §5.4 · pp. 6–7
5.5 Feature extraction using transformers
- To address unreliable distances in high-dimensional spectrograms, the method adapts Whisper with LoRA and triplet-plus-classification supervision, producing 128-dimensional embeddings for Gaussian-kernel alignment.Triplet loss brings same-type signals together and separates different types; classification supervision covers whistles, burst pulses, echolocation clicks, and noise.Source: §5.5 · p. 7
5.6 Column Analysis and Clustering
- Because frames in the same MSA column may represent the same motif, alignment-informed clustering uses column structure as a prior to stabilize assignments and color-code similar sounds.The passage contrasts this approach with standard clustering methods that operate independently of alignment structure.Source: §5.6 · pp. 7–8
- The method applies spectral clustering to an RBF-kernel affinity matrix, then fits a Bayesian Gaussian mixture that prunes negligible components to infer the effective cluster count.The affinity scores reflect Euclidean distances between embeddings, regardless of their alignment positions.Source: §5.6 · p. 8
- A column prior boosts affinity between embeddings in the same MSA column, using temporal correspondences as soft supervision for clustering.The boost propagates structural information identified by the progressive alignment algorithm into the clustering objective.Source: §5.6 · p. 8
- HMM smoothing encourages cluster labels to persist across consecutive frames, reducing rapid switches that fragment the visualized alignments.The transition matrix uses an expected segment length to set the self-transition probability.Source: §5.6 · p. 8
- Column-informed clustering produces coherent overlays where repeated motifs appear as consistent color bands across aligned sequences.Acoustically distinct segments receive contrasting colors, and the column prior reinforces the MSA’s horizontal correspondences.Source: §5.6 · p. 8
6 Experiments
The experiments compare adult aggression and juvenile play vocalizations using alignment metrics that assess gaps, column homogeneity, and clustering consistency. The curated datasets and complementary measures are intended to evaluate both technical alignment quality and biological relevance.
6 Experiments
- Using 58 adult-aggression and 96 juvenile-play-aggression recordings curated from behavioral episodes, the study compares vocalization consistency across age groups with identical alignment procedures to quantify potential developmental differences.Source: §6.1 · pp. 8–9
6.1 Datasets
- Video-annotated episodes supply vocalization snippets with behavioral labels, including adult aggression and juvenile play aggression.The adult recordings come primarily from two 2021 observation sessions; juvenile recordings span 2020–2023.Source: §6.1 · pp. 8–9
6.2 Alignment Quality Metrics
- Four metric groups assess gap structure, per-sequence statistics, column quality, and clustering consistency as complementary aspects of alignment quality.The evaluation combines quantitative metrics with qualitative observations from MSA visualizations.Source: §6 · p. 8, §6.2 · p. 9
6.2.1 Gap Structure Metrics.
- Gap openings, extensions, their ratio, and gap percentage characterize gap fragmentation, contiguity, and alignment density.Lower opening-to-extension ratios indicate longer, more contiguous gap blocks; the text describes 10%–40% gap percentages as a generally reasonable balance.Source: §6.2.1 · p. 9
6.2.2 Per-Sequence Gap Statistics.
- Per-sequence gap summaries include mean, median, standard deviation, maximum, and consecutive gap lengths to identify typical values and outliers.Long gap runs may reflect partial vocalizations or extended silence.Source: §6.2.2 · pp. 9–10
6.2.3 Column Quality Metrics.
- Column variance measures embedding homogeneity; columns below 0.1 are clean, while those above 1.0 are noisy.Higher percentages of clean columns suggest more successful alignment.Source: §6.2.3 · p. 10
6.2.4 Alignment Statistics.
- Alignment rate reports the percentage of input recordings incorporated into multi-sequence alignments, while also tracking files left as singletons.Higher alignment rates indicate sufficient structural similarity for the algorithm to identify correspondences.Source: §6.2.4 · p. 10
6.2.5 Clustering Quality Metrics.
- Mean and median column purity and entropy, plus purity variability and pure-column percentage, assess whether cluster labels agree within aligned columns.A pure column has the same cluster label for every non-gap entry; lower entropy indicates less label diversity.Source: §6.2.5 · p. 10
7 Experiments and Evaluation
The evaluation compares acoustic feature extractors on adult aggression recordings and contrasts those results with juvenile play vocalizations. Whisper small produced the most structurally balanced adult alignment, while juvenile calls showed greater local variability and higher alignment rates and cluster purity.
7 Experiments and Evaluation
- The evaluation compares feature representations and behavioral groups using gap structure, per-sequence statistics, column variance, and clustering consistency.Table 4 summarizes the baseline small transformer on adult aggression against MFCCs, Wav2vec, Perch, and the juvenile dataset.Source: §7 · pp. 10–11, §7.1 · p. 11
7.2 Model Comparison on Aggression Recordings
- 0.268 gap ratio and 55.72% gap percentage for Whisper small, with 78.93% clean columns, made it the most structurally balanced adult-aggression representation.MFCCs and Wav2vec produced gap percentages above 97%, while Perch reduced gap ratio to 0.015 (-94.37%) but retained heavier gap insertion loads than the fine-tuned transformer encoder.Source: §7.2 · p. 11
7.3 Comparison with Juvenile Play Aggression Data
- 0.564 gap ratio (+110.77% over adult aggression) and 102.8 versus 188.0 columns characterized juvenile and adult alignments, respectively.Juveniles also had 80.21% alignment rate versus 68.97% for adults, and 0.859 mean purity versus 0.757; the authors interpret the pattern as greater juvenile variability and ongoing fine motor refinement.Source: §7.3 · p. 11
8 Discussion
The discussion reports that alignment quality depends strongly on acoustic representation and that adult and juvenile vocalizations differ in stereotypy. It also describes the pipeline’s scope and ethical constraints.
8 Discussion
- MSA 1 and MSA 2 show homogeneous, stereotyped structure, whereas MSA 3 contains multiple clusters and greater acoustic variability.Figure 8 colors encode spectral clusters across aligned recordings; alignment position is not absolute recording time, and black gaps carry no acoustic content.Source: §8 · p. 11, Figure 8 · p. 12
- Perch achieves a 0.015 gap ratio and 99.45% cluster purity, while Whisper small achieves 55.72% gaps and 78.93% clean columns.The authors describe the domain-adapted embeddings as a more structurally robust representation than generic or raw features for capturing natural temporal motifs.Source: §8 · p. 11
- The adult–juvenile comparison supports increasing vocal stereotypy with age, consistent with juveniles still acquiring vocal motor control.Source: §8 · p. 11
- Alignment metrics do not directly establish biological meaningfulness; human-speech pretraining may bias features, and cluster stopping requires manual thresholds.Source: §8 · p. 11
- The current pipeline uses pre-segmented recordings with isolated vocalizations; overlapping vocalizers pose source-separation and sequence-boundary challenges.Source: §8 · p. 12
- The listener-first framework restricts the work to pattern recognition and passive observation rather than signal playback or confident prediction of responses to stimuli.The authors describe playback of aggressive patterns as a risk for intra-pod conflict or long-term social displacement.Source: §8 · pp. 12–13
9 Future Work
The future-work section proposes extensions that use alignments to recognize call types, investigate vocal structure, connect calls to behavior, and compare species.
9 Future Work
- Training profile hidden Markov models on pipeline alignments could support probabilistic motif representations, novel-recording classification, and call-type detection in continuous audio.Source: §9 · p. 13
- Grammar induction could test whether calls have compositional structure by learning rules over symbolic sequences formed from MSA column cluster labels.Source: §9 · p. 13
- Jointly aligning vocalizations with video could reveal correlations between acoustic motifs and physical behaviors, adding context for interpreting call function.Source: §9 · p. 13
- Applying the alignment method to other cetaceans could test whether spotted-dolphin structural patterns generalize or reflect species-specific adaptations.Source: §9 · p. 13
10 Conclusion
The conclusion presents MSA with transformer-derived acoustic embeddings and Gaussian-kernel scoring as a way to visualize shared temporal structure in dolphin vocalizations. It reports that representation choice affects alignment quality and frames the work as listener-first pattern recognition.
10 Conclusion
- The framework adapts ClustalW to continuous acoustic embeddings and uses Gaussian-kernel similarity to visualize conserved motifs that are difficult to detect in standard spectrograms.Source: §10 · p. 13
- Standard bioacoustic and self-supervised features yield high gap percentages, while domain-adapted transformer encoders and Perch achieve more balanced structural representations.Source: §10 · p. 13
- The listener-first approach prioritizes human recognition of dolphin social states without playback, supporting passive, non-invasive research.Source: §10 · p. 13