PaperlayerView exact revision on arXiv

Source-linked AI summary

Structure Across Voices: Comparing acoustic-event type accumulation and sequence dependence across four vocal repertoires using frozen audio encoders

Mudit Sinha, Sanika Chavan

arXiv:2609.16612v1cs.SDcs.AI

TL;DR

Different native event units and unequal sequence lengths make direct comparison of vocal repertoires difficult. The paper matches event and local sequence opportunity while comparing four repertoires with frozen encoders and complementary acoustic analyses. Whale leads type accumulation, Finch leads categorical order measures, and the broader comparison depends on acoustic property and temporal scale.

  • Problem

    Vocal repertoires use events of different durations and complexities and provide unequal sequence opportunity, complicating direct comparison of acoustic-event accumulation and temporal organization.

    Source: §I · p. 1

  • Method

    The study preserves native event units, matches observation opportunity, and compares frozen-encoder groups with direct acoustic measures, continuous analyses, and source-aware controls.

    Source: §I · p. 1, §II · p. 2

  • Results

    Whale leads type accumulation, while Finch leads immediate dependence and subsequence recurrence; continuous prediction and longer discrete context shift the comparison toward Whale.

    Source: A. The repertoires differ in profile… · p. 9, D. Temporal scale changes which… · p. 10, §VI · p. 10

  • Takeaways & Limitations

    The four repertoires form distinct profiles rather than one hierarchy, separating acoustic-event diversity, categorical order, continuous predictability, and longer-context effects.

    Source: §VI · p. 10

Abstract

from arXiv · show

Vocal repertoires can differ in acoustic-event type accumulation and temporal organization, yet direct comparison is difficult because corpora use different native events and unequal amounts of sequence. We compare sperm whale codas, human speech phones, Bengalese finch syllables, and common marmoset calls using the same frozen-audio-encoder procedure while matching event count and local sequence opportunity. Whale shows the fastest type accumulation; Finch shows the strongest immediate dependence and repeated-subsequence recurrence. Physically interpretable acoustics recover complementary parts of this profile, continuous analyses without clustering support broad Whale acoustic coverage, and source- and position-preserving nulls retain both Finch order effects. Extending predictive context shifts the comparison toward Whale. Thus repertoire differences depend on the acoustic property and temporal scale measured rather than forming a single hierarchy.

I. INTRODUCTION

Comparing vocal repertoires is difficult because their annotated events differ in duration and complexity, and their corpora provide unequal amounts of sequence. The paper asks how event-type accumulation and temporal organization compare when native units and observation opportunity are retained and matched.

I. INTRODUCTION

  • Sperm-whale codas, human phones, finch syllables, and marmoset calls differ in duration and internal complexity, so equal-duration segmentation would impose a different assumption about event boundaries.The comparison therefore preserves each corpus’s annotated event scale.Source: §I · p. 1
  • The study asks how quickly new acoustic-event groups appear and how strongly temporal context constrains event streams, including immediate dependence, recurrence, and additional predictive value from the event two positions back.It does not assume the native units are biologically homologous.Source: §I · p. 1

II. RELATED WORK

  • Prior sequence measures depend on event definition and available sequence, while type accumulation asks how rapidly operational acoustic groups continue to expand.The paper pairs frozen encoders with direct acoustic measurements, continuous analyses, source-aware controls, and a Whale timing intervention.Source: §II · p. 2

III. METHODS

The comparison preserves each corpus’s native event annotations while matching observation opportunity and applying the same frozen-encoder procedure. Acoustic groups are formed separately within each repertoire rather than aligned across corpora.

III. METHODS

  • The comparison preserves native acoustic events while equalizing signal-processing procedure, clustering resolution, event count, and block-length profile.Each matched draw contains 1,501 events in 113 blocks, yielding 1,388 adjacent pairs and 1,275 trigrams.Source: §III · p. 2, Figure 1 · p. 3

A. Corpora and source-defined acoustic events

  • Whale contributes 1,501 codas, while the reference pools contain 12,000 human phones, finch syllables, and marmoset calls each; the matched draws use 1,501 events.Native order is retained within human tracks, finch birds/songs, and marmoset recording days.Source: A. Corpora and source-defined acoustic… · p. 2, Table I · p. 2

B. Common acoustic representation and within-repertoire clustering

  • Eleven frozen encoder configurations produce one event vector apiece using identical settings across corpora, and K-means groups vectors independently within each repertoire.K ∈ {16, 24, 32} tests grouping resolution; cluster identities are not aligned across corpora.Source: B. Common acoustic representation and… · p. 2
  • Whale’s encoder-derived groups agree only weakly with historical CodaType and covary modestly with click count, duration, and normalized ICI, motivating direct acoustic validation.Source: B. Common acoustic representation and… · p. 3

C. Matching event count and sequence opportunity

The methods match event counts and local sequence opportunities before measuring acoustic-group accumulation and temporal organization. They compare separate measurement ranks rather than defining one aggregate winner.

C. Matching event count and sequence opportunity

  • Each matched draw uses 1,501 events in 113 blocks, yielding 1,388 adjacent pairs and 1,275 trigrams; duration, spectrum, source composition, propagation, and recording hardware remain corpus properties.Reference draws use non-overlapping spans within native source groups, and blocks do not cross sources or duplicate events.Source: C. Matching event count and sequence… · p. 3

D. Primary measurements

  • The primary measurements ask how quickly distinct acoustic groups accumulate and how strongly temporal order constrains them through local predictability and repeated subsequences.The sequence measures compare observed order with within-block shuffles of the same events and group frequencies.Source: D. Primary measurements · p. 3, Table II · p. 4
  • The Heaps–Herdan exponent β summarizes the growth of distinct clusters with event count; larger β means new clusters continue to appear more rapidly.The same accumulation convention, event budget, and K are used in every corpus.Source: D. Primary measurements · pp. 3–4
  • Shuffle-adjusted immediate dependence subtracts the within-block shuffled predictive gain from the observed one-step gain, so positive values indicate greater predictive information in observed order.The entropy terms use Miller–Madow finite-sample correction.Source: D. Primary measurements · p. 4
  • RePair, Lempel–Ziv, and DEFLATE compare observed and shuffled sequence compressibility; ratios below one indicate recurring subsequences make observed order more compressible.The three compression readouts are treated together for cross-corpus rank summaries.Source: D. Primary measurements · p. 4
  • Each repertoire receives separate ranks for accumulation, immediate dependence, and mean compression rank; the main text does not use an aggregate rank to define an overall winner.An equal-weight mean-rank summary is retained only as a secondary descriptive sensitivity check.Source: D. Primary measurements · p. 4

E. Within-block shuffle baseline and uncertainty

Within-block shuffles preserve recorded events, group frequencies, and block boundaries while removing observed order. The paper separately examines source uncertainty and validates encoder-derived patterns with physical and continuous acoustic analyses.

E. Within-block shuffle baseline and uncertainty

  • Within-block shuffling preserves recorded events, cluster counts, block boundaries, and marginal frequencies while removing observed order; position-varying acquisition or behavior is tested separately.Type accumulation is unchanged by shuffling.Source: E. Within-block shuffle baseline and… · p. 4
  • Stability checks vary encoder, K, reference draw, coarsening, clustering geometry, RMS normalization, and cluster construction, while source uncertainty is resampled at the highest available native-source block.The source blocks are Whale Date × Unit, Human track, Finch bird, and Marmoset recording day.Source: E. Within-block shuffle baseline and… · p. 4

F. Physical acoustic validation beyond the learned representation

  • Physical-acoustic families are analyzed separately on the matched shell, while continuous analyses examine acoustic coverage and predict the next descriptor vector without clustering.The physical families include temporal, spectral-location/extent, spectral-shape, and acquisition-sensitive level measurements; continuous prediction uses leave-one-native-source-out evaluation and source-by-position-preserving nulls.Source: F. Physical acoustic validation beyond… · pp. 4–5

G. Source, acquisition, and conditioned-order controls

Whale ranks in the top two on all three primary measurements, while Finch leads the two order-sensitive measurements. The section describes source-aware controls and tests whether longer context and coda timing add information.

G. Source, acquisition, and conditioned-order controls

  • Source-aware tests measure between-source separation and within-source cluster association, using restricted permutations and alternative cluster constructions.The analysis also tests whether cluster assignments are distributed unevenly across source blocks while preserving block sizes.Source: G. Source, acquisition, and… · p. 5
  • Conditioned-order nulls preserve source identity and stream position while testing immediate dependence and repeated-subsequence recurrence.Assignments are permuted within source-by-position strata, preserving event slots, boundaries, lengths, occupancy, source-specific inventories, and position-bin counts.Source: G. Source, acquisition, and… · p. 5
  • The short-event sensitivity analysis replaces right-zero padding with symmetric-reflect padding to test boundary-treatment dependence.Events shorter than the 0.175-s encoder window require padding.Source: G. Source, acquisition, and… · p. 6

H. Longer predictive context and same-signal timing validation

  • Held-out order-1 and order-2 models compare prediction from one versus two preceding events, with source-block bootstrap intervals and Holm-adjusted Whale contrasts.Order 3 tests whether one additional preceding event adds predictive value.Source: H. Longer predictive context and… · p. 6
  • Whale is top-two on all three primary measurements, while Finch leads both order-sensitive measurements.The figure reports ranks from strongest (1) to weakest (4).Source: Figure 2 · p. 6
  • The coda timing test re-encodes the same recorded clicks after changing their inter-click-interval arrangement.The perturbations are local jitter, replacement from the global ICI distribution, and within-coda ICI permutation.Source: H. Longer predictive context and… · p. 6

IV. RESULTS

The matched comparison yields distinct repertoire profiles rather than one overall ordering: Whale leads type accumulation, while Finch leads the order-sensitive measurements. Whale and Finch also frequently retain top-two placement across analysis variants.

A. Matched primary measurements reveal different repertoire profiles

  • Whale leads type accumulation (β = 0.124) and ranks 1,2,2, while Finch ranks 3,1,1 and leads immediate dependence (0.737 versus 0.365 Whale) and RePair/LZ recurrence.The three-dimensional profile, rather than an aggregate score, is the principal result.Source: A. Matched primary measurements reveal… · p. 6, Table III · p. 7
  • Whale is top-two in 98.2% of encoder/K/reference cells, and Finch is top-two in 90.5%.Source-block bootstrap supports the Whale Heaps direction in 22/33 settings versus Finch and 26/33 versus Human and Marmoset, and the Finch immediate-dependence direction in 24/33 Whale–Finch settings; DEFLATE remains unresolved.Source: A. Matched primary measurements reveal… · p. 7

B. Physical acoustics recover complementary parts of the profile

Physically interpretable acoustic families recover complementary parts of the encoder-derived profile, and continuous analyses also show broader whole-curve acoustic coverage for Whale. Source-aware and position-conditioned analyses locate variation in Whale’s accumulation and retain Finch’s order advantage.

B. Physical acoustics recover complementary parts of the profile

  • Spectral shape has the highest whole-profile agreement (ρ = 0.923), while temporal acoustics and spectral shape recover Whale’s type-accumulation placement and spectral location/extent best recovers Finch’s order-sensitive placement.Metadata agreement is much lower (ρ = 0.246); acquisition-sensitive agreement is retained because archival signals contain both production-related and recording-related variation.Source: B. Physical acoustics recover… · p. 7, Figure 3 · p. 7
  • Whale has the largest mean continuous-coverage AUC in pooled descriptor space (0.308) and after native-source means are removed (0.482).It is not largest at every reference-set size: at n = 1200, Whale and Human are nearly tied in the pooled analysis, and Human is slightly larger after source-mean removal.Source: B. Physical acoustics recover… · pp. 7–8

C. Source and position analyses locate the remaining variation

  • Whale’s type accumulation is source-structured but not reducible to between-source differences alone.Whale has the strongest simple-acoustic source separation (R2 = 0.543), while within-source simple acoustics remain associated with cluster membership in every repertoire.Source: C. Source and position analyses locate… · p. 8
  • After source and normalized position are preserved, Finch retains the largest first-order excess and recurrence effect, with Whale second on both.Finch exceeds Whale by +0.872 bits/token [0.520, 1.223]; the Finch/Whale recurrence-ratio quotient is 0.935 [0.888, 0.989].Source: C. Source and position analyses locate… · p. 8, Table IV · p. 8
  • Replacing right-zero with symmetric-reflect padding preserves the secondary Whale/Finch ordering (1.419/1.698 versus 1.448/1.682).The sensitivity analysis addresses the asymmetric padding required by short events.Source: C. Source and position analyses locate… · p. 8

D. Temporal scale changes the comparison; coda timing provides a same-signal validation

Adding one preceding event shifts the discrete predictive comparison toward Whale relative to each reference repertoire. A separate same-signal test finds that the Whale profile is sensitive to observed inter-click timing.

D. Temporal scale changes the comparison; coda timing provides a same-signal validation

  • Whale’s order-2 minus order-1 increment is more positive than Finch’s by +0.169 bits/token (95% simultaneous interval [0.064, 0.274]).The corresponding Whale contrasts are +0.172 bits/token versus Human and +0.149 versus Marmoset; all three Holm-adjusted comparisons satisfy p ≤0.00084.Source: D. Temporal scale changes the… · p. 8, Figure 4 · p. 9
  • The context result is comparative, not a positive Whale-alone effect: Whale’s own effect is +0.081 bits/token with interval [-0.006, 0.177] (p = 0.1361).The direction persists in 49/50 train-only cluster constructions while all references are negative in all 50; order 3 adds no supported gain.Source: D. Temporal scale changes the… · p. 8
  • Observed-coda values exceed all three timing-perturbed versions across five readouts for 10/11 encoder configurations, with AVES2 excepted.The same recorded clicks are retained while their inter-click-interval arrangement changes.Source: D. Temporal scale changes the… · p. 8, Figure 4 · p. 9

A. The repertoires differ in profile rather than along a single ordering

Across the four repertoires, acoustic diversity and temporal organization form distinct profiles rather than a single ordering; physical-acoustic, continuous, source, and position analyses refine that comparison.

A. The repertoires differ in profile rather than along a single ordering

  • Whale leads type accumulation, while Finch leads both order-sensitive measures; the repertoires therefore do not share one ordering.Whale is the only repertoire in the top two on all three primary measurements; Finch ranks third in type accumulation.Source: A. The repertoires differ in profile… · p. 9
  • The comparison retains each corpus’s source-defined event scale instead of forcing equal-duration segments or assuming biological homology.Equal event counts and matched block lengths make the opportunity to observe type accumulation and local order comparable.Source: A. The repertoires differ in profile… · p. 9

B. Physical acoustics provide representation-independent counterparts

  • Physical acoustics recover complementary parts of the profile: temporal and spectral shape capture Whale’s type accumulation, while spectral location/extent and shape highlight Finch’s order measures.Acquisition-sensitive measurements also recover much of the pattern, so the recordings retain acquisition as well as production-related variation.Source: B. Physical acoustics provide… · p. 9
  • Without K-means, Whale has the broadest whole-curve continuous acoustic coverage, but continuous next-event prediction favors Whale rather than Finch.Human approaches Whale at the largest reference-set size; categorical recurrence and continuous-vector predictability measure different aspects of organization.Source: B. Physical acoustics provide… · p. 9

C. Source and position refine, but do not erase, the primary profile

  • Whale’s pooled type accumulation is partly distributed across Date × Unit source blocks, but within-block acoustic differences also remain associated with cluster membership.The sampled corpus contains three social units.Source: C. Source and position refine, but do… · p. 10
  • Finch retains the largest source-aware excess first-order dependence and strongest repeated-order recurrence after cluster assignments are randomized within source and normalized-position strata.Position carries real structure, but a fixed positional template or source identity does not explain away Finch’s categorical order effects.Source: C. Source and position refine, but do… · p. 10

D. Temporal scale changes which repertoire stands out

Temporal comparisons depend on the representation and context length: discrete order effects favor Finch, while continuous prediction and the longer-context comparison shift toward Whale. These findings remain bounded by the recorded corpora and event scales studied.

D. Temporal scale changes which repertoire stands out

  • Finch has the strongest discrete immediate dependence and recurrence after source and position are preserved in the null, whereas continuous prediction favors Whale.The order-2 increment is more positive for Whale than for each reference repertoire, but Whale’s standalone interval includes zero and order 3 adds no supported gain.Source: D. Temporal scale changes which… · p. 10
  • Observed-coda values exceed all three within-coda timing perturbations across five readouts for 10 of 11 encoder configurations.This same-signal intervention tests timing sensitivity within Whale and is not available in the same form for the other native event definitions.Source: D. Temporal scale changes which… · p. 10

E. Scope and next tests

  • The comparison covers four recorded corpora at source-defined event scales and does not identify a species-wide ordering independent of acquisition.Proposed next tests use multiple source-defined units within repertoires and acquisition-matched recordings with controlled microphones, distances, environments, and systems.Source: E. Scope and next tests · p. 10

VI. CONCLUSION

  • The conclusion separates acoustic-event diversity, categorical order, continuous acoustic predictability, and longer-context effects rather than treating them as one hierarchy.Whale has the fastest discrete type accumulation and broadest whole-curve continuous coverage; Finch has the strongest categorical immediate dependence and recurrence after source and position are preserved.Source: §VI · p. 10

AUTHOR DECLARATIONS

The authors report no conflicts, describe the ethics basis and AI-assisted editorial work, and state the availability and licensing conditions for research materials.

AUTHOR DECLARATIONS

  • The authors report no conflicts of interest.Source: AUTHOR DECLARATIONS · p. 11
  • This secondary analysis uses publicly released recordings and metadata; approvals for original collections are described by their source publications.Source: AUTHOR DECLARATIONS · p. 11
  • An AI-assisted language tool supported editorial restructuring and refinement, and the authors reviewed and verified the scientific content and final wording.Source: AUTHOR DECLARATIONS · p. 11

DATA AVAILABILITY

  • Derived manifests, results, and scripts are available during peer review and will be archived publicly before publication; audio and embeddings remain subject to source licenses.Source: DATA AVAILABILITY · p. 11
Loading 2609.16612v1…