PaperlayerView exact revision on arXiv

Source-linked AI summary

Audio Tokens as a Budgeted Resource: Marginal-Utility Allocation for Scalable Audio Representations

Mingyu Zhao, Jinchao Zhang, Zhiyong Wu

arXiv:2609.34337v1eess.AS

TL;DR

Fixed-depth RVQ allocates equal capacity across frames despite varying refinement value. UniAdapt predicts marginal utility on a frozen codec and enforces exact serialized-prefix budgets; it improves reconstruction across audio domains and causal speech results.

  • Problem

    Fixed-depth RVQ allocates equal capacity at every frame even though the value of an added refinement varies, raising where to spend a limited representation budget.

    Source: §1 · p. 1

  • Method

    UniAdapt predicts rate-independent marginal utility on a frozen RVQ hierarchy and allocates depths with a causal primal–dual controller and exact serialized-prefix feasibility checks.

    Source: §1 · p. 2

  • Results

    Across speech, music, and environmental audio, reconstruction improves under no-larger budgets; causally, three of four speech rates improve with zero violations and faster-than-real-time streaming.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    The same frozen hierarchy supports multiple rates and acoustic or semantic objectives without codec retraining, with selective downstream gains.

    Source: §5 · p. 10

    At lower realized bitrates, the MTG-Jamendo tagging and Clotho mean-recall comparisons are not statistically significant and do not establish equivalence or non-inferiority.

    Source: §4.5 · p. 9

Abstract

from arXiv · show

Discrete audio tokens are widely used as a representation interface, yet fixed-depth RVQ tokenizers allocate equal capacity to every frame despite varying refinement value. We introduce UniAdapt, which learns marginal utility of RVQ refinements on a frozen codec and allocates them under exact serialized-bit budgets. A rate-independent causal controller predicts acoustic utility, while an optional semantic head supports speech-only utterance-level allocation; measured acoustic and semantic marginal gains on speech have a correlation of 0.42. For causal allocation, a primal-dual allocator selects prefix-valid depths, while an exact guard constrains each sequence prefix to its matched fixed-depth serialized budget. Under utterance-level allocation, UniAdapt reduces Log-STFT distortion by 1.07-4.39 percent across speech, music, and environmental audio without larger budgets. Causally, it improves three of four speech rates with zero violations across 800 utterance-rate evaluations and runs faster than real time. A 20-listener utterance-level MUSHRA study shows a significant 3.52-point speech improvement, with no significant differences on music or environmental audio. These results support separating utility prediction from budget enforcement for scalable, budget-conditioned audio representations.

1 INTRODUCTION

Fixed-depth RVQ allocates the same number of refinements at every frame, although their value varies. UniAdapt predicts refinement utility and allocates under exact serialized-bit budgets, with improvements reported across audio domains and in causal speech evaluation.

Motivation

  • Fixed-depth RVQ assigns equal capacity across frames even though refinements can be valuable near transitions and redundant in stationary audio.The paper asks where to spend the next refinement under a limited representation budget.Source: §1 · p. 1

Approach

  • UniAdapt predicts rate-independent marginal utility, using acoustic utility for causal allocation and an optional semantic head for speech-only utterance-level allocation.Measured acoustic and HuBERT-based semantic marginal gains on speech correlate at 0.4205 overall, nearing zero or slightly negative values in deeper layers.Source: §1 · p. 2

Results

  • 1.07–4.39%: utterance-level allocation reduces Log-STFT distortion across speech, music, and environmental audio without exceeding matched fixed-depth budgets.Causal allocation improves three of four speech rates, has zero budget violations across 800 utterance–rate evaluations, and runs faster than real time; a 20-listener study finds a significant 3.52-point speech improvement but no significant music or environmental-audio differences.Source: §1 · p. 2

Approach

  • A causal primal–dual allocator chooses prefix-valid depths, while serialized per-prefix checks enforce matched fixed-depth budgets and next-frame minimum-depth feasibility.The exact guard accounts for realized serialized cost rather than relying only on nominal rate.Source: §1 · p. 2

Evaluation

  • The paper evaluates the budget-conditioned frozen-token representation across multiple rates and audio domains, downstream tasks, human listening, and real-time waveform streaming.Experiments keep utterance-level and causal settings separate.Source: §1 · p. 2

2 RELATED WORK

Prior work develops codec architectures, adaptive token use, and semantic-aware representations, but does not combine causal per-frame allocation with exact prefix budgets over a frozen hierarchy. UniAdapt targets that intersection while also studying objective-dependent allocation.

2.1 NEURAL AUDIO CODECS AND ADAPTIVE TOKENIZATION

  • UniAdapt extends adaptive token allocation on frozen RVQ hierarchies to causal, irreversible per-frame decisions under per-prefix budgets, unlike BAMU’s non-causal full-utterance allocation.UniStream supplies the causal universal RVQ hierarchy used here but does not address content-adaptive per-frame depth.Source: §2.1 · p. 3

2.2 SEMANTIC-AWARE AUDIO REPRESENTATIONS

  • Unlike semantic-aware tokenizers that incorporate linguistic information or semantic constraints during codec training, UniAdapt keeps the hierarchy frozen and uses semantic preservation to value existing refinements.The semantic branch is restricted to speech and utterance-level allocation; causal streaming uses acoustic utility only.Source: §2.2 · p. 3

2.3 CAUSAL AND EXACT-BUDGET ALLOCATION

  • Streaming requires each committed prefix to stay within its matched fixed-depth serialized budget while preserving minimum-depth feasibility for the next frame.The target combines causal acoustic-utility prediction with exact prefix enforcement, alongside offline speech allocation over acoustic and semantic objectives.Source: §2.3 · p. 3

3 METHOD

UniAdapt selects active prefixes from a frozen RVQ hierarchy, predicts rate-independent marginal utilities, and allocates depths under exact serialized-budget constraints. Its method separates online causal acoustic allocation from offline speech allocation that can include semantic utility.

3.1 FROZEN HIERARCHY AND PROBLEM FORMULATION

  • The frozen causal codec supplies a shared RVQ hierarchy, and each selected depth defines an active prefix while the encoder, codebooks, routing modules, and decoder remain frozen.The reported real-backbone path evaluates all layers and applies a selected prefix mask; runtime results do not rely on computational early exit.Source: §3.1 · p. 4, §3.4 · p. 6

3.2 RATE-INDEPENDENT ACOUSTIC AND SEMANTIC MARGINAL UTILITY

  • A causal trunk predicts non-negative layer utilities without target-bitrate input; acoustic utility drives causal experiments, while normalized semantic utility is available for speech-only utterance-level allocation.Semantic-head training freezes the shared trunk and acoustic and speech-probability heads; HuBERT is absent from token extraction and deployment.Source: §3.2 · p. 5, §3.5 · p. 6

3.3 CAUSAL EXACT-BUDGET ALLOCATION

  • The online allocator scores candidate depths using a rate-control surrogate and switching controls, then an exact guard checks serialized-prefix affordability and feasibility for a next frame at minimum depth.Exact feasibility takes precedence over soft switching controls; the offline utterance-level branch instead uses full-sequence Lagrange–Viterbi allocation and is not used for streaming.Source: §3.3 · pp. 5–6

3.4 PREFIX REPRESENTATION AND DYNAMIC BITSTREAM

  • The dynamic container stores a header, compressed depth and expert maps, and packed active indices; metadata costs are included when matching budgets.Depth decisions are per latent frame, while shared-expert quantization buffers 75 frames (500 ms); reported streaming timings exclude final byte packing and container parsing.Source: §3.4 · p. 6

3.5 LEARNING OBJECTIVE AND TRAINING PROTOCOL

  • The codec and HuBERT teacher remain frozen, and utilities are trained against non-negative marginal labels using non-padded positions and all K outputs.Semantic-head training starts from the acoustic-controller checkpoint, freezes the shared trunk and other heads, and optimizes only the semantic head.Source: §3.5 · p. 6

4 EXPERIMENTS

Experiments evaluate utterance-level and causal allocation separately, alongside subjective listening, semantic allocation, downstream probes, and an external-codec comparison. Dynamic allocation generally improves reconstruction at matched or lower serialized-bit rates, with results depending on the setting and task.

4.2 UTTERANCE-LEVEL EXACT-BUDGET ALLOCATION

  • 1.07–4.39% lower Log-STFT distortion across speech, music, and environmental audio under utterance-level allocation, without exceeding matched fixed-depth budgets.The comparison uses a three-domain controller; the reported gains span all four rates in each domain.Source: §4.2 · p. 7, Table 1 · p. 7

4.3 CAUSAL ONLINE ALLOCATION AND STREAMING

  • Causal allocation improves Log-STFT at D = 2, 4, and 6, but increases it by 0.55% at D = 3; all four rates use fewer bits with zero prefix-budget violations.The evaluation covers 800 utterance–rate pairs.Source: §4.3 · p. 7, Table 2 · p. 7
  • Removing dwell changes D = 4 Log-STFT improvement from 0.91% to 1.21%, while disabling smoothing controls raises switching from 9.10 to 37.05 transitions/s.Randomly permuted utilities degrade Log-STFT by 1.75–1.80%; approximate side-cost accounting underuses budget by 6.50–12.81%.Source: §4.3 · p. 8

4.4 SUBJECTIVE AND OBJECTIVE-DEPENDENT EVALUATION

  • Speech gains 3.52 MUSHRA points significantly, while music, environmental audio, and overall comparisons are not significant.The study uses utterance-level offline reconstructions and listener-paired confidence intervals.Source: §4.4 · p. 8, Table 3 · p. 8
  • At D = 4, semantic-aware allocation reduces HuBERT distortion by 13.47% at a 0.61% Log-STFT cost, in speech-only utterance-level experiments.Acoustic and signed HuBERT semantic refinement gains correlate at 0.4205; causal streaming uses acoustic utility only.Source: §4.4 · p. 8, Table 4 · p. 9

4.5 SCALABLE FROZEN-TOKEN REPRESENTATIONS

  • Dynamic multi-rate tokens lower CER at all four rates, reduce speaker-probe EER significantly at D = 3, 4, and 6, and improve FSD50K macro-mAP by +0.001633.The FSD50K result uses 1.381% fewer serialized bits; the CER probe is a cross-rate diagnostic, not evidence of unseen-rate generalization.Source: §4.5 · pp. 8–9
  • MTG-Jamendo and Clotho primary-metric intervals include zero, while frozen wav2vec2 WER at D = 4 shows a 6.11% relative reduction whose bootstrap interval reaches zero.The MTG-Jamendo and Clotho comparisons establish neither equivalence nor non-inferiority; the WER result is reported as a trend, not a conclusive improvement.Source: §4.5 · p. 9

4.6 EXTERNAL CODEC POSITIONING

  • At D = 4, EnCodec has higher absolute PESQ/STOI than UniAdapt dynamic, while dynamic allocation improves the frozen UniAdapt backbone at a slightly lower rate.The comparison is contextual, not architecture-controlled, because the systems differ in architecture, sample rate, training objective, and quantization design.Source: §4.6 · p. 9

5 CONCLUSION

UniAdapt treats audio tokens as a budgeted resource, predicting marginal utility to allocate RVQ capacity under serialized-bit budgets. Experiments report improved reconstruction across three audio domains, zero causal budget violations at three of four speech rates, and a significant speech-quality improvement in human evaluation.

5 CONCLUSION

  • UniAdapt learns frame- and layer-wise marginal utilities and allocates RVQ capacity causally under the true serialized-bit budget.The same frozen hierarchy supports multiple rates and acoustic or semantic objectives without codec retraining.Source: §5 · pp. 9–10
  • Across speech, music, and environmental audio, utterance-level reconstruction improves under no-larger serialized budgets.Causal allocation improves three of four speech rates with zero budget violations and runs faster than real time.Source: §5 · p. 10
  • Human evaluation shows a significant speech-quality improvement, while frozen-token experiments show selective downstream gains and support cross-rate use.Source: §5 · p. 10

AI USE STATEMENT

The statement describes AI assistance in code debugging, experiment bookkeeping, figure preparation, and language drafting and editing.

AI USE STATEMENT

  • AI tools assisted with code debugging, experiment bookkeeping, figure preparation, and language drafting and editing.Source: AI USE STATEMENT · p. 10
  • The authors designed the research questions, methods, protocols, and statistical analyses, and reviewed AI-assisted text and code.They also verified reported numerical results and citations.Source: AI USE STATEMENT · p. 10
  • AI tools did not generate experimental measurements or listener ratings; the authors take responsibility for the final paper and reported results.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The appendix documents the codec, utility prediction, allocation, serialized-bit accounting, and streaming implementation. It also reports equivalence and feasibility checks for chunked execution and online budgets.

ETHICS STATEMENT

  • The work uses licensed public datasets and voluntary, informed listeners, collects no sensitive information beyond test responses, and limits speaker verification to representation benchmarking.It does not perform demographic or sensitive-attribute inference.Source: ETHICS STATEMENT · p. 10

A IMPLEMENTATION AND REPRODUCIBILITY DETAILS

  • Implementation details specify a frozen 48-kHz UniStream codec with eight RVQ layers and reversible Top-1/shared-expert serialization; the Appendix also documents configurations and checks.The codec has one shared first layer and seven expert-specific acoustic layers; each active index requires 10 bits.Source: A IMPLEMENTATION AND REPRODUCIBILITY… · p. 13, §A.1 · p. 13

A.2 UTILITY TARGETS AND CONTROLLER

  • Acoustic utility measures nonnegative residual-energy reduction, while HuBERT-derived speech-semantic targets are generated offline; the causal predictor receives no target bitrate.The causal controller uses temporal convolutions and retains eight raw latent frames for chunked inference.Source: §A.2 · pp. 13–14

A.3 EXACT JOINT-UTILITY NORMALIZATION

  • Utterance-level joint allocation separately normalizes acoustic and semantic utilities, whereas causal acoustic allocation uses raw predicted utilities without semantic combination or global normalization.The joint utility is not subsequently renormalized, and the first RVQ layer remains mandatory.Source: §A.3 · pp. 14–15

A.4 ONLINE ALLOCATOR CONFIGURATION

  • The online allocator uses a development-selected speech controller configuration, while an exact prefix-feasibility guard—not the rate bucket alone—enforces the hard serialized-budget constraint.The locked configuration includes transition penalty, hysteresis, dwell threshold, dual step, and a 0.5-second bucket.Source: §A.4 · p. 15

A.5 EXACT SERIALIZED-BIT ACCOUNTING

  • Serialized prefixes include a fixed header, depth and expert run maps, and packed active indices; the prefix-size definition is a feasibility check, not separate packetization.Matched fixed and dynamic prefixes share header and expert-map costs.Source: §A.5 · p. 16

A.6 STREAMING STATE AND CHUNKED INFERENCE

  • Streaming carries controller, allocator, and exact-budget state across chunks; shared-expert processing buffers 75 frames, and reported timings exclude final byte packing and container parsing.The decoder retains eleven latent frames of context; the encoder retains 2560 stride-aligned waveform samples.Source: §A.6 · pp. 16–17

A.7 CORRECTNESS AND EQUIVALENCE CHECKS

  • Chunked allocation matched full causal depth maps, discrete end-to-end quantities matched exactly, and no budget violations occurred across 600 streaming conditions.Encoder latents matched exactly in the tested configuration; decoder outputs agreed within numerical tolerance. The authors note these finite checks do not constitute a standalone proof of feasibility.Source: §A.7 · p. 17

A.8 TRAINING SETS, CHECKPOINTS, AND RANDOM SEEDS

This appendix section identifies controller training configurations, sampling seeds, and checkpoints, then describes the setup and measures used for additional online allocation and ablation results.

A.8 TRAINING SETS, CHECKPOINTS, AND RANDOM SEEDS

  • Speech acoustic, semantic, and three-domain experiments use distinct sampling seeds, and the reported controllers use specified training checkpoints alongside a shared frozen codec checkpoint.The data-sampling seeds are 2030, 2038, and 2040, respectively; the codec checkpoint is step 200,000.Source: §A.8 · pp. 17–18

B ADDITIONAL ALLOCATION AND ABLATION RESULTS

  • Additional allocation experiments use 200 LibriSpeech test-clean utterances, target depth D = 4, a speech-specific acoustic controller, and 500-ms input chunks.Codec parameters and expert assignments remain fixed across allocation conditions.Source: B ADDITIONAL ALLOCATION AND ABLATION… · p. 18
  • Percentage changes average per-utterance relative changes, and Table 9 compares guarded online component ablations at D = 4 against matched fixed-depth coding.Negative values indicate lower distortion or bitrate; the table covers the same 200 test utterances.Source: B ADDITIONAL ALLOCATION AND ABLATION… · p. 18, Table 9 · p. 18

B.1 ONLINE ALLOCATOR COMPONENT ABLATIONS

The ablations examine switching controls, utility inputs, budget enforcement, and online versus offline allocation. They show trade-offs between reconstruction quality, switching, and bitrate utilization, while cautioning that some comparisons change multiple factors.

B.1 ONLINE ALLOCATOR COMPONENT ABLATIONS

  • Disabling switching controls raises transitions from 9.10 to 37.05/s; removing only the transition penalty raises switching to 18.93/s and reduces reconstruction improvement.The controls regulate depth-map temporal complexity and associated side-information cost.Source: §B.1 · pp. 18–19
  • Reducing dwell from five to one improves the Log-STFT change from −0.906% to −1.206%, while switching rises from 9.10 to 9.70/s.On the development set, m = 1 improves reconstruction by 0.3126 percentage points and increases switching by 0.4089/s; the main streaming configuration retains m = 5.Source: §B.1 · p. 19, Table 10 · p. 19
  • Without dual updates, bitrate is 17.21% below the matched reference with almost no reconstruction gain; adding an estimated depth-side charge also underuses budget and worsens reconstruction.The final online configuration retains exact depth-RLE accounting and uses zero estimated depth-side charge.Source: §B.1 · pp. 18–19

B.2 UTILITY-INFORMATION BASELINES

  • Shuffled utility controls degrade Log-STFT by approximately 1.75–1.80%, whereas learned utility improves it by 1.21%; the comparison also disrupts layer correspondence.Uniform utility recovers fixed-depth behavior, and ground-truth latent utility improves the relative Log-STFT change by 0.1830 percentage points, though bitrate and switching also differ.Source: §B.2 · pp. 19–20, Table 11 · p. 19

B.3 EXACT VERSUS APPROXIMATE BUDGET ENFORCEMENT

  • The approximate-side-cost alternative uses 6.50–12.81% less bitrate than the matched reference and has no positive reconstruction point-estimate gain at D = 2, 3, or 4.No final serialized-budget overrun occurs across 800 evaluations, but this comparison changes both guard availability and score and bucket costs.Source: §B.3 · p. 20, Table 12 · p. 20

B.4 ONLINE VERSUS OFFLINE ORACLE

  • At D = 4, replacing learned utility with ground-truth latent utility changes online Log-STFT improvement from 1.2062% to 1.3892%, while the offline latent oracle achieves 4.6748%.The oracle is a proxy, not a certified optimum for waveform Log-STFT distortion.Source: Table 13 · p. 20, §B.4 · p. 21
  • The online–offline gap does not isolate an unavoidable cost of causality because the procedures also differ in lookahead, prefix constraints, switching treatment, and optimization strategy.Improving utility prediction alone recovers only a small part of the observed difference under the tested online allocator.Source: §B.4 · p. 21

C STREAMING AND BITSTREAM ANALYSIS

Streaming tests assess equivalence, runtime, latency, serialization overhead, and exact prefix-budget enforcement. Separate speech analyses show objective-dependent refinement value and a semantic–acoustic trade-off under utterance-level allocation.

C.1 CHUNKED–FULL EQUIVALENCE

  • Across the final 600 rate–chunk–utterance conditions, equivalence and serialized-budget checks all pass; chunked and full execution match discrete maps and indices exactly.Continuous tensors agree within numerical tolerance; decoder checks span 45 conditions.Source: §C.1 · p. 22

C.2 END-TO-END REAL-TIME PERFORMANCE

  • All twelve rate–chunk configurations have mean RTF below one, with ranges of 0.636–0.655 for 100-ms chunks, 0.543–0.575 for 200-ms chunks, and 0.491–0.515 for 500-ms chunks.The reported timing includes online exact-budget accounting but excludes final byte packing and container parsing.Source: §C.2 · p. 22

C.3 ESTIMATED STARTUP LATENCY AND OUTPUT PROCESSING INTERVALS

  • 735.9–745.2 ms estimated startup includes expert-block accumulation, while 234.9–244.4 ms mean output intervals measure processing without input pacing.The output interval is not a network packet interval or the cadence of a real-time source.Source: §C.3 · p. 23

C.4 SERIALIZED SIDE-INFORMATION AND PADDING OVERHEAD

  • At 500-ms chunks, side-information overhead is 1.89–6.25% of the serialized container, or 0.165–0.211 kbps, with its relative share largest at the lowest rate.The shared expert block reduces expert-map variation but requires buffering before its quantized output is finalized.Source: §C.4 · p. 24

C.5 RUNTIME PLACEMENT AND COMPONENT PROFILE

  • CPU allocator placement reduces representative total RTF from approximately 4.47 to 0.502; full RVQ evaluation is the largest measured component in the final profile.The earlier GPU allocator profile had an allocator RTF of approximately 4.04.Source: §C.5 · p. 24

C.6 STREAMING BUDGET CORRECTNESS

  • Every final streaming condition satisfies its matched serialized budget, and chunked and full causal executions match on code indices, expert assignments, depth maps, and serialized sizes.The per-prefix guarantee comes from restricting each committed action to the exactly feasible set.Source: §C.6 · pp. 24–25

D.1 MARGINAL GAINS ACROSS RVQ LAYERS

  • Pooled speech acoustic–semantic marginal gains correlate at 0.4205, but within-layer correlations approach zero in deeper layers, motivating separate objective models.The pooled statistic reflects between-layer variation and does not establish statistical independence or semantic disentanglement.Source: §D.1 · p. 25

D.2 ORACLE AND LEARNED SEMANTIC ALLOCATION

  • At D = 4, learned joint allocation reduces mean HuBERT distortion from 0.125546 to 0.108630, a 13.47% reduction, with a 0.61% Log-STFT increase.The reported learned condition uses β = 0.25 as a moderate trade-off, not a universally optimal setting.Source: §D.2 · pp. 25–26

D.3 FROZEN EXTERNAL ASR

The frozen-token evaluations show task- and operating-point-dependent outcomes rather than uniform downstream gains. External ASR has a favorable WER point estimate, but its reported intervals include zero.

D.3 FROZEN EXTERNAL ASR

  • Joint allocation lowers the WER point estimate by 0.1689 percentage points, or 6.11% relative to fixed depth, while using a lower bitrate; all reported WER-difference intervals include zero.The evaluation uses 500 test-clean utterances and paired speaker-cluster bootstrap resampling.Source: §D.3 · p. 26

E.1 CROSS-RATE CHARACTER RECOGNITION

  • Dynamic and joint tokens reduce CER at all four evaluated rates, but the probe is trained on dynamic tokens at those same rates, so this does not test unseen-rate generalization.The paired CER intervals exclude zero, with improvements in all five seeds at every depth.Source: §E.1 · p. 27, Table 21 · p. 27

E.2 UNSEEN-SPEAKER VERIFICATION

  • Speaker EER improves at D = 3, 4, and 6, while minDCF does not improve uniformly; semantic allocation also shows operating-point-dependent EER and minDCF trade-offs.The reported joint-minus-dynamic EER and minDCF differences each have pointwise intervals excluding zero, but in different directions and at different depths.Source: §E.2 · p. 28, Table 23 · p. 28

E.3 FSD50K ENVIRONMENTAL-SOUND CLASSIFICATION

  • Dynamic FSD50K macro-mAP improves in all five seeds while using 1.381% less serialized bitrate; the supported classification gain is specific to macro-mAP.Micro-mAP improves in four seeds, but its paired interval includes zero.Source: §E.3 · p. 29, Table 24 · p. 29

E.4 ARTIST-DISJOINT MUSIC TAGGING

  • Artist-disjoint MTG-Jamendo macro-mAP improves in four seeds and micro-mAP in all five, but both intervals include zero and do not establish equivalence or non-inferiority.The artist-disjoint split applies to downstream probe training and does not establish that test artists were absent from upstream training.Source: §E.4 · p. 29

E.5 CLOTHO AUDIO–TEXT RETRIEVAL

  • Clotho mean recall has a 4.08% higher point estimate with 0.6002% lower bitrate, but its interval includes zero; A2T R@10 is a secondary finding, not overall retrieval evidence.A2T R@10 is the only individual recall metric whose pointwise interval excludes zero.Source: §E.5 · p. 29

E.6 SUMMARY ACROSS TASKS

Frozen-token results vary by task: some speech and environmental-sound measures improve, while primary music-tagging and retrieval summaries remain inconclusive. The listening study uses utterance-level allocation, and its reported bitrates describe source token sequences rather than cropped playback excerpts.

E.6 SUMMARY ACROSS TASKS

  • Frozen-token outcomes are task dependent: shared multi-rate probes show lower CER and, at D = 3, 4, and 6, lower speaker EER, while FSD50K macro-mAP improves with a jointly trained fixed/dynamic probe.Speaker minDCF shows different trade-offs; primary MTG and Clotho intervals include zero despite positive point estimates and lower bitrates.Source: §E.6 · p. 29

F SUBJECTIVE EVALUATION AND EXTERNAL CODEC COMPARISON

  • The listening study evaluates perceptual changes within the same frozen backbone, while its external codec comparison contextualizes absolute speech reconstruction; both use acoustic utterance-level allocation.The study uses 30 trials across speech, music, and environmental audio, completed by 20 listeners for 2,400 ratings.Source: F SUBJECTIVE EVALUATION AND EXTERNAL… · p. 30, §F.1 · p. 30

F.2 LISTENING-SET BITRATES

  • Listening-set bitrates refer to the source token sequences used to generate stimuli, not separately encoded containers for the cropped playback excerpts.Playback excerpts are cropped after reconstruction.Source: §F.2 · p. 30, Table 27 · p. 30

F.3 MUSHRA RESULTS

In the utterance-level MUSHRA study, the subjective benefit is specific to speech; the overall comparison and other listening domains remain inconclusive. The external codec comparison favors EnCodec on absolute PESQ and STOI, while the evaluation scope does not establish universal downstream gains or state-of-the-art reconstruction.

F.3 MUSHRA RESULTS

  • Speech improves by 3.52 MUSHRA points with an interval excluding zero, while music, environmental audio, and the overall comparison remain inconclusive.The environmental point estimate favors fixed coding; the study’s listener-level inference is conditional on the selected items.Source: §F.3 · pp. 30–31, Table 28 · p. 31

F.4 EXTERNAL ENCODEC COMPARISON

  • Dynamic allocation improves PESQ and STOI over the frozen UniAdapt backbone using 0.5390% less mean serialized bitrate, but EnCodec achieves higher absolute PESQ and STOI.UniAdapt dynamic uses 1.1405% less mean bitrate than EnCodec; the systems differ in architecture and other codec characteristics.Source: §F.4 · p. 31, Table 30 · p. 31

F.5 OVERALL EVALUATION SCOPE

  • The evidence supports content-adaptive allocation under explicit serialized budgets, without establishing universal downstream gains or reconstruction state of the art.Other listening domains and primary music-tagging and retrieval summaries remain statistically inconclusive.Source: §F.5 · p. 32
Loading 2609.34337v1…