Source-linked AI summary
MARC: Multi-Bit Watermarking for Autoregressive Audio Generation against Codec Attacks
Liaoran Xu, Weizhi Liu, Zhaoxia Yin
TL;DR
Codec processing can alter retokenized audio tokens, complicating watermark detection and payload recovery, while existing autoregressive methods use separate token-grouping signals and mainly support zero-bit detection. MARC combines intrinsic token structure with codec-induced confusion for multi-bit embedding and recovery; it reports 97.3% average bit extraction accuracy on unmodified audio and robustness to codec and overwriting attacks.
Problem
Codec processing can change recovered token sequences, and existing autoregressive watermarking mainly targets zero-bit detection while modeling intrinsic token structure and transformation-induced confusion separately.
Source: §1 · p. 1Method
MARC combines intrinsic token representations with retokenization- and codec-induced confusion in codec-aware clusters, using payload-driven scheduling and retokenized observations for multi-bit watermarking.
Source: §1 · p. 2Results
97.3% average bit extraction accuracy on unmodified watermarked audio, with extraction under diverse codec attacks and reported robustness to overwriting.
Source: §1 · p. 2Takeaways & Limitations
Across speech, dialogue, and music generation, MARC extends token-level watermarking beyond presence detection to payload recovery under evaluated codec and overwriting attacks.
Source: §6 · p. 9MARC focuses on multi-bit verification and recovery under token mismatch and temporal misalignment, with codec-induced retokenization mismatch as the primary challenge.
Source: §3 · p. 3
Abstract
from arXiv · showhide
Generated audio is now used in a range of applications, creating a need to verify its origin after distribution and signal processing. This task is particularly challenging for autoregressive audio generation because codec processing can alter the token sequence recovered from the waveform. Such changes reduce the reliability of watermark detection and payload decoding. Existing methods construct token groups using either intrinsic token representations or substitution patterns caused by transformations. As a result, intrinsic token relationships and codec induced substitutions are modeled separately. In addition, most methods support only zero bit detection. They can determine whether a watermark is present but cannot distinguish individual generated outputs. We propose \textbf{MARC}, a multi-bit generative watermarking method for autoregressive audio generation. MARC integrates intrinsic token representations with confusion patterns obtained through retokenization and multiple codecs, forming a codec-aware token-cluster space. Within this space, payload-driven cluster scheduling is used to embed a multi-bit watermark, while detection and payload decoding are performed on retokenized observations. Experiments on speech, dialogue, and music generation show that MARC achieves an average of 97.3\% bit extraction accuracy on unmodified watermarked audio and the watermark can still be extracted under diverse codec attacks. MARC also demonstrates robustness to overwriting attacks.
1 INTRODUCTION
MARC addresses codec-driven token changes and the limits of zero-bit watermarking by combining token structure with codec confusion for multi-bit recovery. It reports 97.3% average bit extraction accuracy on unmodified watermarked audio and robustness under codec and overwriting attacks.
1 INTRODUCTION
- Codec processing can change the token sequence recovered from generated audio, making watermark detection and payload decoding less reliable after distribution and signal processing.Multi-bit watermarking adds information that can distinguish and trace individual outputs.Source: §1 · p. 1
- Existing token-grouping methods use either intrinsic token representations or transformation-induced confusion, while autoregressive watermarking mainly supports zero-bit detection rather than distinguishing outputs.Codec processing and retokenization can corrupt embedded bits, making multi-bit extension difficult.Source: §1 · pp. 1–2
- 97.3% average bit extraction accuracy on unmodified watermarked audio, with extraction still possible under diverse codec attacks across speech, dialogue, and music generation.The paper also reports robustness to overwriting, preserving detection and recovery of the original payload when another watermark is embedded in released audio.Source: §1 · p. 2
- MARC combines intrinsic token representations with retokenization- and codec-induced confusion patterns to form codec-aware token clusters for payload-driven multi-bit embedding and recovery.Detection and decoding operate on retokenized observations.Source: §1 · p. 2
- MARC preserves detection and recovery of the original payload when an additional watermark is embedded into released audio.The paper reports this as robustness to watermark overwriting.Source: §1 · p. 2
2 RELATED WORK
Prior audio watermarking methods use continuous or model-specific representations, while autoregressive methods must operate on tokens and remain detectable after waveform retokenization. Existing autoregressive approaches use different token-grouping signals and mainly target zero-bit detection.
2 RELATED WORK
- Diffusion, latent, noise, model-parameter, and diffusion-modulation watermarking approaches are not directly applicable to discrete token sampling in autoregressive generation.Autoregressive audio watermarking operates on token distributions and must remain detectable after waveform retokenization.Source: §2 · p. 2
- Aligned-IS (Wu et al., 2025b) groups tokens by intrinsic representation geometry, whereas HiPT groups them using transformation-aware confusion; neither jointly models both sources.TraceableSpeech generates watermarked speech within TTS, and Wu et al. (2025a) identify retokenization mismatch in statistical token-level watermarking.Source: §2 · p. 3
- Joint modeling of token structure and codec-aware confusion with robust multi-bit recovery remains insufficiently addressed in autoregressive audio watermarking.The paper positions its method as combining these token-information sources and extending watermarking to multi-bit embedding and recovery.Source: §2 · p. 3
3 THREAT MODEL
The threat model considers released, multi-bit-watermarked audio transformed before verification, with an adversary seeking missed detection or incorrect recovery while preserving perceptual usability. It covers codec, signal-processing, and desynchronization attacks that can alter tokens and their temporal correspondence.
3 THREAT MODEL
- The protocol uses a public check value and secret auxiliary suffix for message consistency checking, without assuming either has cryptographic meaning.The adversary does not directly receive the secret suffix or control the verifier’s payload hypothesis.Source: §3 · p. 3
- The adversary aims to cause missed detection or incorrect payload recovery while preserving underlying audio content and perceptual usability; the setting also includes benign post-processing.Verification retokenizes the transformed waveform, and the verifier searches candidate alignments before estimating the payload.Source: §3 · p. 3
- The considered attacks include neural codecs, signal processing, and desynchronization, which can change token identities and their timing relative to embedding steps.Test-time codecs may be unseen during construction of the codec-aware cluster map; codec-induced retokenization mismatch is the primary challenge.Source: §3 · p. 3
4 MARC: CODEC-ROBUST MULTI-BIT WATERMARKING
MARC addresses codec-induced token changes and alignment errors by combining token relationships into codec-aware clusters, then using them for multi-bit watermarking and retokenized verification.
4 MARC: CODEC-ROBUST MULTI-BIT WATERMARKING
- Codec changes can disrupt token identity and alignment; MARC clusters codec-swapped tokens together and uses local shifts to realign frames.This addresses the challenge that multi-bit recovery depends on alignment with embedding positions.Source: §4 · p. 3
4.1 CODEC-AWARE CLUSTERING
- Offline clustering fuses embedding similarity with normalized reconstruction and codec confusion, then spectrally clusters the affinity into a balanced 20-cluster token map.Codec confusion counts pair tokens at matching indices, and the cluster map uses balanced assignment to control cluster cardinalities.Source: §4.1 · pp. 4–5
4.2 WATERMARK EMBEDDING
- A 12-bit payload is split into three symbols scheduled for eight consecutive steps each, while keyed target-cluster selection and reweighting guide generation.The schedule cycles through symbol positions across 16 laps; reweighting changes cluster probabilities while preserving relative probabilities within each group.Source: §4.2 · p. 5
4.3 EXTRACTION AND DETECTION
- Verification retokenizes waveform variants, searches windows and offsets, thresholds the strongest alignment for detection, and decodes offsets individually before key-checked aggregation.The decoder combines candidate payloads by bit-wise majority voting, then refines the vote using pooled symbol scores and the public check value.Source: §4.3 · pp. 5–6
5 EVALUATION
MARC is evaluated across speech, dialogue, and music generation for watermark reliability, fidelity, capacity, codec robustness, overwriting resistance, and ablation behavior.
5.1 EXPERIMENTAL SETUP
- Experiments use SpiritLM for dialogue and speech synthesis and MusicGen for music, compare MARC with three baselines, and assess detection, payload recovery, and task-appropriate fidelity metrics.Detection uses bit accuracy and TPR at 1% FPR; NISQA and DNSMOS are not used for music outputs.Source: §5.1 · p. 6
5.2 RELIABILITY AND FIDELITY
- Across three SpiritLM datasets, MARC outperforms existing approaches in clustering p-values and detectability, with high multi-bit detection and extraction rates despite fidelity trade-offs.The reported fidelity results also show good audio quality and advantages over existing approaches in dialogue, speech synthesis, and music generation.Source: §5.2 · pp. 6–7, Table 1 · p. 7
5.3 MULTI-BIT CAPACITY
- As capacity increases, FAD rises slightly and detection score generally increases, with a decrease at 10 bits; 12 bits was selected to preserve observations per symbol and robustness.Figure 3 reports fidelity and reliability across capacities for SpiritLM on FinanceQA.Source: Figure 3 · p. 7, §5.3 · p. 8
5.4 ROBUSTNESS
- MARC outperforms all baselines on the reported metric under nine codec and neural-regeneration attacks on both SpiritLM and MusicGen, and preserves original-payload recovery under overwriting.The original owner and attacker can each recover their message using their respective private keys under the evaluated overwriting conditions.Source: §5.4 · p. 8
5.5 ABLATION STUDY
- Codec-union clustering improves post-codec recovery over geometric and retokenization-based alternatives, while multi-lap organization, burst scheduling, and structured decoding contribute to robustness.Removing the lap pipeline and HiPT search causes the largest performance drop; the complete configuration has the best overall trade-off between clean extraction and codec robustness.Source: §5.5 · pp. 8–9
6 CONCLUSION
MARC combines codec-aware clustering, payload-driven scheduling, and alignment-aware verification for multi-bit watermarking in autoregressive audio. Experiments report reliable recovery under evaluated codec and overwriting attacks while maintaining audio fidelity.
6 CONCLUSION
- MARC integrates codec-aware token clustering, payload-driven scheduling, and alignment-aware verification for multi-bit watermarking.The framework is evaluated across speech, dialogue, and music generation.Source: §6 · p. 9
- Experiments demonstrate reliable payload recovery under the evaluated codec and overwriting attacks while maintaining audio fidelity.Source: §6 · p. 9
- MARC extends token-level watermarking beyond detecting watermark presence to recovering multi-bit payloads.Source: §6 · p. 9
A.1 ABLATION STUDY
The ablations examine MARC’s key verification, codec-aware clustering, payload scheduling, and decoding components. The full configurations show stronger recovery in the reported comparisons, including under codec attacks.
Key Verification Ablation
- With both keys correct, MARC reaches 0.98 bit acc; keeping only the pub-key correct yields 0.66, while keeping only the pri-key correct yields 0.78.The results support complementary roles: pri-key scoring assesses candidates, while the pub-key guides refinement and verification.Source: §A.1 · p. 12, Table 4 · p. 12
Clustering Ablation
- C0 achieves 97.77% mean bit acc across eight codec attacks, ahead of C1 at 97.24%, C2 at 97.41%, and C3 at 97.06%.On clean audio, C0 also has the highest bit acc (98.17%) and lowest FAD (0.2622) among the evaluated variants.Source: §A.1 · p. 12, Table 5 · p. 13
Encoding Ablation
- The full schedule reaches 97.77% mean post-codec bit acc, versus 97.24% without codec union, 94.71% with one lap, 96.42% with unit bursts, and 92.41% with round-robin scheduling.It outperforms E2–E4 in bit acc and TPR@FPR=1% under every evaluated codec attack.Source: §A.1 · p. 13, Table 6 · p. 14
Unit-Burst Comparison
- The full configuration recovers more accurately than unit bursts under all eight codec attacks, including 99.25% versus 96.50% bit acc under EnCodec.E3 has 99.42% bit acc on clean audio, while the full configuration has the lowest clean-audio FAD among the evaluated encoding variants.Source: §A.1 · p. 13
Decoding Ablation
- Removing the lap pipeline and HiPT grid search lowers mean post-codec bit acc from 97.77% to 74.57%; removing boundary sliding lowers it to 94.84%.With SoCodec, the full decoder reaches 95.50% recovery versus 64.67% for D1; decoder variants retain similar cluster evidence despite different payload recovery.Source: §A.1 · pp. 13–14
A.2 SUPPLEMENTARY ROBUSTNESS EXPERIMENTS
The supplementary experiments test recovery under additional codec settings, noise, temporal shifts, and overwriting. MARC maintains high reported payload accuracy across the evaluated codec and waveform perturbations.
Attack settings
- The evaluation probes conventional processing and codec-induced token and alignment changes, as well as overwriting that tests preservation of the original message against competing attribution claims.The supplied passage describes these experiments’ aims, not their outcomes.Source: §A.2 · pp. 14–15
Supplementary robustness evaluation
- After excluding duplicate codec configurations, MARC has the highest −log10(cluster p) among compared methods and maintains 94.58%–99.25% bit acc across the supplementary settings.The reported tables cover FinanceQA, LibriSpeech, and Dolly.Source: §A.2 · p. 15, Table 8 · p. 16, Table 9 · p. 16
Codec robustness
- Across evaluated codec configurations and three datasets, MARC maintains 96.00%–99.25% bit acc and 84.00%–97.00% TPR@FPR=1%.At 3 kbps EnCodec, bit acc is 97.83% on FinanceQA, 97.83% on LibriSpeech, and 97.58% on Dolly; MARC also exceeds HiPT’s −log10(cluster p) in these settings.Source: §A.2 · pp. 16–17
Robustness to noise and temporal offsets
- With Gaussian noise at standard deviation 0.01, MARC achieves 96.75% bit acc on FinanceQA, 96.75% on LibriSpeech, and 96.58% on Dolly.It has the highest −log10(cluster p) on all three datasets in this condition.Source: §A.2 · p. 17
- Under a half-frame shift, MARC achieves 94.58%, 95.00%, and 95.17% bit acc on FinanceQA, LibriSpeech, and Dolly, respectively.Its cluster-evidence scores exceed those of all baselines in each case.Source: §A.2 · p. 17
A.3 ATTACK SETTINGS
The attack suite independently perturbs 16 kHz watermarked audio across codec/regeneration, signal-processing, and desynchronization conditions, assessing both watermark evidence and payload recovery. MARC's recovery design targets token substitutions and incomplete or misaligned observations.
A.3 ATTACK SETTINGS
- Attacks are applied independently to 16 kHz watermarked audio, and evaluation measures both watermark evidence and payload recovery.Source: §A.3 · p. 17
- Signal-processing conditions include Gaussian and pink noise, 1/3-kHz filtering, smoothing, echo, and 64-level quantization.Source: §A.3 · pp. 17–18
- Cluster-level encoding can preserve symbols through within-cluster substitutions; repeated observations, alignment handling, and cross-lap aggregation support recovery from corruption.Source: §A.3 · pp. 18–19
- Desynchronization tests include speed changes, first-half cropping, 2.5/10 ms shifts, and onset muting, probing alignment and partial evidence.Source: §A.3 · pp. 18–19
- The suite spans codec/regeneration, signal-processing, and desynchronization attacks targeting token consistency, acoustic evidence, and temporal alignment.Source: §A.3 · pp. 17–19