PaperlayerView exact revision on arXiv

Source-linked AI summary

CRAW: Codec Robust Audio Watermarking

David Chernin, Ethan Fetaya

arXiv:2609.03107v1cs.SDcs.CRcs.LG

TL;DR

Post-hoc audio watermarks can fail under neural codecs and denoisers, transformations commonly applied to speech. CRAW combines distortion-aware training, attention-based pooling, perceptual masking, and error correction; it reports state-of-the-art robustness against neural re-synthesis with comparable perceptual fidelity.

  • Problem

    Existing post-hoc watermarking methods can fail under neural codecs and denoisers, despite these transformations being common in speech processing.

    Source: §1 · p. 1

  • Method

    CRAW combines training against realistic distortions with attention-based pooling, inference-time perceptual masking, and error-correcting codes.

    Source: §1 · p. 1

  • Results

    CRAW achieves state-of-the-art robustness against neural re-synthesis, especially codec attacks, while maintaining fidelity comparable to other baselines.

    Source: §6 · p. 7

  • Takeaways & Limitations

    CRAW reports robustness to neural transformations that had limited the practical utility of post-hoc watermarking methods.

    Source: §6 · p. 7

    Inference-time PESQ-gradient masking adds roughly 150 ms of GPU latency per clip, raising encoder-only inference from 21 ms to 171 ms.

    Source: §5.6 · p. 7

Abstract

from arXiv · show

Recent advances in generative speech models have made it increasingly difficult to distinguish authentic from synthetic audio, enabling new forms of fraud and misinformation. Audio watermarking offers a promising defense by embedding an imperceptible signal into generated speech that can later be detected to verify its provenance. However, recent studies have shown that existing post-hoc watermarking methods fail under neural codecs and denoisers, transformations routinely applied during real-world storage, transmission, and processing, severely limiting their practical utility. Here we introduce CRAW, a codec-robust audio watermarking framework that jointly improves robustness against neural re-synthesis while maintaining high perceptual quality. CRAW combines distortion-aware training with an attention-based pooling mechanism, inference-time perceptual mask- ing, and an error-correcting code to recover the fidelity lost during robust training. Experiments demonstrate that CRAW achieves state-of-the-art robustness against neural codecs, denoisers, and vocoders while maintaining perceptual quality comparable to existing post-hoc watermarking methods. The code is available at https://github.com/DavidC1212/craw.

1 Introduction

Existing post-hoc audio watermarks can fail under neural codecs and denoisers, which routinely process speech. CRAW combines robustness-focused training with pooling, masking, and error correction to target neural re-synthesis while preserving perceptual quality.

1 Introduction

  • Audio watermarking embeds an imperceptible signal in generated speech so it can later be detected to verify provenance.Watermarking may occur during generation or post-hoc on an existing sample.Source: §1 · p. 1
  • Neural codecs and denoisers can defeat existing post-hoc methods despite their robustness to traditional distortions, exposing a gap between benchmarks and deployment.Codecs, denoisers, and enhancement models routinely process speech during calls, conferencing, and online delivery.Source: §1 · p. 1
  • CRAW combines neural codec re-synthesis training, attention-based pooling, inference-time perceptual masking, and error-correcting codes to balance robustness and fidelity.Masking removes watermark from spectrogram regions that contribute most to perceptual degradation; error correction compensates for masking-related robustness loss.Source: §1 · p. 1
  • The post-hoc framework is designed to withstand neural codecs, denoisers, vocoders, and other neural re-synthesis attacks.These transformations significantly degrade existing watermarking methods.Source: §1 · p. 1
  • CRAW achieves state-of-the-art robustness across neural and traditional distortions while maintaining perceptual quality comparable to existing watermarking methods.The framework targets neural codecs, denoisers, vocoders, and other neural re-synthesis attacks.Source: §1 · p. 2

2 Related Work

Prior work includes post-hoc methods that embed and detect watermarks after audio creation, as well as in-generation approaches that condition synthesis on a watermark. These approaches differ in how they handle robustness and whether they can watermark existing audio.

2 Related Work

  • Post-hoc methods use varied embedding and detection strategies, including repeated spectrogram messages in TimbreWatermark, waveform signatures in AudioSeal, and invertible networks in WavMark (Chen et al. 2023).AWARE (Pavlović et al. 2025) uses adversarial optimization under a perceptual budget, while WMCodec trains jointly with a neural codec.Source: §2 · p. 2
  • O’Reilly et al. (2025) and RAW-Bench (Özer et al. 2025) report post-hoc methods collapsing under low-bitrate neural codecs, with O’Reilly et al. (2025) also finding collapse under denoisers.The reported codec collapse occurred regardless of embedding scheme.Source: §2 · p. 2
  • In-generation methods embed watermarks during synthesis, trading generality for robustness by construction but requiring generator retraining or fine-tuning and being unable to watermark existing audio.Examples include GROOT, SOLIDO (Li, Liu, and Lin 2025), and Smark (Zhang, Li, and Gu 2025).Source: §2 · p. 2

3 Background

CRAW builds on TimbreWatermark, which embeds a message through a spectrogram-based pipeline and trains extraction to work on clean and distorted audio. Its detector averages per-frame features before decoding, while a distortion layer simulates voice-cloning-style resynthesis.

3 Background

  • TimbreWatermark uses the magnitude spectrogram as the carrier, encodes the message, repeats its features over time, and concatenates them with carrier features and a skip-connected carrier input.The embedder produces a watermarked spectrogram, which is converted back to a waveform using the original phase.Source: §3 · p. 2
  • The extractor produces per-frame features from the received waveform's spectrogram, averages them across time, and passes the pooled feature to a decoder to recover the message.The extraction front end uses the magnitude spectrogram.Source: §3 · p. 2
  • The distortion layer renormalizes the watermarked waveform, converts it to a mel-spectrogram, and reconstructs audio with Griffin-Lim to mimic voice-cloning resynthesis.The same extraction pipeline runs on both clean watermarked audio and its distorted counterpart.Source: §3 · p. 2
  • Training enforces fidelity with waveform MSE and an adversarial loss, while message losses are computed on both distorted and clean branches.The discriminator is trained with its own loss; the sigmoid function is denoted by σ(·).Source: §3 · p. 3

4 Method

CRAW broadens training attacks to improve codec robustness, then combines attention-based pooling, inference-time masking, and error-correcting codes to recover robustness and fidelity. The components are jointly trained except for masking, which is applied only at inference.

4 Method

  • The embedder, detector, Q-Former, and ECC are trained jointly, while masking is applied only afterward at inference.Across the pipeline, broader attacks improve robustness at a fidelity cost; Q-Former pooling and masking recover fidelity, while ECC restores robustness lost to those steps.Source: §4 · p. 3

4.1 Distortion Layer

  • CRAW broadens the training distortion pool to include neural codec resynthesis and classical signal-processing perturbations, supporting generalization to unseen attacks from the same family.The original distortion layer did not train against codec perturbations.Source: §4.1 · p. 3

4.2 Q-Former Pooling

  • Q-Former pooling replaces equal frame averaging with a learnable query that attends over frames, letting the extractor select which timesteps carry the embedding.The original averaging weighs every frame equally, pushing embedding across frames even when it conflicts with reconstruction loss.Source: §4.2 · p. 3, §4 · p. 3

4.3 Inference-Time Masking

  • Inference-time masking uses a PESQ-loss gradient to identify perceptually sensitive spectrogram pixels, then removes watermark signal from the top-k% of those pixels.The masked spectrogram is reconstructed with an inverse STFT; masking is applied after embedding and does not affect training.Source: §4.3 · pp. 3–4

4.4 Error-Correcting Code

  • An ECC encodes the raw message into a longer codeword for embedding and training, then decodes the recovered codeword into the raw message at inference.The code adds redundancy beyond the embedding's existing time repetition to recover robustness degraded by masking.Source: §4.4 · p. 4

5 Experiments

CRAW performs strongly against neural re-synthesis while maintaining high fidelity, with experiments covering component ablations, baseline comparisons, and cross-domain transfer. Its reported codec robustness comes with a fidelity trade-off and added inference latency.

5.1 Experimental Setup

  • Training and the full baseline comparison use LibriSpeech’s train_clean100 split and 2,620-clip test set; ablations and LJSpeech transfer tests use 200-clip subsets.Audio is resampled to 22.05 kHz, and LJSpeech is evaluated without retraining.Source: §5.1 · pp. 4, 6
  • Detection F1 is measured at a threshold calibrated on clean audio for a 1% false-positive rate; fidelity is evaluated with SI-SNR, PESQ, and STOI.F1 uncertainty is estimated by paired bootstrap, whereas fidelity uncertainty uses the standard error of the per-clip mean.Source: §5.1 · pp. 4–6

5.3 Comparison with Existing Watermarking Methods

  • CRAW beats every baseline on nearly every neural re-synthesis attack (best competing codec F1 ≤0.252) and reaches PESQ 3.99 versus baseline scores of 3.17–4.25.Under traditional signal-level distortions, CRAW is comparable in most conditions, with improvements under MP3 compression and median filtering.Source: Table 3 · p. 6, Table 4 · p. 6, §5.3 · p. 7

5.1 Experimental Setup

  • The attack suite spans classical signal processing, three neural codecs, two denoisers, and two vocoders; most neural attacks are held out during training.FACodec is seen during training; TiCodec, EnCodec, HiFi-GAN, Vocos, and denoiser chains are held out.Source: §5.1 · p. 6
  • The baseline comparison uses five watermarking methods, with payload sizes ranging from TimbreWatermark’s 10 bits to AWARE’s 20-bit configuration.AudioSeal and WavMark use 16-bit payloads; WMCodec embeds a 16-bit payload jointly with its neural codec.Source: §5.1 · p. 6

5.2 Component Ablation

  • +DL supplies nearly all robustness gains at a fidelity cost; +QFormer and +Mask recover fidelity, while +ECC regains most codec robustness lost to masking.+Mask raises PESQ to 4.085 versus the TimbreWatermark baseline’s 4.018, but TiCodec F1 falls from 0.948 to 0.684 before +ECC recovers most of the loss across the three codecs.Source: Table 1 · p. 5, Table 2 · p. 5, §5.2 · pp. 6–7

5.4 Ablation Studies

  • Rep3 outperforms the tested Reed–Solomon, LDPC, and uncoded alternatives on codecs, where errors spread roughly uniformly across the codeword; fidelity is comparable across variants.All code variants are tested with k=10% masking.Source: §5.4 · p. 7

5.5 Cross-Domain Generalization

  • Without retraining, CRAW transfers to LJSpeech with PESQ unchanged at 4.005; TiCodec F1 declines modestly from 0.838 to 0.794.LJSpeech is a single-speaker corpus recorded under different conditions from LibriSpeech.Source: §5.5 · p. 7

6 Conclusion

CRAW builds on TimbreWatermark to address robustness gaps against neural re-synthesis, especially neural codecs. It combines robustness-focused training with fidelity-recovery components and reports strong robustness with baseline-comparable fidelity.

6 Conclusion

  • CRAW builds on TimbreWatermark to close its remaining robustness gap to neural re-synthesis, specifically neural codecs.Source: §6 · p. 7
  • CRAW combines distortion-aware training, attention-based pooling, inference-time perceptual masking, and an error-correcting code to recover fidelity lost during robust training.Source: §6 · p. 7
  • CRAW achieves state-of-the-art robustness against neural re-synthesis, notably codec attacks that collapse current methods, while maintaining fidelity comparable to other baselines.Source: §6 · p. 7

S1 Implementation Details and Hyperparameters

The implementation reuses TimbreWatermark’s convolutional audio architecture and STFT front end, with Q-Former pooling and Rep3 coding. Training uses a fixed seed and applies PESQ-gradient masking only at inference.

S1 Implementation Details and Hyperparameters

  • The system reuses TimbreWatermark’s convolutional encoder–embedder–extractor and STFT front end, changing pooling to Q-Former.The carrier encoder, embedder, and extractor have 3, 4, and 6 layers, respectively; the STFT uses a 1024-point FFT and 256-sample hop at 22.05 kHz.Source: S1 Implementation Details and… · p. 9
  • A single learnable query attends over all frames through one multi-head cross-attention block, followed by a feed-forward block, each with a pre-norm residual connection.The block uses 3 heads, 4× feed-forward expansion, and 0.1 dropout.Source: S1 Implementation Details and… · p. 9
  • The 10-bit message is expanded by Rep3 into a 30-bit codeword before embedding.Source: S1 Implementation Details and… · p. 9
  • Training runs for 20 epochs at batch size 1 with Adam and StepLR, using fidelity, distorted-audio message, clean-audio message, and adversarial loss weights of 1.0, 10.0, 0.01, and 0.01.The Adam learning rate is 2 × 10^-5; gradients are clipped to norm 1.0, and the discriminator uses a separate Adam instance.Source: S1 Implementation Details and… · p. 9
  • PESQ-gradient masking at k=10% is inference-only, and all models use the fixed random seed 2022.Experiments run on a single NVIDIA L4 GPU.Source: S1 Implementation Details and… · p. 9

S2 Distortion and Attack Details

The distortion and attack suite combines stochastic training-time augmentations with a broader evaluation suite of signal-processing and neural attacks. FACodec is the only neural codec encountered during training; other codecs and the listed vocoders are unseen.

S2 Distortion and Attack Details

  • Training and evaluation use distinct severities for several attacks, including sampled low-pass cutoffs in training versus a fixed 2 kHz cutoff in evaluation.Training noise uses a sampled standard deviation of 0.01–0.08, whereas evaluation sweeps a fixed-target SNR of 20–40 dB.Source: S2 Distortion and Attack Details · p. 9
  • Training includes identity, additive noise, low-pass filtering, amplitude scaling, mel-spectrogram resynthesis, spectral gating, and FACodec resynthesis.FACodec is sampled 10% of the time, while the other six training attacks each share the remaining 15%.Source: S2 Distortion and Attack Details · p. 9
  • Evaluation covers cropping, resampling, MP3 compression, 8-bit requantization, median filtering, time stretching, and denoising with ClearVoice FRCRN or MossFormerGAN.Cropping is evaluated from 20% to 80% removal, then densely from 90% to 98%; denoising sweeps 0–20 dB SNR.Source: S2 Distortion and Attack Details · p. 9
  • TiCodec and EnCodec are unseen neural codecs at evaluation, and HiFi-GAN and Vocos are unseen neural vocoders.FACodec is the only codec encountered during training.Source: S2 Distortion and Attack Details · p. 9

S3 Masking Threshold Sweep

The masking sweep varies masking ratio while holding Rep3 enabled, measuring robustness under neural and signal-processing attacks and fidelity against a same-size random mask.

S3 Masking Threshold Sweep

  • Rep3 remains enabled throughout the robustness sweeps, with masking ratio as the only varying setting.Source: S3 Masking Threshold Sweep · p. 9
  • Robustness (F1@1%FPR) degrades steadily as masking increases under neural attacks.Source: Figure S1 · p. 10
  • Figure S2 plots robustness (F1@1%FPR) against masking ratio under signal-processing attacks.The supplied passage identifies the plotted relationship but does not state its outcome.Source: Figure S2 · p. 10
  • At equivalent masking ratios, PESQ-gradient masking recovers substantially more fidelity than random masking.The fidelity sweep compares the two masks at the same k.Source: S3 Masking Threshold Sweep · p. 9, Figure S3 · p. 10

S4 Masking Overhead

Table S1 reports the per-clip GPU inference latency added by PESQ-gradient dynamic masking.

S4 Masking Overhead

  • ∼150 ms/clip is added by PESQ-gradient dynamic masking in the full pipeline.The comparison is against the embedder alone.Source: Table S1 · p. 10, S4 Masking Overhead · p. 10
  • Latency was measured on 100 LibriSpeech test clips, 1.82–10.52 seconds long, as mean±SE.Table S1 compares the embedder alone with the full pipeline using 10% masking.Source: Table S1 · p. 10
  • The component-ablation ratio sweep covers five stages, from Baseline through DL, QFormer, Mask, and ECC.Tables S4 and S5 split results between signal-level distortions and neural re-synthesis attacks.Source: S5 Component Ablation: Full Ratio Sweep · p. 10

S6 Baseline Comparison: Full Ratio Sweep

This section provides full ratio sweeps for the main robustness comparison, component ablations, ECC variants, and zero-shot LJSpeech evaluation, alongside fidelity comparisons.

S6 Baseline Comparison: Full Ratio Sweep

  • Separate ECC models use 10% dynamic PESQ masking and compare Rep3, RS(4,2), RS(6,2), LDPC, and no coding; the RS variants correct up to one or two byte errors.LDPC uses a regular rate-1/2 Gallager construction with min-sum belief-propagation decoding.Source: S7 Error-Correcting Code Variants · p. 10
  • Table S2 reports ECC-variant fidelity with 10% dynamic PESQ masking on a 200-clip subset.The comparison is reported as mean±SE.Source: Table S2 · p. 10
  • Table S3 compares CRAW fidelity in-domain on LibriSpeech with zero-shot fidelity on LJSpeech, using 200-clip subsets per domain.The comparison is reported as mean±SE.Source: Table S3 · p. 10
  • Tables S4 and S5 report component-ablation F1±SE ratio sweeps across five stages, split between signal-level distortions and neural re-synthesis attacks.The stages are Baseline→+DL→+QFormer→+Mask→+ECC.Source: Table S4 · p. 11, Table S5 · p. 12
  • Tables S6 and S7 give the main robustness comparison’s full ratio sweep: F1±SE across attacks and all models, split by signal-level and neural re-synthesis attacks.Table S7 marks unseen attacks with †.Source: S6 Baseline Comparison: Full Ratio Sweep · p. 10, Table S6 · p. 13, Table S7 · p. 14
  • Tables S8 and S9 report ECC-variant robustness as F1±SE on a 200-clip subset, split by signal-level and neural re-synthesis attacks.Table S9 marks unseen attacks with †.Source: Table S8 · p. 15, Table S9 · p. 16
  • Tables S10 and S11 report full ratio sweeps of per-attack robustness, comparing in-domain LibriSpeech with zero-shot LJSpeech across signal-level and neural re-synthesis attacks.Both tables report F1±SE on a 200-clip subset per domain.Source: S8 Cross-Domain Generalization: LJSpeech · p. 10, Table S10 · p. 17, Table S11 · p. 18
Loading 2609.03107v1…