PaperlayerView exact revision on arXiv

Source-linked AI summary

Interpreting and Steering a Text-to-Speech Language Model with Sparse Autoencoders

Nikita Koriagin, Georgii Aparin, Nikita Balagansky, Daniil Gavrilov

arXiv:2606.10029v1cs.LGcs.AIcs.CL

TL;DR

TTS language models mix text prefixes and generated speech tokens, but there is no principled way to identify the roles of their learned features. This paper applies sparse autoencoders and modality-aware labeling, finding interpretable features and steering effects on generated speech.

  • Problem

    For TTS language models that process text prefixes and speech tokens in one stream, there is no principled way to identify which roles learned features serve.

    Source: §1 · p. 1

  • Method

    The paper trains BatchTopK SAEs on CosyVoice3 and uses modality-aware auto-interpretation to label features, then intervenes on selected SAE latents during speech generation.

    Source: §1 · p. 1, §3.6 · p. 2

  • Results

    SAEs yielded text-modal, audio-modal, and mixed features; steering changed laughter, gender cues, and speech rate, raising laughter probability from 0.015 to 0.791.

    Source: §5 · p. 4, §4.4 · p. 4

  • Takeaways & Limitations

    SAE features can serve as interpretability objects and practical control directions for TTS synthesis.

    Source: §6 · p. 4

    Results are limited to CosyVoice3-0.5B and may not transfer to larger TTS models.

    Source: §7 · p. 5

Abstract

from arXiv · show

Language models increasingly serve as the backbone of text-to-speech (TTS) systems, yet we understand little about the representations they build when text and generated speech tokens share a single residual stream. We train BatchTopK sparse autoencoders on the LM backbone of CosyVoice3 and introduce a modality-aware auto-interp pipeline that labels each feature from where it fires-text-prefix context, 1-second speech clips, or both. The recovered features are interpretable, spanning phonemes, laughter, accent prompts and speaker gender. Steering through the SAE latent space shows these features are causal rather than merely descriptive: targeted interventions raise laughter probability from 0.02 to 0.79, flip perceived speaker gender, and control speech rate while preserving spoken content. SAE features thus serve both as interpretability objects and as control directions for TTS synthesis.

1 Introduction

This introduction frames TTS language models as a distinct interpretability challenge because their residual streams mix text-prefix and speech-token representations. It presents BatchTopK SAE training, modality-aware feature labeling, and layer-wise modality analysis as contributions toward identifying what those representations encode.

1 Introduction

  • CosyVoice3 processes a text or instruction prefix followed by discrete 25 Hz speech tokens, but there is no principled way to identify which features serve text, speech, or both.The mixed sequence may encode syntactic and semantic information from text alongside acoustic and prosodic properties of generated speech.Source: §1 · p. 1
  • The first contribution trains BatchTopK sparse autoencoders on CosyVoice3 residual-stream activations from approximately 250M tokens.The paper frames SAE training on a TTS language model as its first contribution.Source: §1 · p. 1
  • The layer-wise feature modality analysis categorizes features by whether they fire on text-position or speech-position tokens, tracing how linguistic and acoustic directions emerge across layers.This is the paper's third stated contribution.Source: §1 · p. 1
  • The modality-aware auto-interp pipeline labels features using text-prefix context, 1-second speech clips, or both, depending on where each feature activates.This adapts automatic interpretation to the model's mixed text–speech sequence.Source: §1 · p. 1

2 Background and Related Work

Prior work uses sparse autoencoders to interpret language-model residual streams and LLM-based protocols to label activation patterns; speech research has explored probing, audio SAEs, and steering. This work adapts automatic interpretation to mixed text/audio evidence and presents the first SAE analysis of a generative TTS LM backbone, to the authors’ knowledge.

SAEs for LM interpretability

Automatic interpretation

  • Bills et al. (2023) describe activation patterns with LLMs, while Paulo et al. (2024) score labels using activating versus nonactivating examples; this work adapts the protocol to mixed text/audio evidence.Source: §2 · p. 1

TTS and speech interpretability

3 Method

The method applies BatchTopK SAEs to CosyVoice3’s LM residual stream, extracts modality-specific evidence for feature labeling and evaluation, and tests whether SAE latent interventions can steer speech generation.

3.1 Model and SAE Training

  • CosyVoice3 (Du et al., 2024) uses Qwen2.5-0.5B (896 hidden units, 28 layers); BatchTopK SAEs (Gao et al., 2024) use d=16,384, k=50 on ≈250M Emilia tokens (He et al., 2024).The authors sweep layers for modality and reconstruction analyses, use layer 20 for detailed qualitative examples, and train with reconstruction, sparsity, and auxiliary dead-feature losses.Source: §3.1 · p. 2

3.2 Evidence Extraction

  • Token boundaries distinguish prefix from speech-token activations, represented respectively by marked text windows and short audio clips centered on active speech tokens.The same auto-interpretation pipeline can therefore use text-only, audio-only, or combined evidence.Source: §3.2 · p. 2

3.3 Feature Modality Tagging

  • Because randomly interleaved text and speech make absolute token position semantically unstable, features are tagged by top-activation speech fraction: audio-modal ≥0.8, text-modal ≤0.2, and mixed otherwise.The tag is based on token-type composition among a feature’s strongest activations.Source: §3.3 · p. 2

3.4 Automatic Labeling

  • Gemini 3.0 Pro (Google DeepMind, 2024) receives text context, 1-second speech clips, or both according to feature modality, and labels a mixed feature only when both evidence types are present.It is prompted to produce one concise sentence describing a consistent property, such as lexical, acoustic, phonetic, prosodic, or cross-modal patterns.Source: §3.4 · p. 2

3.5 Detection-Style Evaluation

  • Held-out detection-style evaluation shuffles positive and negative evidence items and scores label matching, reporting AUROC and balanced accuracy on balanced text, audio, and mixed feature samples.The evaluation adapts a task described by Paulo et al. 2024 to multimodal evidence.Source: §3.5 · p. 2

3.6 SAE Feature Steering

  • During generation, selected speech-token feature activations are shifted in the frozen SAE latent space, decoded to the residual stream, and controlled by α while text and speech-prompt tokens remain unchanged.The intervention uses a signed, normalized shift rather than directly adding a residual vector.Source: §3.6 · p. 2

4 Results

SAE features shift from mixed to audio-dominated and back to mostly text-modal across layers, and their labels are most verifiable for text features. Steering interpreted features changes laughter, speaker-gender cues, and speech rate, with spoken content preserved in the rate intervention.

4.1 Layer Sweep

  • Early and middle layers contained many mixed features, layers 16–20 were audio-dominated, and the final hidden state reverted to a mostly text-modal subspace.The SAE dictionary remained a strong reconstruction model across the TTS backbone.Source: §4.1 · p. 3

4.2 Auto-Interp Quality

  • AUROC was 0.921 for text-modal labels, 0.653 for audio-modal labels, and 0.558 for mixed features in the layer-20 held-out evaluation.The same ordering held across completed layers.Source: §4.2 · p. 3, Figure 2 · p. 3
  • Layer-20 features represented individual tokens, words, years, and voice-prompt attributes, alongside audio cues and mixed text-speech events.Audio features included phonemes, gender, laughter, stuttering, breaths, and accent cues; mixed features linked the same word or phoneme-like event across text and speech.Source: §4.2 · p. 3

4.3 Probe-Based Feature Selection

  • Downstream acoustic probes selected SAE features predictive of controllable speech properties, using steered samples scored with external speech metrics.The metrics included laughter probability, emotion classification, and accent classification; probing details and candidate-selection results are in Appendix G.Source: §4.3 · p. 3

4.4 Feature Steering

  • 0.015 to 0.791: feature 14834 raised laughter probability at α = +60.This is one of three layer-20 steering interventions that changed the corresponding acoustic property.Source: §4.4 · p. 4
  • 0.629 baseline to 0.944 at α = −50 and 0.063 at α = +50: feature 11402 shifted wav2vec2 P(male).The shifts occurred for both male- and female-prompted generations, toward the target gender.Source: §4.4 · p. 4, Figure 3 · p. 4
  • 3.96 s baseline to 10.57 s at α = −50 and 2.75 s at α = +50: feature 3024 changed voiced duration while preserving spoken content.This feature controls speech rate.Source: §4.4 · p. 4

5 Discussion

The modality split suggests that CosyVoice3’s late-layer sparse features encode more than linguistic prefix information: text-modal, audio-modal, and mixed features capture different structures, including cross-modal text-to-speech mapping.

What TTS features encode

  • Text-modal features capture lexical and prompt-side structure, while audio-modal features often correspond to phonetic, acoustic, or prosodic properties.Source: §5 · p. 4
  • Mixed features may reflect cross-modal structure involved in mapping text to speech.Source: §5 · p. 4
  • The steering analysis examines laughter, speaker-gender cues, and speech rate, using Layer-20 SAE features.Figure 4 reports steering effects over a 40-voice × 10-text prompt grid, with error bars showing ±1 SEM.Source: Figure 4 · p. 4, Table 2 · p. 4

6 Conclusion

SAEs recover interpretable text-modal, audio-modal, and mixed features in a TTS language model, while steering experiments show that some features can causally control perceptual properties of generated speech.

6 Conclusion

  • Intervening on some interpreted SAE features causally controls perceptual properties of generated speech, making them practical control directions for TTS synthesis.The conclusion frames SAE features as both interpretability objects and practical control directions.Source: §6 · p. 4
  • SAEs recover interpretable text-modal, audio-modal, and mixed features, which a modality-aware auto-interp pipeline labels with descriptions testable by a detection-style scorer.Source: §6 · p. 4

7 Limitations · Appendix · A Pipeline Diagram

The paper’s findings are limited by their evaluation scope, incomplete auto-interpretability sweep, and measurement and sampling choices.

7 Limitations

  • Results come only from CosyVoice3-0.5B and may not transfer to larger TTS models.Source: §7 · p. 5
  • The labeler and scorer share the same Gemini model, so systematic hallucinations could inflate scores.Source: §7 · p. 5
  • Human evaluation and a scorer-model ablation are needed to address the circular-evaluation risk.Source: §7 · p. 5
  • Modality and reconstruction statistics span layers, but detection-style auto-interp scores cover only the completed subset of layers.Source: §7 · p. 5
  • Speech-token timestamps are sampled at 25 Hz and cannot localize sub-token acoustic onsets at 40 ms resolution.Source: §7 · p. 5
  • Negative examples come from other features, testing label specificity but not robustness to representation-neighbor confounds.Source: §7 · p. 5

B Experimental Protocol Details · C SAE Steering Implementation

The protocol extracts feature evidence from top-activating positions, assigns features to text, speech, or mixed modalities, and scores labels on held-out activations. Steering intervenes in selected SAE coordinates at speech-token positions and decodes the modified latent vector back into the residual stream.

B Experimental Protocol Details

  • For each feature, the protocol finds its top 20 activating positions in teacher-forced sequences and maps speech activations to centered 1-second audio windows, masking padded positions.Sequences mark the text-to-speech transition with a special task-boundary token; instruction, text, and speech lengths are recorded for each sample.Source: B Experimental Protocol Details · p. 6
  • Features are classified by the fraction of top activations in speech: at least 0.8 is audio-modal, at most 0.2 is text-modal, and the rest are mixed.Speech positions are counted from the speech-start position and only before each sample’s valid sequence length.Source: B Experimental Protocol Details · p. 6
  • Label scoring reserves the five strongest activations for labeling and evaluates lower-ranked examples using AUROC and balanced accuracy, with ratings thresholded at 5.The layer-20 balanced comparison uses all 668 mixed features and matched 668-feature text-modal and audio-modal samples; scoring prompts include held-out positives and negatives from other features.Source: B Experimental Protocol Details · p. 6

C SAE Steering Implementation

  • The frozen-SAE hook modifies only speech-token positions, during speech-prompt prefill and at the current generated speech-token position during decoding, leaving instruction, text-prefix, and task-token positions unchanged.The hook is registered at the same layer as the interpreted features.Source: C SAE Steering Implementation · p. 6
  • Steering computes SAE activations and perturbs selected coordinates by signed, feature-wise amounts scaled by steering strength, while setting all unselected feature entries to zero.The sign vector specifies polarity and the feature-wise activation scales set the perturbation amounts.Source: C SAE Steering Implementation · p. 6
  • The modified latent vector is decoded into a residual vector that replaces the original at hooked positions, keeping the intervention local to the SAE feature subspace.The model otherwise follows its normal autoregressive generation procedure.Source: C SAE Steering Implementation · p. 6

D Additional Layer-Sweep Details · E Layer-Wise Auto-Interp Scores

Across the 24-layer sweep, feature modality shifts from early mixed/audio activity to late audio commitment and then a text-modal final state, while reconstruction quality and auto-interpretation scores vary by layer and modality.

D Additional Layer-Sweep Details

  • At layers 0–14, mixed and audio features dominate; layer 0 is 12.3% text-modal, 45.1% audio-modal, and 42.6% mixed, with mixed features peaking at 47.3% at layer 12.Text-modal features remain the minority at 12–33% across this early and middle band.Source: D Additional Layer-Sweep Details · p. 6
  • At layers 16–20, audio-modal features dominate, reaching 76.1% at layer 16, 65.0% at layer 18, and 74.3% at layer 20 as mixed features fall to 4.1%.Mixed features drop from 40.9% at layer 14 to 4.1% at layer 20, marking an audio-commitment zone.Source: D Additional Layer-Sweep Details · pp. 6–7
  • Layer 23 reverses the late-layer pattern: 83.1% of features are text-modal, 14.3% audio-modal, and 2.6% mixed.Its shorter activation sequence leans toward text-prefix positions, suggesting a text-vocabulary-aligned final residual stream.Source: D Additional Layer-Sweep Details · p. 7
  • Overall explained variance stays at 0.97–0.99 across layers 0–8, then declines to a minimum of 0.82 at layer 20.Explained variance is measured per token against the modality-restricted mean on a 5,000-sample sweep.Source: D Additional Layer-Sweep Details · p. 7
  • Text-position activations are generally reconstructed at least as well as audio positions, with text–audio gaps of 0.065 at layer 16 and 0.080 at layer 20.The early-layer exception is layer 4, where audio EV is 0.992 versus text EV of 0.986; audio is generally harder to compress with k = 50 active features.Source: D Additional Layer-Sweep Details · p. 7
  • At layer 23, overall explained variance rebounds to 0.945 and the text/audio gap narrows to 0.015.This sharp rebound breaks the preceding downward trend.Source: D Additional Layer-Sweep Details · p. 7

E Layer-Wise Auto-Interp Scores

  • Across layers, text-modal labels are easiest to verify (AUROC 0.90–0.94), audio-modal labels score 0.65–0.72, and mixed labels vary most (0.53–0.69).This supports cautious interpretation of examples: single-modality features are often captured by one sentence, while mixed features combine correlated textual and acoustic properties.Source: E Layer-Wise Auto-Interp Scores · p. 7

F Representative Features

Representative features capture prompt conditioning and lexical structure in text, acoustic and vocal events in speech, and selected patterns shared across text and audio. Mixed features are often harder to interpret, but some align clearly across modalities.

F Representative Features

  • Text features track both prompt-bound directions and prompt-independent lexical patterns, often firing on individual tokens or short contexts and sometimes describing acoustic events or speaker attributes.Examples include “British,” “shrill,” the substring “ang,” four-digit years, and written laughter, accent, and voicestyle cues.Source: F Representative Features · pp. 7–8
  • Audio features span phonemes and short phoneme sequences, whole-word and non-speech vocal events, and occasional accent cues.Examples include /k/, /if/–/ef/, /ing/, laughter, screams, heavy breathing, stuttering, and breaths.Source: F Representative Features · p. 8
  • Mixed features are often harder and polysemantic, but some link stutters, the word “middle,” or an /ohl/-like sequence across transcript and audio.Punctuation-to-pauselike features are not treated as primary examples unless held-out scores support their labels.Source: F Representative Features · p. 8

G Concept Probing Experiments · H Prompts

The probing experiments measure how well speech-style concepts can be decoded from model activations and whether individual SAE features carry those signals. The prompts specify how features are labeled across text and audio evidence and how their presence is scored.

G Concept Probing Experiments

  • Supervised binary logistic-regression probes assess laughter, emotion, and accent using final speech-prompt-token activations and stratified 5-fold ROC-AUC.They compare raw residuals with SAE latents; data use a shared neutral-speech negative pool and text-prefix control, with concept-specific positive sets. Welsh and New Zealand accent probes use single-speaker buckets, partly confounding accent with speaker identity.Source: G Concept Probing Experiments · pp. 8–9
  • The top-k analysis selects SAE coordinates by mean absolute probe coefficient and refits probes using k ∈ {1, 5, 10, 25, 50, 100}.Coefficients are averaged across cross-validation folds; k = 1 is the monosemanticity test.Source: G Concept Probing Experiments · p. 9

H Prompts

  • The paper reproduces the auto-interpretation labeling prompt and the detection-style scorer prompt.The prompts guide feature description and evaluation using text-token contexts, audio clips, or both.Source: H Prompts · p. 9

H.1 Auto-interp Labeling Prompt

  • The auto-interp prompt asks for a concise, specific property shared by activating evidence, distinguishing text, audio, and mixed features.It treats the relevant audio moment as near the middle of a 1-second clip, marks active text tokens, and provides unrelated audio clips as contrasts.Source: §H.1 · p. 9

H.2 Detection-Style Scorer Prompt

  • The detection-style scorer evaluates whether each text, audio, or combined evidence item exhibits a feature on a 0–10 scale.The prompt defines score bands from no trace (0–2) to unambiguously prominent (9–10) and requires the SCORES line first, with exactly one integer per item.Source: §H.2 · p. 9

G Concept Probing Experiments

  • All three concepts exceed 0.99 ROC-AUC in raw-residual probes by layer 8, while SAE-latent probes closely track them from layer 8 onward.The concepts are laughter, emotion, and accent; raw-residual probes cross 0.99 between layers 4 and 8 and are essentially saturated from layer 8 onward.Source: G Concept Probing Experiments · p. 9, Figure 7 · p. 10
  • Top-1 SAE feature ROC-AUC tests whether one coordinate separates a concept from neutral speech; laughter and emotion peak around L = 12–16, while accent is most localized around L = 8–12.The k = 1 test treats high ROC-AUC as evidence that a single feature carries most of the class-separating signal.Source: G Concept Probing Experiments · p. 9, Figure 8 · p. 10, Table 5 · p. 10
Loading 2606.10029v1…