Source-linked AI summary
Interpreting and Steering a Text-to-Speech Language Model with Sparse Autoencoders
Nikita Koriagin, Georgii Aparin, Nikita Balagansky, Daniil Gavrilov
TL;DR
TTS language models share a residual stream between text prefixes and generated speech tokens, but the roles of their representations are poorly understood. This paper trains sparse autoencoders with modality-aware interpretation and tests whether recovered features can steer synthesis. The features span linguistic and acoustic properties, and interventions control laughter, speaker-gender cues, and speech rate while preserving content.
Problem
The roles of representations in a TTS language model’s shared text-and-speech residual stream are not well understood.
Method
The paper trains BatchTopK sparse autoencoders and labels features from text context, 1-second speech clips, or both before steering selected SAE latents.
Results
Interpreted SAE features span text, phonetic, acoustic, and prosodic properties, and steering them controls laughter, speaker-gender cues, and speech rate while preserving spoken content.
Takeaways & Limitations
SAE features can function both as interpretable representations and as practical control directions for TTS synthesis.
Takeaways & Limitations
Results are limited to CosyVoice3-0.5B, and labeler and scorer share Gemini, creating a circular-evaluation risk.
Abstract
from arXiv · showhide
Language models increasingly serve as the backbone of text-to-speech (TTS) systems, yet we understand little about the representations they build when text and generated speech tokens share a single residual stream. We train BatchTopK sparse autoencoders on the LM backbone of CosyVoice3 and introduce a modality-aware auto-interp pipeline that labels each feature from where it fires-text-prefix context, 1-second speech clips, or both. The recovered features are interpretable, spanning phonemes, laughter, accent prompts and speaker gender. Steering through the SAE latent space shows these features are causal rather than merely descriptive: targeted interventions raise laughter probability from 0.02 to 0.79, flip perceived speaker gender, and control speech rate while preserving spoken content. SAE features thus serve both as interpretability objects and as control directions for TTS synthesis.
1 Introduction
TTS language models process text prefixes and generated speech tokens in a shared residual stream, but the roles of their learned features are not yet identifiable in a principled way. This work applies sparse autoencoders and modality-aware interpretation to recover and analyze those roles.
- TTS language models may encode linguistic, acoustic, prosodic, or cross-modal information in one mixed residual stream.
- The paper addresses the lack of a principled way to identify which features serve text, speech, or both.
- BatchTopK SAEs are trained on approximately 250M CosyVoice3 residual-stream tokens.
- A modality-aware auto-interp pipeline labels features from prefix-token context, 1-second speech clips, or both.
- Layer-wise modality analysis categorizes features by whether they activate at text positions, speech positions, or both.
2 Background and Related Work
Prior work established sparse autoencoders and automatic interpretation as tools for analyzing neural representations, while speech interpretability work had not yet applied SAEs to a generative TTS language-model backbone.
- Sparse autoencoders decompose dense residual-stream activations into sparse, approximately monosemantic features.
- Automatic interpretation uses language models to describe activation patterns and detection-style scoring to evaluate those labels.
- This paper adapts automatic interpretation from text-only activations to mixed text/audio evidence.
- The authors position this work as the first SAE analysis of a generative TTS LM backbone residual stream.
3 Method
The method trains BatchTopK SAEs on CosyVoice3, converts strongest activations into modality-specific evidence, labels features with modality-aware prompts, evaluates labels on held-out evidence, and steers generation through SAE latents.
- SAE training: BatchTopK SAEs are trained across Qwen2.5-0.5B layers on approximately 250M Emilia tokens, with detailed qualitative analysis at layer 20.
- Evidence construction: Strongest feature activations are converted into text windows, speech clips, or mixed evidence using exact sequence boundaries.
- Feature modality tagging: Features are categorized as audio-modal at speech fraction ≥0.8, text-modal at ≤0.2, and mixed otherwise.
- Automatic labeling: The labeler receives prefix evidence for text-modal features, 1-second clips for audio-modal features, and both evidence types for mixed features.
- Label evaluation: Held-out detection-style evaluation scores proposed labels using shuffled positive and negative evidence and reports AUROC and balanced accuracy.
- SAE feature steering: Steering shifts selected encoded SAE activations by signed strength α, decodes them back to residual states, and leaves input-text and speech-prompt tokens unchanged.
4 Results
The analyses find layer-dependent text, audio, and mixed features, interpretable modality-specific labels, and controllable speech properties produced by steering selected SAE features.
- Layer-wise modality analysis: Middle layers contain mixed text–speech features, layers 16–20 become audio-heavy, and the final hidden state becomes mostly text-modal.
- Auto-interp quality: Text-modal labels reach AUROC 0.921, audio-modal labels 0.653, and mixed features 0.558 in the layer-20 held-out evaluation.
- Qualitative feature analysis: Interpretable features cover individual tokens, phonemes, gender, laughter, stuttering, breaths, accents, and cross-modal word or phoneme events.
- Probe-based feature selection: Acoustic probes identify SAE features predictive of controllable properties including laughter, emotion, and accent.
- Feature steering: Feature 14834 raises laughter probability from 0.015 to 0.791 at α = +60.
- Feature steering: Feature 3024 changes voiced duration from 3.96 s to 10.57 s or 2.75 s while preserving spoken content.
5 Discussion
The discussion identifies modality-specific SAE features as representations of linguistic, acoustic, and prosodic properties, and shows that selected features steer corresponding speech attributes.
- Late-layer SAE features distinguish text-modal linguistic structure, audio-modal acoustic or prosodic properties, and mixed cross-modal structure.
- Feature 14834 increases laughter, feature 11402 shifts speaker-gender cues, and feature 3024 changes speech rate while preserving spoken content.
- Table 2 links three layer-20 features to laughter-like vocal events, speaker-gender cues, and speech-rate variation.
- Figure 4 reports steering effects for laughter, speaker-gender cues, and speech rate over a 40-voice × 10-text prompt grid with ±1 SEM error bars.
6 Conclusion
The paper concludes that SAEs recover interpretable features in a TTS language model and that some features can serve as causal control directions for speech synthesis.
- SAEs recover interpretable text-modal, audio-modal, and mixed features, while modality-aware auto-interp labels them using testable descriptions.
- Steering selected SAE latent activations causally controls perceptual properties of generated speech.
7 Limitations
The reported findings have limited scope because the study uses one small TTS model and includes evaluation and coverage constraints.
- Results come from CosyVoice3-0.5B and may not transfer to larger TTS models.
- The labeler and scorer share the same Gemini model, so systematic hallucinations could inflate detection-style scores.
- Detection-style auto-interp scores cover only the completed subset of layers, whereas modality and reconstruction statistics span the layer sweep.
- Speech-token timestamps at 25 Hz do not localize sub-token 40 ms acoustic onsets.
- Negatives drawn from other features test label specificity but not robustness against representation-neighbor confounds.
B Experimental Protocol Details
The protocol extracts modality-specific activation evidence and performs SAE-latent interventions at speech-token positions while leaving text-side positions unchanged.
- Activation evidence: Top-20 activating token positions are identified from teacher-forced residual-stream activations, with speech positions mapped to centered 1-second audio windows.
- Modality assignment: Features are classified as audio-modal at speech fraction ≥0.8, text-modal at ≤0.2, and mixed otherwise.
- Held-out scoring: Held-out scoring reserves the five strongest activations for labeling and evaluates lower-ranked activations against negatives from other features.
- SAE steering: The hook operates only at speech-token positions during prefill and decoding, leaving instruction, text-prefix, and task-token positions unmodified.
- SAE steering: Steering encodes hooked residual vectors with the frozen SAE, shifts selected coordinates by signed normalized amounts, and decodes them back to residual space.
D Additional Layer-Sweep Details
Across layers, SAE features shift from multimodal and mixed representations toward audio specialization, while the final layer sharply reprojects toward text-modal features. Reconstruction quality is generally higher for text positions than audio positions in the audio-commitment layers, and auto-interp verification is strongest for text-modal labels.
- 12.3% of layer-0 features are text-modal, compared with 45.1% audio-modal and 42.6% mixed features, indicating an already multimodal residual stream.
- 47.3% of features are mixed at layer 12, while audio-modal features dominate layers 16–20 and mixed features fall to 4.1% at layer 20.Audio-modal features reach 76.1% at layer 16, 65.0% at layer 18, and 74.3% at layer 20.
- 83.1% of layer-23 features are text-modal, versus 14.3% audio-modal and 2.6% mixed, after the intermediate audio-commitment regime.Per-sample activations suggest the final residual stream re-projects toward a text-vocabulary-aligned subspace before the output head.
- Overall explained variance declines from 0.97–0.99 at layers 0–8 to 0.82 at layer 20, then rebounds to 0.945 at layer 23.At layer 20, the text–audio reconstruction gap is 0.080; at layer 23, it closes to 0.015.
- Text-modal labels achieve AUROC 0.90–0.94, audio-modal labels 0.65–0.72, and mixed labels 0.53–0.69 across layers.The ordering supports more conservative interpretation of mixed features, which more often combine correlated textual and acoustic properties.
F Representative Features
Representative SAE features capture localized text, phonetic, vocal-event, and accent-related properties, while some mixed features connect corresponding events across transcript and speech. Text-modal labels are easiest to verify, whereas mixed labels are more variable.
- Text-modal features encode prompt-bound attributes such as “British” and “shrill” alongside lexical patterns such as “ang” and four-digit years.Most text-modal features activate on individual BPE tokens, words, punctuation, or short token contexts.
- Text-modal labels are consistently easiest to verify, audio-modal labels are above chance but weaker, and mixed labels are the most variable.
- Audio features span phonemes, phoneme sequences, laughter, screams, breathing, stuttering, breaths, and occasional accent cues.
G Concept Probing Experiments
The probing experiments test whether laughter, emotion, and accent are linearly recoverable from raw residuals and SAE latents, and whether one SAE coordinate carries most of each concept. Concepts become highly decodable early, while top-1 results assess feature-level concentration.
- The experiments probe three speech-style concepts: laughter, emotion, and accent.
- Binary logistic-regression probes compare raw residual vectors h_L ∈ R^896 with SAE latent vectors z_L ∈ R^16,384 from the final speech-prompt token.Scores use stratified 5-fold cross-validation with training-fold-only MaxAbs scaling.
- Accent evaluation uses eleven VCTK 0.92 accent groups, but Welsh and New Zealand are single-speaker buckets that partially conflate accent with speaker identity.
- The top-k analysis selects coordinates by mean absolute probe coefficient and re-fits probes using k ∈ {1, 5, 10, 25, 50, 100}.The k = 1 setting tests whether one SAE feature carries most class-separating signal.
- Top-1 SAE ROC-AUC tests whether a single dictionary coordinate separates each concept from neutral speech, with laughter and emotion peaking around layers 12–16 and accent around layers 8–12.
- Raw-residual probes exceed 0.99 ROC-AUC between layers 4 and 8, while SAE-latent probes closely track them from layer 8 onward.This indicates that the sparse code preserves the relevant speech-style information in dictionary coordinates.