PaperlayerView exact revision on arXiv

Source-linked AI summary

Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoders

Georgii Aparin, Vadim Popov, Tasnima Sadekova, Assel Yermekova

arXiv:2606.07473v1cs.SDcs.AI

TL;DR

Whisper can produce fluent, coherent transcriptions disconnected from non-speech input, and existing filtering is insufficient. This paper tests raw and SAE representation steering, finding that SAE-based steering consistently reduces hallucinations without fine-tuning.

  • Problem

    The paper asks whether Whisper’s internal representations can detect and mitigate hallucinations, which existing confidence-based filtering fails to reliably prevent.

  • Method

    The authors analyze raw encoder activations and SAE latents for linear separability, then steer either representation space to suppress hallucination-promoting directions.

  • Results

    SAE-based steering consistently outperforms activation steering across evaluated non-speech datasets, reaching 19.88% HR on UrbanSound8K for Whisper large-v3 without fine-tuning.

  • Takeaways & Limitations

    SAE-based steering is a viable fine-tuning-free approach to hallucination mitigation in ASR.

  • Takeaways & Limitations

    Chinese CER degrades under both steering strategies, partly because the SAE treats Chinese speech as out of domain.

Abstract

from arXiv · show

Whisper, a widely adopted ASR model, is known to suffer from hallucinations - coherent transcriptions generated for non-speech audio entirely disconnected from the input. We investigate whether hallucinations can be detected and mitigated through Whisper's internal representations. We extract audio encoder activations and evaluate two representation spaces: raw Whisper activations and Sparse AutoEncoder (SAE) latents. We show that both spaces encode linearly separable hallucination-related information, with discriminative power concentrated in a sparse feature subset and increasing toward deeper encoder layers. We propose two steering strategies: activation-space steering and SAE latent-space steering. SAE-based steering reduces hallucination rate from 72.63% to 14.11% for Whisper small and from 86.88% to 27.33% for Whisper large-v3 on the full non-speech test set, with small WER degradation on speech data, approaching the performance of fine-tuning-based methods.

1. Introduction

Neural ASR has advanced beyond classical approaches but exhibits hallucinations, especially coherent transcripts assigned to non-speech segments. This paper studies Whisper representations and proposes fine-tuning-free steering methods to mitigate them.

  • Background: Modern ASR relies on sophisticated neural networks trained on large speech corpora, following earlier HMM-, transducer-, and hybrid-based approaches.
  • Hallucinations: ASR hallucinations are fluent, coherent outputs disconnected from the input, with this paper focusing on coherent transcripts assigned to non-speech segments.
  • Mitigation approaches: Existing mitigation approaches include input preprocessing, transcription or intermediate-output postprocessing, and fine-tuning neurons associated with hallucinations.
  • Contributions: Hallucination-prone inputs are linearly separable in raw Whisper activations and SAE latents, with sparse discriminative features that strengthen in deeper encoder layers.
  • Contributions: The paper proposes two fine-tuning-free steering strategies and reports consistent hallucination reduction across evaluated non-speech datasets and both model variants.

2. Background

This section introduces Sparse AutoEncoders (SAEs) as tools for decomposing dense activations into sparse latent representations, and activation steering as inference-time intervention on model representations. It also motivates SAE-based steering as a more interpretable, fine-grained alternative to raw activation steering.

  • Sparse AutoEncoders: SAEs decompose dense, polysemantic activation vectors into sparse, human-interpretable latent representations, motivated by the superposition hypothesis.The approach has been applied in NLP, image, and audio domains.
  • Sparse AutoEncoders: An SAE maps an activation h ∈ R^d to a sparse latent representation z ∈ R^m with m ≫ d, then reconstructs the original activation using a linear decoder.This formalization expands the representation into a higher-dimensional latent space while enforcing sparsity.
  • Sparse AutoEncoders: SAE training combines reconstruction loss with a sparsity penalty, whose strength is controlled by λ and commonly uses an L1-based regularizer.The penalty balances reconstruction fidelity against sparse latent activations.
  • Activation Steering: Activation steering adds a scaled behavior-relevant direction to hidden states during inference, while SAE-based steering targets individual sparse latent dimensions for finer-grained control.Contrastive Activation Addition commonly derives steering vectors from differences between mean activations of two contrastive input sets.

3. Methodology · 3.1. Whisper Hallucinations

This section introduces Whisper’s hallucination failure mode, its inference-time filtering signals, and evaluation metrics for detection and hallucination. It also outlines a dual-data strategy for steering hallucination-prone inputs while monitoring ASR quality.

  • 3.1. Whisper Hallucinations: Whisper is a Transformer-based ASR model trained on 680,000 hours of weakly supervised Internet audio.The training data supports broad generalization across languages, domains, and acoustic conditions.
  • 3.1. Whisper Hallucinations: Weak supervision can pair silence, background noise, or music with arbitrary text, causing coherent hallucinations on non-speech audio.These outputs are disconnected from the input’s meaningful speech content.
  • 3.1. Whisper Hallucinations: Whisper’s inference-time filtering uses no speech prob and avg logprob as scalar indicators for suppressing unreliable segments.No speech prob is the probability of the <|nospeech|> token, while avg logprob is the average log-probability of generated tokens.
  • 3.1. Whisper Hallucinations: Detection Rate (DR) measures the fraction of samples classified as speech-containing using thresholds on no speech prob and avg logprob.The metric is defined for both speech and non-speech data.
  • 3.1. Whisper Hallucinations: For non-speech data, Hallucination Rate (HR) equals DR, whereas for speech data, HR equals 1−DR.These definitions represent hallucinated outputs and genuine speech samples incorrectly suppressed, respectively.
  • 3.1. Whisper Hallucinations: Non-speech data supports hallucination analysis and steering, while speech data measures transcription-quality effects using WER for English and CER for Chinese.The dual-data strategy balances hallucination reduction with ASR-performance control.

3.2. Representations

The analysis uses raw Whisper audio-encoder activations and sparse autoencoder latents as complementary representation spaces for studying hallucinations. Activations are temporally pooled, while SAE latents use non-zero average pooling to preserve sparsity and expose disentangled structure.

  • Representation spaces: Two representation spaces are analyzed: raw Whisper audio-encoder activations and sparse latent representations obtained from an SAE trained on those activations.Raw activations directly access model internal states, whereas SAE latents use a higher-dimensional overcomplete basis under a sparsity constraint.
  • Raw activations: Residual-stream activations capture hidden states after each layer’s residual addition and are average-pooled across time into fixed-size audio-segment representations.The activation at layer l is denoted h_l ∈ R^T×d, where T is sequence length and d is hidden dimension.
  • SAE latents: SAEs are trained across all Whisper encoder layers on diverse audio, producing sparse latents that capture acoustic, semantic, and paralinguistic information in a somewhat disentangled manner.The latent representations are denoted z.
  • SAE latents: Non-zero average pooling aggregates SAE latents across time by averaging only active elements, preserving their sparsity structure in fixed-size representations.The mean excludes zero-valued latent elements.

3.3. Classification

The classification analysis tests whether Whisper’s internal representations distinguish hallucination-prone from non-hallucinating inputs. It evaluates separability using classifier probabilities and AUC on diverse non-speech samples labeled by the DR rule.

  • Classification: The classification task tests whether Whisper representations contain information distinguishing hallucination-prone inputs from non-hallucinating ones.This diagnostic assesses the premise that activation-space intervention could reduce hallucination prevalence.
  • Classification: Labels are assigned per diverse non-speech sample according to the DR rule defined in Section 3.1.
  • Classification: AUC Score measures threshold-independent separability between hallucination and non-hallucination classes using classifier-assigned probability distributions.

3.4. Steering

The section presents activation-space and SAE latent-space steering methods for suppressing hallucinations while preserving speech transcription quality. Both methods modify Whisper encoder representations toward non-hallucinating behavior during inference.

  • Activation-space steering: Activation steering adds a scaled contrastive steering vector to the residual stream at a selected encoder layer.The vector is the difference between mean activations from hallucinating and non-hallucinating samples, shifting representations toward the non-hallucinating regime.
  • Activation-space steering: A positive α controls steering toward the non-hallucinating direction, with its magnitude determining intervention strength.The modified residual activation is passed through subsequent Transformer blocks, propagating the intervention through the encoder.
  • SAE latent-space steering: SAE-based steering intervenes on classifier-identified discriminative latent dimensions, then decodes the modified sparse representation back into the residual stream.Positive classifier importance scores identify latent features that increase hallucination probability, and the intervention retains the top-k features by absolute importance.
  • SAE latent-space steering: The multiplicative SAE method scales selected latent dimensions by α, 1/α, or 1 to amplify, suppress, or leave features unchanged.The resulting latent representation is decoded to reconstruct the modified activation for injection into the model.

4. Experimental Setup

Experiments use Whisper small and large-v3 across non-speech data for hallucination analysis and speech data for ASR-quality monitoring. Strict train/test separation, layer-wise classification, and final-layer steering evaluation support assessment of generalization and intervention effects.

  • Models and inference: Experiments evaluate Whisper small and Whisper large-v3 to test whether observed effects generalize across substantially different model scales.Both model variants are included throughout the experimental evaluation.
  • SAE configuration: SAE inference uses Whisper small SAE1 and Whisper large-v3 SAE2 with Batch-Top-k architecture, expansion coefficient 8, and k = 50 active latent dimensions per token.The resulting latent dimensions are m = 6144 for Whisper small and m = 10240 for Whisper large-v3.
  • Datasets: Non-speech data supports hallucination analysis, classifier training, and steering-vector calculation, while speech data monitors ASR quality.Speech transcription quality is evaluated using WER for English and CER for Chinese.
  • Data splits and tuning: All classifiers, steering vectors, and hyperparameters use only non-speech training data, while non-speech test data is reserved for final evaluation of unseen domains.Speech data is monitored concurrently during tuning to detect severe transcription-quality degradation.
  • Classification: Classification uses independently extracted encoder-layer activations, logistic regression with MaxAbsScaler, and stratified 5-fold cross-validation.SAE latent feature coefficients are averaged across folds to identify the most discriminative dimensions for steering.
  • Steering evaluation: Both steering strategies intervene at each model’s final encoder layer, selected as the most discriminative layer, to avoid conflicting multi-layer perturbations.Activation steering tunes α, whereas SAE steering jointly tunes α and the number of intervened latent dimensions k.

5. Experimental Results · 5.1. Classification · 5.2. Steering

Classification results show that hallucination information becomes increasingly linearly separable in deeper Whisper encoder layers and is concentrated in a small subset of SAE features. Steering reduces hallucinations, with SAE-based methods outperforming activation steering across models and non-speech datasets while balancing speech-recognition quality.

  • 5.1.1. Whisper Activations: Classification performance improves with encoder depth, reaching its highest AUC scores in the final layers of raw Whisper activations.This trend holds across train and test datasets for Whisper large-v3.
  • 5.1.2. SAE Representations: SAE representations show the same depth-related AUC improvement and achieve competitive or superior scores on several datasets.These results indicate that sparse representations preserve, and sometimes enhance, discriminative structure from raw activations.
  • 5.1.2. SAE Representations: 50–100 top-k SAE features are sufficient for stable classification performance in both Whisper small and large-v3.Adding more dimensions yields little additional gain, indicating concentration of hallucination information in a small latent subset.
  • 5.1.3. Comparison: Both raw activations and SAE latents achieve above-chance AUC across all datasets and model variants, with a slight SAE advantage for Whisper large-v3.Whisper small produces comparable results in the two representation spaces.
  • 5.2. Steering: Final-layer activations are used to tune steering hyperparameters because they consistently provide the strongest discriminative performance.The resulting hyperparameters are applied uniformly across layers despite possible coefficient-transfer differences.
  • 5.2.1. Whisper Activations: α = 8 provides the best activation-steering balance for Whisper small, whereas α = 2 is sufficient for Whisper large-v3.The tuning balances hallucination reduction against WER and CER degradation.
  • 5.2.2. SAE Representations: α = 3 with top-k = 25 is selected for Whisper small, while α = 5 with top-k = 10 is selected for Whisper large-v3 in SAE additive steering.Small top-k values support the view that hallucination-related signal is concentrated in a limited subset of sparse features.
  • 5.2.3. Comparison: SAE-based steering consistently achieves greater hallucination reduction than activation steering across both models and all non-speech test datasets.For Whisper small, both steering approaches improve English WER, while Whisper large-v3 shows stable or marginally degraded ASR metrics.

5.3. Reduction of Hallucinations

The section compares the proposed encoder-side SAE steering with Calm-Whisper, a decoder-side intervention targeting hallucination-related attention heads. It reports results across hallucination rate and speech recognition metrics for Whisper small and large-v3.

  • Comparison with Calm-Whisper: Calm-Whisper is the closest alternative, targeting hallucinations by masking a small subset of decoder self-attention heads identified on UrbanSound8K.The comparison includes Calm-Whisper results before fine-tuning, obtained by zeroing the identified hallucinatory heads without parameter updates.
  • Intervention location: Encoder-side SAE steering achieves competitive performance with Calm-Whisper despite intervening before decoding rather than modifying decoder behavior.This result suggests that hallucinations are reflected in encoder representations produced for non-speech inputs, not only in decoder-level generation.
  • Evaluation metrics: The evaluation compares activation and SAE-based steering for Whisper small and Whisper large-v3 using hallucination rate on non-speech data and WER or CER on speech data.The unsteered model is the baseline; WER and CER are reported for English and Chinese speech datasets, respectively.
  • Comparison protocol: The Calm-Whisper comparison reports hallucination rate on UrbanSound8K and WER on LibriSpeech test clean and other.Table 5 compares all methods using these task-specific metrics.

6. Conclusion

The paper finds that Whisper’s raw activations and SAE latents encode sparse, linearly separable hallucination information that strengthens in deeper encoder layers. SAE latent-space steering reduces hallucinations without fine-tuning and establishes a viable mitigation approach.

  • Raw activations and SAE latent representations encode linearly separable hallucination-related information, with classification performance improving toward deeper encoder layers.
  • 10–25 SAE latent dimensions suffice for near-optimal classification, indicating that hallucination-relevant information is concentrated in a sparse feature subset.
  • 19.88% HR on UrbanSound8K, 27.05% on WHAM!, and 33.92% on FSD50k are achieved by SAE steering for Whisper large-v3 without model fine-tuning.SAE-based steering consistently outperforms activation steering across evaluated non-speech datasets and approaches Calm-Whisper fine-tuned at 15.51% on UrbanSound8K.
  • SAE-based steering is established as a viable fine-tuning-free approach to hallucination mitigation in ASR.Future work targets multilingual SAE training and multi-layer steering strategies.
Loading 2606.07473v1…