Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?

Anton Korznikov; Andrey Galichin; Alexey Dontsov; Oleg Rogov; Ivan Oseledets; Elena Tutubalina

Source-linked AI summary

Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?

Anton Korznikov, Andrey Galichin, Alexey Dontsov, Oleg Rogov, Ivan Oseledets, Elena Tutubalina

arXiv:2602.14111v1Feb 15, 2026cs.LG

TL;DR

Whether sparse autoencoders (SAEs) recover meaningful internal features remains uncertain despite strong reconstruction and standard evaluation results. This paper tests them against synthetic ground truth and random baselines, finding weak feature recovery and parity between random and fully trained SAEs across evaluations.

Problem
It remains unclear whether SAE reconstruction and standard evaluations demonstrate recovery of meaningful feature decompositions.
Method
The paper combines synthetic experiments with known ground-truth features and random-direction or random-activation baselines evaluated on real LLM activations.
Results
SAEs recover 9% of true features at 71% explained variance, while random baselines match fully trained SAEs across interpretability, sparse probing, and causal editing.
Takeaways & Limitations
These findings suggest current evaluation practices may be insufficient and that reconstruction may be a poor proxy for meaningful decomposition.
Takeaways & Limitations
The study assumes independent synthetic feature activations and evaluates standard SAEs, not transcoders or crosscoders.

Abstract

from arXiv · show

Sparse Autoencoders (SAEs) have emerged as a promising tool for interpreting neural networks by decomposing their activations into sparse sets of human-interpretable features. Recent work has introduced multiple SAE variants and successfully scaled them to frontier models. Despite much excitement, a growing number of negative results in downstream tasks casts doubt on whether SAEs recover meaningful features. To directly investigate this, we perform two complementary evaluations. On a synthetic setup with known ground-truth features, we demonstrate that SAEs recover only $9\%$ of true features despite achieving $71\%$ explained variance, showing that they fail at their core task even when reconstruction is strong. To evaluate SAEs on real activations, we introduce three baselines that constrain SAE feature directions or their activation patterns to random values. Through extensive experiments across multiple SAE architectures, we show that our baselines match fully-trained SAEs in interpretability (0.87 vs 0.90), sparse probing (0.69 vs 0.72), and causal editing (0.73 vs 0.72). Together, these results suggest that SAEs in their current state do not reliably decompose models' internal mechanisms.

1. Introduction

This introduction argues that SAEs may not reliably recover meaningful feature decompositions despite strong reconstruction, motivating systematic tests against synthetic ground truth and random baselines. Across both settings, the paper finds substantial failures and parity between simple baselines and fully trained SAEs.

Motivation: SAEs aim to decompose dense model activations into sparse, human-interpretable features, but whether they recover meaningful decompositions remains unclear.The motivation is strengthened by the growing importance of interpreting increasingly capable and deployed language models.
Synthetic evaluation: 9% of ground-truth features were recovered despite 71% explained variance, showing a disconnect between reconstruction fidelity and feature recovery.The SAEs recovered only the highest-frequency features in the synthetic experiment.
Results: 0.87 vs 0.90 interpretability, 0.69 vs 0.72 sparse probing, and 0.73 vs 0.72 causal editing show that the baselines match fully-trained SAEs.These comparisons were conducted across multiple SAE architectures.
Implications: Together, the synthetic failures and baseline parity challenge the premise that current SAEs reliably learn meaningful features.The findings also suggest that current evaluation practices may be insufficient.
Real-activation evaluation: The paper introduces three easy-to-implement baselines that constrain SAE feature directions or activation patterns to random values for evaluating real activations.The baselines include Frozen Decoder, Soft-Frozen Decoder, and Frozen Encoder SAEs.

2. Background

Sparse autoencoders (SAEs) are designed to decompose overcomplete neural activations into sparse, interpretable feature directions, addressing polysemanticity under the superposition hypothesis. Their reconstruction-and-sparsity objective has motivated broad applications and scaling, despite growing evidence that the underlying decomposition may be unreliable.

Model Architecture and Decomposition: SAEs address polysemanticity by representing activations as sparse combinations of more feature directions than the activation dimensionality.This approach is motivated by the superposition hypothesis, which posits that networks encode more features than they have dimensions.
Model Architecture and Decomposition: The decoder vectors dj estimate underlying feature directions fj, while a sparsity-inducing encoder produces sparse activations z and reconstructs x as x̂ = Wdecz + bdec.The encoder may use ReLU, and the decoder and encoder biases are learned.
Model Architecture and Decomposition: Expansion factors k = m/n > 1, including k ∈ {16, 32, 64}, let SAEs learn overcomplete dictionaries with more features than the original dimensionality.The overcomplete representation is intended to capture feature directions in activation space.
Training Objective: SAEs minimize reconstruction loss while penalizing latent activity, with variants using L0 constraints or adaptive thresholds to enforce sparsity.The standard objective combines mean squared error with an L1 sparsity penalty, balancing reconstruction against sparsity through λ.
Related Work and Motivation: SAEs have progressed from interpretable features in small transformers to deployments on frontier and open-source models, while critical work questions whether they faithfully represent model computations.This tension motivates testing whether SAEs learn meaningful feature decompositions or merely optimize reconstruction metrics.

3. Case Study #1: Toy Model Experiments

In synthetic experiments with known ground-truth features, SAEs achieved strong reconstruction but recovered very few true features, especially under realistic heavy-tailed activation frequencies.

Synthetic setup: The toy model generated 100-dimensional activations from an overcomplete dictionary of 3200 ground-truth features, with 20 active features expected per sample.Feature activity followed either constant probability pi = 0.00625 or variable probability pi ∼ Log-Uniform(10^-5.5, 10^-1.2).
SAE variants: Both BatchTopK and JumpReLU used dictionary size 3200 and target L0 = 20, matching the synthetic ground truth.These were the two evaluated state-of-the-art SAE architectures.
Conclusion: The experiments show that SAEs fail at their core objective despite strong reconstruction metrics.The synthetic setting directly tests recovery against known generative features.
Constant probability: In the constant-probability setting, both architectures achieved explained variance ≈0.67 but recovered almost none of the ground-truth features.Only 3 out of 3200 true features exceeded cosine similarity 0.8 for BatchTopK and JumpReLU.
Variable probability: In the variable-probability setting, both architectures achieved explained variance = 0.71 while recovering only the most highly activated features.JumpReLU recovered about 7% (225 out of 3200), while BatchTopK recovered about 9% (297 out of 3200).

4. Case Study #2: Validating SAEs on LLMs

The study evaluates whether SAEs recover genuine features in real LLM activations, where ground truth is unknown, by comparing established proxy metrics against randomized frozen-component baselines. It tests this question across three SAE architectures.

Evaluation setup: The evaluation targets the assumption that reconstruction fidelity, latent interpretability, sparse probing, and causal editing indicate recovery of true model features.The study directly tests this assumption on real LLM activations, where ground truth is unavailable.
Evaluation setup: Three baselines use key SAE components that are randomly initialized and frozen to test whether SAEs decompose internal mechanisms rather than exploit spurious data correlations.The baselines are described as easy to implement and are compared against standard SAEs.
SAE variants and baselines: The comparison covers BatchTopK, JumpReLU, and traditional ReLU SAEs.BatchTopK and JumpReLU are identified as state-of-the-art variants, while ReLU is described as traditional.

1. Frozen Decoder: Decoder vectors Wdec

This section evaluates SAEs with decoder vectors that are randomly initialized and frozen throughout training. It tests performance when latent representations are fixed to random vectors.

Frozen Decoder: Decoder vectors Wdec: Decoder vectors are randomly initialized and remain frozen throughout training.This fixes the latent representations to random vectors during SAE training.
Frozen Decoder: Decoder vectors Wdec: The setup tests how well SAEs perform when latent representations are fixed to random vectors.

2. Soft-Frozen Decoder: Decoder vectors Wdec

The soft-frozen decoder baseline randomly initializes decoder vectors and constrains them to remain within cosine similarity τ of their initial values during training. This tests whether early loss plateaus and near-initial decoder vectors indicate a lazy training regime.

2. Soft-Frozen Decoder: Decoder vectors Wdec: Decoder vectors are randomly initialized and constrained to remain within cosine similarity τ of their initial values throughout training.The constraint is enforced via projection.
2. Soft-Frozen Decoder: Decoder vectors Wdec: The baseline is motivated by SAE loss plateauing early while decoder vectors remain near initialization.
2. Soft-Frozen Decoder: Decoder vectors Wdec: The authors hypothesize that this behavior reflects a lazy training regime, where reconstruction loss is reduced through small latent adjustments without substantially changing decoder vectors.

3. Frozen Encoder: Encoder vectors Wenc

Frozen-component SAE baselines perform comparably to fully trained SAEs across reconstruction, interpretability, sparse probing, and causal editing, even when activation patterns or directions are fixed randomly. These results challenge the view that strong SAE evaluations necessarily reflect learned meaningful feature decompositions.

Frozen Encoder: Frozen encoders predetermine each feature’s activation pattern through random initialization, leaving only activation thresholds and decoder parameters to be learned.
Overall Findings: Frozen-component SAEs perform comparably to fully-trained SAEs across reconstruction, interpretability, sparse probing, and causal editing.
Reconstruction: 0.85 Explained Variance is achieved by the original JumpReLU SAE at L0=160, compared with 0.60 for its Frozen Encoder variant.At the same sparsity, Soft-Frozen Decoder reaches 0.79 and Frozen Decoder reaches 0.58.
Interpretability: 0.88 mean AutoInterp score is achieved by Soft-Frozen BatchTopK, closely matching 0.90 for the fully-trained variant.The result indicates high interpretability can emerge when latent directions remain near random initialization.
Sparse Probing: 0.70 sparse probing accuracy is reached by BatchTopK Frozen Decoder at L0=225, matching the fully-trained model and exceeding the 0.50 random-guessing baseline.The Frozen Encoder variant reaches 0.65, while gaps across settings remain between 0.001 and 0.07.
Causal Editing: 0.57-0.62 RAVEL scores are achieved by the BatchTopK Frozen Decoder across higher sparsity levels, versus approximately 0.72–0.74 for the fully trained model.This shows effective causal editing can occur even with decoder vectors frozen as random directions.

5. Limitations

The synthetic experiment assumes independent feature activations, omitting correlations likely present in real neural networks. Modeling these covariances appropriately remains unclear, while current SAEs already fail in this simplified setting.

Limitations: The synthetic experiment assumes independent feature activations, omitting correlations likely present in real neural networks.The authors note that appropriately modeling these covariances in a synthetic setup remains unclear without arbitrary assumptions.

6. Discussion and Conclusion · A. SAEs: Claimed Benefits vs. Documented Limitations

The paper finds a disconnect between SAE reconstruction quality and meaningful feature recovery, while simple random baselines can approach trained SAEs on real-activation evaluations. It argues for more rigorous evaluation without treating these results as a final verdict on SAEs.

6. Discussion and Conclusion: 71% explained variance accompanied recovery of only 9% of true features in the synthetic setting.This exposes a disconnect between reconstruction fidelity and feature recovery when ground-truth features are known.
6. Discussion and Conclusion: Standard metrics make fully-trained SAEs appear effective, but comparison with baselines shows that learning-attributable gains are modest.The comparison challenges conclusions drawn from trained SAE performance viewed in isolation.
6. Discussion and Conclusion: The reconstruction objective may favor any sparse input-recovering representation rather than alignment with the model’s true features.The authors identify the absence of an explicit reward for true-feature alignment as a hypothesized explanation.
6. Discussion and Conclusion: The baselines are simple to implement, so substantial future outperformance would provide stronger evidence for meaningful feature learning.The authors frame this as a proposed standard for evaluating improved SAE architectures.
A. SAEs: Claimed Benefits vs. Documented Limitations: Table 1 summarizes claimed SAE advantages alongside documented challenges that raise doubts about their reliability.Its purpose is to contrast motivations for widespread adoption with limitations documented in the paper.

B. Extended Comparison for Toy Model Experiments · C. Frozen Baselines for TopK SAE on Real Activations

Extended toy-model experiments show that TopK SAE can recover nearly all features under uniform activation probabilities but only a limited high-frequency subset under heavy-tailed probabilities. On real Gemma-2-2B activations, frozen TopK SAE variants remain competitive with fully trained models, especially the Soft-Frozen Decoder.

B. Extended Comparison for Toy Model Experiments: The extended synthetic comparison adds simple TopK and hierarchical Matryoshka SAEs to BatchTopK and JumpReLU using the Section 3.1 setup.These variants are used to assess how architectural choices affect feature recovery in controlled experiments.
B. Extended Comparison for Toy Model Experiments: TopK SAE achieves near-perfect recovery in the uniform synthetic setting, with 99.9% of features aligned at cosine similarity ≥0.8.This substantially exceeds the other tested architectures in the constant-probability setting.
B. Extended Comparison for Toy Model Experiments: In the heavy-tailed synthetic setting, all architectures recover only 7 to 43% of ground-truth features, exclusively from the high-frequency tail.TopK SAE performs best but remains far from complete feature coverage.
B. Extended Comparison for Toy Model Experiments: Matryoshka SAE’s hierarchical decomposition recovers slightly more mid-frequency features than BatchTopK but sacrifices some high-frequency recovery.The result indicates a trade-off in which feature recovery varies across frequency ranges.
B. Extended Comparison for Toy Model Experiments: TopK SAE’s uniform-setting success does not transfer to real LLM activations, where frozen components perform comparably to the fully trained counterpart.This suggests the toy-model result is specific to the simplified controlled setting.
C. Frozen Baselines for TopK SAE on Real Activations: The real-activation evaluation compares Fully Trained, Soft-Frozen Decoder, Frozen Decoder, and Frozen Encoder TopK SAEs with L0=160 on Gemma-2-2B layer 12 activations.These are evaluated using the paper’s standard metrics.
C. Frozen Baselines for TopK SAE on Real Activations: Frozen TopK SAE variants remain competitive with the fully trained model across metrics, and the Soft-Frozen Decoder maintains particularly strong performance.The pattern mirrors the earlier BatchTopK frozen-baseline results.
C. Frozen Baselines for TopK SAE on Real Activations: The real-activation results show that even simple SAE architectures can achieve high scores without meaningful feature learning.This conclusion follows from the strong performance of frozen variants relative to the fully trained model.

D. The Soft-Frozen Decoder: Testing the Lazy Training Hypothesis … G. Random SAEs on CLIP

Soft-Frozen Decoder results support the lazy-training hypothesis: SAEs can achieve strong reconstruction while decoder directions remain close to random initialization. Frozen baselines also perform competitively on additional language-model metrics, while matched random-versus-trained comparisons extend the analysis to CLIP.

D. The Soft-Frozen Decoder: Testing the Lazy Training Hypothesis: After 5% of training, explained variance had plateaued at 0.80 versus a final value of 0.86, while most decoder vectors remained minimally changed.At 10% training, Explained Variance = 0.82 and cosine similarities remained concentrated near 0.8.
D. The Soft-Frozen Decoder: Testing the Lazy Training Hypothesis: τ = 0.8 constrains each decoder vector to a spherical cap around its random initialization, limiting the directions available for feature representation.Representing a ground-truth feature requires at least one decoder vector to align closely with its true direction.
D. The Soft-Frozen Decoder: Testing the Lazy Training Hypothesis: Exponentially small cap probability makes alignment with arbitrary semantic features mathematically unlikely for τ = 0.8, n = 2304, and m = 73728.The baseline’s performance is therefore attributed to combining nearly fixed random directions with vanishingly small encoder and decoder adjustments.
D. The Soft-Frozen Decoder: Testing the Lazy Training Hypothesis: Soft-Frozen Decoder SAEs perform competitively across multiple evaluation dimensions, challenging the assumption that reconstruction fidelity demonstrates meaningful feature discovery.The discussion emphasizes validation against constrained baselines in interpretability research.
E. Frozen Models Ablation: The frozen-model ablation compares fully-trained SAEs with frozen baselines under iso and cov initialization schemes at L0=160 across three language-model layers.The reported metrics are means ± standard deviations for Gemma-2-2B layers 12 and 19 and Llama-3-8B layer 16.
F. Additional Metrics: Frozen Decoder and Frozen Encoder baselines show modest degradation in cross-entropy and KL-divergence relative to fully-trained SAEs, whereas Soft-Frozen Decoder closely matches them.These measures follow the SAEBench framework and evaluate substituting original activations with SAE reconstructions.
G. Random SAEs on CLIP: The CLIP analysis compares pretrained SAEs with matched Kaiming-initialized random SAEs using identical architectures across layers 3, 5, 7, and 9.Rare features with activation frequency below 0.1% are selected and visualized using ImageNet-10k images.
G. Random SAEs on CLIP: Figures 15–18 compare random and trained SAE features on CLIP ViT-B/32 at layers 3, 5, 7, and 9.The supplied figure descriptions identify these as visual comparisons of random versus trained features.