Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry

Background

• SAEs are motivated by producing interpretable, monosemantic latents that enumerate concepts underlying model predictions and enable surgical representation interventions for behavioral control [28] [29].Reported examples include meaningful concepts in language, vision, and protein models.Source: §1 · p. 1

Motivation

• Because different SAEs can have similar fidelity and sparsity yet qualitatively different biases, architecture choice may help explain reported algorithmic instability [32] and lack of causality [33] [34].The introduction presents these issues as motivating questions about whether SAEs recover all concepts used in model computation.Source: §1 · p. 2

Core framework

• SAEs are framed as bilevel optimization solutions whose encoder receptive fields create a duality between SAE architecture and the organization of concepts in model representations.Thus, each architecture is implicitly biased toward concepts organized in particular ways, rather than passively extracting all concepts.Source: §1 · p. 2, Figure 1 · p. 2

Empirical validation

• Experiments across synthetic setups and real model activations show that SAEs systematically miss concepts with heterogeneous intrinsic dimensionality or nonlinear separability when these properties are not modeled.Heterogeneous intrinsic dimensionality means concepts occupy subspaces of varying dimension.Source: §1 · p. 2

Task-specific SAE design

• SpaDE incorporates heterogeneous dimensionality and nonlinear separability into its encoder, identifying concepts that other SAEs fail to detect and supporting task-specific SAE design.The contribution argues that no single SAE architecture may be universally optimal.Source: §1 · p. 2

Sparse Coding

• Sparse coding models data as sparse combinations of dictionary atoms, jointly optimizing nonnegative sparse codes z and dictionary D with a sparsity-promoting regularizer.The regularizer is typically ||z||1.Source: §2 · p. 2

Sparse Autoencoders

• Encoder architectures project activations onto different constraint sets, so ReLU, TopK, and JumpReLU respectively impose positivity, K-sparsity, and hypercube-related restrictions on extracted features.These projections determine which features an SAE can extract and which it suppresses.Source: Figure 2 · p. 3
• SAEs approximate sparse dictionary learning with a single hidden layer that computes a sparse code z = g(W^T x + b_e) and reconstructs x̂ = Dz + b_d.Sparsity is enforced on the SAE latent code.Source: §2 · p. 3
• SAEs differ primarily in encoder nonlinearity g and sparsity regularizer R, which shape the latent-code constraints used during training.The encoder and decoder matrices satisfy W, D ∈ R^(d×s).Source: §2 · p. 3

Unified framework

• ReLU SAE [11], TopK SAE [13] [37], and JumpReLU SAE [14] [38] differ through projection nonlinearities onto distinct constraint sets that determine activation behavior, sparsity structure, and implicit data assumptions.The framework represents each encoder nonlinearity as an orthogonal projection onto a constraint set.Source: §3 · p. 3, Table 1 · p. 3
• Different SAE architectures solve distinct constrained sparse dictionary-learning problems, restricting the solution space and potentially missing the full sparse-coding solution.The encoder architecture imposes the constraint that differentiates SAEs and shapes solution quality.Source: §3 · pp. 3–4

Duality

• Encoder properties constrain SAE latents’ receptive-field structure, and matching those fields to concept structure is necessary for monosemanticity [21].A receptive field is the region of input space where a latent is active; its structure is dictated by the encoder architecture.Source: §3 · p. 4

Fundamental Limitation of SAEs

• Each SAE encoder therefore makes implicit assumptions about data organization, fundamentally shaping which concepts it can identify and which remain obscure.Effective SAE design requires aligning these architectural assumptions with the true structure of the data.Source: §3 · p. 4

4 Implicit SAE Assumptions and Data Properties

• SAEs impose architecture-specific receptive-field assumptions: ReLU and JumpReLU assume linearly separable concepts, while TopK assumes angular separability and equal dimensionality per concept.These assumptions are summarized in Table 2 and determine which concept structures each SAE can represent.Source: §4 · pp. 4–5, Table 2 · p. 5
• Model concepts can be nonlinearly separable and heterogeneous in intrinsic dimensionality, spanning one-dimensional, multidimensional, or higher-dimensional subspaces.Examples include magnitude-dependent or otherwise nonlinearly separable features, truth as a unidimensional feature, days of the week in two dimensions, and higher-dimensional safety features.Source: §4 · p. 4, Figure 3 · p. 4

4.1 SpaDE, or How to Design A Geometry-Driven SAE

• SpaDE is constructed as a geometry-driven SAE with sufficient conditions for capturing both nonlinear separability and heterogeneous concept dimensionality.Its design is intended to perform better than other SAEs when these data assumptions hold.Source: §4.1 · p. 5
• SpaDE combines competitive projection, Euclidean distances to learned prototypes, and simplex-based sparsemax to provide flexible receptive fields and adaptive sparsity across concepts.Simplex projection permits any nonzero sparsity level, allowing different concepts to activate different numbers of latents; encoder columns act as prototypes or landmarks.Source: §4.1 · p. 5, Figure 4 · p. 5
• SpaDE retains an additional implicit assumption that Euclidean distances meaningfully separate concepts and can be used for disentanglement.Its outer optimization uses K-Deep Simplex with a distance-weighted ℓ1 regularizer, while the inner optimization uses one-sided sparsity-regularized optimal transport.Source: §4.1 · p. 6

5 Results: Empirical Validation of SAE behavior

• The experiments train ReLU, JumpReLU, TopK, and SpaDE on synthetic clusters, formal-language activations, and natural vision activations to test whether SAE assumptions generalize across settings.The progression spans controlled tests of nonlinear separability and heterogeneity, semi-synthetic language activations, and large-scale DINOv2 vision activations.Source: §5 · p. 6, §5.1 · p. 7, §5.2 · p. 7

5.1 Separability Experiment

• SpaDE achieves perfect F1 on both linearly and nonlinearly separable synthetic concepts, whereas ReLU and JumpReLU are bounded above by 0.5 on the nonlinear concept.ReLU and JumpReLU also show cross-concept correlations, while SpaDE separates concepts without cross-correlations, though it may overspecialize into subclusters.Source: §5.1 · p. 7

5.2 Heterogeneity Experiment

• TopK uses uniform sparsity across heterogeneous concepts and reconstructs higher-dimensional concepts worse, while SpaDE can nearly match each concept’s intrinsic dimension.TopK falls below 20% normalized MSE only when k exceeds a concept’s dimension; other SAEs remain below this threshold across nearly all concepts.Source: Figure 6 · p. 7, §5.2 · p. 8

5.3 Formal Languages

• On formal-language model activations, SpaDE tiles part-of-speech clusters, avoids latent co-occurrence across parts of speech, and reaches perfect F1 in its most monosemantic latents.Different parts of speech have different intrinsic dimensions and separability, requiring TopK’s k and ReLU or JumpReLU’s sparsity to vary by concept.Source: Figure 7 · p. 8, §5.3 · p. 9

5.4 Vision

• SpaDE identifies the most monosemantic visual concepts across classes, while ReLU, JumpReLU, and TopK show widespread latent co-occurrence; discovered concepts include object parts and foreground/background.These concepts include hands, faces, fins, windows, stairs, eyes, ears, and snouts.Source: §5.4 · pp. 9–10, Figure 8 · p. 9

6 Discussion and Limitations

• SpaDE achieves higher concept specialization than ReLU, JumpReLU, and TopK by addressing nonlinear separability and heterogeneous concept dimensionality.ReLU and JumpReLU show low F1 scores, while TopK has high MSE when concept dimension exceeds K; SpaDE instead achieves high F1, low co-occurrence, and adaptive sparsity.Source: §6 · p. 10
• The results do not establish a universally best SAE; they support tailoring architecture to prior knowledge about the underlying data geometry.The paper presents SpaDE as a concrete example of geometry-aware design rather than an optimal SAE for every scenario.Source: §6 · p. 10
• The receptive-field analysis linking SAE encoders to concept geometry can also study interpretable models such as transcoders [51] [52].The authors frame this analysis as general beyond SAEs.Source: §6 · p. 10
• SpaDE remains limited by its Euclidean-separation assumption, potential over-specialization under aggressive sparsity, and focus on mutually exclusive concepts.For overlapping concepts, the expected co-occurrence structure may differ, although receptive-field analysis can be applied separately to concept presence and absence.Source: §6 · p. 10

A Dictionary Learning

• Sparse coding uses a sparsity-promoting regularizer; with an overcomplete dictionary (s > n), recovering sparse codes is nontrivial, whereas undercomplete codes can be obtained by pseudo-inverse when atoms are linearly independent.The L1 penalty is generally used because the L0 penalty makes the problem NP-hard [54], and overcomplete sparse coding commonly uses ISTA, FISTA, and LISTA [58].Source: A Dictionary Learning · p. 16
• SAEs are dictionary-learning tools that use sparse representations to support concept-level explainability [7] [8] [19] [20].Overcomplete sparse coding learns a data-adaptive basis, and related methods include ReLU, TopK, gated, JumpReLU, Batch TopK, and ProLU SAEs.Source: B Related Work · p. 16, A Dictionary Learning · p. 16

B Related Work

• Prior work reports both useful SAE concepts and major limitations, including multidimensional or nonlinear features, instability across random seeds, and overly specialized concepts [32] [33] [34] [69] [70] [72] [73].Disentanglement results likewise show that autoencoders need assumptions about the generative process, motivating approaches such as weak supervision and cross-layer information [75] [76] [77] [78].Source: B Related Work · pp. 16–17

C Experimental Setup

• Synthetic separability and heterogeneity experiments, vision experiments, and formal-language experiments used NVIDIA A100 40GB and RTX A6000 48GB GPUs, respectively.A100 GPUs supported the synthetic and vision experiments; RTX A6000 GPUs supported the formal-language experiments.Source: C Experimental Setup · p. 17

C.1 Separability experiment

• The synthetic benchmark contains six isotropic Gaussian clusters in 2D, with alternating center norms of 1 and 3 and 6 million total samples.Adjacent cluster centers differ by 2π/6, each cluster has variance 2^-5.5, and 70% of the data is used for training.Source: §C.1 · p. 17
• The comparison includes ReLU, JumpReLU, TopK, and SpaDE SAEs, all using hidden width 128 with normalized decoder activations except for SpaDE’s distinct distance-based SparseMax encoder.A pre-encoder bias is used for all architectures except SpaDE.Source: §C.1 · p. 17
• Concept detection computes precision and recall by thresholding latent activations at 10^-6.Source: §C.1 · p. 18
• The analysis probes latent organization through receptive-field visualizations and pairwise cosine similarities between latent representations, grouped by class.A 2D meshgrid is passed through each model to examine receptive fields.Source: §C.1 · p. 18
• Stable rank and spectral clustering are applied to latent similarity matrices to estimate intrinsic dimensionality and the number of representation clusters.The spectral-clustering cluster count is the stable rank rounded up.Source: §C.1 · p. 18

C.2 Heterogeneity experiment

• The heterogeneity dataset contains five 128-dimensional Gaussian concepts with intrinsic dimensions 6, 14, 30, 62, and 126.Each concept contains 6.4 million samples, yielding 32 million total points, with approximately 22 million used for training.Source: §C.2 · pp. 18–19
• These concepts occupy nested subspaces, so each lower-dimensional structure is fully contained within every subsequent higher-dimensional structure.Concept variance is scaled inversely with intrinsic dimension to keep total variance comparable across concepts.Source: §C.2 · p. 19
• The comparison includes ReLU, JumpReLU, TopK, and SpaDE SAEs, each using width 512 but differing in encoder activations, normalization, bias, and regularization.SpaDE uses Euclidean distances and SparseMax, whereas the other architectures follow their original formulations, [14], and [13].Source: §C.2 · p. 19
• Heterogeneity evaluation reports normalized MSE and per-concept L0 sparsity, then examines per-input and global latent co-occurrence to assess specialization.Global latent labels are assigned according to the concept that most frequently activates each latent.Source: §C.2 · p. 19

C.3 Formal Languages experiment

• The formal-language experiment trains a 2-layer nanoGPT from scratch on English-like strings generated by a probabilistic context-free grammar.The strings follow the grammar used in [34] and are represented with one-hot tokenization.Source: §C.3 · pp. 19–20
• Grammar sentences are generated by repeatedly expanding randomly selected nonterminals according to production-rule probabilities until only terminal symbols remain.The grammar is specified by nonterminals, terminals, production rules, a start symbol, and rule probabilities.Source: §C.3 · p. 20
• Formal-language SAEs use 256 latents for a 128-dimensional residual stream, with regularization sweeps producing different sparsity levels.SpaDE uses synthetic-dataset hyperparameters, whereas ReLU, JumpReLU, and TopK report best-case top-10 per-concept F1 scores from sweeps; other hyperparameters did not transfer.Source: §C.3 · p. 20

C.4 Vision experiment

• The experiment uses DINOv2-base with registers on Imagenette, a 10-class ImageNet subset containing 1.5k images per class and 261 tokens per image.Representations are extracted for images from these classes.Source: §C.4 · p. 20
• SAEs are trained for 50 epochs with 200 latent dimensions on all spatial, CLS, and register tokens, totaling approximately 200M training tokens.The best reconstruction is selected using a sparsity-controlled learning-rate sweep.Source: §C.4 · p. 20
• TopK, ReLU, and SpaDE use an optimal learning rate of 5 × 10−4, whereas JumpReLU performs best with 10−4 under Adam optimization.The JumpReLU implementation uses a Silverman kernel with bandwidth 10−2.Source: §C.4 · p. 20

D.1 Projections and Nonlinearities

• ReLU, TopK, and JumpReLU are orthogonal projections onto distinct constraint sets, whose geometry determines activation behavior, sparsity, and implicit data assumptions.ReLU projects onto the positive orthant, TopK onto the union of k-sparse subspaces, and JumpReLU onto shifted hypercube corners.Source: §D.1 · p. 21, Table 4 · p. 21

D.2 Receptive fields of various SAEs

• SAE receptive fields are unions of piecewise-affine regions, so under monosemanticity their shapes directly encode assumptions about the structure of concepts.Projection-set geometry determines receptive-field properties, while gating selects the active-index pattern defining each affine region.Source: §D.2 · p. 23

D.2.1 ReLU, JumpReLU SAE

• ReLU and JumpReLU receptive fields are half-spaces, with ReLU regions forming convex polytopes and, without bias, unbounded hyperpyramids.Bias allows ReLU regions to have multiple corners and occur anywhere in the input space.Source: §D.2.1 · p. 24

D.2.2 TopK SAE

• TopK receptive fields are unions of hyperpyramids, while each active-index region is an unbounded hyperpyramid whose faces are determined by differences between active and inactive weight vectors.Typical pre-encoder bias shifts the hyperpyramids’ corner away from the origin.Source: §D.2.2 · p. 24

D.2.3 SpaDE

• SpaDE receptive fields are unions of local convex polytopes around prototypes, incorporating locality and flexible shapes that can capture nonlinearly separable concepts.Its active-index selection is input-adaptive and based on Euclidean distances to prototypes.Source: §D.2.3 · p. 25

D.3 KDS and Sparse Coding

• K-Deep Simplex can represent any standard sparse-coding generative model after rescaling the data, possibly with one additional latent dimension.The constructive equivalence uses a zero dictionary atom and assigns the residual to the simplex-constrained code.Source: §D.3 · p. 25

D.4 SpaDE

• SpaDE is exactly a continuous, piecewise-linear ReLU SAE with input-adaptive, piecewise-constant transformations, while its simplex projection and KDS outer objective support heterogeneous concepts and locality.The probability simplex provides edges and corners with varying sparsity, and KDS’s distance-weighted regularizer induces a soft clustering bias while retaining standard sparse-coding expressivity.Source: §D.4 · pp. 26–27
• SpaDE captures concept heterogeneity through sparsity and supports nonlinear separability using prototype-local receptive fields formed from unions of convex polytopes.These properties allow SpaDE latents to become monosemantic for concepts that are nonlinearly separable from the rest of the data.Source: §D.4 · p. 27

E Further Results

• The section provides a more detailed analysis of the results from each of the paper’s four experiments.Source: E Further Results · p. 27

Concept-level reconstruction

• All SAEs reach similar final NMSE levels, but their concept-level training dynamics differ, with below-variance MSE indicating that within-concept variation is being reconstructed.MSE near the concept variance suggests predicting each concept’s mean, while reconstruction relative to cluster centers should match cluster variance when points are reconstructed well.Source: Figure E.2 · p. 27, Figure E.3 · p. 28

Reconstruction and sparsity

• TopK’s NMSE decreases significantly more at dense representations because it learns a redundant two-latent orthogonal-basis solution, despite comparable dense-regime MSE-sparsity curves across SAEs.Across SAEs, denser representations generally lower NMSE, but reconstruction performance does not capture their differing concept-extraction abilities.Source: §E.1 · p. 27, Figure E.2 · p. 27

Separability metrics

• SpaDE has significantly fewer dead latents and much higher latent stable rank than ReLU, JumpReLU, and TopK, indicating greater latent specialization.The other SAEs have stable ranks well below the desirable value of 6, the number of data clusters.Source: §E.1 · p. 28

Activation profiles

• Latent activation histograms share SAE-specific profiles across concepts, with SpaDE exhibiting a pointy distribution suggestive of a second mode beyond zero.The profiles vary across concepts but retain a common structure for each SAE.Source: §E.1 · p. 28, Figure E.4 · p. 29

Correlation structure

• SpaDE produces the lowest cross-concept correlations, whereas ReLU and JumpReLU retain higher correlations that do not decrease substantially with sparsity.Latent co-occurrences broadly decrease as sparsity increases, while increasing sparsity also leaves fewer active latents because dead latents accumulate.Source: §E.1 · p. 28

E.2 Heterogeneity Experiment

• Heterogeneity training has two phases: first matching each concept’s data scale and mean, then explaining within-concept variance.The transition appears as a kink in the per-concept error curves, with the gray reference lines distinguishing mean prediction from variance explanation.Source: §E.2 · p. 31, Figure E.8 · p. 31, Figure E.9 · p. 32
• TopK fails to allocate latents adaptively across heterogeneous concepts, whereas other SAEs form latent-correlation blocks with concept-dependent sizes.With k = 32, TopK overactivates smaller numbers of concepts because it cannot allocate enough latents to large-intrinsic-dimension concepts, especially at moderate sparsity.Source: §E.2 · p. 31, Figure E.10 · p. 33, Figure E.12 · p. 34

E.3 Formal Language Experiments

• SpaDE achieves highly monosemantic features and more effectively discerns a concept’s sentence position than protocols learning more uniform representations.Formal-language results also track normalized MSE, stable ranks, monosemanticity, dead latents, concept-level parts-of-speech behavior, and sparse-code correlations across sparsity.Source: §E.3 · p. 35, Figure E.13 · p. 35, Figure E.14 · p. 35
• Across formal-language sparsity sweeps, fidelity, data and latent stable ranks, monosemanticity, and dead-latent fractions vary at both overall and parts-of-speech levels.The concept-conditioned dead-latent count includes both always-inactive latents and latents inactive for the particular concept.Source: §E.3 · p. 35, Figure E.13 · p. 35, Figure E.14 · p. 35
• Formal-language correlation heatmaps reveal position-dependent concept structure and latent specialization by parts of speech.Datapoints are ordered by concept and sentence position, while latents are ordered by the parts of speech for which they achieve their strongest F1 scores.Source: §E.3 · p. 35, Figure E.17 · p. 37, Figure E.18 · p. 38
• Most well-performing SAEs organize formal-language latents structurally in PCA, but SpaDE maintains this organization throughout the experiments.The visualizations project either datapoint activation patterns or the datapoints activated by each latent into two or three dimensions.Source: §E.3 · p. 35, Figure E.19 · p. 38, Figure E.20 · p. 39

E.4 Vision Experiment

• SpaDE’s top concepts are visualized with feature attribution maps for five Imagenette classes: Tench, Chainsaw, Church, Golf, and Springer.The corresponding visualizations are provided in Figures E.23–E.27.Source: §E.4 · p. 41, Figure E.23 · p. 41, Figure E.24 · p. 42

• The paper states that its abstract and introduction accurately reflect theoretically and empirically supported contributions and scope, with limitations discussed in Section 6.Theoretical results appear in Sections 3–4 and Appendix D, while experiments appear in Section 5 and Appendix E.Source: NeurIPS Paper Checklist · p. 46

Theory assumptions and proofs

• Theoretical limitations of existing SAEs are supported by complete proofs in Appendix D through derivations of their receptive-field structure.Source: NeurIPS Paper Checklist · p. 46

Experimental reproducibility

• The paper provides experimental setup and training details in Section 5 and Appendix C, reports error bars when experiments are not too expensive, and supplies anonymized code for formal-language experiments while promising full release after review.Compute-resource information is also included in Appendix C.Source: NeurIPS Paper Checklist · pp. 47–48

Responsible release

• The checklist indicates that existing assets are properly credited, new synthetic datasets and the SpaDE SAE are documented in code, and the paper conforms to the NeurIPS Code of Ethics.Source: NeurIPS Paper Checklist · pp. 49–50

Ethics and declarations

• The paper reports no direct societal impact, misuse risk, human-subject or crowdsourcing involvement, or important LLM use in its core contributions.Accordingly, societal-impact, safeguard, human-subject, IRB, and LLM declarations are marked NA.Source: NeurIPS Paper Checklist · pp. 49–51