Feature Superposition in Neural Networks: From Theory to Practice

1 Introduction

• Sparse feature activity and nonlinear decoders can limit interference from overlapping directions, allowing useful features beyond a layer’s width, though not one feature per neuron.Recovery claims depend on the feature distribution, tolerated error, and decoder class.Source: §1 · p. 4
• Activation reconstruction alone does not establish feature identity or computational role, so those properties require separate evaluation.The survey also examines documented failures and applications in light of the evidence for each claim.Source: §1 · pp. 4–5
• The survey connects theoretical conditions for representing and recovering superposed features with methods and evidence from trained networks.It covers feature geometry, statistics, decoder choice, and computation with superposed inputs.Source: §1 · pp. 4–5
• The practical review groups feature-recovery methods into probes, unsupervised dictionaries, functional replacement, and parameter decomposition.It compares these methods with their evaluation tests and examines interventions that change recovered directions and measure behavioral effects.Source: §1 · p. 5
• The survey complements prior reviews by comparing representation and computation results, recovery methods beyond sparse autoencoders, and evidence for their uses in trained models.Its literature criteria include studies of superposed representations and methods for estimating or evaluating trained-network features, including negative results.Source: §1 · p. 5

2.1 Setup and Notation

• The additive feature model maps given feature values along encoder directions into a lower-dimensional hidden vector, which a decoder uses to reconstruct those values.The model makes the recovery target explicit and connects to sparse coding and compressed sensing.Source: §2 · p. 6, §2.1 · p. 7

2 Background and Definitions

• The definition of superposition uses a feature distribution, encoder, and decoder, and recovery comparisons depend on the permitted decoder family.The survey’s notation distinguishes feature count d from hidden width m.Source: §2 · p. 6, §2.1 · pp. 7–8

2.2 Definitions

• Superposition requires overlapping feature directions and one decoder that recovers each selected feature within tolerance; mixed neuron responses after rotation alone do not suffice.Rotation can make neurons respond to multiple features while leaving the feature directions orthogonal.Source: §2.2 · p. 9

2.3 A Worked Example

• For sufficiently sparse Bernoulli features, an antipodal code recovers both features in one dimension with a ReLU decoder, while the best linear decoder remains capped at R2 = 1/2.The ReLU decoder errs when both features fire; linear recovery remains at that cap for every p.Source: §2.3 · p. 11

2.4 Measures of Superposition

• Packing degree counts simultaneously recoverable features per dimension, while superposition benefit compares reconstruction with the best orthogonal representation at the same width.The two-procedure risk difference in the worked example is not automatically the optimized benefit.Source: §2.4 · p. 12

2.5 Strength of Claims about Features

• Six claim levels distinguish representation and accessibility from identification, functional substitution, causal deployment, and mechanistic faithfulness.A feature can be decodable without its generating direction being identifiable, and preserving reconstruction need not preserve the original computation.Source: §2.5 · pp. 12–13

2.6 Notions of Capacity

• Capacity measures count different resources or targets, including direction overlap, recoverable feature values, and computable functions, so their assumptions and counts must be distinguished.The section’s capacity table separates seven quantities by their input assumptions, recovery criteria, and resource counts.Source: §2.6 · p. 13, Table 2 · p. 14

3.1 Interference and Linear Reconstruction Risk

• Linear reconstruction risk depends on feature covariance and readout, while three planar directions permit exact ReLU recovery on single-feature inputs but collide when all three activate equally.The equal co-activation maps to the same hidden state as the empty input, so no decoder can distinguish them.Source: §3.1 · pp. 15–16

3.2 Lower Bounds on Interference

• For d unit-norm directions in m<d dimensions, squared overlap is at least d(d −m)/m; at tight frames, tied risk is d/m times optimal untied risk.These reconstruction-risk comparisons assume isotropic features and zero residual; other encoders can have a different risk ratio.Source: §3.2 · pp. 16–17

3.3 Mechanisms that Reduce the Realized Cost

• Sparsity and ReLU decoding reduce some realized cross-talk, while untied readouts change the decoder and feature importance changes which reconstruction errors training penalizes.Correlated features may cluster or align constructively, so the outcome depends on the feature law and training objective.Source: §3.3 · pp. 18–19

3.4 Feature Capacity

• Feature capacity compares each direction’s squared self-overlap with its total squared overlaps; its sum is bounded by rank, but it does not specify decoding or reconstruction accuracy.The statistic is geometric, not a measure of causal use or Shannon capacity.Source: §3.4 · pp. 19–20

3.5 Finite Precision and Counting Bounds

• Finite-precision message counts depend on coordinate range and resolution; pairwise direction packing does not guarantee sum recovery, while sparse linear recovery has a distinct width guarantee.For fixed sparsity and error tolerance, Garg et al. (2026) give width Oε(k2 log d); comparisons require matching support, amplitude, accuracy, and decoder assumptions.Source: §3.5 · p. 20

3.6 Observed Geometries

• Observed toy-model geometries vary with the objective and feature statistics, while curved manifolds and multidimensional concepts limit one-direction-per-feature descriptions without establishing their prevalence.Reported toy-model shapes include antipodal pairs, triangles, pentagons, tetrahedra, and square antiprisms; their detailed polytope taxonomy is tentative.Source: §3.6 · pp. 20–21

4.1 Emergence of Superposition in Training

• Elhage et al. (2022)'s phase diagrams depend on the feature law and objective; Scherlis et al. (2022) find fractional capacity above kurtosis 3 but all-or-nothing allocation below it.Fractional capacity means dimensions are shared among features; the threshold applies under the analyzed moment, centering, and objective assumptions.Source: §4.1 · pp. 21–22

4.2 Incidental Polysemanticity

• Lecomte et al. (2024) show that initialization and sparsity-inducing regularization or noise can produce polysemantic neurons even when width permits orthogonal directions.Polysemantic responses alone do not establish more recoverable features than dimensions or nonorthogonal feature directions.Source: §4.2 · p. 22

4.3 Linear Access

• For fixed error, Garg et al. (2026) give nearly quadratic dependence on active features for uniform linear recovery, while fixed k permits exponentially many candidate features to be read.The lower bound also covers independently chosen encoders and decoders; it concerns uniform error and simultaneous activity, not an unconditional cap on readable features.Source: §4.3 · p. 23

4.4 Comparison of Decoder Classes

• Under suitable restricted-isometry conditions, sparse optimization improves recovery's dependence on k from nearly quadratic to linear, but guarantees require appropriate encoder properties and depend on the recovery target.Table 3 separates uniform recovery guarantees with different error criteria and computational procedures from model-dependent limits.Source: §4.4 · p. 23, Table 3 · p. 24

4.5 Identifiability

• Identification results depend on observation assumptions: query-based recovery and activation-generative-model SAE guarantees coexist with model-specific non-identifiability and objective-induced pathologies.Cui et al. (2025) and Tang et al. (2026) locate negative results in specified regimes; they do not show every trained dictionary has those failures.Source: §4.5 · p. 24

4.6 Superposition and Neural Scaling Laws

• Liu et al. (2025)'s toy model predicts 1/m loss under near-isotropic overlaps, but steep power-law feature frequencies shift the observed exponent toward 2(α−1); open-weight language-model head fits give 0.91±0.04.The language-model measurements support an interpretation of the head, not a mechanism for loss scaling throughout the network; the width and training-time results concern different limits.Source: §4.6 · p. 25

5.1 From Recovery to Computation

• In Elhage et al. (2022)'s inspected toy network, shared neurons compute absolute values through asymmetric interference control: inhibition and an output ReLU remove unwanted positive contributions.This mechanism is established only for the inspected toy setting.Source: §5.1 · p. 26

5.2 The Representing-versus-Computing Separation

• For Adler and Shavit (2024)'s specified Boolean-task family, universal computation requires Ω(d′ log d′) parameter bits, while explicit k≤2 networks compute instances with O(√(d′ log d′)) neurons.The result separates a compact representation from the resources needed to specify and compute arbitrary instances; the upper construction chains without error accumulation.Source: §5.2 · p. 27

5.3 Constructive Results

• Hänni et al. (2024) construct ReLU layers that represent each pairwise AND across many neurons, with uniform fixed-sparsity guarantees for designed Boolean encodings, not a training guarantee.For composition, separate routing and error-correction constructions re-encode outputs to avoid accumulated error; the circuit-emulation passage is a proof sketch with draft support.Source: §5.3 · p. 29

5.4 The Compressed-Computation Dispute

• In the compressed-ReLU studies, the initial loss advantage disappears at M=0 and peaks near σ=0.03; higher loss exponents distribute errors more evenly, and interventions support a sparse-code mechanism in the tested model.The mixing experiments are single-run curves without reported seeds or error bars, and the learned motif does not establish the construction's uniform Boolean guarantee.Source: §5.4 · p. 30

5.5 The Existence-versus-Use Gap

• Hänni et al. (2024) show that random one-layer MLP activations can make all pairwise ANDs readable, so a successful probe does not establish that a trained model uses those functions downstream.Adler et al. (2025) identify routing motifs in trained models, while the task and optimization conditions under which training finds such mechanisms remain unresolved.Source: §5.5 · p. 31

5.6 Temporal Superposition

• In the delay task, retaining d features through delay τ requires d(τ+1) feature-age targets; width-2 trained solutions maintain five, three, two, or one feature at delays 0, 1, 2, or at least 3.These are selected small-model solutions; only roughly 8% to 54% of trained models in reported sweeps exhibited the analyzed geometries.Source: §5.6 · pp. 31–32
• Sharma et al. (2026)'s linear-recurrence loss separates task and interference terms; in the sparse-limit ReLU approximation, wrong-age directions opposite the readout avoid projection interference.The scalar identity assumes independent gates and values; correlated sequences add covariance terms, and the ReLU expression is an approximation for nonnegative values.Source: §5.6 · p. 32

6.1 Boundaries with Neighboring Concepts

• Polysemanticity, mixed selectivity, and linear feature format describe different properties; none alone establishes nonorthogonal, above-width feature recoverability.Polysemanticity can arise from basis rotation or spare capacity, and mixed selectivity concerns population activity across task conditions; the linear representation hypothesis permits orthogonal features up to width.Source: §6.1 · pp. 33–34
• Population mixing can increase activity dimension while reducing constituent-recovery accuracy, unlike sparse feature-direction superposition with thresholded interference.The appropriate comparison specifies feature statistics, readout, and target computation.Source: §6.1 · p. 34

6.2 Superposition as Explanation

• Controlled capacity-constrained results link shared-direction interference to adversarial vulnerability, but deep-network evidence is indirect and does not establish that superposition is necessary.In the cited deep-network analysis, attack and interference profiles correlate as the bottleneck tightens, while adversarial robustness remains low and approximately flat.Source: §6.2 · pp. 36–37
• Under stated assumptions, similarity metrics can underrate identical feature content, while head-based scaling fits support only an interpretation of head behavior.Recovered-latent alignment exceeds raw-activation alignment in the reported comparisons, but does not by itself establish feature identity.Source: §6.2 · pp. 35, 37

6.3 Superposition as Resource Allocation

• Sparse representations do not make efficiency a width-only question: readout parameters, computation costs, and which features receive capacity remain distinct concerns.The cited bounds distinguish representation width from explicit readout size, and the allocation results show that architecture width alone does not specify feature capacity.Source: §6.3 · pp. 37–38

6.4 Scope of Further Benefit Claims

• A teacher-student model reports a fitted loss-decay exponent up to ten times the sequential-learning value, but comparable acceleration in trained language models and general performance benefits remain unestablished.The reported observation concerns training dynamics in the constructed model.Source: §6.4 · p. 39

6.5 Superposition as a Premise for Feature Recovery

• Overcomplete dictionaries supply candidate feature directions, not proof of feature identity; recovery claims depend on format, sparsity, identifiability, and layer coverage.A cited identifiability theorem assumes equal latent and representation dimensions, excluding the overcomplete setting.Source: §6.5 · pp. 39–40
• Curved representations challenge linear one-dimensional dictionary atoms, although tested subspace patching changes answers on two templatic tasks without identifying every underlying feature.The circular geometry has a competing interpretation as correlated one-dimensional features compressed into a layer.Source: §6.5 · p. 39

7.1 Supervised Probes and Mean-Difference Dictionaries

• Sparse probes find localized neuron detectors and a three-neuron combination that nearly separates a compound word, suggesting superposition without testing whether feature count exceeds width.Ablating one French-text neuron increases loss by 8% in a 70M-parameter model and 0.2% in a 6.9B-parameter model; prediction alone does not establish causal use.Source: §7.1 · p. 41
• Task-specific TopK and Gated autoencoders approach the supervised IOI reference on three causal tests, while full-distribution autoencoders perform worse, with the comparison partly confounded by tuning.The reference covers only attributes specified in advance and can miss properties distinguished by higher moments; the task-specific models received a wider hyperparameter search.Source: §7.1 · p. 42

7.2 Unsupervised Dictionary Learning

• At matched fidelity, compute-matched Gated SAEs use about half the L0 of baselines, while Matryoshka SAEs reduce first-letter absorption from 0.49 to 0.05 at matched sparsity.Gated SAEs have comparable blinded interpretability to the baseline; Matryoshka incurs two percentage points less explained variance, and its tested first-letter improvement does not distinguish it on board-game ground truth.Source: §7.2 · pp. 43–44
• Dead latents reach 90% without mitigation but fall to 7% at 16 million GPT-4 latents; end-to-end training reports less than 45% of baseline loss increase at matched L0.The end-to-end case study assigns 593 local latents versus two behavioral-objective latents to a weakly consequential direction, but behavioral preservation still requires stability and computational-faithfulness checks. Recovery guarantees are not uniform, and cross-design comparisons differ in protocols without matched combined-gain tests.Source: §7.2 · pp. 43, 44, 46

7.3 Functional-Replacement Methods

• Transcoder attributions factor into an input-dependent activation and a weight-computable factor, but their relevance to the model depends on surrogate accuracy and published fidelity rankings conflict.The factorization assumes the downstream latent is already active and does not show the upstream latent makes it fire; reports disagree on transcoder performance relative to same-site autoencoders.Source: §7.3 · pp. 46–47
• Latent circuits explain a majority of subject-verb agreement behavior with roughly 100 nodes versus roughly 1,500 neurons, but interpreted circuits have faithfulness 0.21 and rely on residual-error nodes.On an 18-layer model, an acausal crosscoder reaches lower loss than per-layer autoencoders at matched total latents, but needs about twice the training FLOPs at large compute to reach the same loss.Source: §7.3 · p. 47

7.4 Parameter-Space Decomposition

• VPD decomposes about 28 million non-embedding parameters into 38,912 rank-one subcomponents, with roughly 6,500–7,000 live across tested decomposition sizes.APD recovers toy mechanisms including superposed and cross-layer representations, but grouping subcomponents across ranks or layers is assumed known in those settings; the motivating interference evidence is low-confidence.Source: §7.4 · p. 48
• VPD subcomponents can span computations across several attention heads, while masks expose narrow components whose out-of-context contributions distort outputs.The report contrasts this with prior analyses that mainly use the head as the unit of analysis.Source: §7.4 · p. 49
• After 320 adversarial-optimization steps, ablating subcomponents marked unimportant can still drastically change model output.Demonstrations reach about 108 parameters; separation of circuit logic from directional interference lacks ground-truth testing, and APD and VPD share a questioned linear-in-weights assumption.Source: §7.4 · p. 49

7.5 Superposition Across Layers

• Residual directions can persist until canceled, leading per-layer dictionaries to duplicate them; cross-layer latents also show gradual directional drift consistent with distributed feature formation (Lindsey et al., 2024).Cross-layer latents replace a seven-step circuit chain at its first layer, though a feature formed earlier and amplified later is also possible (Lindsey et al., 2024).Source: §7.5 · p. 49
• No published capacity bound was identified for cross-layer representations; multi-layer width bounds require a separately specified joint representation rather than the width of one activation site.Existing bounds apply to a specified representation of width m, and a single-site analysis retains the observed width m.Source: §7.5 · p. 49

7.6 Comparison of the Families

• Probes test access to specified properties, SAEs estimate candidate directions through sparse reconstruction, and computation-focused methods require faithfulness checks suited to their intended claims.Changing sparsity or fidelity changes an SAE's estimated dictionary without establishing feature identity; the activation site also limits the representation examined.Source: §7.6 · p. 50

8 Empirical Protocol, Benchmarks, and Complexity

• Known references test recovery directly, while negative controls probe score validity and behavioral tests examine effects that explained variance does not measure.Reconstruction, interpretation, and intervention scores can rank dictionaries differently; residual measurements assess what a dictionary omits.Source: §8 · p. 52

8.1 Synthetic Ground Truth

• Synthetic generators specify feature directions and values, enabling separate comparisons of dictionary directions with ground truth and firing patterns with feature supports.These tests establish recovery for the chosen generator; transferring conclusions to trained networks requires checking their unknown feature law and representation.Source: §8.1 · p. 52
• Matching-pursuit encoders can reconstruct well by exploiting interference between ground-truth directions, without recovering those directions.The generator reproduces a disconnect between reconstruction and latent quality, alongside an L0-dependent precision-recall trade-off.Source: §8.1 · p. 53
• With 3,200 directions and matched-width dictionaries, explained variance reaches 0.67 at a constant firing rate while only 3 directions match above cosine 0.8.Under heavy-tailed firing, explained variance reaches 0.71 and 7 to 9% of directions meet that threshold, almost all among the most frequent features; recovery ranges from 7 to 43%. The independently drawn directions omit correlation and hierarchy.Source: §8.1 · p. 53
• Recovery percentages are not directly comparable across studies because generators, architectures, matching criteria, and sparsity settings differ.For example, the reported constant-rate recovery is 99.9% for TopK and 0.03% for Matryoshka under a different matching criterion.Source: §8.1 · p. 53

8.2 Model Organisms

• In chess, the best dictionaries score 0.48 coverage and 0.85 board reconstruction, versus 0.11 and 0.01 for random-model dictionaries and 0.98 and 0.98 for linear probes.The comparison detects learned board information but finds dictionaries recover less of it than probes; the rebuilding rule can affect the qualitative comparison.Source: §8.2 · pp. 53–54
• On IOI, latent edits are evaluated by whether they reproduce counterfactual activation-patching effects, not by whether the supervised directions are recovered.Successful behavioral control therefore leaves feature identification as a separate question.Source: §8.2 · p. 54
• Chess controls reveal that low coverage can reflect weak target representation, while high board-reconstruction scores can arise from surface-form cues.Probes fall below F1 = 0.95 on 6 of 15 strategy concepts. Including pieces on starting squares yields 0.52 board reconstruction at layer 1, but masking them lowers it to 0.01.Source: §8.2 · p. 54

8.3 Benchmarks on Trained Language Models

• SAEBench compares more than 200 dictionaries across two language models, seven architectures, three widths, six target sparsity levels, and eight metrics.The metrics cover concept detection, interpretability, reconstruction, and disentanglement.Source: A Common Comparison Protocol · p. 54

A Common Comparison Protocol

• At 65k latents and L0 ∈[40, 200], Matryoshka BatchTopK leads on 5 of 8 metrics; greater width improves reconstruction and interpretability but worsens several other metrics.The width-related exceptions and trade-offs differ by metric, and supervised rankings may not transfer across model scale.Source: A Common Comparison Protocol · p. 54
• On a fixed dictionary, reseed coefficients of variation span 0.2% for probing-based scores to 23% for targeted probe perturbation at top-N = 50.A 95%-level difference between two single-seed scores must be about 3.93 times the metric’s reseed standard deviation; only 7 of 34 metric configurations distinguishable from chance show between-variant variance.Source: A Common Comparison Protocol · pp. 54–55
• Two audited metrics can decline during training, and spurious correlation removal ranks an exact-coefficient oracle below 11 of 35 trained dictionaries at top-N = 10.The audit recommends against targeted probe perturbation and spurious correlation removal, while other measurements still favor Matryoshka and have differing audit coverage.Source: A Common Comparison Protocol · p. 55

8.4 Baselines and Negative Controls

• Frozen-random SAE controls approach trained dictionaries on some proxy scores, but their biases, thresholds, and unfrozen matrices still receive training.The comparisons vary by site and task: a frozen-at-initialization decoder is markedly worse on circuit-level sparse control, whereas causal-editing results cluster near 0.5 at some sites.Source: §8.4 · pp. 55–56
• On steering and concept detection, dictionary methods are uncompetitive in the evaluated settings; dictionary probes also underperform logistic regression on average across tested data regimes.Adding dictionary latents to baseline ensembles does not consistently improve them, and application comparisons do not establish whether a latent identifies a feature.Source: §8.4 · p. 56

Sparsity and Feature Recovery

• Changing L0 changes the representation tested: too little activity promotes mixtures of correlated features, while too much admits degenerate mixtures.The cited work therefore treats sparsity level as more than a sparsity-fidelity setting.Source: §8.5 · p. 56

Comparison with Supervised References

• On board-game models, Gated dictionaries score higher on loss recovered, while p-annealed dictionaries match or exceed them on board-state coverage.Dictionaries with 8,192 latents improve supervised scores over 4,096-latent dictionaries despite indistinguishable unsupervised proxies.Source: §8.5 · pp. 56–57

Reliability and Validity

• Reconstruction metrics are highly discriminative but measure activation reconstruction, not feature identity; low evaluation noise alone does not validate a score as a recovery measure.The sae-probes variant is the most reliable remaining metric, while probing-based scores can saturate despite substantial variation in matched cosine.Source: §8.5 · p. 57

Metric Scope and Disagreements

• Table 7 distinguishes reconstruction, probing, interpretation, and intervention tests, and records documented disagreements rather than declaring metrics generally invalid.For example, probing scores may saturate, while disentanglement scores can fail training or oracle checks.Source: §8.5 · pp. 56, 58, Table 7 · p. 57

Causal Editing

• Causal-editing scores measure behavioral effects of reconstruction, removal, or bounded latent edits, but do not alone establish that edited latents are the model’s units.Frozen-random dictionaries match trained dictionaries on RAVEL, while a frozen-at-initialization decoder performs markedly worse on circuit-level sparse control.Source: §8.5 · p. 58

Ground-Truth Recovery

• Recovery figures are not a common rate when studies use different generators, dictionary widths, and matching rules, and recovery beyond supplied ground truth remains unmeasured.The reported normalization also affects interpretation of denominators.Source: §8.5 · pp. 56, 58

The Jacobian Lens

• Exchanging a sparse J-space concept component places the second concept in the top five outputs on 59% of trials, versus 88% for exchanging J-lens vectors and 5% for the remainder.The report constructs J-lens vectors from token-likelihood sensitivity and uses sparse nonnegative combinations to isolate a component for intervention.Source: §8.6 · pp. 58–59
• The J-space result depends on a linearization and concept-vector selection; the report describes its sparsity choice as somewhat arbitrary and uses proprietary checkpoints without identified replication.These conditions limit generalization of the reported variance and intervention rates.Source: §8.6 · p. 59

Behavioral Tests of Recovered Features

• Among 22 features matched above cosine 0.90, the associated latent is causally inert for 2 in a well-trained dictionary and up to 17 in a degraded dictionary.Here, inert means the matched latent remains silent whenever its feature is present, separating direction alignment from activity needed for causal control.Source: §8.6 · p. 59

Explained Variance as a Target

• Explained variance and behavioral effects require separate measurements in the evaluated settings, and extending representation or computation bounds to a specified causal criterion remains open.A high-variance direction can have a small behavioral effect, while a lower-variance component can matter more.Source: §8.6 · pp. 58–59

Residuals and Their Measurements

• A dictionary residual e(h) = h−ĥ is called SAE dark matter, but published dark-matter quantities measure different properties and are not commensurable.Activation variance, weighted paths, and judged prompts have different denominators and cannot be treated as interchangeable unexplained fractions.Source: §8.7 · pp. 59–60, Table 8 · p. 60

Contributions to the Residual

• Residual structure may reflect sparse-objective mismatch with dense activation components, while frequency and layer-local coverage can also constrain what a dictionary captures.The dense-component account is proposed rather than an identified feature decomposition, and cross-layer structure alone does not force local reconstruction error.Source: §8.7 · p. 60

Dictionary-width Scaling and Superposition

• Width-scaling fits leave the residual constant’s origin unresolved; the interference bound concerns isotropic linear coefficient reconstruction, not nonlinear SAE maps or activation residuals.The latent-feature count d, activation width m, and chosen dictionary width d̂ are distinct; extrapolated asymptotes do not prove irreducible error for all recovery methods.Source: §8.7 · p. 61

Predictable Residual Structure

• Held-out mean coordinatewise R2 for affine error-vector prediction ranges from 0.15 to 0.72, while intercept-including prediction of residual squared norm reaches scalar R2 from 0.70 to 0.95.The vector-prediction score decreases with dictionary width or active latent count; residuals are relative to an affine predictor, not an identified feature decomposition.Source: §8.7 · p. 61
• In reported replacement tests, predictable and remainder components each raise cross-entropy in proportion to their norm, while per-token residual norms are predictable across dictionary scales.Dictionaries trained on the remainder reconstruct it worse than dictionaries trained on the predictable component, with fraction of variance unexplained 0.59 against 0.54.Source: §8.7 · pp. 61–62

Residual Paths and Behavioral Coverage

• A 10-million-latent cross-layer transcoder raises the replacement score from 0.37 to 0.61 versus same-size per-layer transcoders, leaving 39% of weighted path influence through residuals.Edge-based completeness changes from 0.78 to 0.80 while average path length falls from 3.7 to 2.3; these graph scores measure a different omission than residual norm.Source: §8.7 · pp. 60, 62

• A full benchmark pass takes hours per dictionary, and one audit spent more compute on evaluation than training; board-game evaluations take minutes but cover specified properties only.Repeated seeds may be needed to resolve differences near the noise levels reported in Section 8.3.Source: §8.8 · p. 62

• Several measurements rely on proprietary checkpoints in self-published institutional reports, limiting independent repetition despite inspectable methods and observations.Open-model studies include refereed work and public preprints.Source: §8.9 · p. 62
• Crosscoder analyses partly trace apparent model-exclusive latents to the objective, while width comparisons show that latents can decompose across widths.The reported objective effect has no estimated fraction; these results qualify treating individual latents as canonical units.Source: §8.9 · pp. 62–63
• Released dictionaries, metric-audit code, and synthetic generators support comparisons under shared conditions, while frozen-random controls show that repeatable high scores can support narrower conclusions.Such resources enable checking reported differences against common data, controls, and matching rules.Source: §8.9 · pp. 63–64

9.1 Feature Splitting, Absorption, and Hedging

• A random 1000-latent dictionary beat the best two-latent objective on the reported distribution, while narrow dictionaries can merge correlated features and width comparisons depend on evaluation targets.The merge pattern also varies with sparsity level, and a reproduced merge may describe the model even if it differs from stipulated generating features.Source: §9.1 · pp. 66–67
• Dictionary objectives can split fixed features or make parent latents silent on joint activations; one spelling latent scored F1 = 0.81 yet missed _short, where another latent carried the intervention effect.A broader sweep reported absorption rates up to about 0.35 under hand-set thresholds, for first-letter spelling measurements.Source: §9.1 · pp. 66–67

9.2 Non-Canonicity and Seed Instability

• Across seeds, 94% of one width-3072 dictionary’s latents had a cosine match of at least 0.7; broader studies make reproducibility conditional on objective, initialization, and comparison level.Experiments report both stable subspaces with unstable directions and stable directions under particular objectives.Source: §9.2 · p. 68

9.3 Interpretability Illusions

• Random directions yield pattern rates of 82% for 33 directions versus 80% for 25 coordinates, and frozen-random parity limits what automated scores establish about feature recovery.One coordinate also received three mutually incompatible descriptions across three corpora; the cited setting reports no confidence intervals.Source: §9.3 · pp. 68–69

9.4 Dark Matter and Accumulated Replacement Error

• Residuals contain predictable omissions; removing residual-stream error nodes severely disrupts the model, while frontier replacement can match prompts by retaining unexplained computation.Exact prompt-level agreement therefore does not establish that recovered latents account for the computation assigned to error nodes.Source: §9.4 · pp. 69–70

9.5 Ontology Mismatch

• Curved manifolds can be covered by local atoms; subspace patching reproduces most whole-layer patching effects in Mistral 7B and Llama 3 8B, but does not show computation with the circle.GPT-2-small’s circles received no causal test, and a competing account leaves underlying feature dimensionality open; Gurnee et al. (2026a) examine a curved manifold in a counting task.Source: §9.5 · p. 70
• Llama-3.1-8B dictionaries showed dilution, while co-activation statistics separated some groups missed by decoder cosine; the Ising interpretation does not hold for fixed-cardinality TopK patterns.Dense features may remain in the residual because their frequent activation incurs sparsity penalties; the proposed residual separation assumes the autoencoder-introduced term is not linearly predictable from input.Source: §9.5 · pp. 70–71

9.6 Scope of the Negative Results

• Splitting, absorption, and control-score failures concern estimators or evaluations; dense and curved structure limit theoretical assumptions without contradicting results proved under them.Applying toy-objective conclusions to trained models requires evidence relating stipulated generating features to the model’s representation.Source: §9.6 · p. 71

10 Interventions and Applications

• Applications require tests of intended use: detection, intervention effects, and circuit prediction support distinct claims, and successful interventions alone do not establish that features outnumber dimensions.The review also reports robustness studies, model-diffing and fine-tuning analyses, and tests against biological, physical, and task-level references, with mixed evidence for useful interventions.Source: §10 · p. 72

10.1 Monitoring and Concept Detection

• Supervised detectors lead on named concepts, while frontier-scale dictionaries propose candidate concepts whose selected interventions support individual effects, not measured monitoring performance.Across 500 concepts, five supervised methods fitted on small synthetic datasets outperform autoencoder latents; some decoder-matched latents also remain silent when their feature is present, and only 7 of 83 GPT-2-small concepts clear the stated cosine threshold.Source: §10.1 · pp. 72–73

10.2 Steering

• 0.894 mean score for prompting exceeds rank-one representation-finetuning (0.543), difference-in-means (0.239), and autoencoder latents (0.165) in the steering benchmark.The benchmark tunes magnitude per concept and method; increasing magnitude reduces instruction following, while concept presence varies by intervention site. Supervised latent selection narrows the gap, and multiplier-based diffusion erasure can produce artifacts despite successful localization.Source: §10.2 · pp. 73–74

10.3 Unlearning and Targeted Editing

• Selected SHIFT latent ablation raises worst-group accuracy from 24.4 to 76.0 in Pythia-70M and from 18.2 to 50.0 in Gemma-2-2B, while random ablation preserves baseline performance.The result concerns one binary task and a supplied gender concept; selected editing beats random ablation, but comparisons with alternative methods vary by model and do not establish general unlearning.Source: §10.3 · pp. 75–76

10.4 Circuit Discovery and Mechanistic Prediction

• Downstream circuit connections improve latent-intervention prediction: normalized rank falls to 0.381 from 0.457 using examples, but substantial ranking error and unexplained computation remain.The preliminary comparison covers ten candidate groups; broader circuit yield is not quantified, and replacement models leave attention selection or error-node computation unexplained.Source: §10.4 · pp. 76–77

10.5 Robustness

• Whether reducing feature overlap at fixed capacity improves robustness while preserving task performance remains unresolved, and real-task perturbations must preserve ground-truth labels.The cited synthetic width control changes available capacity rather than removing overlap in a fixed-width trained model.Source: §10.5 · p. 77

10.6 Model Diffing and Fine-Tuning Analysis

• Model-diffing results can reflect the recovery objective: sparsity penalties favor shared latents, while a geometric fine-tuning filter reduces misaligned outputs from 87 to 57 in Gemma-2 2B.The filtering result uses eight diagnostic prompts and a language-model judge; remaining responses and a comparable judge-based control qualify its effectiveness as a remedy.Source: §10.6 · pp. 77–78

10.7 Architectures and Domains Beyond Language

• Beyond language, biological database matches exceed demonstrated regulatory correspondence (3 of 48 transcription factors), physical features do not align with standard decompositions, and widening barely changes forecasting performance.In the forecasting study, widening changes downstream performance by 0.214% on average and interventions on dominant latents barely change forecasts; the evidence is architecture- and study-specific.Source: §10.7 · p. 78
• Across three message-passing architectures, direction overlap varies with width in a phase pattern and decreases with sharper pooling, but L3 and L5 lack dictionary-recovery or intervention evidence.These measurements concern prescribed concepts; the recurrent-model study sweeps width on a delay task, and no study at scale in a selective state-space language model was identified.Source: §10.7 · p. 79
• Section 11 proposes tests for unmeasured intervention properties and sparsely evaluated architectures, including matched alternatives, magnitude sweeps, preserved behavior, and external representation references.The application evidence leaves these comparisons unmeasured or with little direct evaluation.Source: §10.7 · p. 79
• Supervised directions outperform dictionary latents on reviewed named-concept benchmarks; latent selection changes steering, SHIFT removes a spurious cue, and steering can damage preserved behavior.Biological and physical references also expose gaps between interpretable descriptions and attributed effects.Source: §10.7 · p. 79

11.1 Status of the Field’s Stated Open Problems

• Six negative findings constrain dictionary interpretations in analyzed settings: standard objectives can optimize without recovering directions, and frozen-random decoders match three standard scores.The reported dictionaries also lack demonstrated completeness or atomicity (Leask et al., 2025).Source: §11.1 · pp. 81, 83
• Toy-model optima and large-scale dictionary runs establish feasibility, not recovery quality; how capacity, training, and scaling shape trained representations remains unsettled.The survey identifies these as two positive answers that do not settle the questions about recovery or trained-network explanations.Source: §11.1 · p. 82
• Sharing can be optimal above a fourth-moment threshold, and polysemantic units can form even when m ≥ d; their proportions in trained models remain unmeasured.The survey reports both results in their own settings, while noting the trained-model measurement remains outstanding.Source: §11.1 · p. 84

11.2 Theoretical Questions

• Cross-layer capacity bounds must model the state available and dependencies between layers; simply summing layer widths would not distinguish shared representation from production followed by amplification.The latter is one proposed explanation for cross-layer latents, so single-site analyses may describe the relevant bottlenecks if it applies.Source: §11.2 · pp. 84–85

What Width Suffices for a Linear Map Followed by a Pointwise Nonlinearity?

• Width bounds for a linear map followed by a pointwise nonlinearity remain unresolved unless upper and lower bounds match inputs, nonlinearities, error criteria, and encoder constraints.The surveyed rates concern specified recovery criteria and do not order all decoder families into nested sets; correlated feature laws may also change optimal geometry.Source: What Width Suffices for a Linear Map… · p. 85
• Training guarantees for hand-built computation remain absent; trained computation questions require distinguishing features computed by the network from features merely readable by an external decoder.The survey also calls for identifying appropriate recovery targets beyond individual directions, including subspaces, manifolds, or hierarchies under stated assumptions.Source: What Width Suffices for a Linear Map… · p. 85

11.3 Empirical Questions

• A trained network with an independently justified feature inventory would enable recovery tests and a packing-degree count, but that count must fix the feature list, site, tolerance, decoder, and evaluation distribution.A packing degree above one supports superposition for the declared features and criterion; a failed recovery run alone cannot establish an upper bound.Source: §11.3 · pp. 86–87
• Architecture coverage is uneven, comparable audits of cross-layer replacements are missing, and tests must compare learned with matched-random intervention predictions and measure behavior-preserving reconstruction.The key unresolved measurement is whether dictionary reconstruction preserves the component responsible for tested behavior.Source: §11.3 · p. 87

12 Concluding Remarks

• Feature geometry constrains interference, while feature statistics, error tolerance, and decoder choice determine recoverability; feature count alone does not determine linear recovery or computed functions.The toy-model results also distinguish representations favored by different training objectives from the computation of specified functions.Source: §12 · p. 88
• In trained networks, reconstruction, feature matching, descriptions, and interventions provide different evidence, so their tests support different claims about recovered features and computation.Documented failures include sparse objectives altering recovered units, random directions receiving convincing descriptions, and omitted activation components affecting behavior; applications are specific to their task, alternatives, and measured effects on behavior intended to remain unchanged.Source: §12 · p. 88
• Stronger explanations require explicit feature targets and recovery criteria, evidence that recovered computations predict intervention effects, decoder bounds tied to observed layer activations, and training guarantees linked to computation in superposition.These are remaining empirical and theoretical questions for determining when superposition explains trained networks and when recovery provides only an incomplete behavioral description.Source: §12 · p. 88