PaperlayerView exact revision on arXiv

Source-linked AI summary

Interference Beyond Geometry in Concept Extraction

Valérie Costa, Bahareh Tolooshams

arXiv:2609.35351v1cs.LG

TL;DR

Geometric overlap alone is an incomplete measure of interference because it does not account for how feature codes are used. The paper defines effective interference and analyzes architectural mechanisms that shape it; SAE experiments show constrained architectures selectively reduce realized interactions while bias, gain, and encoder freedom allow constructive contributions to remain.

  • Problem

    Existing interference measures largely capture geometric overlap, although realized interactions also depend on whether and how strongly features co-activate.

    Source: §1 · pp. 1–2

  • Method

    The paper defines effective interference by combining feature geometry with code statistics and analyzes four architectural accommodation mechanisms under local fixed-support conditions.

    Source: §1 · p. 2

  • Results

    Across SAE families, global squared geometric overlap increases during training while squared effective interference falls from its peak, and architectural freedom shifts the balance toward positive net interference.

    Source: §5 · pp. 7–8

  • Takeaways & Limitations

    Interference in learned representations depends on feature use and architecture as well as geometry, with constrained encoders tending to orthogonalize co-active features and architectural freedom allowing constructive cross-contributions to remain.

    Source: §6 · p. 9

    Effective interference diagnoses energy allocation within a learned additive decomposition, not latent ground-truth interactions, representation quality, semantic interpretability, causal relevance, or downstream behavior.

    Source: §6 · p. 9

Abstract

from arXiv · show

Interference is commonly treated as geometric overlap between learned features. We introduce effective interference, which combines feature geometry and code statistics to capture realized interactions, distinguishing constructive from destructive interference and frequent weak interactions from rare strong ones. Under local fixed-support assumptions, we characterize how architectural constraints shape interference through four mechanisms: feature orthogonalization, bias compensation, gain adaptation, and encoder-decoder separation. Experiments with sparse autoencoders show that constrained architectures selectively reduce overlap among co-active features, while bias, gain, and encoder freedom allow constructive cross-contributions to remain. Together, these results show that interference in learned representations depends not only on feature geometry, but also on how features are used and on the architecture that produces their codes.

1 INTRODUCTION

Geometric overlap alone does not capture realized interference, because whether and how strongly features interact depends on their codes. The paper defines effective interference and studies how SAE architecture shapes these interactions.

1 INTRODUCTION

  • Existing measures largely treat interference as geometric overlap, despite evidence that weakly correlated codes can accompany little interference and correlated concepts can exploit overlap constructively.These observations motivate accounting for how concepts are used across data.Source: §1 · p. 1
  • Codes indicate which concepts co-activate and at what magnitudes, so they distinguish frequently realized overlaps from overlaps that rarely or never interact.Geometry alone assigns the same weight regardless of co-activation.Source: §1 · p. 1
  • Effective interference combines feature orientation, co-activation frequency, and conditional activation magnitude to describe realized interactions.The factorization separates dictionary geometry from code statistics.Source: §1 · p. 2, Figure 1 · p. 2
  • Under local fixed-support conditions, the paper derives four non-exclusive architectural routes for accommodating interference: orthogonalization, bias compensation, gain adaptation, and encoder-decoder duality.The analysis links these mechanisms to encoder consistency and cross-talk.Source: §1 · p. 2
  • SAE experiments across Pythia-160M, MNIST, and independently trained language-model features show distinct interference regimes shaped by architectural constraints.The reported patterns illustrate interactions that geometry alone does not capture.Source: §1 · p. 2

2 RELATED WORK

Prior work studies geometric overlap, superposition, feature extraction, and sparse-autoencoder theory, but the paper focuses on measuring realized interactions and the interference structure recovered by an extractor.

2 RELATED WORK

  • Most interference measures emphasize dictionary geometry, while comparatively few account for code statistics; this paper measures interference jointly through geometry and code use.Related approaches include information-theoretic formulations, co-occurrence studies, and conditional orthogonalization [Costa et al., 2025].Source: §2 · p. 2
  • Prior superposition work often starts from prescribed sparse features and code distributions, whereas this paper starts from model activations and asks what structure an extractor recovers.Prieto et al. (2026) show that co-activation statistics shape interference patterns.Source: §2 · p. 3
  • Concept-extraction methods include dictionary-learning approaches and local feature extraction, while SAEs learn concept dictionaries with sparse codes and are widely used in mechanistic interpretability.Architecture and optimization have motivated work on aligning projections with data structure and modeling temporal or hierarchical structure.Source: §2 · p. 3
  • Classical SAE recovery theory often assumes independent sparse codes and dictionary incoherence, while newer analyses identify recovery limits and instability across runs.The paper instead studies interactions in learned additive representations and how an amortized encoder accommodates them across supports.Source: §2 · p. 3

3 EFFECTIVE INTERFERENCE IN ADDITIVE REPRESENTATIONS

Effective interference measures the expected, code-weighted cross-contribution between features in an additive representation. Its sign and profile describe how components combine, not whether the representation is beneficial or harmful.

3 EFFECTIVE INTERFERENCE IN ADDITIVE REPRESENTATIONS

  • Effective interference is the expected, symmetric, code-weighted interaction between two learned features in an additive representation.Its pairwise values can be collected into a matrix whose off-diagonal entries represent pairwise interactions.Source: §3 · pp. 3–4
  • Positive pairwise contributions are constructive and negative contributions destructive in reconstruction energy, but their signs alone do not determine reconstruction error or downstream behavior.The reconstruction-error change also depends on alignment with the input.Source: §3 · p. 4
  • Unlike cosine similarity, effective interference is support-aware and scale-aware because geometry sets orientation while code statistics determine whether and how strongly overlap is realized.The formulation assumes finite second moments and, unless stated otherwise, nonnegative active codes and normalized concepts.Source: §3 · pp. 3–4
  • The interference profile separates geometric orientation, co-activation frequency, and conditional magnitude, so equal average interference can reflect frequent weak or rare strong co-activation.When co-activation frequency is zero, effective interference is zero and conditional magnitude need not be defined.Source: §3 · p. 4
  • Signed interference separates constructive and destructive contributions, while net interference can vanish through cancellation even when both contributions are large.The net value alone can therefore obscure substantial interactions.Source: §3 · p. 4
  • Large geometric overlap with near-zero co-activation describes unrealized overlap, while high co-activation with small conditional magnitude and low co-activation with large magnitude describe frequent weak and rare strong interactions.These regimes are conflated by a single geometric or aggregate statistic.Source: §3 · p. 4

4 HOW SPARSE AUTOENCODERS HANDLE INTERFERENCE

The paper analyzes how SAE encoder and decoder constraints accommodate interactions under local fixed-support conditions. Tying, bias, gain, and encoder-decoder separation yield distinct, potentially overlapping routes for handling cross-talk.

4 HOW SPARSE AUTOENCODERS HANDLE INTERFERENCE

  • The analysis treats the encoder as locally affine on a fixed support and uses code optimality plus encoder consistency to identify architectural contributions to cross-talk.These analyses approximate a shared amortized encoder by per-input code optimality; the control settings relax one constraint at a time.Source: §4 · p. 5
  • With tied, normalized, bias-free weights, spanning code vectors on a support imply orthogonality among concepts that co-activate, rather than across the full dictionary.For two active features, the gradient result predicts a first-order non-increase in their overlap after a support-preserving step.Source: §4 · pp. 5–6
  • The orthogonalizing contribution applies on jointly active examples, so co-activation frequency controls its direct frequency and the tendency is selective, not global.The proposition concerns one-example gradient descent with a support-preserving step; the reported tendency also remains with higher k, batch updates, and Adam.Source: §4 · p. 6
  • With tied weights and a learned encoder bias, a shared bias can compensate for mean incoming cross-talk on active examples but generally cannot cancel it separately for each input.Tying removes self-response mismatch and fixed-support code optimality removes residual coupling in this setting.Source: §4 · p. 6
  • When encoder direction remains tied but its norm can vary, positive mean incoming cross-talk predicts gain below one and negative mean cross-talk predicts gain above one, given a positive denominator.Gain rescales self-response and incoming cross-talk, providing an alternative to decoder angular orthogonalization.Source: §4 · p. 6
  • Untying separates decoder interaction from inference cross-talk, allowing constructive effective interference to remain while incoming cross-talk is suppressed.The four routes are controlled limits of one balance and can coexist in practical encoders.Source: §4 · p. 6

5 RESULTS

SAE architectures redistribute rather than simply eliminate effective interference: constrained models reduce realized interactions, while bias, gain, and encoder freedom allow constructive contributions to remain. These patterns appear alongside approximate, not exact, fixed-support code optimality.

Training dynamics

  • Global squared geometric overlap rises during training while squared effective interference falls from its peak, and the code-optimality gap decreases but remains nonzero.The learned codes therefore approach fixed-support optimality, but only approximately.Source: §5 · p. 7

Decoder interactions

  • Pairs with large co-activation-weighted magnitude concentrate near zero or negative decoder orientation in tied, normalized, bias-free models, but can extend toward positive orientation with architectural freedom.This pattern recurs across other SAEs.Source: §5 · p. 7, Figure 3 · p. 7

Bias and gain

  • Predicted bias and encoder gain track their learned values across SAE families, with closer agreement for TopK and BatchTopK than for Matryoshka and JumpReLU.The larger deviations in the latter families indicate a less exact fixed-support approximation in this controlled setting.Source: §5 · p. 8, Figure 4 · p. 8

Encoder-decoder separation

  • Untying shifts co-active decoder pairs toward positive alignment while their encoder-decoder cross-gains remain near zero, separating constructive reconstruction from inference cross-talk.The pairwise profiles are consistent with approximate support-conditional biorthogonalization, but do not establish exact duality on every support.Source: §5 · p. 8, Figure 5 · p. 8

Interference contributions

  • Untying produces the largest constructive contribution across four SAE families: I+ + |I−| is roughly 40–50% of reconstruction energy, while net interference is approximately 15–25%.Bias, encoder gain, and independent encoder directions also shift the balance toward positive net interference relative to tied baselines; intermediate variants are not strictly ordered. The evidence is in within-family changes, not absolute rankings across SAE families.Source: §5 · pp. 7–8

Support size and dictionary width

  • Increasing support size raises co-activation frequency but lowers conditional magnitude at fixed dictionary width, while net interference shows no universal monotonic trend.At fixed support size, increasing dictionary width reduces co-activation frequency; the largest tied configurations have elevated FVU and warrant caution.Source: §5 · p. 9

6 CONCLUSION

Effective interference measures realized pairwise cross-contributions, but it does not establish representation quality or characterize latent ground-truth interactions. The conclusions support caution when interpreting SAE features and interventions, within a theory and experiment scope limited by local assumptions.

6 CONCLUSION

  • Effective interference diagnoses reconstruction-energy allocation, not representation quality, semantic interpretability, causal relevance, or downstream behavior.It also does not characterize interactions among latent ground-truth concepts or remain invariant to feature splitting or absorption.Source: §6 · p. 9
  • The theory assumes fixed supports and approximates a shared amortized encoder by per-input code optimality, which may degrade under distribution shift.The two-feature orthogonalization result uses a small, support-preserving gradient step and does not establish global convergence or cover larger supports, minibatch training, adaptive optimizers, or support transitions.Source: §6 · p. 9

C APPENDIX - ETHICS STATEMENT

  • The results support cautious interpretation of individual SAE features and interventions because their separability depends on architectural choices.The experiments support the proposed compensation and orthogonalization mechanisms without uniquely identifying one mechanism.Source: §6 · p. 9

A APPENDIX - AI USE STATEMENT

  • The authors used generative AI for manuscript polishing and organizational feedback, not for research formulation, design, derivations, or theoretical development.They also used Claude Code to adapt existing SAE implementation and evaluation code for cached Pythia activations; the architectures, objective, and evaluation quantities were fixed beforehand.Source: A APPENDIX - AI USE STATEMENT · p. 17

B APPENDIX - REPRODUCIBILITY STATEMENT

The appendix describes reproducibility materials and ethics boundaries, then states properties of effective interference and its relationship to geometric measures. The formulation assumes nonnegative codes; signed-code extensions are left unexplored.

B APPENDIX - REPRODUCIBILITY STATEMENT

  • The appendix reports model, data, training budget, architectures, initialization, optimization, and constraints, plus holdout reconstruction, sparsity, and dead-feature statistics.SAEBench results use public checkpoints identified by model, layer, and sparsity target, with SAEBench’s evaluation pipeline.Source: B APPENDIX - REPRODUCIBILITY STATEMENT · p. 17

C APPENDIX - ETHICS STATEMENT

  • The ethics statement says the work uses public pretrained models, datasets, and SAE checkpoints, with no new data collection or human subjects.Language corpora may contain personal or biased content, but analysis uses aggregate geometric statistics rather than individual examples or sensitive attributes.Source: C APPENDIX - ETHICS STATEMENT · p. 17

D APPENDIX - EFFECTIVE INTERFERENCE

  • Effective interference is invariant to positive rescaling of a feature direction and its code when their product is unchanged.If two features never co-activate, their effective interference is zero regardless of their directional overlap; under nonnegative coding, its sign follows the feature dot product when co-activation has nonzero probability.Source: D APPENDIX - EFFECTIVE INTERFERENCE · p. 18

D.1 RELATION TO GEOMETRIC MEASURES

  • Equal pairwise code second moments make the off-diagonal effective-interference matrix proportional to the off-diagonal Gram matrix.In the binary, unit-normalized case, effective interference is cosine similarity weighted by co-activation probability; mutual coherence instead supplies a support-agnostic worst-case bound that can be loose.Source: §D.1 · p. 18

D.2 NONNEGATIVE CODES

  • The paper formulates and reports results for nonnegative codes; signed-code profiles are proposed but not explored.Under nonnegative coding, the signed-code profile parameter κij equals 1 and recovers the original profile.Source: §D.2 · p. 18

D.3 OPTIMALITY AND CONSISTENCY

The appendix states an optimality condition for fixed-support least-squares codes and an encoder consistency identity. It also distinguishes decoder geometry from inference cross-talk and derives average co-activation probabilities under uniformly distributed supports.

D.3 OPTIMALITY AND CONSISTENCY

  • For a fixed decoder and support, any interior least-squares optimum satisfies the code-optimality condition.For nonnegative codes, interiority means active coordinates are positive, so the unconstrained first-order condition applies.Source: §D.3 · pp. 18–19
  • The encoder consistency identity balances self-response mismatch, incoming cross-talk, bias compensation, and residual coupling for each active feature.It applies when the encoder is locally affine.Source: §D.3 · p. 19

D.4 ENCODER-DECODER DUALITY

  • With an untied encoder, the decoder Gram matrix determines decoder geometry while the cross-Gram matrix determines inference cross-talk.These matrices need not share the same off-diagonal structure; when the active decoder has full column rank, the support-specific linear encoder is the ideal encoder conditioned on the current support.Source: §D.4 · p. 19

D.5 AVERAGE CO-ACTIVATION PROBABILITY PAIRS UNDER UNIFORMLY DISTRIBUTED SUPPORT

  • The appendix derives average co-activation probability pairs for a k-sparse autoencoder when support is uniformly distributed across features.The reported average uses the number of used features, excluding dead neurons.Source: §D.5 · p. 19

E APPENDIX - PROOF OF PROPOSITION 4.3

For two active features in a tied, normalized, bias-free model, the appendix analyzes how a normalized gradient step changes feature overlap. Normalization removes radial gradient components, and under stated conditions the step cannot increase the overlap measure to first order.

E APPENDIX - PROOF OF PROPOSITION 4.3

  • A gradient step cannot increase Δ(DS) to first order when the two directions overlap and their code-weighted feature vectors differ.This result assumes a tied, normalized, bias-free model with two active features and a sufficiently small, support-preserving step.Source: E APPENDIX - PROOF OF PROPOSITION 4.3 · pp. 20–21
  • For two active unit-norm features, the proof reduces the overlap-measure change to the first-order change in their direction correlation.It derives each active feature’s normalized update before analyzing that correlation’s dynamics.Source: E APPENDIX - PROOF OF PROPOSITION 4.3 · p. 20
  • Column normalization removes the radial gradient component, leaving only the tangent component to drive first-order feature-direction evolution.The proof uses this normalized update to characterize the change in correlation between the two active directions.Source: E APPENDIX - PROOF OF PROPOSITION 4.3 · pp. 20–21

F APPENDIX - PROPOSITION F.1

The appendix derives local interference dynamics for tied features and clarifies how normalization and tied encoder contributions shape those dynamics. It also shows that feature norms can provide an additional route for reducing interference.

F APPENDIX - PROPOSITION F.1

  • With unit-norm decoder features, the tied, bias-free update decreases the interference measure to first order for positively co-active features at nonzero, noncollinear correlations.Without normalization, the angular dynamics also depend on feature norms.Source: F APPENDIX - PROPOSITION F.1 · pp. 22, 24
  • Feature norms provide an additional degree of freedom through which gradient descent can reduce interference, so local orthogonalization need not be enforced.The update acts on the unnormalized dictionary, while the analysis tracks angles using its column-normalized version.Source: F APPENDIX - PROPOSITION F.1 · pp. 22, 24

G APPENDIX - COLUMN NORMALIZATION

  • Explicit renormalization and tangent-space gradient projection produce the same first-order dynamics on the unit sphere, with differences only in higher-order terms.The discrepancy is O(η^2).Source: G APPENDIX - COLUMN NORMALIZATION · pp. 25–26

H APPENDIX - GRADIENT COMPUTATIONS

  • In the tied setting, a feature affects reconstruction both directly through the decoder and indirectly through its activation.The gradient derivation treats the active support as locally fixed, with inactive features receiving zero gradient.Source: H APPENDIX - GRADIENT COMPUTATIONS · p. 27

I APPENDIX - EXPERIMENTAL DETAILS

The experiments use controlled Pythia-160M SAE runs with architectural variations in encoder weights, bias, and gain. Evaluation reports reconstruction, sparsity, and dead-feature statistics, with sparsity closely matched across settings while FVU generally falls as constraints are relaxed.

I APPENDIX - EXPERIMENTAL DETAILS

  • Pythia-160M layer-8 activations are used to compare TopK, BatchTopK, JumpReLU, and Matryoshka SAEs at width 16,384 and support size 40.Runs train for 200M tokens and use a fixed, disjoint 2M-token holdout set.Source: I APPENDIX - EXPERIMENTAL DETAILS · p. 28
  • The five settings vary weight tying, learned bias, per-feature encoder gain, or an independently learned encoder.Gain rescales encoder columns independently of decoder directions; the untied setting learns W separately.Source: I APPENDIX - EXPERIMENTAL DETAILS · p. 28
  • Figure 2 averages three seeds, while all other experiments use one; aggregate interference patterns are stable across seeds despite substantial dictionary differences.Table 1 reports cross-seed dictionary agreement averaged over three seed pairs.Source: I APPENDIX - EXPERIMENTAL DETAILS · p. 28, Table 1 · p. 28
  • Sparsity is closely matched across settings, while FVU generally decreases as constraints are relaxed.The holdout evaluation also reports the fraction of features that never activate.Source: I APPENDIX - EXPERIMENTAL DETAILS · p. 29, Table 2 · p. 29

J.1 EXTENDED RESULTS

Extended experiments examine interference across sparsity and width, in MNIST, and in independently trained language-model SAEs. The reported qualitative signatures persist in these settings, though absolute interference levels in SAEBench are not directly comparable to the controlled sweep.

J.1 EXTENDED RESULTS

  • The extended figures report the training-time transition of interference factors, a shift toward constructive decoder interactions with architectural freedom, and consistent untying effects across SAE families.The untying pattern is shown for TopK, BatchTopK, Matryoshka, and JumpReLU.Source: Figure 7 · p. 30, Figure 8 · p. 31, Figure 9 · p. 31

J.2 INTERFERENCE UNDER VARYING SPARSITY AND DICTIONARY WIDTH

  • Figure 10 examines TopK SAE interference while varying support size k and dictionary width p on Pythia-160M.The figure separates the k sweep from the p sweep.Source: Figure 10 · p. 32

J.3 GENERALITY OF THE INTERFERENCE SIGNATURES

  • The generality tests use MNIST and independently trained SAEBench language-model SAEs to check whether the main interference signatures persist beyond the controlled setup.The SAEBench checkpoints target k = 40 in Gemma-2-2B layer 12 and Pythia-160M layer 8.Source: §J.3 · p. 33, §J.3.2 · p. 33

J.3.1 A DIFFERENT MODALITY: MNIST

  • On MNIST, effective interference is lowest in the most constrained tied setting and increases as architectural degrees of freedom are added.The figure is presented as a different-modality comparison complementary to Figure 6.Source: Figure 11 · p. 33

J.3.2 INDEPENDENTLY TRAINED SAES: SAEBENCH

  • SAEBench SAEs show substantial effective interference and the same qualitative signatures across both model families, but their absolute levels are not directly comparable to the controlled sweep.The reported levels are generally higher than in the authors’ untied runs, and additional training choices may contribute.Source: §J.3.2 · p. 33, Figure 13 · p. 33, Figure 14 · p. 35
Loading 2609.35351v1…